Patent Application
20080005302
Coordination of media and UI on a presentation device may be understood in the context of a media server network.
Instead of a conventional PC, the media server 106 may comprise a variety of other devices capable of storing and distributing media content including, for example, a notebook or portable computer, a tablet PC, a workstation, a mainframe computer, a server, an Internet appliance, or combinations thereof. The media server 106 may also be a set-top box capable of delivering media content to a computer where it may be streamed, or the set-top box itself could stream the media content. As the media server 106 may be a full function computer running an operating system, the user may also have the option to run standard computer programs (e.g., word processing and spreadsheets), send and receive emails, browse the Internet, or perform other common functions.
In addition to storing media content, the media server 106 may be connected with a variety of media sources, for example, a cable connection 114, a satellite receiver 116, an antenna (not shown for the sake of graphic clarity), and/or a network such as the Internet 118. A user may thus control a live stream of media content (e.g., TV content) received, for example, via the cable connection 114, the satellite receiver 116, or antenna. This capability is enabled by one or more tuners residing in the media server 106. The one or more tuners may alternatively be located remote from the media server 106. In either case, the user may choose a tuner to fit any particular preferences. For example, a user wishing to watch both standard definition (SD) and high definition (HD) content may employ a tuner configured for both types of contents. Alternately, the user may employ an SD tuner for SD content and an HD tuner for HD content separately.
The TV content may be received as an analog (i.e., radio frequency) signal or a digital signal (e.g., digital cable). The received TV content may include discrete content packets, where each content packet includes actual TV content (i.e., audio and video data) and a policy or policies associated with the actual TV content. If TV content is received as an analog signal, discrete content packets may be created from the analog signal.
Digital rights management (DRM) policies may be employed to protect the actual TV content or video content digitally stored on the media server 106. Licenses may therefore be associated with the actual TV or video content. A license identifies keys used to decrypt video and TV content (e.g., content packets) that are encrypted as part of DRM. In particular, the keys are used to allow consumption or use of the actual video and TV content. In certain implementations, the content packets of received TV content may be encrypted or compressed. Encrypted content packets are typically decrypted with keys transmitted to or resident at the playback device or home network devices 122, 126.
The home environment 100 may also include one or more network devices functioning as media receivers 122, 126 placed in communication with the media server 106 through a network 128, for example, a local area network (LAN). In an exemplary embodiment, the media receivers 122, 126 may be a Media Center Extender device, for example, an Xbox 360™ (Microsoft Corporation, Redmond, Wash.). The media receivers 122, 126 may also be implemented as any of a variety of conventional media rendering or computing devices, including, for example, a set-top box, a television, a video gaming console, a desktop PC, a notebook or portable computer, a workstation, a mainframe computer, an Internet appliance, a handheld PC, a cellular telephone or other wireless communications device, a personal digital assistant (PDA), or combinations thereof. Furthermore, the media receivers 122, 126 may include a tuner as described above.
The network 128 may comprise a wired and/or wireless network, for example, cable, Ethernet, WiFi, a wireless access point (WAP), or any other electronic coupling means, including the Internet. The network 128 may enable communication between the media server 106, the media receivers 122, 126, and any other connected device through packet-based communication protocols, such as transmission control protocol (TCP), Internet protocol (IP), real-time transport protocol (RTP), and real-time transport control protocol (RTCP). Communications may be transmitted directly between devices over a LAN, or they may be carried over a wide area network (WAN), for example, the Internet 118.
One or more video display devices, for example a main TV 120 in the living room 102, a secondary TV 124 in the bedroom 104, and a video monitor 112 may be situated throughout the home environment 100. These video display devices may be connected with the media server 106 via the network 128 either directly or via the media receivers 122, 126. As shown in the example of
The media receivers 122, 126 may be configured to receive streamed media content, including video and TV content, from the media server 106. Media content, and particularly video and TV content, may be transmitted from the media server 106 to the media receivers 122, 126 as streaming media comprised of discrete content packets via any of the network protocols described above. The streamed media content may comprise video IP, SD, and HD content, including video, audio, and image files, decoded on the home network devices 122, 126 for presentation on the connected TVs 120, 124. The media content may further be “mixed” with additional content, for example, an EPG, presentation content related to the media content, a web browser window, and other user interface environments transmitted from the media server for output on the TVs 120, 124 or the monitor 112. Such additional media content may be delivered in a variety of ways using different protocols, including, for example, standard remote desktop protocol (RDP), graphics device interface (GDI), or hypertext markup language (HTML).
In addition to the media receivers 122, 126 and the video display devices 112, 120, 124, the media server 106 may be connected with other peripheral devices, including components such as digital video recorders (DVR), cable or satellite set-top boxes, speakers, and a printer (not shown for the sake of graphic clarity). The media server 106 may also enable multi-channel output for speakers. This may be accomplished through the use of digital interconnect outputs, such as Sony-Philips Digital Interface Format (S/PDIF) or TOSLINK® enabling the delivery of Dolby Digital, Digital Theater Sound (DTS), or Pulse Code Modulation (PCM) surround decoding.
As indicated in
In addition to the remotely sourced media 212 and remotely sourced UI information 214, command and control information 216 may also be transmitted over the network link 208 between the media server 202 and the media receiver 204. This command and control information 216 traffic may be two-way in order to allow for coordination of a composite presentation to the user on the display device 206. As used herein, the term “composite” is not meant to refer to the analog video format called “composite video.” Instead, “composite” herein is meant to describe the combination or aggregation of video and UI components in a single presentation.
The remote UI 310 related to the audio media, for example, a music CD, is shown as generated by the media server 302. The remote UI 310 may be composed of playback information 322 at the bottom of the screen, which in the example of
Once the audio stream and remote UI information 310 from the media server 302 are received at the media receiver 304, the audio stream is decoded by an audio decoder module 318 within the media receiver. The audio decoder module 318 may translate the audio stream, which is generally digitally compressed, into an analog signal for transformation into sound by the loudspeakers 308 associated with the display device 306. In alternate embodiments, the loudspeakers 308 may be digital loudspeakers with the ability to directly transduce digital audio signals without intermediate decoding by the audio decoder module 318. In such embodiments, the digital audio signals may be forked by the media receiver 304 in order to provide the digital audio signals to both the loudspeakers 308 and the visualization engine 320.
The analog signal may also be passed from the audio decoder 318 to a visualization engine module 320 within the media receiver 304. The visualization engine 320 may be a software program that analyzes the frequencies and intensities of the audio signals and translates the frequencies and intensities into animated graphics. An exemplary graphic visualization 326 is depicted in
The media receiver 304 may output the remotely generated UI 310 and the locally generated UI 328 as a combined graphic output 330 for display as a composite UI 334 on the display device 306. The media receiver 304 may simply overlay the remotely generated UI 310 on top of the locally generated UI 328 or vice versa. As shown in
In this exemplary implementation the graphic visualization 326 of the locally generated UI 328 is synchronized with the audio output 332 rendered by the media receiver 304. The remotely generated UI 310 is also synchronized with the audio output 332 played over the loudspeakers 308 to graphically move the vertical bar of the playback information 322 horizontally and increment the time elapsed of the song while the song plays and stop both actions when the song is paused. When the song selection changes, the track name is changed in the remote UI 310 component of the composite UI 334. Further, when the album selection changes, the album title and the album picture 324 change in the remote UI 310 component of the composite UI 334 to reflect a different album.
The remote UI 410 related to the audio/video media, for example, a movie or television program, is shown as generated by the media server 402. The remote UI 410 may be composed of a video playback window 418 at the bottom of the screen in which the movie is presented. The remote UI 410 may also include interactive menu graphics 420 related to the software on the media server 402 for control of the home entertainment environment 400. A logo 422 identifying the presence of the home entertainment environment 400 framework may also be a part of the remotely generated UI 410. The remote UI 410 may further include a graphic or textual information (not shown) derived from metadata associated with the particular movie, for example, the movie title, the actors, director, producers, etc.
Once the audio/video stream and remote UI information 410 from the media server 402 are received at the media receiver 404, the audio/video stream is decoded by an audio/video decoder module 424 within the media receiver. The audio decoder module 424 may also translate the audio/video stream, which is generally digitally compressed, into analog video signals rendered as video frames on the presentation device 406 and analog audio signals transduced into sound by the associated loudspeakers 408. In alternate embodiments, the display device 406 and the loudspeakers 408 may be digital with the ability to directly render the digital audio/video signals without intermediate digital/analog conversion by the audio/video decoder module 424, or with partial intermediate decoding by the audio/video decoder module 424.
Once decoded, metadata associated with the video signal may also be passed from the audio/video decoder 424 to a metadata decoder module 426 within the media receiver 404. The metadata decoder 426 may be a hardware chipset or a software program that analyzes the video signal and extracts embedded data, for example, closed captioning information 428. The closed captioning information 428 or other metadata may be incorporated into a locally generated UI 432 by a graphic UI generator module 430 in the media receiver 404. The media receiver 404 may further be configured to generate additional local UI 432 components. For example, the media receiver 404 may accept remote control or push button commands from a user to control playback of the video or volume of the audio at the presentation device 406. As depicted in
The media receiver 404 may output the audio/video media, the remotely generated UI 410, and the locally generated UI 432 as a combined audio/video graphic output 438 for display as a composite UI 440 on the presentation device 406. The audio portion of the combined audio/video graphic output 438 is separated for playback on the loudspeakers 408 associated with the presentation device 406.
The media receiver 404 and the media server may coordinate the layout of the remotely generated UI 410 with the locally generated UI 428 via data exchange and command and control communications transmitted via the backchannel 436. For example, as shown in
In the exemplary implementation of
An exemplary hardware and operating environment for implementing the invention is depicted in
The system bus 518 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, a switched fabric, point-to-point connections, and a local bus using any of a variety of bus architectures. The system memory 504 may also be referred to as simply the memory, and includes read only memory (ROM) 506 and random access memory (RAM) 505. A basic input/output system (BIOS) 508, containing the basic routines that help to transfer information between elements within the computer 500, such as during start-up, is stored in ROM 506. The computer 500 further includes a hard disk drive 530 for reading from and writing to a hard disk, not shown, a magnetic disk drive 532 for reading from or writing to a removable magnetic disk 536, and an optical disk drive 534 for reading from or writing to a removable optical disk 538 such as a CD ROM, DVD, or other optical media.
The hard disk drive 530, magnetic disk drive 532, and optical disk drive 534 are connected to the system bus 518 by a hard disk drive interface 520, a magnetic disk drive interface 522, and an optical disk drive interface 524, respectively. The drives and their associated computer-readable media provide nonvolatile storage of computer-readable instructions, data structures, program modules and other data for the computer 500. It should be appreciated by those skilled in the art that any type of computer-readable media that can store data that is accessible by a computer, for example, magnetic cassettes, flash memory cards, digital video disks, RAMs, and ROMs, may be used in the exemplary operating environment.
A number of program modules may be stored on the hard disk 530, magnetic disk 532, optical disk 534, ROM 506, or RAM 505, including an operating system 510, one or more application programs 512, for example, a multimedia software package, other program modules 514, for example, a visualization engine, an audio/video decoder, a UI generator, and a metadata decoder, and program data 516, for example, media content. A user may enter commands and information into the personal computer 500 through input devices such as a keyboard 540 and pointing device 542, for example, a mouse. Other input devices (not shown) may include, for example, a microphone, a joystick, a game pad, a tablet, a touch screen device, a satellite dish, a scanner, a facsimile machine, and a video camera. These and other input devices are often connected to the processing unit 502 through a serial port interface 526 that is coupled to the system bus 518, but may be connected by other interfaces, such as a parallel port, game port, or a universal serial bus (USB).
A monitor 544 or other type of display device is also connected to the system bus 518 via an interface, such as a video adapter 546. In addition to the monitor 544, computers typically include other peripheral output devices, such as a printer 558 and speakers (not shown). These and other output devices are often connected to the processing unit 502 through the serial port interface 526 that is coupled to the system bus 518, but may be connected by other interfaces, such as a parallel port, game port, or a universal serial bus (USB). In the case of a media server, a tuner 560 may also be connected with the system bus in order to receive and convert analog content signals into digital format and similarly convert digital content into analog signals for transmission to connected devices.
The computer 500 may operate in a networked environment using logical connections to one or more remote computers, such as remote computer 554. These logical connections may be achieved by a communication device coupled to or integral with the computer 500; the invention is not limited to a particular type of communications device. The remote computer 554 may be another computer, a server, a router, a network personal computer, a client, a peer device, or other common network node, and typically includes many or all of the elements described above relative to the computer 500, although only a memory storage device 556 has been illustrated in
When used in a LAN 550 environment, the computer 500 may be connected to the local network 550 through a network interface or adapter 528, which is one type of communications device. When used in a WAN 552 environment, the computer 500 typically includes a modem 548, a network adapter, or any other type of communications device for establishing communications over the wide area network 552. The modem 548, which may be internal or external, is connected to the system bus 518 via the serial port interface 526. In a networked environment, program modules depicted relative to the personal computer 500, or portions thereof, may be stored in a remote memory storage device. It is appreciated that the network connections shown are exemplary and other means of and communications devices for establishing a communications link between the computers may be used.
The technology described herein may be implemented as logical operations and/or modules in one or more systems. The logical operations may be implemented as a sequence of processor-implemented steps executing in one or more computer systems and as interconnected machine or circuit modules within one or more computer systems. Likewise, the descriptions of various component modules may be provided in terms of operations executed or effected by the modules. The resulting implementation is a matter of choice, dependent on the performance requirements of the underlying system implementing the described technology. Accordingly, the logical operations making up the embodiments of the technology described herein are referred to variously as operations, steps, objects, or modules. Furthermore, it should be understood that logical operations may be performed in any order, unless explicitly claimed otherwise or a specific order is inherently necessitated by the claim language.
The above specification, examples and data provide a complete description of the structure and use of exemplary embodiments of the invention. Although various embodiments of the invention have been described above with a certain degree of particularity, or with reference to one or more individual embodiments, those skilled in the art could make numerous alterations to the disclosed embodiments without departing from the spirit or scope of this invention. In particular, it should be understand that the described technology may be employed independent of a personal computer. Other embodiments are therefore contemplated. It is intended that all matter contained in the above description and shown in the accompanying drawings shall be interpreted as illustrative only of particular embodiments and not limiting. Changes in detail or structure may be made without departing from the basic elements of the invention as defined in the following claims.
