Patent Application
20130198342
Media designed for distribution over the Internet by servers come in many forms and formats. Media players residing on client devices interpret incoming media objects and convert the media into human-perceivable form, i.e., into audio and video form for outputting to a user.
Media players residing on client device may have individual capabilities and support a set of media formats, while media servers may offer media content in a number of different media formats. Conventionally, media servers specify to the media player the different media formats in which media content is available. The user then selects the proper media format(s) that is to be used by the media player on the client device. For example, an iOS media player may support the QuickTime media format while a Windows media player may support WMV media formats. Furthermore, different iOS devices may support different media formats i.e., iPhone supports MPEG-4 AVC baseline profile while iPad supports MPEG-4 AVC main profile. Accordingly, to allow the client device to play the media content, the user needs to understand the media formats and capabilities supported by client devices and then select the appropriate format for the device.
In accordance with the present invention, a method and apparatus is provided for negotiating a media format to be used by a client device to access a media object. The method includes receiving data over a communications network from a client device. The data specifies at least one media format in which the client device is able to render the media object. Based on the data received from the client device, one or more media formats is determined in which the media object is available so that the media object is renderable by the client device. The media object is delivered to the client device over the communications network in at least one of the one or more media formats.
In accordance to another aspect of the invention, a client device is provided which includes a network interface for communicating over a communications network and at least one media player for rendering media objects received by the network interface over the communications network. The client device also includes at least one output device for presenting the media object rendered by the media player. A processor is provided which is configured to send one or more parameters reflecting a media format that is supported by the media player to a media content source over the communications network.
To address the aforementioned problems and limitations, a client device receiving a media object and a server delivering the media object undergo a negotiation process to determine the media format in which the media object should be delivered to the client device. As detailed below, as part of the negotiation process the client device informs the server of the media formats it supports. In addition, the client device may inform the server of its media related capabilities i.e., native screen size, network preference, buffer size, and CPU load. The server in turn responds by selecting the proper representation for the media content, which may or may not require the server to perform transcoding and media format conversion. and before delivering the media object to the client. In this way media management on the part of the user can be simplified.
Multiple devices may communicate over network 4. As illustrated in the example of
In some examples the client device 6 is a mobile communications device such as a wireless telephone that also contains other functions, such as PDA and/or music player functions. To that end the device may support any of a variety of applications, such as a telephone application, a video conferencing application, an e-mail application, an instant messaging application, a blogging application, a digital camera application, a digital video camera application, a web browsing application, a digital music player application, and/or a digital video player application.
Furthermore, media content source 10 may communicate with client device 6 over network 4. In the example of
In the example of
A user 16 of client device 6 may request that web browser application 14 present a web page provided by server 10. In response to the request from user 16, web browser application 14 may cause client device 6 to send a request to server 10 via network 4. The request may be a Hypertext Transfer Protocol (“HTTP”) request, a HTTP with Secure Sockets Layer (“HTTPS”) request, or a request that employs another network protocol. In response to the request from web browser application 14, server 10 may send to web browser application 14 a response that includes the requested web page. Upon receiving the web page, web browser application 14 may cause client device 6 to render the web page.
The web page may include a media object that is to be presented as part of the web page. For example, the web page may identify the media object using one or more Hypertext Markup Language (“HTML”) tags. The media object may be audio data (e.g., a song, sound effect, musical composition, etc.), audio/video data (e.g., a movie, clip, animation, etc.), or other types of media. As illustrated in the example of
Each media object is a data structure that has a particular media format. Media objects may be made available in a variety of different media formats. The format of a media file is often evidenced by its file extension, which is the two or three-character code in the portion of the filename preceded by a dot (.) that indicates the type of file, such as the format in which the file was created. The file extensions for some examples of common media formats are shown in Table. 1. A media player can generally access the content of a limited number of media objects. Examples of media players include RealPlayer, Macromedia Shockwave Player, Windows Media Player, and MusicMatch Jukebox Plus. To access the content of a media object in a particular media format, the media player employs a compressor/decompressor standard commonly called a “codec.” Table 1 also lists some examples of common media formats for audio, video and multimedia and indicates the media format it is able to access.
Common codecs for Video includes MPEG-2, AVC/h.264, WMV, MPEG-4 ASP. Each codec supported by media player may associate a set of profiles and levels. For example, iPhone may support Baseline Profile with Level 3 while iPad may support Main Profile with Level 4. Common codecs for Audio includes AC3, AAC, and MP3.
In order to cause client device 6 to render the media object, media player 18 may cause client device 6 to output to server 10 a request for media data units of the media object. A media data unit (“MDU”) may be one or more video frames, a set of audio samples, or a unit of another type of media data. In response to this request, server 10 may send MDUs of the requested media object to client device 6. Server 10 may send MDUs of the requested media object to client device 6 in a variety of ways. For example, server 10 may use a media streaming protocol to send MDUs of the requested media object to client device 6. These media streaming protocols may include, by way of example, HTTP Live Streaming, Smooth Streaming, the Real-Time Streaming Protocol (“RTSP”), Real-Time Transport Protocol (“RTP”), Real-time Transport Control Protocol (“RTCP”), Real-Time Messaging Protocol (“RTMP”), Advanced Systems Format (“ASF”), Real Data Transport (“RDT”), Motion Pictures Experts Group (“MPEG)-2 transport stream, and other protocols. In another example, server 10 may send MDUs of the requested media object to client device 6 as a progressive download via HTTP or another network protocol. During a progressive download, all of the MDUs in the requested media object may be stored to a hard disk of client device 6, but playback of the MDUs may begin before all of the MDUs in the requested media object have been stored to the hard disk of client device 6. Of course, in other implementations transmission techniques such live streaming and video-on-demand may be employed as an alternative to progressive downloading, in which case details of the media player 18 shown in
After client device 6 begins to receive the media object, media player 18 may cause client device 6 to begin rendering the media object. Media player 18 may begin rendering the media object within a window embedded within the web page, within a media player window that is separate from the web page, within a full-screen window, or in another type of window.
User 16 may interact with web browser application 14 to request that web browser application 14 present a web page. When web browser application 14 receives a request from user 16 to present a web page hosted by web server 10, web browser application 14 may cause network interface 30 to send one or more messages to web server 10. The messages sent by network interface 30 include a request to retrieve the web page. Subsequently, network interface 30 may receive one or more messages that include the web page. When network interface 30 receives the messages that include the web page, web browser application 14 may begin to render the web page. While rendering the web page, web browser application 14 may determine that the web page includes an embedded media object. When web browser application 14 determines that the web page includes an embedded media object, web browser application 14 may cause media player 18 to start executing on client device 6.
When media player 18 starts executing on client device 6, a media input module 32 in media player 18 may cause network interface 30 to send one or more media request messages to web server 10. The media request messages instruct web server 10 to start sending a stream of MDUs to client device 6. In response to these media request messages, web server 10 may start to send a stream of MDUs to client device 6. When network interface 30 receives MDUs in the stream of MDUs, media input module 32 may temporarily store the MDUs in a media buffer 34.
In the example of
Server 10 typically makes media objects available in a limited number (.e.g., one or two) of media formats and allows the client to select the format that is compatible with the media player or players available to it.
Instead of simply having the server 10 present all of its available media formats to the client and allowing the client device to choose among them, in the present case the client device provides details of its media capabilities to the server. Based on these capabilities the server provides the client device with a list of media format options that is best suited for it. If there is more than one option, the client device can then select the most suitable option.
As shown in
The media capabilities that are provided by the client device 6 to the server 10 by the media format negotiation module 40 will typically include the codecs available to it and any other information needed by the client device to render the media object. These capabilities may include, by way of example, parameters such as mime type, sampling rate, language capabilities, bandwidth capabilities and so. In addition, physical device-specific parameters and attributes characterizing the client device may also be included in the media capabilities provided to the server 10. Such parameters may include, for instance, parameters describing the device display such as screen height and width and native resolution, client memory size, video buffer size, processing capabilities (i.e. number of processor cores, hardware A/V accelerator), A/V coding tools (i.e. ability to support CABAC, B pictures etc.), audio channel parameters and so on. Even if not required to render the media object, these additional parameters may allow the presentation of the media object to be better optimized.
The media capabilities may be provided in a device description document that is delivered by the media format negotiation module 40 to the server over the communications network *. The device description document may employ any format that allows the server to parse the information that it needs to select one or more media formats for the client device. In some implementations the device description document may be employ a format that presents the capability information as structured data, which is data that is organized in accordance with a schema. Examples of suitable device description formats that can present the media capability information as structured data include Extensible Markup Language (XML), JavaScript Object Notation (JSON), Ordered Graph Data Language (OGDL) and Comma-Separated Values (CSV). Of course, the information in the device description document may be presented in accordance with other formats and schemas, including those which do not employ structured data formats.
Table 2 shows an illustrative schema of a device description document that is formatted in accordance with XML. Likewise, Table 3 shows an example of a device description document for a particular client device which uses the schema of Table 2.
After receiving the device description document and determining the most suitable media format or formats in which it can provide the requested media object or objects to the client device, the server can respond to the client device with a media format description document which specifies the media format or formats it has selected as being most compatible with the capabilities of the client device. Table 4 shows an illustrative schema of a media format description document that is formatted in accordance with XML. Likewise, Table 5 shows an example of a media format description document for a particular server/client device pair which uses the schema of Table 4.
By informing the server of the client device's capabilities, the server will be better able to optimize the delivery of the media object to the client device. For example, if the client device has a native resolution of 640×480, even although its media player supports a resolution of 720p, the server may deliver a video object at a resolution of 640×480 in order to best utilize the capabilities of the client device while saving network bandwidth.
If the client device is presented with multiple media format options from which to choose, the selection may be performed in any of a variety of different ways. In one example, the user may provision the client device with one or more client profiles which specify a preferred media format that is to be used under a given set of conditions. The client profiles may be used by the media format negotiation module 40 when communicating with the server 10. For example, one client profile may be used in a scenario where a minimum bandwidth, processing power or other quality-of-service parameters are guaranteed, whereas another client profile may be used when one or more quality-of-service parameters are not guaranteed. Instead of using client profiles, in another implementation the media format negotiation module 40 may simply maintain a list of preferred media formats ranked from most preferable to least preferable. In yet another implementation the user may be prompted to manually select a preferred media format.
One illustrative example of a message exchange process used to negotiate a media format between a client device and server is shown in
Referring now to
Processor 201, memory 202, display 203, user controls 204, network interface and controller 212 and nonvolatile storage 205 are all coupled by an interconnect 206, such as one or more buses and/or a network connection, and are interoperable. The client device architecture 200 is constructed and operates according to known techniques, including a basic input/output system (BIOS), and operating system (OS), and one or more applications or user programs.
Those skilled in the art will recognize that, for simplicity and clarity, the full structure and operation of the client device is not being depicted or described herein. Instead, only so much of the client device is described as needed to facilitate an understanding of the systems and method being depicted and described herein. The remainder of the construction and operation of the client device may conform to any of the various implementations and practices known in the art.
Nonvolatile storage 205 conventionally contains a variety of user programs and user data 207, where the user programs are loaded into memory 202 for execution and may be employed in customizing the operation of such user programs. In the context of the present disclosure, programs 207 loaded into memory 202 include a browser 208 or similar application within which a media player 209 operates as a plug-in as well as a media format negotiation module or component 215.
Any of the disclosed methods can be implemented as computer-executable instructions stored on one or more computer-readable storage media (e.g., non-transitory computer-readable media, such as one or more volatile memory components (such as DRAM or SRAM), or nonvolatile memory components (such as hard drives) and executed on a processor. Any of the computer-executable instructions for implementing the disclosed techniques as well as any data created and used during implementation of the disclosed embodiments can be stored on one or more computer-readable storage media (e.g., non-transitory computer-readable media). The computer-executable instructions can be part of, for example, a dedicated software application or a software application that is accessed or downloaded via a web browser or other software application (such as a remote computing application). Such software can be executed, for example, by a processor on a single local computer (e.g., any suitable commercially available computer) or in a network environment (e.g., via the Internet, a wide-area network, a local-area network, a client-server network (such as a cloud computing network or other such network) using one or more network computers.
| Number | Date | Country | |
|---|---|---|---|
| 61591249 | Jan 2012 | US |
