Patent Application
20130340012
The present invention generally relates to systems, software and methods and, more particularly, to mechanisms and techniques for providing main media content with a predetermined quality to a display or alternate media content when the main media content cannot be provided with the predetermined quality.
The mobile and fixed user terminals are capable, for example, to provide media content (data, video, text, games, etc.) from various sources to the user. There is a tendency to stream media content over a communication network to the user so that the user can watch on the user terminal a video or other multimedia from a variety of sources. One such application streams multimedia that is being constantly received by the user terminal over a packet switched network (e.g., Internet or a mobile network). Examples of popular services that provide streaming media include Youtube (www.youtube.com), SVT play (www.svtplay.se), BBC iPlayer (http://www.bbc.co.uk/iplayer/radio), Google TV (www.google.com/tv/) and Voddler (www.voddler.com). These are only some of the possible services available on the market. Those skilled in the art would appreciate that other services exist and also may appear in the future to provide media content. As noted above, these services may be offered to fixed or mobile user-terminals.
Mobile internet protocol television (IPTV) is another service that extends the Internet-based multimedia services into telecommunication provider networks by introducing network support for streaming media. Standardization activities in this area are ongoing as noted, for example, in Open IPTV Forum (see http://openiptvforum.org/) and ITU-T FG IPTV (http://www.itu.int/ITU-T/IPTV/). A general overview of mobile IPTV may be found in S. Park and S. Jeong, “Mobile IPTV: Approaches, Challenges, Standards and QoS Support”, IEEE Internet Computing, Vol. 13, Issue 3, pp. 23-31, May-June 2009, the entire content of which is incorporated herein by reference.
Transferring multimedia content from a server to a user terminal is schematically illustrated in
The communication network capacity determines how long time it takes before the user can start watching (or continuing to watch after a buffering event) a given content as the time to build up the play-out buffer depends on the available network capacity. The choice of the codec (i.e., multimedia quality) used to compress/decompress the media stream also influences the time necessary before the user can start watching a given content.
However, given the large increase of users that streamline the desire content and the slow increase of capacity of the exiting communication networks, there are instances when the communication network cannot keep up with the bandwidth demand or the servers providing the media content cannot provide the media content with a predetermined quality. Current multimedia systems react to these communication network/server events by either decreasing the quality of the video down to some lower limit or by temporarily pausing a play out process. While the streaming is paused, the image displayed at the user terminal is either frozen or showing a message such as “buffering”.
This reaction of the current multimedia systems is undesirable as the users do not like to pause the watched video because of the buffering problems. Thus, there is a need for finding a solution to avoid freezing the image or displaying annoying messages to the users. Accordingly, it would be desirable to provide devices, systems and methods that avoid the afore-described problems and drawbacks.
Assets as, for example, media content are desired to be provided to the users from various locations over a communication network. However, the existing systems may fail to provide the desired content at a desired quality and thus have to reduce the quality or not provide, temporarily, the desired content.
According to one exemplary embodiment, there is a method for selectively downloading media content through a communication network and selectively displaying the media content at a user terminal connected to the communication network. The method includes receiving main media content at the user terminal; determining at the user terminal level whether there are streaming resources available to maintain a predetermined quality of streaming the main media content; receiving secondary media content when the streaming resources are above a predetermined level; and displaying the main media content when the streaming resources are above the predetermined level and automatically displaying the secondary media content when the streaming resources are below the predetermined level. A switching functionality of the user terminal is configured to select which media content to be displayed as a function of the streaming resources.
According to another exemplary embodiment, there is a user terminal configured to selectively download media content through a communication network and selectively display the media content. The user terminal includes a main play-out buffer configured to receive main media content; a streaming detection unit configured to determine whether there are streaming resources available to maintain a predetermined quality of streaming the main media content, wherein the streaming detection unit is also configured to determine a state of the main play-out buffer; a secondary play-out buffer configured to receive secondary media content when the streaming resources are above a predetermined level, wherein the streaming detection unit is also configured to determine a state of the secondary play-out buffer; and a media switching unit connected to the streaming detection unit and configured to select whether to display the main media content or the secondary media content based on a result of the streaming detection unit. The main media content is displayed when the streaming resources are above the predetermined level and the secondary media content is automatically displayed when the streaming resources are below the predetermined level.
According to still another exemplary embodiment, there is a media application in a user terminal configured to selectively download media content through a communication network and selectively display the media content. The media application includes an interface configured to receive a state of a main play-out buffer that receives main media content, wherein the main media content is selected by a user of the user terminal; a streaming detection unit configured to determine whether there are streaming resources available to maintain a predetermined quality of streaming the main media content, wherein the streaming detection unit is also configured to receive the state of the main play-out buffer; the interface is further configured to receive a state of a secondary play-out buffer that receives secondary media content when the streaming resources are above a predetermined level, wherein the streaming detection unit is also configured to receive the state of the secondary play-out buffer; and a media switching unit connected to the streaming detection unit and configured to select whether to display the main media content or the secondary media content based on a result of the streaming detection unit. The main media content is displayed when the streaming resources are above the predetermined level and the secondary media content is automatically displayed when the streaming resources are below the predetermined level.
It is an object to overcome some of the deficiencies discussed in the previous section and to provide a mechanism for switching between a main media content and a secondary media content when a display quality of the main media content cannot be maintained. One or more of the independent claims advantageously provides a switching functionality or a media switching unit in a user terminal that achieves this function.
The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate one or more embodiments and, together with the description, explain these embodiments. In the drawings:
The following description of the exemplary embodiments refers to the accompanying drawings. The same reference numbers in different drawings identify the same or similar elements. The following detailed description does not limit the invention. Instead, the scope of the invention is defined by the appended claims. The following embodiments are discussed, for simplicity, with regard to the terminology and structure of a user terminal that receives from a communication network streams of desired content and display them. However, the embodiments to be discussed next are not limited to a user terminal but may be applied to other existing systems, e.g., a communication network, a node, etc.
Reference throughout the specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the present invention. Thus, the appearance of the phrases “in one embodiment” or “in an embodiment” in various places throughout the specification is not necessarily all referring to the same embodiment. Further, the particular features, structures or characteristics may be combined in any suitable manner in one or more embodiments.
According to an exemplary embodiment, a user terminal is configured to utilize extra network capacity, when available, for building up not only a main play-out buffer (a buffer that receives the desired media content selected by the user) but also for building alternate media content (e.g., advertisements, weather information, traffic information, sport information or other information of possible interest to the user) in a secondary play-out buffer. In addition, the user terminal may be configured to display the alternate media content when a minimum streaming-media rate for the main media content is temporarily not available. Thus, according to this exemplary embodiment, during the time the alternate media content is displayed, the main play-out buffer has the opportunity to rebuild itself. The user terminal may be also configured to switch back to the main media content when the main play-out buffer has enough data to sustain the desired quality of the displayed media content or when the communication network has enough streaming resources to meet the desired display quality.
A general functionality for achieving a media switching as discussed above is now presented in regard to
The user terminal may be configured (as will be discussed later) to have various functionalities for determining when the network has spare capacity, i.e., above the minimum capacity required for providing the main media content with the desired quality. When this condition is met, the user terminal uses the spare network capacity not required by the main media content download for downloading alternate or secondary media content in step 202. The alternate media content may be provided by the original or main server that provides the main media content or by a different server, the secondary media server. The secondary media content may be, for example, advertisements or other personal content.
In step 204, a determination may be made by the user terminal whether the main media content stream is distorted, e.g., there is a decrease in the network capacity, or there is a congestion at the main media server prevents the main media content to arrive at the user terminal, or the main play-out buffer is under-run, or other causes. When a distortion is determined, a switching functionality of the user terminal switches the stream fed to the display from the main media content stream to a secondary media content stream (from a secondary play-out buffer). The secondary play-out buffer, by storing, for example, advertisement information, is able to display this information on the display until the main play-out buffer has enough data to sustain a desired streaming quality.
In step 206, the switching functionality switches back to the main media content stream to be provided to the display when the distortion in step 204 is removed. This may happen when the data stored by the main play-out buffer is large enough or when the streaming rate reaches above a desired threshold. The above noted algorithm may be implemented in software, hardware or a combination thereof as discussed later.
One possible implementation of the above noted algorithm is now discussed with reference to
The user terminal 100 may include a main play-out buffer 102, a secondary play-out buffer 104, and a media application module 120. Alternatively, the user terminal 100 may include a display 130. However, it is noted that the display 130 does not have to be part of the user terminal 100. The main play-out buffer 102 may be part of a media player functionality 103 while the secondary play-out buffer 104 may be part of a media player functionality 105. These functionalities involve dedicated software for receiving, manipulating and feeding media content. In this respect, the main play-out buffer 102 is configured to receive the main media content, via the network 200, from the main media server 300. Similarly, the secondary play-out buffer 104 is configured to receive the secondary media content, via the network 200 or via another network, from the secondary media server 302. In one application, the main media server 300 and the secondary media server 302 are the same server.
For the sake of simplicity, it is considered in this exemplary embodiment that the main media server and the secondary media server are different entities and both of them are connected to the same network. Also, this embodiment shows the media servers being outside the network. However, in one application, one or both of the media servers are part of the network. Such a server or database 202 is illustrated in
As noted above, the user selects a desired media content, the main media content. The secondary media content may be either selected by the user or by the operator of the network. The operator of the network may select the secondary media content, to be displayed when the main media content cannot be displayed, based, for example, on preferences of the user, or a geographical location of the user terminal, or based on the network operator interest, or based on other considerations.
Both the main play-out buffer and the secondary play-out buffer are configured to communicate with the display 130 to provide the corresponding media content. Links 140 and 142 are illustrated in the figure for providing the communication between the buffers and the display. For controlling which play-out buffer to provide the media content to the display 130, a state of each play-out buffer is determined by a streaming detection unit 106. For this reason, the streaming detection unit 106 is connected via links 144 and 146 to the play-out buffers. The streaming detection unit 106 may include a state buffer detector unit 106a and a measurement unit 106b. An interface 122 may be provided to receive information from the two buffers. The state buffer detection unit and the measurement unit are discussed next.
The state buffer detector unit 106a is configured to determine a state of the play-out buffers. These states may be indicative of the buffer reaching a minimum threshold, i.e., buffer under-run and/or of the buffer reaching a high threshold, i.e., buffer is ok. These states are transmitted from the buffer detector unit 106a to a media switch unit 108, which may be part of the media application module 120 or may be a unit outside this module.
The measurement unit 106b may be configured to receive a state of the main play-out buffer (see link 148) and also a state of the network 200 via link 150. The state of the main play-out buffer may include a buffer trend, e.g., buffer is being depleted of information faster than receiving new information, which signals that soon the supply of the main media content to the display is going to be impacted (e.g., delayed). The state of the network may include a bandwidth capability of the network for streaming the main media content, loss, delay, etc. The measurement unit 106b may be configured to actively perform these measurements.
For example, suppose that a bandwidth 400 of the network 200 available for downloading the main media content varies over time as shown in
The communications between the various units of the media application 120 and also between the media application 120 and the network, the media servers, and the display are illustrated in
With regard to the switching unit 108, this unit is configured to coordinate the various states of the main and secondary play-out buffers and to send control messages to them, e.g., when to download information and when to provide their information to the display. It is noted that the switching unit 108 and the streaming detection unit 106 may be implemented in dedicated circuitry, a processor, software or a combination thereof as dictated by the circumstances. A possible algorithm for controlling the main and secondary buffers is now discussed with
Starting with step 500, a detection is made whether a buffer under-run state is present. If not, the process advances to step 506. This detection may be made in the buffer detection unit 106a or the media switching unit 108 based on states S1 and S2. Assuming that such a state is detected in step 500, the process continues to step 502 where it is determined whether the received state is related to the main buffer. If the answer is no the process advances to step 504 in which it is determined whether the main buffer is ok. If the answer is no, the process advances to step 506 or otherwise to step 508. In step 508, as the main buffer is ok, the control message C2 is sent by the media switching unit 108 to the main media player application 103 to instruct the main buffer 102 to provide its media content M1 to the display 130. Next, the media switching unit 108 sends the control message C3 to the alternate media player application 105 to instruct the secondary buffer 104 to pause sending media content M2 to the display 130. After this, the process advances to step 506.
However, if the result of step 502 is that the received state is from the main media player application 103, then the process advances to step 512 in which it is determined whether the secondary buffer is ok. If the answer is no the process advances to step 506 or otherwise to step 514. In step 514, the switching unit 108 sends a pause control message to the main media player application 103 to stop the main buffer and in step 516 it sends a play message to the alternate media player application 105 for instructing the buffer 104 to send its media content to the display 130.
From step 506, the process advances to step 520 where a check is made to determine whether a buffer ok state was received. If the answer is no the process returns to step 500 or otherwise it advances to step 522. In step 522 the state of the buffers are set to true or false as is the case and in step 524 various policy matters may be checked for the alternate media player application. An example of such a policy is when to start/stop the secondary media content. Other policies may be envisioned. If the answer in step 524 is to continue, the process returns to step 500 or otherwise advances to step 526 in which the secondary buffer is instructed to stop and to step 528 in which the main buffer is instructed to play. Then, the process returns to step 500.
Having discussed a possible configuration of the media application 120 and an algorithm for controlling the play-out of the main and secondary media content to a display 130, next some details about the various elements of the media application 120 are discussed. The buffer detection unit 106a may be configured that, upon a buffer reaching a minimum threshold, the buffer detection unit 106a sends a message buffer under-run(buffer_id) to the media switching unit 108 via a C1 control message where buffer_id either corresponds to the main media application buffer or to the alternate media application buffer. When the main media application buffer reaches a high threshold, the buffer detection unit 106a sends a message buffer OK(buffer_id) to the media switching unit 108 via the C1 control message.
The media switching unit 108 may be configured to switch back and forth between the main buffer and the secondary buffer based on various parameters and/or policies determined by the user or the network operator. The switching is determined at least based on the states of the two buffers. The media switching unit obtains buffer information from the buffer detection unit 106a via the C1 control message. The media switching unit may start/stop/pause the media applications via C2 and C3 control messages as discussed with reference to
The measurement unit 106b may be configured to obtain state information from the main media player application 103 via S3 (e.g., by observing buffer trends) and from the network 200 via S4 (e.g., through active measurements that provides parameters such as loss, delay, available capacity, etc.). The measurement unit 106b can communicate via control message C4 with the alternate media player application to start/stop/pause downloading of ads or other personal content from the secondary media server.
A state machine describing a possible functional implementation of the measurement unit 106b is now discussed with regard to
With regard to the main media server 300, this may be a standard server for streaming content. However, the secondary media server 302 may still be a traditional server but, for example, oriented towards advertisements, which could be personalized. For this reason, an end user database 304 may be provided to this server and may include information on user preferences with respect to advertisements. This database may be co-located with the advertisements and personal content server.
According to one or more embodiments, a mechanism is advantageously provided to mask from the user network problems and seamlessly integrate advertisements or other information into the streaming experience. Another aspect provides a mechanism to “at least show some media”, even if the user enters some geographical area not covered e.g., by mobile broadband, or if the media quality drops below some threshold due to network congestion. Still another aspect capitalizes on unused network capacity while not impacting on the quality of the media stream.
According to an exemplary embodiment illustrated in
An example of a representative user terminal capable of carrying out operations in accordance with the exemplary embodiments is illustrated in
The exemplary user terminal 800 suitable for performing the activities described in the exemplary embodiments may include a server 801. Such a server 801 may include a central processor (CPU) 802 coupled to a random access memory (RAM) 804 and to a read-only memory (ROM) 806. The ROM 806 may also be other types of storage media to store programs, such as programmable ROM (PROM), erasable PROM (EPROM), etc. The storage devices 804 and 806 may be used to implement the buffers 102 and 104. The processor 802 may communicate with other internal and external components through input/output (I/O) circuitry 808 and bussing 810, to provide control signals and the like. The processor 802 carries out a variety of functions as is known in the art, as dictated by software and/or firmware instructions.
The server 801 may also include one or more data storage devices, including hard and floppy disk drives 812, CD-ROM drives 814, and other hardware capable of reading and/or storing information such as DVD, etc. In one embodiment, software for carrying out the above discussed steps may be stored and distributed on a CD-ROM 816, diskette 818 or other form of media capable of portably storing information. These storage media may be inserted into, and read by, devices such as the CD-ROM drive 814, the disk drive 812, etc. The server 801 may be coupled to a display 820, which may be any type of known display or presentation screen, such as LCD displays, plasma display, cathode ray tubes (CRT), etc. A user input interface 822 is provided, including one or more user interface mechanisms such as a mouse, keyboard, microphone, touch pad, touch screen, voice-recognition system, etc.
The server 801 may be coupled to other computing devices, such as the landline and/or wireless terminals and associated watcher applications, via a network. The server may be part of a larger network configuration as in a global area network (GAN) such as the Internet 828, which allows ultimate connection to the various landline and/or mobile client/watcher devices.
The disclosed exemplary embodiments provide a user terminal, a method and a computer program product for switching between buffers for providing continuous media content, if possible, to a display. It should be understood that this description is not intended to limit the invention. On the contrary, the exemplary embodiments are intended to cover alternatives, modifications and equivalents, which are included in the spirit and scope of the invention as defined by the appended claims. Further, in the detailed description of the exemplary embodiments, numerous specific details are set forth in order to provide a comprehensive understanding of the claimed invention. However, one skilled in the art would understand that various embodiments may be practiced without such specific details.
As also will be appreciated by one skilled in the art, the exemplary embodiments may be embodied in a wireless communication device, a fixed device, a telecommunication network, as a method or in a computer program product. Accordingly, the exemplary embodiments may take the form of an entirely hardware embodiment or an embodiment combining hardware and software aspects. Further, the exemplary embodiments may take the form of a computer program product stored on a computer-readable storage medium having computer-readable instructions embodied in the medium. Any suitable computer readable medium may be utilized including hard disks, CD-ROMs, digital versatile disc (DVD), optical storage devices, or magnetic storage devices such a floppy disk or magnetic tape. Other non-limiting examples of computer readable media include flash-type memories or other known memories.
Although the features and elements of the present exemplary embodiments are described in the embodiments in particular combinations, each feature or element can be used alone without the other features and elements of the embodiments or in various combinations with or without other features and elements disclosed herein. The methods or flow charts provided in the present application may be implemented in a computer program, software, or firmware tangibly embodied in a computer-readable storage medium for execution by a specifically programmed computer or processor.
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/IB11/00348 | 2/22/2011 | WO | 00 | 9/3/2013 |
