Patent Application
20150304249
The present invention generally relates to systems, software and methods and, more particularly, to mechanisms and techniques for offering a user scrolling capabilities for smooth and fast access to various portions of a media file.
The evolution of terminals such as tablets, computers, smart phones, game consoles, personal digital assistants, etc., resulted not only in the possibility to consume video or audio content but also to fast forward (scroll) this content.
Progressive download is a way to download desired content from a server to the terminal. Using progressive download, the client application can start playback of the desired content before the entire content is downloaded. When the download starts, the client application stores the beginning of the desired content (e.g., media file) in a playout buffer. This phase is called initial buffering.
When the playout buffer contains a certain amount of the desired content (e.g., the first few seconds), the client application can start playback, while at the same time it continues to download the remaining of the desired content into the playout buffer. If the download speed is high enough, the download is “ahead” of the playback, and hence the user experiences a continuous media experience
If, on the other hand, the download speed is insufficient, or there are temporary connectivity problems between the terminal and the server, then the media playback can “catch up” with the download. This means that the playback can come to a point in the media file that is missing from the playout buffer. At this point, the media playback has to be paused until the download process acquires the necessary content and fills up the playout buffer with a few seconds of media content again. This process is called rebuffering. During this process, the image shown to the viewer is paused or frozen, which is undesirable.
The process of rebuffering is also used when a user scrolls ahead in the media file to a point that is not yet downloaded into the playout buffer. In this case, the playout process is paused and the terminal starts to download the media content from the point where the user scrolled. During this time, no content is shown to the viewer.
Thus, the rebuffering process significantly impacts the user's perceived service quality. For this reason, most users would accept temporary quality degradation rather than having to wait for the rebuffering. As many users actually have this habit of scrolling ahead the media files that they are watching, the network and service providers are looking for a solution to provide content to these users even when scrolling ahead.
Accordingly, it would be desirable to provide devices, systems and methods that are capable of downloading content and providing a continuous supply of the content even when the user scrolls ahead to any point of the content.
Assets as, for example, content are desired to be provided to the users and to be continuously (e.g., with no interruptions) available as the user scrolls ahead the content. Thus, a terminal used by the user would ideally be able to download the content when selected by the user and also be able to display the content as the user scrolls it ahead.
To address this problem, there is disclosed a method for scrolling ahead a desired content while the desired content is downloading to a terminal. The method includes a step of selecting the desired content for being downloaded in the terminal and starting at least first and second download processes. The first download process is configured to download a first variant of the desired content and the second download process is configured to download a second variant of the desired content. The second variant is different from the first variant. Thus, the terminal is configured playback the first variant and to playback the second variant when a user scrolls ahead the desired content or the first variant becomes unavailable.
Such a method may be implemented in a terminal that includes an interface configured to receive a user command for downloading the desired content at the terminal and a playout and download controller connected to the interface. The controller is configured to initiate the first and second download processes to obtain first and second variants of the desired content. The controller is also configured to instruct the media player which of the two variants to play. The controller may also be implemented in a server in order to provide the first and second variants to the terminal, depending on whether the user scrolls ahead or not.
The method may also be implemented in a computer readable medium that includes computer executable instructions, wherein the instructions, when executed by a processor, implement the method discussed above.
Thus, it is an object to overcome some of the deficiencies discussed in the previous section and to provide a terminal that downloads various variants of the same content at the same time. One or more of the exemplary embodiments advantageously provides a terminal that continuously provides a user with the desired content as the user scrolls it ahead as a short or no re-buffering process may be needed.
The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate one or more embodiments and, together with the description, explain these embodiments. In the drawings:
The following description of the exemplary embodiments refers to the accompanying drawings. The same reference numbers in different drawings identify the same or similar elements. The following detailed description does not limit the invention. Instead, the scope of the invention is defined by the appended claims. The following embodiments are discussed, for simplicity, with regard to the terminology and structure of a terminal connected to a server from which content is desired to be viewed and thus this content is downloaded onto the terminal via a communication network. However, the embodiments to be discussed next are not limited to these systems but may be applied to other systems.
Reference throughout the specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the present invention. Thus, the appearance of the phrases “in one embodiment” or “in an embodiment” in various places throughout the specification is not necessarily all referring to the same embodiment. Further, the particular features, structures or characteristics may be combined in any suitable manner in one or more embodiments.
According to an exemplary embodiment, there is a terminal that is configured to implement a method for scrolling ahead a desired content while the content is downloading at the terminal. The terminal may include a playout and download controller configured to start a first download process for the desired content. The first download process is configured to download a first variant of the desired content. The controller also starts a second download process for the same desired content. The second download process is configured to download a second variant of the desired content, and the second variant is different from the first variant. In this way, the controller may instruct a media player to playback the first variant and also may instruct the media player to playback the second variant instead of the first variant when a user scrolls ahead the desired content or the first variant becomes unavailable.
Thus, two variants of the desired content are available at the same time for viewing, the first variant including the desired content from its beginning while the second variant includes parts of the desired content. These parts may start from a point later in time than the beginning of the desired content or from the beginning of the desired content. The second variant may also be of a lower quality than the first variant so that its download does not significantly affects the downloading of the first variant, which may be the full-scale version of the desired content. In this way, the viewer is offered the first variant for viewing and when the viewer scrolls ahead, the terminal or server is configured to immediately or continuously offer the second variant for viewing and avoid the rebuffering process.
This process is possible as the desired content may be coded using a layered media coding. Layered media coding (or scalable media coding) is a technology where media is coded such that a subset of the coded bit stream can be decoded and used to reconstruct the media, albeit with poorer media quality. For example, the media may be coded to have a basic layer and extra layers. The basic layer, when available, may be used by the playback application of the terminal to provide the user with a viewing of the desired content. If an extra layer is also available, the viewing quality produced by the basic layer and the extra layer is improved. Thus, by having further extra layers, the quality of the displayed desired content continues to increase. In one application, an extra layer can be used by itself by the playback application to display the desired content. In other words, it is possible to have only an extra layer and not the base layer and still offer a viewing of the desired content (at a low quality). In another application, the playback application may use the extra layer “n” only if the basic layer and the previous “n-1” layers are available.
In still another application, instead of using the basic layers and extra layers, the playback application may be configured to use different streams of data for displaying the desired content. For example, a server that stores the desired content may store a first stream which is the full-rate of the desired content but also a second stream that is a low-rate of the desired content. The two streams may be played independently of each other in the terminal. In still another application, the first stream may be coded with a first bit rate while the second stream may be coded with a second bit rate. In yet another application, there is a single stream that is provided by the server storing the desired content but there is an intermediary server that is configured to transcode the single stream, i.e., can change the bit rate of the single stream to provide the first stream and also may allow the stream with the original bit rate to be the second stream.
In the following, the novel concepts discussed in the exemplary embodiments are equally applicable to layered media coding (i.e., content coded to have a base layer and extra layers) but also to the scenario in which different streams are available. For simplicity, in the following, only the layered media coding scheme is discussed.
According to an exemplary embodiment, the traditional single process of sequential download of the desired content (e.g., media file), which starts at the beginning of the media file to be download, is replaced with a plurality of download processes starting at different points in the media file, and using different coding rates. As long as the user is viewing the desired content sequentially, the terminal (or server) plays out the full-rate version of the media file, but at the same time, the terminal receives and stores a low-rate version of another part of the media file, ahead of the playing time.
As a result of this novel concept, if the user scrolls ahead in the media file, there is a high likelihood that the low-rate version is available for immediate viewing and thus, the playout application may start immediately, albeit with inferior media quality, providing the scrolled ahead desired content. The trade off for offering this convenient feature is that the initial buffering may take a longer time (depending on the capacity of the network, the terminal, etc.) than with a traditional single download process.
The novel process discussed above is now described with regard to
As the two processes advance, at a time t=T1, the first download process 12 has downloaded 5 seconds of the media file at full rate, and the second download process 14 has downloaded the base layer between 5 s and 25 s of the media file 10. At this time, the media playback application in the terminal may start a playout process 16 to display the desired content. The playout process starts from the beginning of the media file 10, while both download processes 12 and 14 continue. It is noted that the first process 12 downloads now the differential layers (i.e., the extra layers and not the base layer as the base layer has already been downloaded by the second process 14). This change in the first process 12 starts at the time reference point 5 s in the media file 10, while the second process 14 continues downloading the base layer from 25 s onwards.
As a result of having two simultaneous download processes 12 and 14, when the user starts viewing the media file 10, a large portion of the media file is available in a buffer of the terminal, coded at base layer. If the user now scrolls ahead in the media file or the first download process 12 is interrupted (e.g., connectivity problems with the server storing the media file), the playback process may start immediately, without waiting to download the content as that content is already available due to the second process 14. This process is illustrated in
At the same time, the first download process 12 also jumps to the time mark 20 s of the media file and starts downloading the differential layers from that point in the media file. Thus, at a time t=T3, the first download process 12 has downloaded the differential layers up to, for example, the time mark 28 s of the media file 10, and the playout process 18 can resume displaying the media file using the full rate coding again. In this particular case, it is noted that the second download process 14 has finished downloading the base layer of the media file 10 and for this reason is not illustrated in
Alternatively, if the terminal loses connectivity to the server at time t=T3, the controller of the terminal may be configured to present the user with the part of the media file 10 downloaded by the second download process 14. In other words, although there is no connectivity between the terminal and the server, the playout continuously presents the media file 10 to the user even if some parts of it are missing, e.g., between 7 s and 20 s in
The embodiments disclosed with regard to
Another variation may be adjusting the download strategy based on a predicted probability of (i) scrolling ahead or (ii) temporary connectivity problems. The network (e.g., server) or the terminal itself may be configured to determine how often the user scrolls ahead and also what is an average scrolling time for the user, i.e., how much in advance the user scrolls ahead. Thus, this prediction on scrolling can be based on past user behavior or the type of media or other factors. Assuming that a certain user scrolls ahead in average 20 s per movie, the server or the terminal may configure the second download process to download the media file 20 s from the beginning.
The prediction of temporary outage can be based on wireless connectivity conditions, user mobility, etc. Depending on the prediction, the number of download processes, as well as their start and stop times can be optimized. For example, if the server or the terminal determines that a connection between the terminal and the server providing the media file is poor, the second download process may be delayed, or even suppressed. All these predictions may be stored in a server and the terminal may have access to this data.
According to another exemplary embodiment, it is possible that one or more layers are downloaded as distinct parts of the file, not as a continuous file. For example, if the user scrolls to some position of the media file for which the cache does not yet have any downloaded data, the playout starts at a position of the media file where data is available and which is as close as possible to the position the user scrolled to.
This novel concept is illustrated in
At time t=T5, the user scrolls ahead to a time mark 20 of the media file that is not in the cache. Thus, the server or the terminal determines that the new playout process 18 should start from a different time mark in the media file that corresponds to available portion 10b. One criterion for determining where to start from when the portion where the user has scrolled to is not available may be the closest available portion, i.e., 10b in this case. In this way, a waiting time is decreased (at the expense of the precision of the scrolling). Note that there is a trade-off between video quality and precision in scrolling. Exactly how to balance between the waiting time and the scrolling precision may depend on user and/or operator preferences and/or may depend on an algorithm which calculates the maximum user perceived quality based on subjective perception of waiting time before the sequence starts playing and the decrease in subjective quality due to scrolling to a position that is not exactly the desired position in the sequence.
Using layered coded media, the total amount of data to be downloaded is not increased compared to state-of-the-art progressive media download. If the media file is not coded this way, as already discussed above, the novel concepts can still be applied by downloading two different streams, a full-rate version and a low-rate version. In this case, however, parts of the media will be downloaded multiple times, which adds to the total amount of data to be downloaded, and to the total download time. However, for a terminal having a high speed connection with the server this might not be a disadvantage. Optionally, the communication network can perform transcoding so that the second stream is obtained by transcoding the first stream.
In this respect, it is possible that the desired content is coded at different bit rates, a first stream being coded at a high bit rate and a second stream being coded at a low bit rate. Also, it is possible that the desired content is transcoded at different bit rates, a first stream being coded or transcoded at a high bit rate and a second stream being coded or transcoded at a low bit rate.
A system 30 that includes a terminal 32 and a server 34 that may be configured to implement the above noted novel concepts are now discussed with regard to
The terminal 32, as illustrated in
The controller 50 may also be in communication with a database 56 of the network or a third party provider that supply behavior information about the user of the terminal 32 or any other information relevant for estimating user scrolling behavior, such as terminal type, geographical information, service characteristics etc. In another embodiment, the network 36 or the third party provider may have a controller 58 that controls the download processes in the terminal so that no or minimal decisions are made in the terminal about the download processes and other processes.
A playout position 60 is acquired by the controller 50 in order to be able to coordinate the scroll process. In other words, based on the playout position 60 and on the contents of the cache 54, the controller 50 controls the media player 16 and creates/stops the download processes 12 and 14. The same may be achieved by the controller 58 in the network or in the third party provider. In another embodiment, the playout and download controller is split, having a component 50 in the terminal and a component 58 in the network.
In order to be able to receive the base layer and the extra layer noted above, the terminal 32 uses the layered media cache 54. Cache 54 is illustrated in
The Playout and Download Controller 50 may include software commands for achieving the desired download processes. Alternatively, the controller may be hard wired to achieve the desired processes. Also, a combination of both is possible. Assuming that tt denotes a point in the media file that the user wants to play out, TT is the media file length, K and P are pre-set constants (P is the required playout buffer time), then, for any media file that the user has opened, the following algorithm may be implemented:
if (the base layer is not available for the entire media) then
download the base layer's first missing portion after tt, modulo TT;
if (the extension layer is not available between tt and tt+K) then
endif
else
if (the extension layer is not available for the entire media) then
endif
endif
if (base layer is available between tt and tt+P) then
playout media at tt using the layers available
endif
Of course, the above example is simplistic and provided only to illustrate one possible approach. Other algorithms, more sophisticated, may be used.
Having the above noted structure in place, a terminal may reduce or eliminate a waiting time at rebuffering, when a user scrolls ahead in a media file. Alternatively, the terminal may reduce or eliminate a waiting time when temporary connectivity problems between the terminal and the server appear. For example, if the second downloading process has downloaded the base layer and the connectivity to the server is lost while the first downloading process still downloads the extra layers, the terminal is still capable of providing the user with the base layer, thus assuring that the user views the content but not necessary with at the best quality available.
A method for scrolling ahead a desired content while the desired content is downloading to a terminal is now discussed with regard to
For purposes of illustration and not of limitation, an example of a representative terminal capable of carrying out operations in accordance with the exemplary embodiments is illustrated in
The exemplary terminal 800 may include a processing/control unit 802, such as a microprocessor, reduced instruction set computer (RISC), or other central processing module. This unit may include the controller 50 discussed with regard to
The processing unit 802 may control the basic functions of the system as dictated by programs available in the storage/memory 804. Thus, the processing unit 802 may execute the functions described in
One of the programs that may be stored in the storage/memory 804 is a specific program 806. As previously described, the specific program 806 may receive a playout position for controlling a media player. The program 806 and associated features may be implemented in software and/or firmware operable by way of the processor 802. In one exemplary embodiment, the programs 806 and data 808 are stored in non-volatile electrically-erasable, programmable ROM (EEPROM), flash ROM, etc. so that the information is not lost upon power down of the parallel computing system 800.
The processor 802 may also be coupled to user interface 810 elements associated with a mobile terminal. The user interface 810 of the mobile terminal may include, for example, a display 812 such as a liquid crystal display, a keypad 814, speaker 816, and a microphone 818. These and other user interface components are coupled to the processor 802 as is known in the art. The keypad 814 may include alpha-numeric keys for performing a variety of functions, including dialing numbers and executing operations assigned to one or more keys. Alternatively, other user interface mechanisms may be employed, such as voice commands, switches, touch pad/screen, graphical user interface using a pointing device, trackball, joystick, or any other user interface mechanism.
The terminal 800 may also include a digital signal processor (DSP) 820. The DSP 820 may perform a variety of functions, including analog-to-digital (A/D) conversion, digital-to-analog (D/A) conversion, speech coding/decoding, encryption/decryption, error detection and correction, bit stream translation, filtering, etc. The transceiver 822, generally coupled to an antenna 824, may transmit and receive the radio signals associated with a wireless device. However, the transceiver 822 and the antenna 824 may be optional.
The terminal 800 of
The disclosed exemplary embodiments provide a terminal, a method and a server for scrolling ahead a desired content while the content is downloading to the terminal. It should be understood that this description is not intended to limit the invention. On the contrary, the exemplary embodiments are intended to cover alternatives, modifications and equivalents, which are included in the spirit and scope of the invention as defined by the appended claims. Further, in the detailed description of the exemplary embodiments, numerous specific details are set forth in order to provide a comprehensive understanding of the claimed invention. However, one skilled in the art would understand that various embodiments may be practiced without such specific details.
As also will be appreciated by one skilled in the art, the exemplary embodiments may be embodied in a wireless communication device, a telecommunication network, as a method or in a computer program product. Accordingly, the exemplary embodiments may take the form of an entirely hardware embodiment or an embodiment combining hardware and software aspects. Further, the exemplary embodiments may take the form of a computer program product stored on a computer-readable storage medium having computer-readable instructions embodied in the medium. Any suitable computer readable medium may be utilized including hard disks, CD-ROMs, digital versatile disc (DVD), optical storage devices, or magnetic storage devices such a floppy disk or magnetic tape. Other non-limiting examples of computer readable media include flash-type memories or other known memories.
Although the features and elements of the present exemplary embodiments are described in the embodiments in particular combinations, each feature or element can be used alone without the other features and elements of the embodiments or in various combinations with or without other features and elements disclosed herein. The methods or flow charts provided in the present application may be implemented in a computer program, software, or firmware tangibly embodied in a computer-readable storage medium for execution by a specifically programmed computer or processor.
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/IB2011/002060 | 9/6/2011 | WO | 00 | 7/14/2014 |
