Patent Grant
11671681
This Application is a Section 371 National Stage Application of International Application No. PCT/FR2015/050009, filed Jan. 5, 2015, the content of which is incorporated herein by reference in its entirety, and published as WO 2015/104490 A1 on Jul. 16, 2015, not in English.
The invention belongs to the field of digital content distribution and relates in particular to a method for selecting a representation of a digital content from a plurality of representations available for this same content when there is a rendering error.
With the democratization of high bitrate Internet access, users are increasingly using continuous broadcasting technologies referred to as “streaming”, which may be used to consult content without prior downloading. It may also be noted that users today have multiple terminals available from which this content may be consulted. It is, for example, possible to access a video from a personal computer, a connected television or even from a smartphone or tablet mobile terminal. These terminals have heterogeneous capabilities, notably in terms of display, connectivity and processing capacity. In order to make it possible to consult the same content in streaming from heterogeneous terminals, adaptive streaming technologies have been developed, such as the HTTP Live Streaming (HLS) solution developed by Apple® or also the MPEG-DASH (Dynamic Adaptive Streaming over HTTP) standard as described in the standard ISO/IEC 23009-1:2012.
In particular, the MPEG-DASH standard provides for splitting the contents to be broadcast into segments of short duration. Each segment is made available at different bitrates so that for each of them, the terminal may render the version most suited to its capabilities. For example, a connected television with a high bitrate Internet access may render a high quality representation of a digital content whereas a mobile terminal will select a low bitrate representation of the same content. MPEG-DASH also enables terminals to adapt dynamically and transparently to changing network conditions.
The addresses and characteristics of the various representations of the same content are published in a descriptive information medium such as a manifest file. In the case of MPEG-DASH, this file is called an MPD (Media Presentation Description). This file is intended for terminals and describes information on the segments available, such as the addresses of the segments, the video resolution or also the encoding bitrate of the various segments.
It may happen under certain conditions that a particular representation of a content announced in a manifest is no longer available at a given time. For example, an operator of a streaming server may decide to suspend the broadcasting of a high bitrate representation of a digital content when it finds that the server's load reaches a critical threshold. The sudden disappearance of a representation most of the time causes a playback error on terminals on which the suspended representation was in the process of rendering. This unexpected termination of playback is detrimental to the user experience. Indeed, on receiving such an error, a rendering system according to the prior art may, for example:
These behaviors generally induce a break in rendering leading to the abandonment of the consultation by the user when they occur too frequently. So techniques have been developed for informing a terminal of the suspension of broadcasting a particular representation.
A first technique, implemented on the terminal, consists in regularly downloading an update of the manifest file so as to have an up-to-date description of the various representations of a content. The HLS solution from Apple®, for example, performs an update of the manifest file each time a segment is downloaded. This technique, however, has the drawback of generating additional traffic for processing an exceptional problem.
Another technique, implemented on the streaming server, consists in inserting information into the stream itself. In particular it is possible to indicate that the manifest file has expired, so that the terminal may download a modified version of it. This technique also makes it possible to indicate to a terminal that a particular representation of a content is no longer available. However, this technique has drawbacks: first, it requires a particular formatting of the streams to be broadcast for inserting the information therein, and secondly the information may only be known to the terminal when it decodes the corresponding stream. When the broadcast of a particular representation is suspended, a terminal can no longer receive the information inserted in the corresponding stream.
There is therefore a need for a simple solution that minimizes the impact on the user experience in case of unavailability of a representation of a content in the process of rendering.
A first aspect of the invention relates to a method for processing an error in the course of the rendering of a digital content during a multimedia session on a terminal, said session offering a plurality of representations of said digital content described in a descriptive information medium for the multimedia session transmitted independently of the representation data, the method being such that after obtaining the descriptive medium, it comprises steps of obtaining, for at least one representation indicated in the descriptive information medium, at least one item of information regarding guarantee of availability of the data of the representation on a server; of comparing the information regarding guarantee of availability of the data of the representation on the server with at least one predetermined value; of selecting a fallback representation according to the result of the comparison, and on detection of a rendering error, of switching to the selected fallback representation.
The method thus provides for extracting from a descriptive information medium, such as a manifest file, an item of information regarding guarantee of availability associated with a particular representation of the content. Unlike the prior art, when a rendering error occurs following the sudden unavailability of a particular representation in the process of rendering, the terminal may select a representation the availability of which is guaranteed and continue the rendering of the content from this representation without risking a further interruption that could eventually cause an abandonment of the playback by a weary user. The method thus advantageously makes it possible to avoid a degradation of the user experience related to the termination of the rendering or to the selection of an alternative representation which is also not available.
According to another embodiment, the method is such that the fallback representation is selected prior to the error detection.
The terminal may thus switch very quickly to a representation the availability of which is guaranteed when a rendering error occurs.
According to a particular embodiment, the method is such that the fallback representation is selected at the time of the error.
In this way, if an up-to-date version of the manifest has been downloaded by the terminal between the start of rendering and the occurrence of the error, the selection of a fallback representation may be performed on the basis of this new manifest.
According to another embodiment, the method is such that the information regarding guarantee of availability includes an item of information representative of a type of guarantee belonging to the group including at least a guarantee of availability of the representation; a guarantee of speed of access to the representation; and a guarantee of availability from any access points, as well as an item of information of value associated with the type of guarantee.
The availability of a particular representation may thus relate to its availability over time and guarantee, for example, that the broadcast of the representation will not be suspended. In this way, following a rendering error, the terminal may continue the playback of a content by selecting a representation the broadcast of which will not be suspended.
The information regarding guarantee of availability may also correspond to a guarantee of reduced latency when accessing the content. In this way, a terminal may select an immediately available stream with the aim of improving the user experience when selecting an alternative representation. This guarantee of reduced latency may, for example, correspond to placing the corresponding representation in a network cache.
Availability may further correspond to an availability related to the network accessing the representation. For example, when some representations are only available in some parts of the network, the information regarding availability may enable the terminal to select a representation available locally and avoid the selection of a representation only available on some other PoPs (Points of Presence).
According to another particular embodiment, the method is such that the information regarding guarantee of availability further includes an item of data representative of a temporal period of validity associated with the type of guarantee.
The descriptive information medium may include an item of temporal information, such as a duration, associated with the information regarding availability. A terminal may thus know the duration during which the availability of a particular representation is guaranteed. This information is particularly useful when a content is composed of different sequences, since these sequences may not be available in all the representations. For example, an advertising spot may only be available in low quality whereas the content in which it is inserted may be available in high quality. The method according to the invention enables the terminal to select, for a predetermined period, a representation compatible with the advertising spot the availability of which is guaranteed.
According to another embodiment, the method is such that the step of selecting the fallback representation is repeated at the expiration of the period of validity associated with the information regarding guarantee of availability of the selected fallback representation.
When the availability of the representation in the process of rendering is no longer guaranteed, the terminal may thus select a new representation with a guarantee of availability associated with a future or unexpired period of validity.
The invention also relates to a carrier signal of descriptive information of a plurality of representations of the same digital content during a multimedia session, transmitted independently of the representation data to a terminal and characterized in that it includes an item of information regarding guarantee of availability associated with at least one representation of the digital content, said terminal being arranged for selecting, in case of rendering error, a fallback representation according to said received information regarding guarantee of availability.
According to another embodiment, the carrier signal of descriptive information is such that it further comprises an item of data representative of a temporal period associated with the guarantee of availability of at least one representation of the digital content.
The invention also relates to a device for processing an error in the course of the rendering of a digital content during a multimedia session, said session offering a plurality of representations of said digital content described in a descriptive information medium for the multimedia session transmitted independently of the representation data, the device being such that it comprises units for obtaining (306), for at least one representation indicated in the descriptive information, from the descriptive information, at least one item of information regarding availability of the data of the representation on a server; for comparing (305) the information regarding availability of the data of the representation on the server with a predetermined value; for selecting (307) a fallback representation according to the result of the comparison; and on detection of a rendering error, for switching to the selected fallback representation.
According to another aspect, the invention relates to a computer program comprising the instructions for the execution of the error processing method, when the program is executed by a processor.
Another aspect also relates to an information medium readable by a computer on which a computer program is recorded including instructions for the execution of the steps of the error processing method.
The signals, devices, programs and information media present advantages similar to those of the corresponding method described above.
Other features and advantages of the invention will appear more clearly on reading the following description of a particular embodiment of the invention, given as a simple, illustrative and non-restrictive example, and the accompanying drawings, in which:
In the description that follows, “representation” should be understood to mean a version of an encoded content at a particular resolution and/or bitrate. For example, in the case of a system of an MPEG-DASH type of adaptive streaming, the same digital content may be available in multiple resolutions and/or bitrates, each of these versions then corresponding to a representation.
In the rest of this disclosure, “manifest” should be understood to mean a description of the various representations of a digital content stored in a descriptive medium such as a file. The manifest may correspond, for example, to an MPD file according to the MPEG-DASH standard or also a “Playlist” file in the case of the Apple® HLS solution. Generally speaking, the manifest as understood in this document is a file constituting the entry point for a terminal wishing to render a digital content and contains in particular the access addresses to the various segments corresponding to the various representations of a content available to streaming playback as well as characteristics associated with these representations.
The notion of “guarantee of availability” referred to in this document particularly relates to a guarantee of making representation data available by the broadcasting server. As for the distribution of representation data from the broadcasting server to the rendering terminal this may not be guaranteed. It may also be noted that the information regarding guarantee of availability may also be interpreted as a risk of unavailability of a representation, a guarantee of availability of maximum value corresponding to a minimum risk of unavailability and vice versa.
The server 103 may, for example, host a digital content C1 (105) prepared for an adaptive streaming broadcast. In most of the adaptive streaming standards, the preparation of the content C1 for an adaptive streaming broadcast consists in producing different versions of this content, encoded, for example, with different qualities, resolutions or bitrates, so as to offer the terminals wishing to consult a version of this content suited to their rendering capability. The preparation of the content also includes a step of segmentation of the different representations into sections, called “media segments” or “chunks” that may represent a few seconds of broadcast. In the rest of this disclosure, “representation” will refer to a particular version of the same content. For example, the content C1 in
The server places at the disposal of the terminals a description of the available contents including in particular the characteristics of the different representations available for a given content. This description is, for example, termed an “MPD” (Media Presentation Description), “Playlist” or “manifest” according to the standard used. In the rest of this disclosure, “manifest” will refer to such a description. A manifest 121 may be transmitted in the form of a carrier signal of descriptive information. This manifest is, for example, downloaded by the rendering device 100 when it wishes to access the content C1 and may include an item of descriptive information 120 for the content C1 including the description (112, 113) of the different representations available. For example, the description of the representation R1 of the content C1 may include at least one URL (Universal Resource Identifier) 114 for accessing the segments composing the representation and an item of information 115 relating to the quality of the representation. Likewise, the description of the representation R2 of the content C1 may include at least one URL (Universal Resource Identifier) 117 for accessing the segments composing the representation and an item of information 118 relating to the quality of the representation. Thus, from the manifest 121, the rendering system 100 may access the representation of the content C1 most suited to its characteristics and begin the rendering.
It should be noted that the manifest describes the different representations of a content at a given instant and that the availability of a particular representation may evolve over time. In other words, a representation described in a manifest at a given instant may no longer be available at a following instant when the rendering has begun, which generally causes a playback error on the rendering system. For example, a server may decide to suspend the distribution of a high quality representation when its load exceeds a critical threshold. In other scenarios, some segments of a content may only be available in some representations. For example, in
The invention provides for associating with at least one representation an item of information regarding guarantee of availability of this representation. For example, the description of the representation R1 present in the manifest 121 includes an item of information regarding availability over time 116 making it possible to indicate to the terminal that the availability of all the segments of the representation is guaranteed. More generally, the information regarding guarantee of availability consists, for example, of a type of availability and an associated value, the associated value being, for example, a value included in a set of predefined values, a probability, a Boolean, a character string, or any other value capable of providing an item of information regarding availability.
According to one particular embodiment, the information regarding guarantee of availability may be accompanied by an item of temporal information representative of a period during which the information regarding guarantee of availability is valid.
According to one embodiment, the information regarding guarantee of availability relates primarily to the availability over time of all the segments of a given representation. For example, the information regarding availability 119 in
According to another embodiment, the information regarding guarantee of availability may also correspond to a latency when accessing a particular representation. Its value then reflects the fact that a representation is systematically cached so as to reduce the access time to the segments and thus to improve the convenience of playback. The information may, for example, take a value such as:
According to another embodiment, the information regarding availability may further relate to the availability of a representation from a particular access. For example, a high definition representation of a video content may not be accessible from a 3G mobile access or be accessible only from a residential access. The information regarding availability may then be expressed in the following way in the manifest:
It should be noted that the person skilled in the art is well aware of other means of expressing a type/value association in a document such as a manifest and that the examples above are given only as a guide for the sole purpose of illustrating this disclosure. On the other hand, these various embodiments may be combined, since the various items of guarantee information may cohabit in the same descriptive medium.
In this example, the rendering system 100 first downloads a manifest from the server 103, by transmitting, for example, a GET request 202 to the server. In the context of this invention, the location of the manifest is of no importance, it may, for example, be downloaded from a separate server to the server 108. In response 203 to this request, the server 103 transmits the manifest 121 to the rendering system which optionally stores it in a memory. From the manifest 121, the rendering system chooses according to certain criteria to render the representation R2107. The rendering system selects, for example, a high quality representation of the content C1 because the consultation is performed from a domestic environment with a high bitrate Internet access and compatible hardware. The rendering system begins the rendering by transmitting a GET request 204 to download the first two segments R2.1 and R2.2 of the representation R2 of the content C1, in response 205 to which it may receive the segments to be rendered. The rendering system 100 then transmits a new request 206 in order to obtain the next two segments 110 and 111. The requested segments may have been described in the manifest but no longer be available at the time of the request, e.g. for reasons previously cited. In this case, the server indicates the unavailability of the requested segments by an error 207.
According to the invention, the rendering system obtains from the manifest during a step 208, an item of information regarding guarantee of availability as described above associated with at least one representation of the content. For example, the rendering system may consult the manifest 121 and extract from it the availability values 116 and 119 corresponding to the representations of the content C1.
In step 209, the method according to the invention provides for performing a comparison with a predefined availability value. For example, the rendering system may compare the availability values 116 and 119 of the representations R1 and R2 (112, 113) with the value “ALW” indicating in this example a guaranteed availability. Still in this example, the rendering system may determine that the representation R1 has a guaranteed availability.
The rendering system may then select during a step 210 a representation the availability of which is guaranteed that will advantageously make it possible to continue the consultation of the content without risk of a new incident. In this example, the rendering system selects the representation R1 and transmits, during a step 213 a request 211 to download in response 212 the segments R1.3 and R1.4 (108, 109) and switch to the rendering of the representation R1.
According to one particular embodiment, the rendering system may also extract an item of temporal information associated with the item of availability data. This may be, for example, a period during which the availability of a representation can be guaranteed. This advantageously makes it possible to respond to the problem of inserting some advertising spots available only according to one representation as mentioned earlier, since the rendering system is thus capable of reselecting the deficient representation after the passage of a certain duration measured, for example, using a clock.
On initialization, the instructions of the computer program 303 are, for example, loaded into a RAM (Random Access Memory in English) memory, before being executed by the processor of the processing unit 302. The processor of the processing unit 302 implements the steps of the error processing method, according to the instructions of the computer program 303. For this, the device includes, in addition to the memory 301, a communication unit 304 (COM) enabling the device to connect to a communication network and exchange data with other devices via this communication network. This communication unit may, for example, be a network interface and is used notably to obtain a carrier signal of descriptive information, to transmit requests and receive responses through the communication network. The device also includes a unit for interpreting 306 (INT) the content of a manifest and particularly for reading an item of information regarding availability, a unit for comparing 305 (COMP) an availability value with a predefined value, a unit for selecting 307 (SEL) a representation according to the result of the comparison and a unit for switching 308 (SWITCH) to the selected representation.
According to one particular embodiment, the device may be incorporated in a digital decoder terminal, Set Top Box, connected television, tablet, mobile phone, personal computer, or even any other rendering equipment connected to a communication network.
An exemplary embodiment of the present invention improves the situation described in the Background section.
Although the present disclosure has been described with reference to one or more examples, workers skilled in the art will recognize that changes may be made in form and detail without departing from the scope of the disclosure and/or the appended claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 1450083 | Jan 2014 | FR | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/FR2015/050009 | 1/5/2015 | WO |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2015/104490 | 7/16/2015 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 8683542 | Henry | Mar 2014 | B1 |
| 8909736 | Bosch | Dec 2014 | B1 |
| 9338209 | Begen | May 2016 | B1 |
| 9432961 | Rao | Aug 2016 | B2 |
| 10939178 | Long | Mar 2021 | B2 |
| 20100188936 | Beppu | Jul 2010 | A1 |
| 20110138020 | Pantos | Jun 2011 | A1 |
| 20110246563 | Keum | Oct 2011 | A1 |
| 20120179833 | Kenrick | Jul 2012 | A1 |
| 20130007223 | Luby | Jan 2013 | A1 |
| 20130182705 | Avni et al. | Jul 2013 | A1 |
| 20130325609 | Sokolov | Dec 2013 | A1 |
| 20140130099 | Kunisetty | May 2014 | A1 |
| 20140150044 | Takahashi | May 2014 | A1 |
| 20140230003 | Ma | Aug 2014 | A1 |
| 20140282772 | Chen | Sep 2014 | A1 |
| 20150074732 | Green | Mar 2015 | A1 |
| 20150296269 | Tokumo | Oct 2015 | A1 |
| 20150365450 | Gaunt | Dec 2015 | A1 |
| 20150372927 | Lv | Dec 2015 | A1 |
| 20150381680 | Thornburgh | Dec 2015 | A1 |
| 20160149978 | Wissingh | May 2016 | A1 |
| 20160373790 | Tsukagoshi | Dec 2016 | A1 |
| 20200359062 | Pantos | Nov 2020 | A1 |
| Number | Date | Country |
|---|---|---|
| 2013004260 | Jan 2013 | WO |
| Entry |
|---|
| International Search Report and Written Opinion dated Apr. 21, 2015 for corresponding International Application No. PCT/FR2015/050009, filed Jan. 5, 2015. |
| Intel, “Another Potential Solution for Server and Network Assisted DASH”, 3GPP Draft; S4-130952 DASH SAND Solution, 3rd Generation Partnership Project (3GPP), Mobile Competence Centre; 650, Route Des Lucioles; F-06921 Sophia-Antipolis Cedex; France, vol. SA WG4, No. Vancouver Canada; Sep. 23, 2013-Sep. 27, 2013 Sep. 18, 2013, XP050727640. |
| Telefon AB LM Ericcson et al., “IMS Based Adaptive HTTP Streaming”, 3GPP Draft; S4-100783 IMS HTTP Streaming, 3rd Generation Partnership Project (3GPP) Mobile Competence Centre; 650 Route des Lucioles; F-06921 Sophia-Antipolis Cedex; France, vol. SA WG4, No. Barcelona, Spain; Nov. 8, 2010, Nov. 2, 2010, XP050527312. |
| English translation of the International Written Opinion dated Apr. 21, 2015 for corresponding International Application No. PCT/FR2015/050009, filed Jan. 5, 2015. |
| French Search Report and Written Opinion dated Aug. 8, 2014 for corresponding French Application No. 1450083, filed Jan. 7, 2014. |
| Number | Date | Country | |
|---|---|---|---|
| 20160337715 A1 | Nov 2016 | US |
