Patent Application
20080109556
Implementations described herein relate generally to streaming media and, more particularly, to adaptive buffering of streaming media. More specifically, the invention as described by means of examples relates to methods and devices for processing of streaming media, in order to adaptively insert content in streaming media.
Streaming media typically includes audio and video transmitted over a communications link, such as the Internet, in a streaming or continuous fashion. Unlike media data files, such as sound files or video files, that are played after they are downloaded, streaming media is played within a few seconds of requesting it, and the data is not stored permanently in the computer. If the streaming media is broadcast live, it is sometimes called “real-time media.” However, technically, real-time means no delays, and there is a built-in delay in streaming media. The basic idea behind the concept of streaming media is that is annoying to listen to momentary blips in music or a conversation, or frozen or staggering frames in a video signal, and the only way to compensate for that over an erratic network, such as the Internet, is to get some of the audio data into the user device before starting playback. In streaming audio, both the client, i.e. the user device, and server cooperate for uninterrupted sound. The client side stores a few seconds of sound in a buffer before it starts sending it to the dedicated output device, such as a speaker or a display. Throughout the session, it continues to receive media data ahead of time. Therefore, in streaming media applications, streaming audio and/or video data may be played back without the data being completely downloaded first. Streaming media may, thus, be viewed or listened to in quasi “real-time” as the data is received. Streaming media may be user-controlled, e.g., on-demand, pay-per-view movies, etc., or server-controlled, e.g. webcasting.
There are several Internet-based streaming services including, for example, audio books, news and podcasts. Podcasting may include the distribution of real-time streaming audio and/or video data over the Internet for user viewing and/or listening. Podcasting typically involves a subscription feed of automatically delivered new audio and/or video content. Audio streaming of voice or music content may include the distribution of media containing voice or music over the Internet for user listening.
A problem related to streaming media is that sometimes the bandwidth of the communications link over which the streaming media is provided may be uneven and insufficient. Delivery of streaming media over a low bandwidth connection can lead to a poor user experience if the continuous stream is constantly interrupted because buffering is needed due to depletion of data in the buffer. This may be the case both for wired connections to a communications network such as the Internet, and over the air communication links via radio, such in a IEEE 802.11 WLAN connection or a connection over a mobile telephony network such as a WCDMA network, and the problem is typically more common in mobile applications where bandwidth generally is more restricted.
A general object of the invention is therefore to provide a solution for processing streaming media signals such that the user experience is enhanced.
According to a first aspect, this object is fulfilled by means of a method for processing streaming media in a user device, comprising the steps of:
In one embodiment the method may comprise the steps of:
In one embodiment the step of analyzing remaining media data in the first buffer is carried out when the first buffer is emptied to a predetermined level.
In one embodiment the predetermined level is a certain percentage of the storage capacity of the first buffer.
In one embodiment the step of analyzing remaining media data in the first buffer comprises the step of identifying a location in the first buffer that stores an inactive period.
In one embodiment the step of analyzing remaining media data in the first buffer comprises the step of identifying a halt flag in the media data.
In one embodiment the inactive period comprises a silent period in an audio portion of the streaming media data.
In one embodiment the silent period comprises a pause in speech contained in the audio portion.
In one embodiment the inactive period comprises a still period in a video portion of the streaming media data.
In one embodiment the certain percentage of the first buffer's capacity comprises 20%.
In one embodiment the step of receiving streaming media data comprises the steps of:
In one embodiment the step of receiving streaming media data comprises the steps of:
In one embodiment the method comprises the step of:
In one embodiment the method comprises the steps of:
In one embodiment the step of determining a suitable re-filling time period comprises the steps of:
In one embodiment the step of receiving streaming media data comprises the steps of:
In one embodiment the second content is selected based on which of said additional sets of content have been previously played while re-filling the first buffer.
In one embodiment the second content is selected based on a priority order between said additional sets of content.
In one embodiment the method comprises the steps of:
In one embodiment the step of resuming playback of the first content is carried out when the second content has reached an end point.
In one embodiment the step of resuming playback of the first content is carried out when the first buffer has been re-filled to a predetermined level.
In one embodiment the predetermined level is a certain percentage of the storage capacity of the first buffer.
In one embodiment the certain percentage of the first buffer's capacity comprises 80%.
According to a second aspect, the object is fulfilled by means of a user device for playback of streaming media, comprising:
In one embodiment the user device comprises:
In one embodiment the user device comprises:
In one embodiment the buffer controller is configured to analyze remaining media data in the first buffer when the first buffer is emptied to a predetermined level.
In one embodiment the predetermined level is a certain percentage of the storage capacity of the first buffer.
In one embodiment the buffer controller is configured to analyze remaining media data in the first buffer by identifying a location in the first buffer that stores an inactive period.
In one embodiment the buffer controller is configured to analyze remaining media data in the first buffer by identifying a halt flag in the media data.
In one embodiment the inactive period comprises a silent period in an audio portion of the streaming media data.
In one embodiment the silent period comprises a pause in speech contained in the audio portion.
In one embodiment the inactive period comprises a still period in a video portion of the streaming media data.
In one embodiment the certain percentage of the first buffer's capacity comprises 20%.
In one embodiment the user device comprises:
In one embodiment the buffer controller is configured to
In one embodiment the buffer controller is configured to select the second content from one of the plurality of separate buffers, based on which of said additional sets of content have been previously played while re-filling the first buffer.
In one embodiment the buffer controller is configured to select the second content from one of the plurality of separate buffers, based on a priority order between said additional sets of content.
In one embodiment the user device comprises:
In one embodiment the buffer controller is configured to resume playback of content from the first buffer when the second content has reached an end point.
In one embodiment the buffer controller is configured to resume playback of content from the first buffer when the first buffer has been re-filled to a predetermined level.
In one embodiment the predetermined level is a certain percentage of the storage capacity of the first buffer.
In one embodiment the certain percentage of the first buffer's capacity comprises 80%.
In one embodiment the user device is configured to receive a data stream comprising:
According to a third aspect, the object is fulfilled by means of a streaming media data signal, comprising:
In one embodiment the header is transmitted before the media data of the additional content, and where the media data of the additional content is transmitted before the media data of the first content.
In one embodiment the header includes information on playback time for the additional content.
In one embodiment the header includes information on the size of the additional content.
In one embodiment the header includes information on a location of an end portion of the additional content in the stream.
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate one or more embodiments of the invention and, together with the description, explain the invention. In the drawings,
The following detailed description of the invention refers to the accompanying drawings. The same reference numbers in different drawings may identify the same or similar elements. Aspects of the invention may be implemented in methods and/or computer program products. Accordingly, the invention may be embodied in hardware and/or in software (including firmware, resident software, microcode, etc.). Furthermore, the invention may take the form of a computer program product on a computer-usable or computer-readable storage medium having computer-usable or computer-readable program code embodied in the medium for use by or in connection with an instruction execution system. The actual software code or specialized control hardware used to implement aspects consistent with principles of the invention is not limiting of the invention. Thus, the operation and behavior of the aspects were described without reference to the specific software code, it being understood that one of ordinary skill in the art would be able to design software and control hardware to implement the aspects based on the description herein.
Furthermore, certain portions of the invention may be implemented as “logic” that performs one or more functions. This logic may include hardware, such as an application specific integrated circuit or field programmable gate array, software, or a combination of hardware and software.
No element, act, or instruction used in the present application should be construed as critical or essential to the invention unless explicitly described as such. Also, as used herein, the article “a” is intended to include one or more items. Where only one item is intended, the term “one” or similar language is used. Further, the phrase “based on” is intended to mean “based, at least in part, on” unless explicitly stated otherwise. It should also be emphasized that the terms “comprises” and “comprising” when used in this specification is taken to specify the presence of stated features, integers, steps, components or groups but does not preclude the presence or addition of one or more other features, integers, steps, components or groups thereof.
Furthermore, different embodiments and aspects of the invention will be described herein with reference to a user device in which a streaming media data signal is received and processed for playback. In this context, the user device may be connected by wire to a source delivering the streaming media signal, or it may be wirelessly connected over the air, typically by means of a radio communications link. The user device may be a computer, stationary or portable, or it may be some other form of media presentation unit devices for reception and presentation of streaming media according to the aspects of the invention. However, the invention will mainly be described herein in the context of mobile communications, where the user device preferably is a portable electronic device configured to operate in a radio network. More specifically, in a preferred embodiment the user device is a mobile phone configured to operate in a radio network, preferably in a packet switched network service. Examples thereof include a 3GPP WCDMA network, or a GPRS GSM network. Furthermore, it should be noted that the invention as described with reference to the outlined embodiments relate to processing of a received streaming data signal in a user device before playback, but that it is not essential that the user device actually incorporates the means required for presentation of the content the streaming media signal represents. On the contrary, the user device may well be connected to another separate presentation device, such as one or more speakers and/or a separate display device, on which the media content is displayed.
Consistent with aspects of the invention, an adaptive buffering technique is provided that halts media playback and re-fills a first, or main, buffer with streaming media data when the first buffer has been depleted to a certain extent, and which switches to provide media data representing additional content from a second buffer, for playback during re-filling of the first buffer. Preferably, this is performed when it is appropriate from a media context point of view. For example, playback may be halted when there is a natural pause is speech contained in the media data regardless of how full or empty the buffer is. Consistent with aspects of the invention, the natural pause may be extended with an amount of time required to buffer additional streaming media data. Existing buffering techniques wait until the buffer is emptied of streaming data before re-filling the buffer. This re-filling of the buffer happens without any relation to the actual media that is being played, which means that the re-filling may occur, for example, in the middle of a sentence or even in the middle of a word. In contrast, re-filling the buffer at, for example, natural pauses in the media is likely to be less intrusive than completely re-filling the buffer when the buffer is emptied or has reached a predetermined level. Furthermore, by presenting additional content when playback of the first buffer is halted to be re-filled allows additional content or information to be presented. Typically, the time during which playback of the content of the first buffer needs to be halted in order to provide adequate re-filling is fairly short, maybe in the order of just a few seconds. Therefore, the additional content may be very brief, and a suitable type of content for this purpose is a commercial advertisement.
Some general aspects related to the invention will now be described with reference to
User device 120 may include a personal computer, a telephone, a cellular radiotelephone, a Personal Communications System (PCS) terminal or the like. A PCS terminal may combine a cellular radiotelephone with data processing, facsimile and/or data communications capabilities. User device 120 may further include a personal digital assistant (PDA), a conventional laptop and/or palmtop receiver, or another appliance that includes a radiotelephone transceiver, or the like. A PDA may include a radiotelephone, a pager, Internet/intranet access, a web browser, an organizer, calendars and/or a global positioning system (GPS) receiver.
Streaming media source 110 may include a server entity that may transmit streaming media to user devices 120-1 through 120-N via network 130. Streaming media source 110 may connect to media storage 140. Media storage 140 may store media content that may be retrieved by media source 110 and transmitted to user devices 120-1 through 120-N as streaming media.
Cellular network 130 may consist of components conventionally used for transmitting data to and from user device 120 and streaming media source 110 (not shown). Such components may include base station antenna arrays 215a-215f, which transmit and receive, via appropriate data channels, data from mobile stations within their vicinity. Base stations 210a 210f connect to their respective antenna arrays 215a 215f, and format the data transmitted to, or received from the antenna arrays 215a-215f in accordance with conventional techniques, for communicating with BSCs 205a-205b or a mobile station, such as user device 120. Among other functions, BSCs 205a-205b may route received data to either MSC 220 or a base station (e.g., BSs 210a-210c or 210d-210f). MSC 220 routes received data to BSC 205a or 205b. GW 225 may route data received from an external domain (not shown) to an appropriate MSC (such as MSC 220), or from an MSC to an appropriate external domain. Streaming media source 110 (not shown in
User device 120 is in one embodiment configured to receive streaming media from the Internet 32, corresponding to network 130, by wire connection 35. As an optional or additional feature, user device 120 is configured to operate in a radio communications network 30, for the purpose of receiving streaming media. The radio communications network 30 may e.g. be a 3G network, capable of providing streaming media from the Internet 32. For streaming audio, this is already available today through many mobile phone operators. For streaming video over the mobile network, one possible technology is Multimedia Broadcast and Multicast Service (MBMS) specified in 3GPP for use in WCDMA networks. Another alternative is Broadcast and Multicast Service (BCMCS) specified in 3GPP2 for CDMA2000. Both of these technologies are known and in many aspects similar to each other, and may both be used for providing streaming video to mobile terminals over a 3G mobile phone network. The terms broadcast and multicast are used alternatively, even though they do not mean exactly the same thing. For video streaming of television channels over a 3G network using e.g. MBMS, multicast will most likely be used, as it generally would be a subscription service. In addition, or as an optional feature, user device 120 may be provided with a radio signal receiver configured to operate in a IEEE 802.11 WLAN to receive streaming media including audio and video from a WLAN transmitter, or access point, 33 connected to the Internet 32. As yet another option, or additional feature, user device 120 may include a receiver for receiving broadcast or multicast media signals from a broadcast station 34, such as a radio station or a television station. For traditional radio transmissions, headset cords have been used as antennas for receiving FM radio. For digital transmissions of video, DVB-h (Digital Video Broadcasting-handheld) is a technology based on DVB, which has been adopted by ETSI as a standard specification for bringing broadcast services to handheld devices, is at least partly described in ETSI EN 302 304. DVB-h is basically a mobile version of the digital technology used in digital television networks. DMB (Digital Multimedia Broadcasting) is another digital transmission system for sending data, radio and TV to mobile devices such as mobile phones, whithout making use of the mobile phone network. It can operate via satellite (S-DMB) or terrestrial (T-DMB) transmission. DMB is based on the Eureka 147 DAB standard and has some similarities with DVB-h.
In summary, there are a number of different ways of receiving digital media, both audio and video, which may be used in user device 120. The actual technology or bearer used is not of primary importance to the invention, though, and only one single streaming media data receiver 20 is therefore depicted in
The user device 120 is configured to receive streaming media data, and to convert that data into a user-perceivable signal, such as sound or images. For this purpose, one embodiment of the user device preferably includes a processing unit 22, which in a preferred embodiment represents a microprocessor system including associated memory space, operation software, and application software, and is configured to control the processes carried out in the user device 120. Processing unit 22 is, inter alia, configured to the control buffering according to the invention, as will described in more detail. User device 120 further includes a buffer unit 23 including two or more buffers configured to be filled with input streaming media data under control of processing unit 22, and to be drained of streaming media data for playback. Buffer unit 23 is therefore connected to a media player 24, configured to convert media data into media signals adapted for output through a media output device. For audio media content, media player 24 may therefore be connected to speaker 14, loudspeaker 15, and/or to a connector 16, where connector 16 is communicatively connectable to an external headset 17 or an external speaker (not shown). Such a headset 17 or speaker may be connectable by means of Bluetooth® or the like to a radio signal transceiver of connector 16. Alternatively, or additionally, connector 16 may include a socket for wire-bound connection to the external headset 17 or speaker. For video content, media player 24 may be connected to display 12 of the user interface, or to a connector 18 for connection to an external display 19. Connector 18 may also provide connection for audio associated with the video content, to be played back via a speaker in external display 19. The media output device of user device 120 may therefore be speaker 14 or 15, display 12, or connector 16 or 18. As a power supply, user device 120 preferably further includes a battery (not shown), and/or a connector for a mains outlet.
The block diagram of
In accordance with aspects of the invention, the content stored in the at least one additional buffer 232-235 is provided to the media player when the first buffer 231 needs to be re-filled, and playback of the content of the first buffer 231 therefore is halted. When a user want to take part of the first, or main, content stored in first buffer 231, buffer controller 221 is triggered to extract media data from first buffer 231 and provide the extracted media data to media player 24, which plays the streaming media data to the user via the dedicated output device. For example, media player 24 may decode audio data from first buffer 231 before using speaker 15 to convert the decoded data from an electrical signal to an auditory output signal. As another example, media player 24 may decode video data from first buffer 231 before using display 12 to convert the video data to a visual representation.
During operation of user device 120 to receive and present streaming media of a main content, media data is sequentially stored in first buffer 231, and extracted from first buffer 231 to media player 24. Dependent on inter alia the bit rate of the media data and the bandwidth of the communication link over which streaming media is transported to receiver 20, first buffer 231 may be depleted faster than it is re-filled. When this is the case, first buffer 231 will finally be completely drained unless playback of the content from first buffer 231 is halted, so that first buffer 231 is re-filled only. In one embodiment, re-filling only of first buffer 231 is performed once first buffer 231 is completely emptied. In another embodiment, re-filling only is performed when the first buffer 231 has been emptied to a predetermined level. That level may be a certain percentage of the total capacity of first buffer 231, such as 5%, 10%, 15%, 20%, 25%, or some other percentage level.
In another embodiment, an adaptive approach is used for re-filling first buffer 231. In this embodiment, user device 120 includes a buffer analyzer 222, configured to analyze the data stored in first buffer 231 to identify specific periods in the streaming media where it is suitable to halt playback of the content, preferably corresponding to “inactive” periods. Such inactive periods may include, for example, periods in which audio data is silent, e.g. a natural pause in speech, or periods in which video data is “still”, e.g. little or no change in the video frame by frame for a number of frames, or the presence of a freeze frame representing e.g. a camera change or a scene change. As an alternative, or additional, solution the buffer analyzer 222 may be configured to analyze remaining media data in first buffer 231 to locate a halt flag, stored in the media data. Such a halt flag may be inserted by the producer of the main content, or by a content provider.
Buffer analyzer 222 may be configured to continuously analyze the data presently stored in first buffer 231. Alternatively, buffer analyzer 222 is triggered to analyze the remaining data in the first buffer 231 when the first buffer 231 has been emptied to a predetermined level, such as to a certain percentage of the total capacity of first buffer 231. As an example, buffer controller 221 may be configured to trigger buffer analyzer 222 to analyze the remaining data in first buffer 231 when the first buffer has been emptied to 20% of its total capacity. When first buffer 231 then subsequently has been depleted to the point where media data representing the inactive period or halt flag, is to be extracted, or has just been extracted, buffer controller 221 controls media player 24 to halt playback of the first content, while receiver 20 still passes media data to first buffer 231 to re-fill it.
When playback of media data content from first buffer 231 is halted, buffer controller 221 is configured to instead extract media data from a second buffer to media player 24. In case there is only one additional buffer 232, that additional buffer 232 represents the second buffer. In an alternative embodiment, in which there are a plurality of additional buffers 232, 233, 234, 235, one of those additional buffers will be selected as the second buffer, or one at a time will de selected as the second buffer. While the content of the second buffer is played back using media player 24, first buffer is re-filled with media data representing the first content. It should be noted, that in this context re-filled need not mean completely re-filled. When a certain criteria has been met, buffer controller is configured to switch back to extracting media data from first buffer 231, to resume playback of the first content. This criteria may include that buffer analyzer determines that first buffer 231 has been re-filled to at least a predetermined degree. An alternative, or additional, criteria to be met may be that the second content played back while re-filling first buffer 231 has reached an end point. That end point may be an identified end point location in the media data of the second buffer, or it may simply be when the second buffer has been emptied. By means of any of these embodiments, any user taking part of the content played by user device 120 will be provided with additional content while the first buffer is re-filled, which means that there will be fewer pauses in the played content. In a preferred embodiment, playback of the first content is halted at a located inactive period, as outlined above. This provides the additional benefit that the additional content as inserted from the second buffer will be perceived as less intrusive in the first, or main, content.
In one embodiment, the additional content stored in the additional buffer(s) 232, 233, 234, 235 represent commercial ads, preferably of the same media type, i.e. audio and/or video, as the main content stored in first buffer 231. When playback of the main content is halted, the commercial ad is instead played back for a short period of time. The playback of the commercial ad may be halted when first buffer 231 has been re-filled to at least a predetermined level, and/or after the commercial ad has been played to its end. Preferably, the or each commercial ad stored in the additional buffer(s) 232, 233, 234, 235 is/are quite short, in terms of playback time, such as between 2 and 10 seconds, which in many applications will be sufficient to re-fill the first buffer 231 adequately.
In the illustrated embodiment of
The following describes one example of how processing of streaming media may be achieved according to an embodiment of the invention:
1) A service provider having a streaming media source 110 creates a data stream 50 including main content and additional content. This step may be preceded by a user of user device 120 selecting the main, or first, content to play. In this embodiment, the additional content of the data stream includes four different and separate commercials A, B, C, and D. In this example the main content is a sitcom episode, comprising audio and video data. Typically, the additional sets of content, i.e. the commercials, therefore also include video content, but they may merely include audio content. In one example the playback time of the different commercials vary, where A has a playback time of 3 seconds, B has a playback time of 3 seconds, C has a playback time of 5 seconds, and D has a playback time of 8 seconds. The service provider also streams settings to be used by the receiving user device 120, preferably in a header part 51 of the stream 50.
2) The service provider first streams the media data of the separate content A-D in the stream 50, preferably preceded by header 51, and then starts streaming the comic episode, i.e. the main content, over the communication link
3) Streaming media of data stream 50 is received in user device 120, and buffer controller 221 stores the received media in different buffers 231-235 of buffer unit 23. In this embodiment, the number of ads receiver matches exactly with the number of additional buffers in buffer unit 23. Should that not be the case, additional rules would have to be applied. Should there be more additional sets of content provided in stream 50 than the number of additional buffers, the additional content to store in those additional buffers may be determined by a priority order given in header 51. Alternatively, the additional sets of content are stored sequentially, such that the first number of additional sets of content matching the number of additional buffers are stored therein, while any further sets of additional content in stream 50 are neglected. Preferably, the settings streamed in stream 50, such as in header 51, are analyzed by buffer controller 221 in a pre-buffer 236, from which the received media data is passed to the different buffers 231-235. Since the media data of the additional content is streamed first, and therefore also received first, the media data representing that additional content is first passed to respective separate buffers, such that for instance content A is stored in buffer 232, content B is stored in buffer 233, content C is stored in buffer 234, and content D is stored in buffer 235. When the media data of the main content is finally received, it is stored in first buffer 231.
4) Media data is then extracted from first buffer 231 and passed to media player 24, in order to begin playback of the sitcom. Extraction of data from first buffer 231 may, theoretically, begin as soon as there is any media data stored in first buffer 231. In a preferred embodiment, though, extraction of data does not begin until first buffer 231 is filled to an upper threshold, such as e.g. 80% of its total capacity.
5) After a while of playback of the main content, the first buffer 231 has been depleted to an extent where it is suitable to re-fill the first buffer before any more media data can be extracted. As previously mentioned, this may be when first buffer 231 is completely empty. However, in a preferred embodiment a suitable place to halt playback in the main content is defined either by a halt flag or by an inactive period in the main content. In one of those latter embodiments, buffer analyzer 222 is triggered to start looking for such a suitable place to halt when the first buffer reaches a lower threshold, such as e.g. 20% of its total capacity. In one embodiment, buffer controller 221 also calculates an estimate of expected length in time of the necessary re-filling of first buffer 231 when the playing of the main content halts. This may be based on an estimation of average network bandwidth during a suitable time-period before the halt, and an estimation of the average bit rate of the main content during the same time-period. This may provide a measure of the time needed to re-fill the first buffer e.g. to the upper threshold.
6) The media player 24 halts playback of content from the first buffer 231 when first buffer 231 has been depleted to a detected suitable place to halt, whereby first buffer 231 is only re-filled with media data for the main content. At this point in time, playback of additional content is commenced. In one embodiment, the different sets of additional content are played sequentially as received and stored in the additional buffers 232-235. In another embodiment, they are played in an order as determined by information in the header 51 of the received stream 50. In another embodiment, they are played in an order determined by their respective size, in terms of playback time, correlated with the estimated required length in time to re-fill the first buffer 231. In one embodiment, as here disclosed, a combination of these selection rules apply. In that embodiment, it has been estimated that it 2.5 seconds are required for adequate re-filling of first buffer 231. Buffer controller 221 therefore selects the first additional content having a playback time of at least 2.5 seconds as the second content, in this case commercial A. Media data from additional buffer 232 are thereby extracted to media player 24, whereby commercial A is played, preferably on the same output device as the main content was played on, e.g. display 12. It should be stressed that the estimate of required time, as well as the selection of which additional content to play when the main content is halted, are preferably performed before the main content is actually halted, enabling a fast and smooth switch from the first buffer 231 to the (selected) second buffer 232.
7) Playback of the first content is then resumed when certain criteria are met. Preferably, the additional content selected matches the estimated required re-filling time, and the switch back to play the main content from first buffer 231 is not performed until the selected content, i.e. commercial A, has been played to an end point. In this context, an end point may mean the final end of content A, or an intermediate end point within that additional content, separating different content portions of commercial A, e.g. by flags in the media data. Alternatively, playback of content A is terminated when it has been determined that first buffer 231 has been adequately re-filled, e.g. to a predetermined threshold level, without considering whether or not content A is interrupted. When the buffer is filled and commercial A is played, it starts playing the sitcom. Should it be possible to play the remainder of the main content without having to halt and re-fill, the method ends when the last of media data of the main content has been extracted and played. However, for a longer show such as a sitcom, that would not necessarily be the case, and more halts would therefore be required. At those halts, different rules of selecting which additional content to play may be employed. The following steps 8-10 outline one illustrative example of how one embodiment could be realized.
8) Once playback of the main content has been resumed, the process will eventually come to step 5 again. Also this time, the estimated re-filling time required is 3 seconds in step 6. However, since commercial A was played the last time, commercial B is played instead. Step 7 is then repeated in order to resume playback of the main content.
9) When the process comes back to step 5 again. This time the estimated time for buffering is 4 seconds. The buffer controller 221 could then select commercial C, but instead it selects commercial D because a priority order provided in header 52 of the received stream 50 specifies this order of choice. The background may e.g. be that the service provider gets better paid by the company behind commercial D. When the entire content of commercial D has been played back, Step 7 is repeated in order to resume playback of the main content.
10) The sitcom episode provided in the main content is finished. In one embodiment, the user device 120 reports to the content provider, before ending the session, that it has played commercials A, B and D once, information which the buffer controller preferably has stored in memory 25. Alternatively, this information is provided to the content provider at a later stage, and possibly using another communication medium. The transmission of data related to which content has been played may e.g. at certain intervals for regular subscribers to a media service.
The foregoing description of implementations consistent with principles of the invention provides illustration and description, but is not intended to be exhaustive or to limit the invention to the precise form disclosed. At least some of the merits of the invention include the effect that, instead of freezing picture and sound when buffering, the users get something to watch and/or listen to. Service providers and media owners are furthermore provided with a potential revenue source linked to the concept of providing additional content including commercial advertisements, without actually extending the playback time of the main content into which it is inserted. Using such a service for streaming media, where additional content is provided when the image or audio otherwise would freeze, could also form a basis for subsidized media services, provided that a user accepts to be presented with advertisements when the user device is buffering media data. Modifications and variations are possible in light of the above teachings, or may be acquired from practice of the invention.
