Patent Application
20070130597
The invention is directed to communication networks and in particular to a multimedia delivery system with instant replay and time-shifted playback.
Systems for delivering broadcast television over the Internet Protocol are emerging, along with new services that enhance and diversify the entertainment experience of subscribers to such systems. Generally, a broadcast provider obtains multimedia signals (e.g. TV signals) from traditional sources such as satellite or cable and encodes the signals for delivery to subscribers over an IP enabled broadband network. Subscribers enabled with television sets and set top boxes (STB) or PCs are fitted with appropriate software to decode the respective IP format for program viewing. An interactive electronic program guide, viewable on the subscriber's monitor or TV, provides access to a range of viewing features and system management functions.
Currently, program delivery and the services available to the subscribers are at the convenience of the service provider or broadcaster and, hence, the subscriber must arrange his/her schedule to coincide with program availability. On the other hand, because of the proliferation of broadband backbone and delivery networks, emergence of specialty services (such as pay per view and first run movies), and increase in usage of PCs by consumers, an increasing demand is evolving for on-demand multimedia services. These services enable consumers to plan their entertainment to their own schedule and interests rather than tailor their entertainment viewing habits to a service provider's broadcast schedule.
The service providers use unicast and multicast protocols to deliver their services to subscribers. Multicast protocols are used for conventional TV programming and near video on demand (NVOD) movie selections, as they enable numerous subscribers to have access to a program at the same time. Unicast IP protocols are preferably used for true video-on-demand services such as VOD, VDVR, timelessTV and TV-on-Demand.
Currently, subscribers viewing a multicast stream cannot freely interrupt the stream to review content previously delivered, because there are other viewers joined to the same transport stream. Subscribers must actively store and manage the broadcast content for the purposes of instant-replay and time-shifted playback. In recent years, one popular approach to offer some form of VCR-like interactivity is to add a storage unit to the subscriber equipment so as to cache all the available content being broadcast. Many time-shifted systems that exist today are based on having the clients record in real-time a channel of interest; only by this means can the content be played back.
For example, TiVo™ set top boxes (STB) available today function similarly to VCRs, but use non-removable hard-disk storage, and contain sophisticated software to record programs—not only those the user specifically requests, but also other material the user is likely to be interested in. Programs being watched “live” can be paused or “rewound” to repeat a sequence just watched. A TiVo digital video recorder (DVR) allows a user to specify which programs to record by time, by program title, and by specifying combinations of genre, actors, directors, etc. Programs may be stored until internal storage is filled, at which time the unit will dispose of older programs (unless flagged to be saved until manually deleted), to allow for new programs to be recorded.
However, standalone TiVo systems can only record one channel at a time. In addition, “time-shifted playback” in order to view/review a certain past content, can be performed only for the channels currently viewed. Furthermore, TiVo requires special hardware (hard drive) in the STB to enable this feature, which is costly, and suffers from operational problems like storage unit failure and management.
For example, US Patent Application Publication # 20030170003 entitled “Time-Shifted Video Signal Processing” (Levesque et al.) describes a time-shifted playback method where real-time video frames are delivered for display during a real-time mode, and time-shifted video frames are delivered for display during a time-shifted mode. The time-shifted video frames are delayed relative to the real-time video frames. A real-time frame is paused during a transition from the real-time mode to the time-shifted mode. This method enables a client-side implementation of trick play and instant replay.
Most “time-shifted” viewing offered by digital providers is purely based on TV networks broadcasting in different time-zones, letting you watch e.g. the 18:00 clock news in Vancouver at the 21:00 Toronto time.
There are also some entertainment delivery systems that use network storage devices for providing instant-replay and time-shifted playback, as described in e.g. US Patent Application Publication # 20040254999 entitled “System For Providing Content To Multiple Users” (Bulleit et al). The delivery system described by Bulleit et al. comprises a grid computing platform, which provides storage of the content across network storage devices and consumer storage devices. The grid computing platform monitors distribution of the content to one or more of the consumer networks and determines when and where to store non-real time video. Processor-based network element(s) determine, based on customer preference, customer viewing habits or other factors, when to store a video program on the consumer storage device and where to store content that is not resident on the user's local consumer storage device. This is a very complex system and as such is prone to errors and expensive.
An example of a combination of network DVR and download VOD is presented in the U.S. Pat. No. 6,611,654 entitled “Time- and location-driven personalized TV” (Shteyn). This patent describes a server system that enables a subscriber to select a specific broadcast program for recording and a specific location and time frame for play-out of the recorded program. In one embodiment, the content information can be recorded at a first recording system, e.g., at a server. The selected content information recorded on the recording system can then be streamed over a data network, e.g., over the Internet or a private network like AOL to a location specified by the user. Alternatively, or subsidiarily, a program can be recorded and stored temporarily at a server and then streamed over a data network, e.g., the Internet, using a low-bandwidth protocol, to a recording device specified by the user as destination. Accordingly, the Shteyn's system attempts to contribute to the user's needs by means of enabling shifting of recording and play-out locations, in addition to the time-shifting provided by the known services. The main disadvantage of this system is that it provides an arbitrarily delayed and location-independent experience.
There is a need to enable a subscriber with additional services such as instant replay and time shifted playback that is controllable by the user, does not require expensive subscriber terminals and provides an inexpensive and reliable network solution.
It is an object of the invention to provide a simple method for providing a time-shifted view of a live audio/video stream for the purpose of instant-replay review and short-term time-shifting. The implementation of this simple mechanism is transparent to the end-user unless invoked, implying no additional hardware cost or complexity at the STB.
Accordingly, the invention provides, a time-shift unit for a video delivery server for distributing multimedia content streams to a plurality of subscriber premise equipment (SPE) units, where SPE units select and join multimedia content streams using interactive program guide capabilities at the server. The time-shift unit comprises a circular buffer for storing multimedia content received in a live multimedia content stream; a first sender for bypassing the circular buffer and multicasting the live multimedia content stream directly to the SPE units; and a second sender for accessing the multimedia content from a specific position in the circular buffer, and multicasting a time-shifted variant of the live multimedia content stream to the SPE units, wherein the specific position in the circular buffer confers a predetermined time-shift for the variant of the live multimedia content stream with respect to the live multimedia content stream.
Further, the invention also provides the video server with an instant reply unit comprising: a replay buffer for storing a length T of multimedia content; an instant replay controller for receiving an instant reply request from the CPE unit, taking a snapshot copy of length T of the multimedia content from the circular buffer to the reply buffer and directing a pointer associated with the SPE unit to the reply buffer; and a sender associated with the replay buffer for transmitting the snapshot copy to the SPE unit from the reply buffer.
The invention also provides a for distributing multimedia content streams to a plurality of subscriber premise equipment (SPE) units, where SPE units select and join multimedia content streams using interactive program guide capabilities at the server, comprising: continuously storing multimedia content received in a live multimedia content stream into a circular buffer; identifying a specific access point in the circular buffer that provides a predetermined time-shift of multimedia content in the circular buffer from the access point of multimedia content in the live multimedia content stream; and accessing the multimedia content from the specific offset in the circular buffer, and multicasting a time-shifted variant of the live multimedia content stream to the SPE units from the specific access point.
The method of distributing multimedia content streams further comprises: providing a plurality of reply buffers of various lengths Tn; receiving an instant reply request from the CPE unit, wherein the instant reply specifies a length T for instant reply of selected content of interest; identifying a certain reply buffer from the plurality of reply buffers that can store content of the length T; taking a snapshot copy of length T of the multimedia content from the circular buffer to the certain reply buffer and directing a pointer associated with the SPE unit to the certain reply buffer; and transmitting the instant reply content of interest to the SPE unit.
Instant replay and time-shifted playback would likely be considered as mandatory features for IPTV users. Advantageously, the invention provides a network implementation of instant-replay and time-shifted playback, while enabling the subscriber with a VCR-like functionality for any multimedia (TV) broadcast program. This enables reduced complexity in the equipment located at the subscriber premises, since the STB will need less memory. Therefore the present solution enables implementing these additional features more cost-effectively in the network compared to traditional client-side implementations.
Since it is implemented in the network, the system according to the invention permits real-time statistical reporting so that the service providers can further adapt their offerings to subscribers' needs. In addition, the system is client-agnostic, in that it does not require any special custom hardware and only minimal software changes.
The foregoing and other objects, features and advantages of the invention will be apparent from the following more particular description of the preferred embodiments, as illustrated in the appended drawing
Definition of some terms is provided for better understanding the invention. The term “instant replay” refers to a feature that enables a viewer of a TV program/channel to jump back in time by a few seconds (8-10 seconds), and resume viewing of the respective channels from that point in time. The term “time-shifted playback” refers to a feature that enables a viewer to switch from one version of the TV signal to another version which is delayed or ahead of the previous version by a specified amount of time. This can be used to effectively rewind the signal when real-time viewing is interrupted (e.g. whilst speaking on the phone, or while answering the door) and to fast-forward the signal subsequently. The term “trick play” refers to using controls such as pause, instant replay, rewind, fast forward, slow, skip to end, while watching live TV.
The term “multimedia content” is used here to designate units of information (packets, frames, etc) transported by the multimedia content streams according to the respective transport protocol.
For large amounts of data, including point-to-multipoint (P2P) video transmissions, IP multicast is more efficient than normal Internet unicast transmissions because a server can broadcast the multimedia content to many recipients simultaneously. Unlike traditional Internet traffic that requires separate connections for each source-destination pair, IP multicasting allows many recipients to share the same source. This means that just one set of packets is required to be transmitted for all the destinations, and that the clients join the multicast stream upon request.
As well known, the subscriber premise equipment SPE (or subscriber terminal, or user terminal, or end-client) denoted with 2, 2n, 2o, 2ir, may include a set-top box (STB) 21, 21n, 21o, 21ir, and a TV monitor 22, 22n, 22o, 22ir, respectively, for enabling a subscriber to view and listen to the multimedia content streamed from system 10. The subscriber terminal is not limited to the examples provided in
Also shown on
Relevant to this invention, server 10 includes a time shift unit 20, and an instant reply unit 25, that provide server 10 with additional functionality by implementing instant-replay and time-shifted playback, for enabling the subscribers with a VCR-like functionality for any multimedia (TV) broadcast program.
Time-shift unit 20 includes a time-shift controller 11, a time-shift transmit unit 15 and a circular buffer 14. Time-shift transmit unit 15 includes a plurality n of senders 5, 51, . . . 5n, for transmitting predefined time shifted variants of the live stream to the SPE's 2, 21, . . . 2n. The circular buffer 14 caches continuously the live stream. Each sender is associated with an access point P, P1, . . . Pn that enables a certain time delay D1 to Dn in rendering the content to the users by sending from a different offset within the circular buffer. The number of senders 5 is a design parameter and is only limited by the amount of memory, CPU and bandwidth available at the server. A STE can choose to view the original live broadcast with any of the available time-shifts (delay) D. For example, in the case that three buffers are used, D1 could be 5 minutes, D2, 15 minutes and D3, 30 minutes. As indicated above, D could have any value desired, memory limited. A variation of this ability is micro time-shifts of 30 seconds, 1 minute, 2 minutes, and 5 minutes.
In operation, time-shift unit 20 receives a “live” broadcast through a respective medium and continuously caches copies of the live stream in its circular buffer 14. The senders transmit the delayed versions using preferably IP multicast, so that more than one SPE unit can view the same delayed version of the live stream. Upon receipt of a request message 4, let's say from subscriber terminal 21n, demanding a transmission delayed by Dn, controller 11 identifies the buffer start point Pn for the version time-shifted by Dn, and provides the respective terminal with the address where this stream can be joined. This enables STB 21n to playback the stream with the respective delay Dn by just joining the appropriate multicast stream (transport medium chosen in this example).
Alternatively, the system may be configured so that the client 2n can infer what multicast address each of the delayed streams is on, and join the respective stream so that the interaction with controller 11 is optional.
Since it is implemented in the equipment at the edge of the provider's network, the instant replay system of the invention requires fewer resources and is more practical to do in memory, yet is still quite powerful. This configuration of the video delivery system 10 allows subscribers to replay the content of interest by jumping to and from a time-delayed version, to take breaks for answering the phone, and to activate a VCR, etc. without missing the show. It also allows the user to catch-up with real-time during commercial breaks, which are usually even multiples of 30 seconds.
Performance of the time-shifting mechanism described above can be optimized using techniques described in the co-pending U.S. Patent application Ser. No. N/A entitled: “Client Side PID Translation” (Furlong et al.), filed on Nov. 2, 2005. This co-pending Patent Application describes a mechanism whereby program guide and system information is send to subscriber terminals before the respective content. The SPE is provided with an input buffer for storing the input data until a milestone is detected and a multimedia content stream and a system information controller that detects the milestones in the stream. In this way, the decoder does not need to establish the correspondence between the PID of the elementary streams to re-assemble the respective stream upon receipt of the content, since this is performed in advance upon detector initialization. n the case time-shifting is used, the advance knowledge of the decoding information for the respective stream allows the subscriber to perform the time-shift faster than with traditional SPE's.
It is to be noted that any number of such buffers may be provided, their number being limited by the memory availability. Instant reply transmit unit 30 comprises senders 6, 6′ that provide the instant reply content to the SPE's 2ir; each sender is associated with a respective buffer 13, 13′.
In operation, when an end-user wants to perform an instant replay function, he/she initiates this event (e.g. a key press on the channel selector), as shown by instant reply request 7. This can be for example implemented using the RTSP (real time streaming protocol), which is a client-server multimedia presentation control protocol, designed to address the needs for efficient delivery of streamed multimedia over IP networks. Once the user initiates the instant replay, controller 12 takes a snapshot copy 26 of the “live” stream from the circular buffer 14 to a certain arbitrary buffer 13 of Tn seconds, and directs a user pointer to it.
At this point sender 6 in this example, commences sending buffer 13 in a continuous loop of 10 seconds (T=10 seconds in this example). With RTSP optional trick play support, the user can pause, fast-forward and rewind the sender within the T second buffer. Once done with the instant-replay, the user may rejoin the live stream. The instant replay stream is sent to STB 21ir preferably as a unicast transmission, since only user 2ir wishes the instant replay at that particular moment.
Performance of the instant replay mechanism described above can be optimized using techniques described in a co-pending U.S. patent application Ser. No. : N/A (Parker et al) entitled “Milestone Synchronization in Broadcast Multimedia Streams” filed on Mar. 28, 2005. This co-pending Patent Application describes a mechanism whereby the beginning of a circular buffer contains the beginning of a GOP sequence (i.e. an MPEG2 I-Frame). This involves flushing data until a GOP sequence is found and keeping track of the PAT/PMT frames separately. As the circular buffers fills-in, the server keeps notes of where the GOP sequences start and end. When a join request arrives from a STE, the requester is added to a list of clients, and the time of request arrival is associated with the closest GOP start in the buffer. Since requests come in randomly, each client is at a specific (different) packet in the buffer. The same circular buffer 14 used for implementing instant reply capabilities may be also used for synchronizing the clients.
