The present invention relates to digital video communication, and more particularly to the insertion of advertisements and the like into a multiplex of digital video streams. The invention also relates to the grooming of digital video streams, such as for recombining selected programs from one or more sources.
The process of inserting advertisements into video programming became much more complex after the transition from analog video to compressed digital formats such as MPEG1 and MPEG2. Instead of substituting analog waveforms (or digitized pixels) from one signal to another, it became necessary to first identify a suitable exit point in a first compressed digital stream, and then to align this exit point with a suitable entrance point into a second compressed digital stream. In addition, many parameters needed to be added, modified, or replaced, in order to maintain seamless continuity during the splicing transition. In fact, most digital splicing products in existence today not only modify these parameters, but also regenerate the entire stream. This is done to insure that the data rate of the resulting stream remains within the limits of the communication channel used to convey the signal from the splicer to one or more receiving devices. This data rate modification process is often referred to as transrating.
The combination of splicing and transrating capabilities is particularly advantageous in products designed for ad insertion as well as products designed for grooming. Grooming refers to the recombination of selected programs from one or more sources. It is an application which typically depends on transrating to match the combined rate of the selected programs with the data rate of the communications channel. In this case, if changes are made to the selection of programs, or if advertisements are inserted into one or more of these selected programs, then the transrating process will automatically insure that the capacity of the communication channel is never exceeded.
In many applications, video and audio streams are encrypted prior to distribution in order to prevent unauthorized access to the content. Ad-insertion and grooming operations are usually performed before encrypting, since both processes generally require access to certain headers while they remain in the clear. For example, certain header fields must be modified if an ad insertion splice is to be seamless, and many transraters are designed to perform a full recompression process and therefore require unrestricted access to the entire video stream.
In some cases, it would be advantageous to insert ads or perform grooming and transrating functions after a stream has been encrypted. For example, if the same stream is to be broadcast to multiple recipients, then it may be more effective to insert multiple customized ads in order to better address the needs of each individual recipient or group of recipients. This matching of ads with attributes of known recipients is known as targeted advertising.
Unfortunately, it may not be desirable or even possible to defer the encryption process until each of the customized ads has been inserted. For example, the ads may be inserted at various downstream locations, and it may be necessary to protect the content before it is transmitted to these downstream locations. Additional considerations are the cost of encryption, and the availability of bandwidth. Each time a customized ad is inserted into the stream, a new version of the stream is created. Therefore, the cost of encrypting multiple versions must be considered. Furthermore, if the multiple versions are created upstream by co-locating the ad servers with the encrypter, then additional bandwidth will be needed to accommodate each of the customized versions as they are transmitted to their intended viewers.
For these and other reasons, it is often desirable to perform ad insertion and/or grooming close to the edge of a video distribution network and to encrypt content before it is transmitted from a more central point in the same network. This would require the ad insertion and grooming to be applied to already encrypted video streams.
Accordingly, it would be advantageous to provide a solution to the problem of transmitting a single encrypted version of a program from a central point in the network to multiple recipients, while still realizing the benefits of targeted ad insertion and/or grooming. The present invention provides a solution with these and other benefits.
A method is provided for inserting content into a video stream. The video stream is received, and a cueing message is extracted therefrom. The cueing message is indicative of a splice point within the video stream. The video stream is conditioned in response to the cueing message to indicate the location of the splice point in a manner adapted to survive the subsequent transrating of the video stream. The conditioned video stream is transrated, and conveyed to a splicer. The content to be inserted is also conveyed to the splicer for insertion into the transrated conditioned video stream at the splice point.
The conditioning step can condition the video stream in response to the cueing message to indicate the location of the splice point in a manner adapted to survive both the subsequent transrating and the subsequent encryption of the video stream.
The video stream can be received at a pre-splicer which performs the extracting and conditioning steps. The pre-splicer can receive and convey the content to the splicer via one of an inband or out-of-band channel. The content can be conveyed to the splicer without being transrated. The content can also be conveyed to the splicer without being encrypted.
The conditioned video stream can be encrypted after the conditioning step and before the content is inserted therein. Alternatively, the conditioned video stream can be encrypted after the conditioning and transrating steps, and before the content is inserted therein. The content can be conveyed to the splicer without being transrated. The content can also be conveyed to the splicer without being encrypted.
Apparatus is provided for inserting content into a video stream. The apparatus includes a pre-splicer adapted to (i) receive the video stream; (ii) extract information indicative of a splice point from the video stream; and (iii) condition the video stream to indicate the location of the splice point in a manner adapted to survive the subsequent transrating of the video stream.
A transrater is coupled to receive and transrate the conditioned video stream from the pre-splicer. A splicer is coupled to receive the transrated video stream from the transrater and to receive content to be inserted into the transrated video stream from a content server. The splicer inserts the received content into the transrated video stream at the splice point.
The apparatus can further comprise an encrypter between the transrater and the splicer.
In one embodiment, the pre-splicer receives the content from the content server and conveys the received content to the splicer via one of an inband or out-of-band channel independent of the transrater.
In another embodiment, the splicer receives the content directly from the content server.
The pre-splicer can condition the video stream to indicate the location of the splice point by providing splice identifier information in an unencrypted adaptation header carried in a transport packet of the video stream. The pre-splicer can further provide insertion interval duration information in the unencrypted adaptation header. The pre-splicer can also provide (e.g., in the unencrypted adaptation header) attributes of the content which must be satisfied for compatibility with the insertion interval defined by the insertion interval duration information. The attributes of the content can include, for example, a maximum data rate of the content.
Where a maximum data rate of the content is provided, the transrater can examine this parameter and carve out room for the content before the content is inserted at the splicer.
The pre-splicer can insert the splice identifier information in adaptation headers at intervals which are prior to but do not immediately precede the splice points. This will subsequently allow sufficient time for communication and synchronization with the content server prior to inserting the content into the video stream.
Although the invention is described with multiple references to the process of ad insertion, it should be realized that the solution is equally applicable to grooming applications where splicing occurs between video streams consisting of arbitrary unrelated video content of limited or unlimited durations.
A block diagram of a prior-art ad insertion system is shown in
For cable systems, the widely adopted standard specifying the format of the cueing messages in MPEG-2 transport streams is referred to as SCTE 35, entitled “Digital Program Insertion Cueing Message for Cable”, a copy of which is available at http://www.scte.org/documents/pdf/ANSISCTE352004.pdf. Similarly, the adopted standard specifying the communication protocol between the splicer and the ad server is SCTE 30, entitled “Digital Program Insertion Splicing API”, available at http ://www.scte.org/documents/standards/approved/ANSISCTE302006.pdf. Both of these standards have been extended for compatibility with newer video compression standards. An example of such a video compression standard is the well known H.264 standard, which is equivalent to MPEG-4 Part 10, or MPEG-4 AVC (for Advanced Video Coding).
The pre-splicer also performs another important task. It conditions the output data stream in such a way that the precise location of splicing points, as well as all parameters needed to implement a seamless splice, will survive the subsequent transrating and encryption processes.
A transrater 24 and encrypter 26 are also shown in
Note that the ads which are stored on the ad server 12 may be either encrypted or in the clear. In most applications, there is little interest in protecting the ad content from unauthorized access. However, if it is desirable to encrypt multiple ads in real time using the same encrypter as used for the network stream, then this may be accomplished using a multiplexed encryption design as described in US patent publication 2005-0180568 entitled “Time-Multiplexed Multi-Program Encryption System” and incorporated herein by reference. In this approach, the most important segments of multiple ads are selected and combined for transmission during the ad insertion interval of a particular network stream.
An alternate embodiment of the invention is shown in
A preferred method for conveying information from the pre-splicer 22 (
In the example implementation of
Therefore, an adaptation header is created (if one does not already exist) immediately prior to the desired splice point in a particular video, audio or data stream. The adaptation header 42 includes a length field 42a, flags 42b, and a private data field 42c. Within the flags 42b is a transport_private_data_flag 44, which is set to ‘1’. The relevant information is then inserted into the private data field 42c, as specified by the ISO/IEC 13818-1 specification (available at www.iso.org).
As shown in
Note that in many systems, timing information such as presentation and decoding time stamps are subsequently encrypted, and therefore will not be available to the splicer. For this reason, the pre-splicer must examine the SCTE-35 cue messages, as well as the time stamps present in the video, audio, and data streams, in order to determine the precise location of a splicing exit or entrance point. Although the location of splicing points may be precisely conveyed from the pre-splicer to the splicer by inserting adaptation headers immediately before the splice entrance and exit points, it may also be important to allow time for communication and synchronization with the ad server. Therefore, it may be advantageous to insert the adaptation headers at intervals preceding the splice points. Additional information can be inserted into the header to specify either the time interval before the splice is to be executed, or if the splice is to be executed immediately.
The ad processing system of
In a typical system, the transrater would be placed after the ad insertion splicer and before the encrypter. In this way, the transrater is able to process the video signals while they remain in the clear, and is able to even out the data rate fluctuations introduced by the insertion of new video content. However, the example implementations disclosed herein place the transrater in front of the ad servers and the ad splicers. Therefore, the transrater should be modified in order to carve out room for the ad before the ad is inserted at the splicer. This can be done by examining the max data rate tag 46c that was described previously with reference to
The amount of transrating that is needed depends on the max data rate tag, as well as the rate of each component stream, including the stream(s) that is being replaced. Consider the example in
Assume that a segment of the first stream (video stream 1) and a segment of the forth video stream (video stream 4) are to be replaced by advertisements. The result after inserting these new segments is shown in the data rate diagram 70 of
Typically, the transrater operates as a statistical remultiplexer and attempts to deliver consistent video quality across all programs. As the combined data rate begins to exceed the data rate capacity of the transmission channel, the transrater increases the recompression ratio, resulting in a slight degradation in video quality on all programs. This is generally considered to be a better result than a larger degradation to the quality of one or more channels, while there is little or no degradation to others.
A conventional statistical remultiplexing transrater can be modified to allow for substitution of programming content at subsequent downstream locations. A simple solution is to modify the channel capacity parameter that is supplied to the transrater.
For example, immediately after the new content is spliced into video stream 1 at time instant TA shown in
Once the first splice is implemented for program 1 at time TA, the buffer fullness calculation is modified. Packets continue to be subtracted from the buffer at a constant rate C, while packets arriving from streams 2 to 6 continue to be added to the buffer. Although the new content for program 1 is not yet available for inclusion in the multiplex, the transrater rate control system can account for its presence by assuming a constant packet arrival rate R1. These hypothetical new packets to replace program 1 are added to the buffer fullness calculation. At the same time, the original packets corresponding to program 1 are subtracted from the buffer fullness result as they are processed. This new buffer fullness result is then used in place of the original fullness result when determining whether to increase or decrease the amount of video recompression. An example of a preferred method for mapping the multiplexer buffer fullness level to a video quality setting applicable to transrater systems is described in US patent publication 2008-0068997 entitled “Methods and Apparatus for Rate Estimation and Predictive Rate Control”, incorporated herein by reference.
In the example of
At time instant TC, the transrater adjusts the rate control calculation to account for the splice from ad stream 1 back to the original video stream 1. The channel capacity is now adjusted to become C−R4+r4. Finally, after completion of the ad for video stream 4 at time TD, the channel capacity reverts back to the nominal setting C, and normal transrater operation resumes until the next insertion interval. The result after transrating is illustrated in the data rate diagram 80 of
Depending on encoder settings used during compression of the ad segments, it may be necessary to consider transient data rates at each of the splicing points. This potential problem can be dealt with either by strictly enforcing the rate constraint at the beginning and end of each insertable ad, or by assuming worst case transition events during the transrating process. This is discussed in commonly owned co-pending U.S. provisional patent application No. 61/133,614 filed on Jun. 30, 2008 and entitled “Preconditioning Ad Content for Digital Program Insertion”, incorporated herein by reference.
It should now be appreciated that the present invention provides apparatus and methods for inserting advertisements and/or to perform grooming functions after a video, audio and/or data stream has been transrated and/or encrypted. In this manner, ad insertion and grooming can be performed close to the edge of a video distribution network. Transrating and encryption of a program into which content is to be later inserted can be accomplished before the program is transmitted. Thus, a single encrypted version of a program can be transmitted from a central point in the network to multiple recipients, while providing the benefits of subsequent targeted ad insertion or grooming downstream of the central point.