Patent Application
20050025463
In any communications medium, one may want to send different copies of content to different receivers. This may be useful, for example, for forensics when copies have been stolen. If the copy that has been stolen can be uniquely identified, then the circumstances of the theft can be more easily narrowed down. For example, if each movie shown in a theatre could be uniquely watermarked, it would be possible to identify the theatre from which a pirated copy was taken. This is of particular importance in the movie industry because piracy amounts to several billion dollars a year worldwide. Then, if a movie were stolen, at least the point of theft could be easily determined by looking at the pirated copies. In order for this unique watermarking to be useful, it must be done in a secure way such that the content owner, not the distributor or exhibitor, can guarantee the uniqueness of the copy. If someone other than the content owner, e.g., the exhibitor, can control the watermark, then they can steal the copy without leaving a watermarked trail.
Therefore, there is a need in the art to provide uniquely marked copies of content in the compressed and uncompressed domain.
The present invention generally relates to a method and apparatus to provide uniquely marked copies of content in the compressed and uncompressed domain. In one embodiment, at least two content versions are provided. The at least two content versions are alternated in order to provide uniquely marked content.
In another embodiment, a plurality of content versions are provided. Content is unlocked in accordance with a keyset.
In another embodiment, a content stream having a plurality of segments is provided. One or more segments are selected from the content stream in accordance with a keyset.
So that the manner in which the above recited features of the present invention can be understood in detail, a more particular description of the invention, briefly summarized above, may be had by reference to embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments of this invention and are therefore not to be considered limiting of its scope, for the invention may admit to other equally effective embodiments.
The present invention generally relates to uniquely marking content. Watermarking may occur in the compressed domain or watermarking may be applied to uncompressed content. One having skill in the art would understand that the marking techniques disclosed herein could be applied to video and/or audio content.
Assuming for simplicity that watermark carriers do not overlap in time, i.e., each frame contributes to at most one watermark carrier, which can be modulated to carry a single bit of payload (0 or 1). Results can be generalized to non-binary modulation and also to carriers overlapping in space and/or time. Let Ui be the i-th frame of the unmarked copy and let Mij be the i-th frame of the j-th marked copy. Then marking can be viewed as the combination of two mathematical operators:
Mij=embed(Ui, select ij(0,1))
The function select ij(a,b) returns its first argument if the j-th marked copy should contain a 0 payload in the i-th frame and its second argument if it should contain a 1. The function embed(F,m) embeds a watermark carrier modulated for payload m in frame F.
Since embedding distributes over selection, the above formula can be rewritten as:
Mij=select ij(embed(Ui,0), embed(Ui,1))
Note that the original formula contains one (computationally expensive) embedding operation per marked copy per frame, while the second formula contains two such embedding operations. At first glance, it would appear that the first formula is more efficient. However, note that the embeddings in the second formula are independent of the particular marked copy. Thus, one would need two embeddings per marked frame whether one produces one marked copy or a million marked copies. In the case of non-binary modulation and/or carriers overlapping in time and/or space, the above discussion holds if the select function has an appropriate arity and returns one of all possible combinations of payloads and if the embed function takes a vector as its second argument rather than a single number. For example, if there are at most k binary carriers overlapping in time, the select function must have an arity of 2k and the embed operation must take as second argument a vector of dimension k.
The above conclusions lead to a method to create marked copies.
In one embodiment, the marked copies are compressed. The present invention discloses several methods to create two compressed marked copies (all-0s and all-1s) shown as inputs in
Care must be taken to ensure that switching from the all-0s sequence to the all-1s sequence is feasible. There are many ways to achieve this. One possible way is to ensure that watermark carriers are always separated by at least one I-frame and always effect switching at one of the I-frames that separates the carriers.
In one embodiment, one copy is marked by embedding all-0s in the pixel domain and another copy by embedding all-1s in the pixel domain. Each copy is compressed separately. The resulting compressed files should be similar enough to allow multiplexing of frames from either copy, because the differences introduced by watermarking are relatively small. The compatibility of the two versions can be verified, but it cannot be guaranteed. The approach is illustrated in
In one embodiment, one copy is marked (e.g., by embedding all-0s in the pixel domain) and compressed. A slave encoder that mimics all the encoding decisions of the original encoder is used and the method proceeds in either of the following ways: 1) Mark the uncompressed copy with all-1s in the pixel domain and feed it into the slave encoder, which creates a compressed all-1s marked copy compatible with the all-0s copy created by the master encoder. This approach is illustrated in
In the slave-encode once method, compatibility can be guaranteed and the content need be encoded only once. This is important, since human intervention may be required.
In one embodiment, the unmarked copy is compressed and the method proceeds in either of the following ways: 1) Create uncompressed all-0s and all-1s marked copies of the original sequence and run each of them through the slave encoder, to create compressed all-0s and all-1s marked copies that are compatible with the unmarked compressed copy created by the master encoder and with each other. The approach is illustrated in
Either way, in the slave-encode twice method, the resulting marked copies are compatible with each other (and with the original copy). As in the slave encode once method, compatibility can be guaranteed and the content need be encoded only once. In addition, a carrier list (i.e., a spatiotemporal list of watermark locations) need not be generated until after compression, i.e., compression is independent of watermarking, which provides operational workflow advantages.
For certain applications, it is desirable not to change the size of the compressed file upon insertion of the watermarks in the compressed domain. This size preservation may be desirable to meet bandwidth, buffering or recording medium size constraints. There are two ways to deal with this problem: 1) Leave additional space during compression (or pad the compressed file), so that the resulting bit stream is compliant with the standard but still has some “slack” space in it. When the watermark is added in the compressed domain, the slack space can be used to absorb the expansion while preserving the size of the bit stream. One way of adding slack space in an MPEG-2 bitstream is to intentionally add picture-layer stuffing, which is in an integer number of zero bytes added to the end of a coded picture. As a conservative estimate, the amount of stuffing should be greater than the largest increase that the low-frequency watermark is expected to add to the bit rate (on the order of 2-4%); or 2) Locally adjust the bit rate. The bit rate may be locally adjusted so that the total size of the compressed picture (with watermark) is less than or equal to that of the original frame. If the adjusted frame size, after requantization, is less than the original frame, it can be made equal to the original frame by picture-layer stuffing. Local adjustment of bit rate can be done in many ways and can be done in a single pass or in multiple passes for coded frame. Since the addition of the watermark will generally increase the bit rate, the way to compensate for this is to more heavily quantize certain parts of the image, compared to their original quantization state. For instance, the edges of the image or busier regions can be quantized more heavily with little degradation. When the picture is requantized, the total coded size of each watermarked frame is compared to its original. Requantizing by a single large factor (like 1.2× the original) will virtually guarantee that the output frame size will be smaller than the original in a single pass; however, the quality degradation may be too severe. In a multipass embodiment, selected regions are more heavily quantized and the output size is iteratively compared to the original until it is equal or lower in size. If desired, image and video quality tools, like peak signal-to-noise ratio (PSNR) and just noticeable difference or just noticeable distortion (JND), can be used to assess the impact of the requantization. Thresholds can be used to ensure that degradation due to requantization stays below some predetermined value.
For some applications, it may be desirable to process content frames (in order to embed the watermark) in an order different from the order in which the content frames were processed when the carrier list for the watermarks was created. For example, it may be desirable to process the content in the order in which it is stored on a physical medium, which may be different from the order in which it is played out. An example is a DVD image file, where blocks of compressed video may not be stored consecutively because of navigation or interleaving.
In step 115 the at least two content versions are alternated in order to provide uniquely marked content. In one embodiment, the original (unmarked) copy is marked twice, once with all watermark carriers modulated to carry a 0-bit as payload and once with all watermark carriers modulated to carry a 1-bit as payload. Creating an individually marked copy with an arbitrary payload essentially reduces to assembling the copy by selecting frames from the all-0 marked sequence (if the copy to be created has a 0 payload in the corresponding location) or from the all-1 marked sequence (if the copy to be created has a 1 payload in the corresponding location). This is illustrated in
In one embodiment two compressed watermarked master streams are created, one marked entirely with is and the other entirely with 0s. To create a uniquely watermarked stream from these two, the streams are selected alternately using a pattern of alternation that represents a unique code sequence.
In another embodiment, two uncompressed video streams are provided, each stream marked with either an “all ones” watermark sequence or an “all zeros” watermark sequence.
In one embodiment, two master VTRs 805, 825 playing in sync are fed to a 2×N routing switcher 820 that is remote-controlled by a PC 815 with a timecode reader 810. In one embodiment, timecoder reader 810 is a PCI timecode reader card. Timecode from one VTR 805 (or perhaps house timecode) is fed to the PC 815. PC 815 is also given a file containing a sequence of 1s and 0s unique for each of the N router outputs, corresponding to which VTR 805, 825 should be connected to a given router output, and the times (timecodes) at which those connections should be made. An example of the file PC 815 is given is illustrated in
In one embodiment, more than two uncompressed video streams may be provided (not shown). In this embodiment each stream would be marked with either an “all ones” watermark sequence, an “all zeros” watermark sequence, or no watermark sequence. This operation may be necessary if any of the ‘one’ or ‘zero’ watermarks is visible, in which case the unwatermarked sequence would be switched in.
In step 1015 content is unlocked in accordance with a keyset. Along with the versions of the data is a set of keys, transmitted separately, that unlocks some, but not all of the segments.
In the embodiments illustrated in
Note that the keys are sent entirely separately from the content. In order to ensure security, the keys would be sent separately from content, perhaps using public key encryption technology to securely send a keyset over the Internet to each venue where the content was to be displayed. Because the keys are very small, they can be sent efficiently to a large number of locations. Thus, using the movie example, if a movie is shown on three thousand screens, the large content files can be broadcast to all locations, but the small keyset files can be sent, individually, over the Internet to each theatre. The content owner would have a record of how and to whom that keyset was distributed. In this example, each location would have a unique keyset that would allow only the particular subset of content to be played.
If pirated content were found with a particular set of watermarks, it would be instantly clear which keyset was used on the content. In one embodiment, watermarking could be implemented using a combination of marked segments where the marked segments are marked with a watermark representing a “one” or a “zero”. In another embodiment, watermarking could be implemented using a combination of marked and unmarked segments since it may be necessary to replace a marked segment with an unmarked segment if the marked segment is visible.
In the above embodiment, the entire content comprises paired segments and it would be necessary to send twice as much content to the receiver in order to have one unique piece of content. However, for purposes of distinguishing individual versions of the content, it may not be necessary to have the entire content paired up, only small parts of it that might, for example, contain distinguishing watermarks.
The content bitstreams in
In one embodiment, watermarking could be implemented using a combination of marked segments where the marked segments are marked with a watermark representing a “one” or a “zero”. In another embodiment, watermarking could be implemented using a combination of marked and unmarked segments since it may be necessary to replace a marked segment with an unmarked segment if the marked segment is visible.
Besides watermarking, as in the previous examples, it may be useful to broadcast multiple versions of data, for example movies with different ratings. The content provider may want to send a single data stream that plays differently in different locations, e.g., with a PG-13 rating in one location and an R rating in another. This is easy to do with the proposed scheme, by only sending keys for the segments to be displayed. Note that there may be some segments that do not appear at all in some reconstructed versions of the content. A movie studio might wish to send out a movie with several different ratings and perhaps charge differently for each rating, or restrict distribution of some ratings to particular geographic areas.
There could be more than two versions of any given piece of the content, so that keysets would be selected from more than two sets of keys.
Note that in
Different versions and different ratings and unique watermarking can all be applied simultaneously to the same content.
The present invention need not be applied only to image content. It could apply equally well to audio or other data that is broadcast to sites. For example, the content owner could control the language in which a movie is shown. From a business point of view, a distributor might want to charge more for a movie with multiple language sound tracks as opposed to a single language. The present invention allows the content owner to control which audio goes with the movie as it is shown.
The invention disclosed in
The invention disclosed in
Thus, image processing device or system 1900 comprises a processor (CPU) 1910, a memory 1920, e.g., random access memory (RAM) and/or read only memory (ROM), unique content creation module 1940, and various input/output devices 1930, (e.g., storage devices, including but not limited to, a tape drive, a floppy drive, a hard disk drive or a compact disk drive, a receiver, a transmitter, a speaker, a display, an image capturing sensor, e.g., those used in a digital still camera or digital video camera, a clock, an output port, a user input device (such as a keyboard, a keypad, a mouse, and the like, or a microphone for capturing speech commands)).
It should be understood that the unique content creation module 1940 can be implemented as one or more physical devices that are coupled to the CPU 1910 through a communication channel. Alternatively, the unique content creation module 1940 can be represented by one or more software applications (or even a combination of software and hardware, e.g., using application specific integrated circuits (ASIC)), where the software is loaded from a storage medium, (e.g., a magnetic or optical drive or diskette) and operated by the CPU in the memory 1920 of the computer. As such, the unique content creation module 1940 (including associated data structures) of the present invention can be stored on a computer readable medium, e.g., RAM memory, magnetic or optical drive or diskette and the like.
While the foregoing is directed to embodiments of the present invention, other and further embodiments of the invention may be devised without departing from the basic scope thereof, and the scope thereof is determined by the claims that follow.
This application claims benefit of U.S. provisional patent application Ser. No. 60/480,307, filed Jun. 20, 2003 and 60/515,144, filed Oct. 28, 2003, which are herein incorporated by reference.
| Number | Date | Country | |
|---|---|---|---|
| 60480307 | Jun 2003 | US | |
| 60515144 | Oct 2003 | US |
