Apparatus to process images and video

Information

  • Patent Grant
  • 7991184
  • Patent Number
    7,991,184
  • Date Filed
    Wednesday, January 10, 2007
    17 years ago
  • Date Issued
    Tuesday, August 2, 2011
    12 years ago
Abstract
The present disclosure relates generally to digital watermarking. One claim recites an apparatus including: an input for receiving imagery or video; and an electronic processor for processing received imagery or video. The electronic processor is programmed for detecting first digital watermarking and second digital watermark from the imagery or video, in which the first digital watermarking utilizes a relationship between at least a first color space and a second color space. Another claim recites an apparatus including: an input for receiving imagery or video; and an electronic processor for processing received imagery or video. The electronic processor is programmed for embedding first digital watermarking and second digital watermark in the imagery or video, in which the first digital watermarking comprises a geometrically linear assignment of pixels, while the second digital watermarking comprises a geometrically random assignment of pixels. Of course other claims and combinations are provided too.
Description
FIELD OF THE INVENTION

The present invention relates to steganography, and more particularly relates to the use of multiple watermarks to determine the authenticity or history of a particular document or electronic object (e.g., image, motion picture, audio track).


BACKGROUND OF THE INVENTION

Steganographic and digital watermarking technologies are well known. For example see U.S. Pat. No. 5,636,292 and the extensive references cited therein. Also see patent application Ser. No. 08/327,426 which was filed Oct. 21, 1994 (now U.S. Pat. No. 5,768,426) and application Ser. No. 08/436,134 which was filed May 8, 1995 (now U.S. Pat. No. 5,748,763).


The technology for inserting digital watermarks in images and the technology for reading or detecting digital watermarks in images is well developed, well known and described in detail in public literature. Furthermore, there are commercially available products which include programs or mechanisms for inserting digital watermarks into images. For example the commercially available and widely used products “Adobe Photoshop” which is marketed by Adobe Corporation of San Jose Calif. and “Corel Draw” program which is marked by Corel Corporation of Ontario Canada, include a facility for inserting digital watermarks into images.


The technology for making high quality copies of documents is widely available. The technical quality of scanners and color printers has been increasing rapidly. Today for a relatively low cost one can purchase a high quality scanner and a high quality color printer. Thus, it is becoming increasingly easy to duplicate documents. The ability to create high quality copies has created a need for technology which can differentiate between original documents and copies of the original.


It is known that watermarks can be used to help differentiate genuine documents from copies. However, the prior art techniques for using digital watermarks to differentiate genuine documents from copies have serious limitations. The present invention is directed to an improved technique for using steganography and digital watermark technology to facilitate differentiating original documents from copies of the original.


The present invention can also be used for various other purposes such as to embed multiple types of information in a single document or to provide watermarks that enable documents to perform special functions.


SUMMARY OF THE INVENTION

With the present invention multiple digital watermarks, each of which has a different character, are embedded in a document. The characters of the two watermarks are chosen so that the watermarks will be affected in different manners by what may subsequently happen to the document.


The detection process or mechanism reads the two digital watermarks and compares their characteristics. While wear and handling may change the characteristics of the individual watermarks, the relationship between the characteristic of the two watermarks will never-the-less give an indication as to whether a document is an original or a copy of an original.


For example according to the present invention two digital watermarks in a document may have different energy levels. The absolute energy level of a digital watermark in an original image may be decreased if a document is subject to wear. Likewise the energy level of the digital watermark in an image may be decreased if an image is scanned and reprinted on a color printer. However, the relationship between the energy level of the two digital watermarks will be different in an image that has been subject to wear and in a reproduced image. Likewise if two digital watermarks are introduced into an image where the bit pattern used to construct the digital watermarks have different patterns, the ratio between the signal to noise ratio of the watermarks will be different in an original subject to wear and in a copy generated by scanning the original and printing the scanned image. Other characteristics of multiple digital watermarks can also be used to differentiate original documents from copies.


In other embodiments, a watermark-independent assessment of wear can be performed, and the results used to aid in differentiating original documents from copies.





BRIEF DESCRIPTION OF THE FIGURES


FIG. 1 shows the paths that a document and a copy may follow.



FIGS. 2A and 2B show a fine grain and a course grain watermark.



FIGS. 3A and 3B show a geometrically linear and a geometrically random assignment of pixels to a bit in a digital watermark.



FIG. 4 illustrates a fourth embodiment of the invention.



FIGS. 5A and 5B illustrate watermark strength v. wear charts.





DETAILED DESCRIPTION

The problem of differentiating an original document from a copy is made more difficult in situations where the original document is subject to being handled, worn, folded and otherwise damaged. Many original documents such as identification documents and currency are extensively handled. The wear to which such documents is subjected reduces the quality of images on the document and therefore reduces the quality of any information embedded in the document using conventional steganographic techniques.


With the present invention, a number of different watermarks are embedded in a document. Each of the watermarks embedded in the document has a different character. All watermarks are somewhat affected when a document is subjected to wear, and all watermarks are somewhat affected when a document is duplicated by being scanned and reprinted. However, the magnitude of the effect caused by being scanned and reprinted on watermarks with certain characteristics is much greater than the effect on watermarks with different characteristics. Likewise, wear and handling of a document affects watermarks with certain characteristics much more than it affects watermarks with different characteristics.


Thus, if multiple watermarks with different characteristics are inserted into a document, it is possible to differentiate a copy from an original document that has been subjected to wear by examining the ratios of characteristics of the watermarks in the image being examined.


In order to print a document on a color printer, the document is put through a transformation from a color space such as the RGB color space to a different color space such as the CMYK (cyan, magenta, yellow, black) color space. Such transformations are well known. For example see chapter 3 entitled “Color Spaces” in a book entitled “Video Demystified, A handbook for the Digital Engineer,” Second Edition, by Keith Jack, published by Harris Semiconductor/Hightext Publications of San Diego, Calif., and “The Color PC” by Marc Miller and published by the Hayden Press.


When an image is transformed from one color space to another color space, noise is introduced into the image. Among the reasons for this is the fact that each color space has its own distinctive gamut (or range) of colors. Where the gamut of two color spaces overlap, the conversion from one color space to another color space can in theory be precise. However, there will be some areas that are in the gamut of one color space but not in the gamut of another color space. Such situations definitely introduce noise into the conversion process. Even in areas that are in the gamut of two color spaces, conversion from one color space to another color space introduces noise because of such things as round off errors. The present invention takes advantage of the fact that if an original is copied and then a copy is printed, the image on the printed copy will have gone through several conversions to which the original will not have been subjected. For example, the conversions to which a copy may be subjected are:

  • 1) a document to RGB conversion (i.e. scanning the document into the computer),
  • 2) a RGB to CMYK conversion,
  • 3) a CMYK to copy conversion (i.e. printing the document).


Any characteristics of the two digital watermarks that will be affected differently by the additional conversion process to which copies are subjected can be used to differentiate copies from an original. Since the two watermarks with different characteristics are affected in a different manner by the additional conversion step, a comparison of the characteristics of the two watermarks in a document being examined will indicate if the document is an original (which has not gone through the additional conversions) or a copy which has gone through the additional conversions. While the characteristics of each watermark will have been changed by wear and by the copying process, the comparison between the characteristics of the two watermarks will still be able to differential a copy from an original.


Four embodiments of the invention are described below. Each of the embodiments utilizes two watermarks in a document. The differences between the two watermarks in the document are as follows:


In the First Embodiment





    • First watermark: Has fine grain

    • Second watermark: Has a course grain





In the Second Embodiment





    • First watermark: Has geometrically linear assignment of pixels

    • Second watermark: Has geometrically random assignment of pixels.





In the Third Embodiment





    • First watermark: Has low power

    • Second watermark: Has higher power





In the Fourth Embodiment





    • First watermark: uses standard RGB to HSI and HSI to RGB transformations

    • Second watermark is biased before being transformed from HSI to RGB.






FIG. 1 shows the steps to which documents and copies are typically subjected. In the normal course, a document 10 may be subjected to handling and wear 11 resulting in a worn document 10A. Document 10 may also be scanned as illustrated by box 12. The scanning produces a digital image that can be printed, as illustrated by box 13. The printed image may be subjected to handling and wear 14 resulting in a copy 10B. It is noted that the document 10 may also be subject to handling and wear prior to the scanning operation 12. The task to which this invention is directed is the task of differentiating the worn document 10A from the copy 10B.


The document 10 includes an image (not explicitly shown) that has two digital watermarks inserted therein. In the first embodiment of the invention, the first watermark has a fine grain and the second watermark has a course grain. The grain of the two watermarks is illustrated in FIG. 2. FIG. 2A shows the grain of the first watermark and FIG. 2B shows the grain of the second watermark. The first watermark uses blocks of 9 pixels (a 3 by 3 block). Each of the pixels in each 9 pixel block has its gray value changed by the same amount. For example FIG. 2A shows that the first 9 pixel block has its gray value increase and the second 9 pixel block has its gray value decreased. The amount of increase and the selection of blocks that is increased and decreased is conventional.


As shown in FIG. 2B, the grain of the second watermark is in blocks that are 6 pixels by 6 pixels or 36 pixels. All of the pixels in each 36 pixel block are changed by the same amount.


In the original document 10, the two watermarks have power ratios of 1 to 1. After wear and handling, the power of the first watermark will be degraded somewhat more than the power of the second watermark. For example, as illustrated in FIG. 1, after document 10 is subjected to handling and wear, a detector which reads the watermarks might find that the power ratio of the water marks is 1 to 2.


If the document 10 is scanned and the resulting digital image is printed to make a copy of the document 10, the ratio of the power of the watermarks will be affected much more than the effect of handling and wear. For example as illustrated in FIG. 1, the power ratio of the watermarks may be 1 to 10, thereby allowing one to differentiate the worn original document 10A from the copy 10B.


It is noted that the mechanism for inserting watermarks into an image is well known, as is the technique for reading a watermark and using correlation techniques to determine the signal to noise ratio (i.e. the power) of a watermark.



FIGS. 3A and 3B show an alternative technique for implementing the present invention. In the second embodiment of the invention, the two watermarks inserted into the image on a document have different patterns of assigning pixels to the bits of the payload represented by the watermark. The first watermark utilizes a geometrically linear assignment of pixels to each bit. For example FIG. 3A shows an image that has 500 by 500 pixels. Considering a watermark payload with 50 bits, each bit of the watermark would have 5000 pixels assigned to represent that bit. A linear assignment could have each fifth bit in each row (100 bits per row) and each fifth row (50 rows) assigned to each bit of the watermark. Thus 5000 pixels would be assigned to each bit in a very orderly or linear manner.


In the second watermark the pixels would be assigned to each bit in a random manner as shown in FIG. 3B. Each bit in the watermark would still have 5000 assigned bits; however, the pixels would be a random location over the image. Naturally it should be understood that FIGS. 3A and 3B illustrate how pixels are assigned to one bit of the watermark. The other bits of the watermarks would have pixels assigned in a similar manner.


Similar to the first embodiment of the invention, the watermark with a linear assignment of pixels and the watermark with a random assignment of pixels would be affected differently by handling and wear on the original document than they would be by being scanned and reprinted.


The third embodiment of the invention described herein utilizes watermarks that have different power levels. Handling and wear as contrasted to scanning and printing would affect a watermark with a low power level differently than a watermark with a high power level. Watermarks with different power levels can be inserted into a document in order to practice the present invention utilizing commercially available programs such as Adobe Photoshop or Corel Draw. In the Adobe Photoshop and Corel Draw programs, the power or intensity of the watermark can be adjusted by setting a simple control setting in the program.


The fourth embodiment of the invention introduces different characteristics into two watermarks by modifications made to one of the watermarks during the initial step during which the watermarks are introduced into an image. The operation of the fourth embodiment can be explained as shown in FIG. 4. First as illustrated by equation 1 there is a conversion from RGB to HSI as is conventional. This is illustrated by equation 1. As illustrated by equation 2, the first watermark is inserted into the image in a conventional manner by modifying the I value in the HSI representation of the image using the first watermark values (designated as WM1 Δ). A first RGB value designated RGB(1) is then calculated using a conventional transformation designated T. As indicated by equation 3, the second watermark WM2 is then biased toward a particular color and the biased watermark is then combined with the HSI values and transformed to a second set of RGB values designated RGB(2). Finally as indicated by equation 4, the values RGB(1) and RGB(2) are combined to form the watermarked image designated RGB(F).


The transform used to go from RGB to HSI color space (indicated in equation 1 in FIG. 4) can be anyone of a variety of known other techniques. For example, the RGB to HSI conversion can be one of the techniques explained in the above referenced text book such as the following: (which assumes that RGB and Intensity have a value range of 0 to I and that Red equals 0°):


First Calculate:

M=max(R, G, B)
m=min(R, G, B)
r=(M−R)/(M−m)
g=(M−G)/M−n)
b=(M−B)/(M−m)


Then calculate I, S, and H as follows:

I=(M+M)/2  a)
if M=m then S=0 and H=180
if I< or =0.5 then S=(M−m)/(M+m)
if I>0.5 then S=(M−m)/(2−M−m)  b)
if R=M then H=60(b−g)
if G=M then H=60(2+r−b)
if B=M then H=60(4+g−r)
if H> or =360 then H=H−360
if H<0 then H=H+360  c)


The first watermark in inserted into the RGB values in a conventional manner by modifying the I value of appropriate pixels so as to combine the watermark Δ values with HSI values. This is indicated by equation 2 in FIG. 4. Next as indicated by equation 3 in FIG. 4, the HSI values are converted to RGB values using a transform “T”. The transform “T” can be conventional and it can for example be done as follows:


First Calculate:

if I< or =0.5 then M=I(I+S)
if I>0.5 then M=I+S−IS
m=2I−M
if S=0 then R=G=B=I and H=180°


Then calculate R, G and B as follows:

if H<60 then R=M
if H<120 then R=m+((M−m)/((120−H)/60))
if H<240 then R=m
if H<300 then R=m+((M−m)/((H−240/60))
otherwise R=M  a)
if H<60 then G=m+((M−m)/(H/60))
if H<180 then G=M
if H<240 then G=m+((M−m)/((240−H/60))
otherwise G=m  b)
if H<120 then B=m
if H<180 then B=m+((M−m)/((H−120/60))
if H<300 then B=M
otherwise B=m+((M−m)/((360−H)/60))  c)


Next the values which represent a second watermark are used to calculate a second set of RGB values designated RGB2. In order to calculate RGB2, the values of H and S are modified so that they are slightly biased toward a particular color designated H1 and S1 New values for H and S are calculated as follows:


(Note, H1 must be between 0 and 360, S1 must be non-negative and can be between 0 and 1 and X is a value between 0 and 1)


Calculate new values for H and S as follows:

If H>H1 then H=H−(H−H1)x
else H=H+(H1−H)x
If S>S1 then S=S−(S−S1)x
else S=S+(S1−S)x

:


Next add the second watermark to the values of HSI and transform these values to the RGB color space as indicated by equation 3 The transformation from HSI color space to RGB color space is done as previously indicated.


Finally as indicated by equation 4 in FIG. 4, the final RGB value (designated RGBF) is calculated by combining the values of RGB1 and RGB2. This combination can be done in a variety of known ways.


It is noted that in the above example the difference between the transformation used for the first and the second watermarks involves biasing the values of H and S. Alternatively a wide variety of different changes could also be made. The key to this fourth embodiment of the invention is that in effect a different transformation is used for the first and the second watermarks.


In more sophisticated embodiments, the wear of the document can be independently assessed and used to aid in distinguishing an original from a copy.


There may be cases in which the wear-based degradation to the watermarks in a worn but original document can yield results similar to the scan/print degradation to the watermarks in a crisp copy. For example, consider the case of an original document having watermarks A and B of equal energy, but tailored so that watermark B is more frail and falls-off rapidly in energy when photocopied. On finding a suspect document with a ratio of energy between the two documents in excess of 2:1 (or a commensurate difference in signal-to-noise ratios), a counterfeit may be presumed. However, this ratio may also result from extreme wear of an original document. See, e.g., the watermark strength v. wear chart of FIGS. 5A and 5B for an original document, and the same document after scanning on a 600 dpi scanner and printing on a 720 dpi printer. The original document degrades to a watermark energy ratio of 2:1 at point x. A crisp copy has the same ratio, resulting in a potential ambiguity.


To distinguish these two cases, the wear of the document can be assessed. Various means can be used to distinguish document wear. One is high frequency content, as can be determined by high pass filtering the document image data, or performing an FFT, DCT, etc. A worn document typically loses some high frequency energy. Another is contrast—a worn document typically loses contrast. Still another is color gamut—a worn document may fade to a less varied gamut. Still another is luminance—the soiling of a document can decrease the overall document brightness. Yet another is physical integrity—a worn document droops when only partially supported. Yet another means is a quick human assessment of wear, with human entry of a corresponding datum into a system (e.g., on a wear scale of 0 to 10, or simply “crisp,” “used,” or “very worn”). Still other means can similarly be employed.


The wear can be graded on an arbitrary scale, depending on the particular measurement means used. In an illustrative case, wear may range from 0 (“crisp”) to 7(extreme). In the FIG. 5 example, the x point may be at wear value 5. In distinguishing the documents, a look-up table, microprocessor-implemented algorithm, or other arrangement can be provided that takes as its input the ratio and wear values, and produces outputs, e.g., as follows:






















Wear = 0
Wear = 1
Wear = 2
Wear = 3
Wear = 4
Wear = 5
Wear = 6
Wear = 7
























Ratio =
Original
Original
Original
Original
Error?
Error?
Error?
Error?


1.0


Ratio =
Original
Original
Original
Original
Original
Error?
Error?
Error?


1.25


Ratio =
Original
Original
Original
Original
Original
Original
Error?
Error?


1.5


Ratio =
Copy
Copy
Original
Original
Original
Original
Original
Error?


1.75


Ratio =
Copy
Copy
Copy
Copy
Original
Original
Original
Original


2.0


Ratio =
Copy
Copy
Copy
Copy
Copy
Original
Original
Original


2.25


Ratio =
Copy
Copy
Copy
Copy
Copy
Copy
Original
Original


2.5


Ratio =
Copy
Copy
Copy
Copy
Copy
Copy
Original
Original


2.75


Ratio =
Copy
Copy
Copy
Copy
Copy
Copy
Copy
Original


3.0


Ratio =>
Copy
Copy
Copy
Copy
Copy
Copy
Copy
Copy


3.25










(The “Error?” outputs corresponds to cases that should not occur in actual practice, e.g., a very worn document in which the ratio of watermarks is 1.0.)


While four embodiments and a further enhancement of the invention have been shown herein, it should be understood that many other characteristics and attributes of a digital watermark could be used to practice the present invention in addition to the characteristics and attributes described herein. Furthermore other known digital watermarking techniques can be used together with and applied to the digital watermarks used for the present invention. It is also noted that while in the above examples only two watermarks were used; in some situations one could use three, four five or more watermarks. That is, the embodiments of the invention specifically described herein utilize two watermarks. It should be understood that any number of watermarks could be utilized in like manner. Furthermore while the embodiments shown herein utilize two separate watermarks, the two watermarks used to practice the present invention could be combined into one watermark which has a plurality of separate identifiable and measurable characteristics.


Still further, while the invention was particularly illustrated with reference to watermarking that is effected in the pixel domain, the same techniques are likewise applicable to watermarking effected in the DCT, wavelet, or other domain (e.g., as shown in U.S. Pat. No. 5,930,369). Moreover, some documents may include watermarks effected in two different domains (e.g., pixel and DCT).


Still further, the different watermarks can be of entirely different types. For example, one watermark can comprise slight alterations to the image normally printed on a document, and the second can comprise a texture formed on the document substrate, or a background weave or tint pattern—both of which convey watermark data. (Examples of texture-, weave- and tint-based watermarks are shown, e.g., in copending application Ser. No. 09/074,034 (filed May 6, 1998; now U.S. Pat. No. 6,449,377), Ser. No. 09/127,502 (filed Jul. 31, 1998; now U.S. Pat. No. 6,345,104), Ser. No. 09/151,492 (filed Sep. 11, 1998), U.S. Pat. No. 5,850,481, and laid-open PCT publication WO 99/53428.


It is noted that while the present invention utilizes multiple watermarks with different characteristics to differentiate original documents from copies of the original, one can also utilizes multiple watermarks with different characteristics for other reasons. Documents may include multiple similar watermarks in addition to the watermarks having different characteristics according to the present invention. As used herein, the term “document” generally refers to a physical entity. However, the same methodologies can also be applied to purely digital images—e.g., to detect losses that an image has suffered through a lossy compression/decompression process such as JPEG or MPEG, color re-balancing, etc., and thereby discern something about the history of a digital image.


It will be recognized that the principles of the invention can be incorporated into an apparatus used at cash registers and other points of sale to assess the genuineness of banknotes, food stamps, coupons, and other documents. Such an apparatus can include a scanning 1D, or stationary 2D image sensor (e.g., CMOS or CCD), and a microprocessor suitably programmed to discern first and second watermarks in image data provided by the sensor (as well as wear, if desired). (In some cases, a stationary 1D sensor may be employed.) Such apparatus further includes an output device—such as a display screen, indicator light, audible tone, voice synthesizer, or equivalent device—to provide an appraisal of the document's validity based on the sensed information.


A similar apparatus can be provided for use by Customs officials at ports of entry to check merchandise tags, packaging, labels, and other printed indicia associated with clothing, purses, electronic components, software, and other readily-counterfeitable goods, to determine whether the sensed tag/package/label is an original, or a copy. While such a determination may not provide the confidence needed to seize a shipment as counterfeit, it could flag the goods as suspect and needing further inspection and/or forensic analysis.


The idea in each of the foregoing apparatuses is, of course, to provide an indication of possible non-genuineness more reliable than the typical casual and semi-casual human inspection during very fast point-of-sale transactions and other similar high traffic volume situations, where it is unrealistic to expect human observation to be efficient “flaggers” of suspect product and documents.


To provide a comprehensive disclosure without unduly lengthening this specification, applicants incorporate by reference the documents (including applications) cited above.


While the present invention has been described with respect to four specific embodiments of the invention, it should be understood that various changes in forma and detail could be made without departing from the spirit and scope of the invention. The scope of the present invention is limited only by the appended claims.

Claims
  • 1. An apparatus comprising: an input configured to receive imagery or video;a processor configured to embed a first digital watermarking and a second digital watermarking in the imagery or video, wherein the first digital watermarking is embedded according to first color characteristics, wherein the second digital watermarking is embedded according to second color characteristics, and wherein a comparison between characteristics associated with the first digital watermarking and characteristics associated with the second digital watermarking provide clues regarding the imagery or video; andan output for configured to output the embedded imagery or video.
  • 2. The apparatus of claim 1, wherein the comparison comprises calculating a ratio.
  • 3. The apparatus of claim 1, wherein the processor steganographically embeds the first digital watermarking in a first color gamut.
  • 4. The apparatus of claim 1, further comprising a memory configured to store the imagery or video.
  • 5. The apparatus of claim 1, wherein the clues pertain to authentication.
  • 6. An apparatus comprising: an input configured to receive imagery or video;a processor configured to:detect at least a first digital watermarking and a second digital watermarking from the imagery or video, wherein the first digital watermarking is embedded according to first color characteristics, and wherein the second digital watermarking is embedded according to second color characteristics;compare characteristics associated with the first digital watermarking and characteristics associated with the second digital watermarking; andprovide clues regarding the imagery or video; andan output configured to output information associated with the clues.
  • 7. The apparatus of claim 6, wherein the comparing comprises calculating a ratio.
  • 8. The apparatus of claim 6, further comprising a memory configured to store the imagery or video.
  • 9. The apparatus of claim 6, wherein the clues pertain to authentication.
  • 10. An apparatus comprising: an input configured to receive imagery or video; anda processor configured to detect a first digital watermarking and a second digital watermarking from the imagery or video, wherein the first digital watermarking utilizes a relationship between at least a first color space and a second color space.
  • 11. The apparatus of claim 10, further comprising a memory configured to store the imagery or video.
  • 12. An apparatus comprising: an input configured to receive imagery or video; anda processor configured to embed a first digital watermarking and a second digital watermarking in the imagery or video, wherein the first digital watermarking comprises a geometrically linear assignment of pixels, and wherein the second digital watermarking comprises a geometrically random assignment of pixels.
  • 13. The apparatus of claim 12, further comprising a memory configured to store the imagery or video.
  • 14. A method comprising: receiving imagery or video;embedding a first digital watermarking and a second digital watermarking in the imagery or video, wherein in which the first digital watermarking is embedded according to first color characteristics, wherein the second digital watermarking is embedded according to second color characteristics, and wherein a comparison between characteristics associated with the first digital watermarking and characteristics associated with the second digital watermarking provide clues regarding the imagery or video; andoutputting the embedded imagery or video.
  • 15. A non-transitory computer-readable medium having instructions stored thereon, the instructions comprising: instructions to receive imagery or video;instructions to embed a first digital watermarking and a second digital watermarking in the imagery or video, wherein in which the first digital watermarking is embedded according to first color characteristics, wherein the second digital watermarking is embedded according to second color characteristics, and wherein a comparison between characteristics associated with the first digital watermarking and characteristics associated with the second digital watermarking provide clues regarding the imagery or video; andinstructions to output the embedded imagery or video.
  • 16. A method comprising: receiving imagery or video;detecting at least a first digital watermarking and second digital watermarking from the imagery or video, wherein the first digital watermarking is embedded according to first color characteristics, and wherein the second digital watermarking is embedded according to second color characteristics;comparing characteristics associated with the first digital watermarking and characteristics associated with the second digital watermarking; andproviding clues regarding the imagery or video; andoutputting information associated with the clues.
  • 17. A non-transitory computer-readable medium having instructions stored thereon, the instructions comprising: instructions to receive imagery or video;instructions to detect at least a first digital watermarking and second digital watermarking from the imagery or video, wherein the first digital watermarking is embedded according to first color characteristics, and wherein the second digital watermarking is embedded according to second color characteristics;instructions to compare characteristics associated with the first digital watermarking and characteristics associated with the second digital watermarking; andinstructions to provide clues regarding the imagery or video; andinstructions to output information associated with the clues.
  • 18. A method comprising: receiving imagery or video; anddetecting a first digital watermarking and a second digital watermarking from the imagery or video, wherein the first digital watermarking utilizes a relationship between at least a first color space and a second color space.
  • 19. A non-transitory computer-readable medium having instructions stored thereon, the instructions comprising: instructions to receive imagery or video; andinstructions to detect a first digital watermarking and a second digital watermarking from the imagery or video, wherein the first digital watermarking utilizes a relationship between at least a first color space and a second color space.
  • 20. A method comprising: receiving imagery or video; andembedding a first digital watermarking and a second digital watermarking in the imagery or video, wherein the first digital watermarking utilizes a relationship between at least a first color space and a second color space.
  • 21. A non-transitory computer-readable medium having instructions stored thereon, the instructions comprising: instructions to receive imagery or video; andinstructions to embed a first digital watermarking and a second digital watermarking in the imagery or video, wherein the first digital watermarking utilizes a relationship between at least a first color space and a second color space.
  • 22. An apparatus comprising: an input configured to receive imagery or video; anda processor configured to embed a first digital watermarking and a second digital watermarking into the imagery or video, wherein the first digital watermarking utilizes a relationship between at least a first color space and a second color space.
  • 23. A method comprising: receiving imagery or video; andembedding a first digital watermarking and a second digital watermarking in the imagery or video, wherein the first digital watermarking comprises a geometrically linear assignment of pixels, and wherein the second digital watermarking comprises a geometrically random assignment of pixels.
  • 24. A non-transitory computer-readable medium having instructions stored thereon, the instructions comprising: instructions to receive imagery or video; andinstructions to embed a first digital watermarking and a second digital watermarking in the imagery or video, wherein the first digital watermarking comprises a geometrically linear assignment of pixels, and wherein the second digital watermarking comprises a geometrically random assignment of pixels.
  • 25. A method comprising: receiving imagery or video; anddetecting a first digital watermarking and a second digital watermarking from the imagery or video, wherein the first digital watermarking comprises a geometrically linear assignment of pixels, and wherein the second digital watermarking comprises a geometrically random assignment of pixels.
  • 26. A non-transitory computer-readable medium having instructions stored thereon, the instructions comprising: instructions to receive imagery or video; andinstructions to detect a first digital watermarking and a second digital watermarking from the imagery or video, wherein the first digital watermarking comprises a geometrically linear assignment of pixels, and wherein the second digital watermarking comprises a geometrically random assignment of pixels.
  • 27. An apparatus comprising: an input configured to receive imagery or video; anda processor configured to detect a first digital watermarking and a second digital watermarking from the imagery or video, wherein the first digital watermarking comprises a geometrically linear assignment of pixels, and wherein the second digital watermarking comprises a geometrically random assignment of pixels.
RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No. 10/723,731, filed Nov. 26, 2003 (now U.S. Pat. No. 7,171,020), which is a continuation of application Ser. No. 10/012,703, filed Dec. 7, 2001 (now U.S. Pat. No. 6,744,906), which is a continuation of application Ser. No. 09/433,104, filed Nov. 3, 1999 (now U.S. Pat. No. 6,636,615), which is a continuation in part of application Ser. No. 09/234,780, filed Jan. 20, 1999 (now abandoned), which claims the benefit of application 60/071,983 filed Jan. 20, 1998.

US Referenced Citations (305)
Number Name Date Kind
4238849 Gassmann Dec 1980 A
4296326 Haslop et al. Oct 1981 A
4689477 Goldman Aug 1987 A
4723072 Naruse Feb 1988 A
4908873 Philibert et al. Mar 1990 A
5228056 Schilling Jul 1993 A
5237164 Takada Aug 1993 A
5284364 Jain Feb 1994 A
5291243 Heckman et al. Mar 1994 A
5337361 Wang et al. Aug 1994 A
5354097 Tel Oct 1994 A
5384846 Berson et al. Jan 1995 A
5436970 Ray et al. Jul 1995 A
5450490 Jensen et al. Sep 1995 A
5471533 Wang et al. Nov 1995 A
5481377 Udagawa et al. Jan 1996 A
5490217 Wang et al. Feb 1996 A
5493677 Balogh et al. Feb 1996 A
5502576 Ramsay et al. Mar 1996 A
5510900 Shirochi et al. Apr 1996 A
5530759 Braudaway et al. Jun 1996 A
5541741 Suzuki Jul 1996 A
5598526 Daniel et al. Jan 1997 A
5617119 Briggs et al. Apr 1997 A
5636292 Rhoads Jun 1997 A
5646997 Barton Jul 1997 A
5652626 Kawakami et al. Jul 1997 A
5659726 Sandford, II et al. Aug 1997 A
5687236 Moskowitz et al. Nov 1997 A
5694471 Chen et al. Dec 1997 A
5721788 Powell et al. Feb 1998 A
5745604 Rhoads Apr 1998 A
5748763 Rhoads May 1998 A
5751854 Saitoh et al. May 1998 A
5768426 Rhoads Jun 1998 A
5772250 Gasper Jun 1998 A
5799092 Kristol et al. Aug 1998 A
5822436 Rhoads Oct 1998 A
5825892 Braudaway et al. Oct 1998 A
5841886 Rhoads Nov 1998 A
5850481 Rhoads Dec 1998 A
5857038 Owada et al. Jan 1999 A
5862260 Rhoads Jan 1999 A
5864622 Marcus Jan 1999 A
5892900 Ginter et al. Apr 1999 A
5901224 Hecht May 1999 A
5905800 Moskowitz et al. May 1999 A
5907149 Marckini May 1999 A
5912974 Holloway et al. Jun 1999 A
5933798 Linnartz Aug 1999 A
5943422 Van Wie et al. Aug 1999 A
5946414 Cass et al. Aug 1999 A
5949055 Fleet et al. Sep 1999 A
5949885 Leighton Sep 1999 A
5951055 Mowry, Jr. Sep 1999 A
5974548 Adams Oct 1999 A
5991426 Cox et al. Nov 1999 A
6024287 Takai et al. Feb 2000 A
6026193 Rhoads Feb 2000 A
6064764 Bhaskaran et al. May 2000 A
6065119 Sandford, II et al. May 2000 A
6104812 Koltai et al. Aug 2000 A
6122392 Rhoads Sep 2000 A
6185683 Ginter et al. Feb 2001 B1
6188787 Ohmaie et al. Feb 2001 B1
6226387 Tewfik et al. May 2001 B1
6233347 Chen et al. May 2001 B1
6233684 Stefik et al. May 2001 B1
6237786 Ginter et al. May 2001 B1
6240121 Senoh May 2001 B1
6243480 Zhao et al. Jun 2001 B1
6246775 Nakamura et al. Jun 2001 B1
6246777 Agarwal et al. Jun 2001 B1
6256110 Yoshitani Jul 2001 B1
6266430 Rhoads Jul 2001 B1
6272176 Srinivasan Aug 2001 B1
6272634 Tewfik et al. Aug 2001 B1
6275599 Adler et al. Aug 2001 B1
6278792 Cox et al. Aug 2001 B1
6285775 Wu et al. Sep 2001 B1
6285776 Rhoads Sep 2001 B1
6289108 Rhoads Sep 2001 B1
6292092 Chow et al. Sep 2001 B1
6304345 Patton et al. Oct 2001 B1
6314192 Chen et al. Nov 2001 B1
6314457 Schena et al. Nov 2001 B1
6330335 Rhoads Dec 2001 B1
6332031 Rhoads et al. Dec 2001 B1
6332194 Bloom et al. Dec 2001 B1
6334187 Kadono Dec 2001 B1
6343138 Rhoads Jan 2002 B1
6343204 Yang Jan 2002 B1
6345104 Rhoads Feb 2002 B1
6353672 Rhoads Mar 2002 B1
6363159 Rhoads Mar 2002 B1
6374036 Ryan et al. Apr 2002 B1
6389151 Carr et al. May 2002 B1
6398245 Gruse et al. Jun 2002 B1
6400827 Rhoads Jun 2002 B1
6404898 Rhoads Jun 2002 B1
6418232 Nakano et al. Jul 2002 B1
6425081 Iwamura Jul 2002 B1
6427012 Petrovic Jul 2002 B1
6427020 Rhoads Jul 2002 B1
6427140 Ginter et al. Jul 2002 B1
6430302 Rhoads Aug 2002 B2
6439465 Bloomberg Aug 2002 B1
6442284 Gustafson et al. Aug 2002 B1
6449377 Rhoads Sep 2002 B1
6449379 Rhoads Sep 2002 B1
6456726 Yu et al. Sep 2002 B1
6487301 Zhao Nov 2002 B1
6496591 Rhoads Dec 2002 B1
6504941 Wong Jan 2003 B2
6512837 Ahmed Jan 2003 B1
6519352 Rhoads Feb 2003 B2
6522771 Rhoads Feb 2003 B2
6535618 Rhoads Mar 2003 B1
6539095 Rhoads Mar 2003 B1
6542618 Rhoads Apr 2003 B1
6542620 Rhoads Apr 2003 B1
6560349 Rhoads May 2003 B1
6560350 Rhoads May 2003 B2
6567534 Rhoads May 2003 B1
6567535 Rhoads May 2003 B2
6567780 Rhoads May 2003 B2
6571001 Murphy May 2003 B2
6574350 Rhoads et al. Jun 2003 B1
6577744 Braudaway et al. Jun 2003 B1
6577746 Evans et al. Jun 2003 B1
6580819 Rhoads Jun 2003 B1
6587821 Rhoads Jul 2003 B1
6590996 Reed et al. Jul 2003 B1
6590997 Rhoads Jul 2003 B2
6594373 Gustafson Jul 2003 B1
6614914 Rhoads et al. Sep 2003 B1
6625295 Wolfgang et al. Sep 2003 B1
6636615 Rhoads et al. Oct 2003 B1
6647129 Rhoads Nov 2003 B2
6654480 Rhoads Nov 2003 B2
6654887 Rhoads Nov 2003 B2
6675146 Rhoads Jan 2004 B2
6700995 Reed Mar 2004 B2
6718046 Reed et al. Apr 2004 B2
6721440 Reed et al. Apr 2004 B2
6724912 Carr et al. Apr 2004 B1
6728390 Rhoads et al. Apr 2004 B2
6738495 Rhoads et al. May 2004 B2
6744906 Rhoads et al. Jun 2004 B2
6744907 Rhoads Jun 2004 B2
6750985 Rhoads et al. Jun 2004 B2
6754377 Rhoads Jun 2004 B2
6757406 Rhoads Jun 2004 B2
6762124 Kian et al. Jul 2004 B2
6763123 Reed et al. Jul 2004 B2
6768808 Rhoads Jul 2004 B2
6771796 Rhoads Aug 2004 B2
6778682 Rhoads Aug 2004 B2
6785815 Serret-Avila et al. Aug 2004 B1
6804376 Rhoads et al. Oct 2004 B2
6804377 Reed et al. Oct 2004 B2
6804379 Rhoads Oct 2004 B2
6850626 Rhoads et al. Feb 2005 B2
6882738 Davis et al. Apr 2005 B2
6891959 Reed et al. May 2005 B2
6912295 Reed et al. Jun 2005 B2
6915002 Gustafson Jul 2005 B2
6944298 Rhoads Sep 2005 B1
6959100 Rhoads Oct 2005 B2
6959386 Rhoads Oct 2005 B2
6970573 Carr et al. Nov 2005 B2
6978036 Alattar et al. Dec 2005 B2
6983051 Rhoads Jan 2006 B1
6987862 Rhoads Jan 2006 B2
6993152 Patterson et al. Jan 2006 B2
6996252 Reed et al. Feb 2006 B2
7003132 Rhoads Feb 2006 B2
7006656 Fridrich et al. Feb 2006 B2
7027614 Reed Apr 2006 B2
7054462 Rhoads et al. May 2006 B2
7054463 Rhoads et al. May 2006 B2
7055034 Levy May 2006 B1
7072487 Reed et al. Jul 2006 B2
7076084 Davis et al. Jul 2006 B2
7113615 Rhoads et al. Sep 2006 B2
7130087 Rhoads Oct 2006 B2
7171020 Rhoads et al. Jan 2007 B2
7181022 Rhoads Feb 2007 B2
7184570 Rhoads Feb 2007 B2
7213757 Jones et al. May 2007 B2
7227671 Zolla et al. Jun 2007 B2
7239734 Alattar et al. Jul 2007 B2
7242790 Rhoads Jul 2007 B2
7263203 Rhoads et al. Aug 2007 B2
7266217 Rhoads et al. Sep 2007 B2
7269275 Carr et al. Sep 2007 B2
7286684 Rhoads et al. Oct 2007 B2
7305117 Davis et al. Dec 2007 B2
7313253 Davis et al. Dec 2007 B2
7321667 Stach Jan 2008 B2
7340076 Stach et al. Mar 2008 B2
7359528 Rhoads Apr 2008 B2
7372976 Rhoads et al. May 2008 B2
7391880 Reed et al. Jun 2008 B2
7400743 Rhoads et al. Jul 2008 B2
7415129 Rhoads Aug 2008 B2
7418111 Rhoads Aug 2008 B2
7424132 Rhoads Sep 2008 B2
7427030 Jones et al. Sep 2008 B2
7499564 Rhoads Mar 2009 B2
7532741 Stach May 2009 B2
7536555 Rhoads May 2009 B2
7537170 Reed et al. May 2009 B2
7539325 Rhoads et al. May 2009 B2
7548643 Davis et al. Jun 2009 B2
7555139 Rhoads et al. Jun 2009 B2
7567686 Rhoads Jul 2009 B2
7570784 Alattar Aug 2009 B2
7602940 Rhoads et al. Oct 2009 B2
7602977 Rhoads et al. Oct 2009 B2
7606390 Rhoads Oct 2009 B2
7639300 Yumiki Dec 2009 B2
7639837 Carr et al. Dec 2009 B2
7672477 Rhoads Mar 2010 B2
7676059 Rhoads Mar 2010 B2
7702511 Rhoads Apr 2010 B2
7720249 Rhoads May 2010 B2
7720255 Rhoads May 2010 B2
7724919 Rhoads May 2010 B2
7738673 Reed Jun 2010 B2
7762468 Jones et al. Jul 2010 B2
7796826 Rhoads et al. Sep 2010 B2
20010008557 Stefik et al. Jul 2001 A1
20010020270 Yeung et al. Sep 2001 A1
20010021144 Oshima et al. Sep 2001 A1
20010024510 Iwamura Sep 2001 A1
20010026377 Ikegami Oct 2001 A1
20010028725 Nakagawa et al. Oct 2001 A1
20010028727 Naito et al. Oct 2001 A1
20010047478 Mase Nov 2001 A1
20010051996 Cooper et al. Dec 2001 A1
20010052076 Kadono Dec 2001 A1
20010053235 Sato Dec 2001 A1
20010054144 Epstein et al. Dec 2001 A1
20020010684 Moskowitz Jan 2002 A1
20020015509 Nakamura et al. Feb 2002 A1
20020023148 Ritz et al. Feb 2002 A1
20020031241 Kawaguchi et al. Mar 2002 A1
20020033844 Levy et al. Mar 2002 A1
20020037091 Terasaki Mar 2002 A1
20020037093 Murphy Mar 2002 A1
20020054355 Brunk May 2002 A1
20020061121 Rhoads et al. May 2002 A1
20020061122 Fujihara et al. May 2002 A1
20020062442 Kurahashi May 2002 A1
20020064298 Rhoads et al. May 2002 A1
20020064759 Durbin et al. May 2002 A1
20020067914 Schumann et al. Jun 2002 A1
20020068987 Hars Jun 2002 A1
20020071556 Moskowitz et al. Jun 2002 A1
20020073317 Hars Jun 2002 A1
20020080396 Silverbrook et al. Jun 2002 A1
20020095577 Nakamura et al. Jul 2002 A1
20020097873 Petrovic Jul 2002 A1
20020097891 Hinishi Jul 2002 A1
20020105679 Haynes Aug 2002 A1
20020106192 Sako Aug 2002 A1
20020112171 Ginter et al. Aug 2002 A1
20020114458 Belenko et al. Aug 2002 A1
20020118381 Shirai et al. Aug 2002 A1
20020122568 Zhao Sep 2002 A1
20020163671 Takaragi Nov 2002 A1
20020176114 Zeller et al. Nov 2002 A1
20020178368 Yin et al. Nov 2002 A1
20030009669 White et al. Jan 2003 A1
20030033529 Ratnakar et al. Feb 2003 A1
20030097568 Choi et al. May 2003 A1
20030128860 Braudaway et al. Jul 2003 A1
20030138128 Rhoads Jul 2003 A1
20040263911 Rodriguez et al. Dec 2004 A1
20050058320 Rhoads et al. Mar 2005 A1
20060062386 Rhoads Mar 2006 A1
20070016790 Brundage et al. Jan 2007 A1
20070172098 Rhoads et al. Jul 2007 A1
20070180251 Carr et al. Aug 2007 A1
20070201835 Rhoads Aug 2007 A1
20080131083 Rhoads Jun 2008 A1
20080131084 Rhoads Jun 2008 A1
20080149713 Brundage Jun 2008 A1
20080253740 Rhoads Oct 2008 A1
20080275906 Rhoads et al. Nov 2008 A1
20090220121 Stach Sep 2009 A1
20090252401 Davis et al. Oct 2009 A1
20100008534 Rhoads Jan 2010 A1
20100008536 Rhoads Jan 2010 A1
20100008537 Rhoads Jan 2010 A1
20100021004 Rhoads Jan 2010 A1
20100027969 Alattar Feb 2010 A1
20100040255 Rhoads Feb 2010 A1
20100119108 Rhoads May 2010 A1
20100131767 Rhoads May 2010 A1
20100142752 Rhoads et al. Jun 2010 A1
20100146285 Rhoads et al. Jun 2010 A1
20100163629 Rhoads et al. Jul 2010 A1
20100172538 Rhoads Jul 2010 A1
Foreign Referenced Citations (36)
Number Date Country
23 04 416 Aug 1973 DE
29 43 436 May 1981 DE
0 614 308 Sep 1994 EP
629 972 Dec 1994 EP
975 147 Jan 2000 EP
1041815 Oct 2000 EP
1077570 Feb 2001 EP
1202250 May 2002 EP
1209897 May 2002 EP
1050005 Sep 2007 EP
2346110 Aug 2000 GB
3-185585 Aug 1991 JP
WO9513597 May 1995 WO
WO9820411 May 1998 WO
WO 9936876 Jul 1999 WO
WO 0044131 Jul 2000 WO
WO0105075 Jan 2001 WO
WO 0108405 Feb 2001 WO
WO0139121 May 2001 WO
WO0173997 Oct 2001 WO
WO0176253 Oct 2001 WO
WO 0188883 Nov 2001 WO
WO0197128 Dec 2001 WO
WO01978175 Dec 2001 WO
WO0207425 Jan 2002 WO
WO0207442 Jan 2002 WO
WO0217631 Feb 2002 WO
WO0219589 Mar 2002 WO
WO0225599 Mar 2002 WO
WO0237309 May 2002 WO
WO 0250773 Jun 2002 WO
WO 02056264 Jul 2002 WO
WO02056264 Jul 2002 WO
WO 02087250 Oct 2002 WO
WO 02089057 Nov 2002 WO
WO02098670 Dec 2002 WO
Related Publications (1)
Number Date Country
20070172098 A1 Jul 2007 US
Provisional Applications (1)
Number Date Country
60071983 Jan 1998 US
Continuations (4)
Number Date Country
Parent 10723731 Nov 2003 US
Child 11621905 US
Parent 10012703 Dec 2001 US
Child 10723731 US
Parent 09433104 Nov 1999 US
Child 10012703 US
Parent 10112647 Mar 2002 US
Child 11420977 US
Continuation in Parts (4)
Number Date Country
Parent 09234780 Jan 1999 US
Child 09433104 US
Parent 11420977 May 2006 US
Child 09234780 US
Parent 09998763 Nov 2001 US
Child 10112647 US
Parent 09442440 Nov 1999 US
Child 09998763 US