The present invention relates to image-related pixel information written to or retrieved from hard-copy image media such as color paper or film using an additional light-sensitive layer.
Display or capture media for images have been known for over 150 years. Today, these images are often organized into color planes, each plane representing the corresponding color components of the image. Black-and-white images have one plane, while color images generally have three, one each in a color primary, typically red, green, and blue. Images are often stored on negative or positive transparent film, color paper, or digitally on magnetic media. When stored digitally, uncompressed images are generally organized as sets of pixels, image lines, or image planes. Each element of the set represents a different color within the image. When stored on film, a separate physical layer of the film incorporates each color plane of the image. Traditional color silver halide paper uses a similar approach. Inkjet and thermal sublimation techniques deposit separate layers or pixels of primary colored inks.
False-color imaging systems are also well known. In these systems, invisible light (typically infrared or ultraviolet) exposes otherwise traditional imaging materials such as color film to produce a visible color in the exposed medium.
It is often useful to associate non-image information about an image with the image. Such information is often known as meta-data and comprises information such as time of image capture, location of capture, an image description, flash and exposure settings, audio clips, and so forth. This information can be used to optimize the display or use of the image itself. When images are stored digitally, associating such information with an image is straightforward. The meta-data is organized in some structured fashion and written as a header in the image file or otherwise associated with the image file.
Associating such information with images stored on hard-copy media is more problematic. Such a system is described in U.S. Pat. No. 5,763,891 issued Jun. 9, 1998 to Yoshinaga et al., entitled Non-Visualized Information Recording Medium and Apparatus for Detecting the Same.
Techniques are known for writing meta-data on a variety of hard-copy image media. For each of these techniques, invisible, light-responsive materials must be coated or otherwise applied to the medium, and methods to expose or otherwise write information in the invisible material, as well as read the information back, are known (e.g. see U.S. Pat. No. 5,995,193 issued Nov. 30, 1999 to Stephany et al., entitled Self-Contained Device for Recording Data Encoded Either in Visible or Invisible Form). Other materials, such as a thermal sublimation donor material coated with an additional, invisible layer capable of information storage on a receiver media, is known as is the use of an inkjet deposition system to provide an additional invisible layer for information storage on a paper inkjet print. U.S. Pat. No. 5,514,860 issued May 7, 1996 to Berson, entitled Document Authentication System Utilizing a Transparent Label describes placing a transparent tape over a medium. U.S. Pat. No. 5,841,885 issued Nov. 24, 1998 to Neff et al., entitled Print and Method and Apparatus for Printing, Storing and Retrieving an Image Record discloses storing additional digital information on the reverse of an image print.
There are also techniques known that utilize invisible magnetic layers to store information. For example, U.S. Pat. No. 5,229,259 issued Jul. 20, 1993 to Yokota, entitled Silver Halide Photographic Material discloses a silver halide photographic material with a magnetic recording layer. However, these systems and their associated meta-data associate information with the image as a whole and do not physically associate information with each image pixel on the recording medium.
In addition to meta-data describing attributes of an image, information related to each pixel can be useful, particularly if the hard-copy image media constrains the image information. Once written to the constrained medium, original image information is lost. For example, a residual image is the difference between one hard-copy representation of an image (typically color paper) and another less constrained representation (typically color film).
In general, it is possible to store any meta-data or additional information associated with the pixels of an image separately from the image pixels, either on the same hard-copy medium or in a separate location. In the latter case, the meta-data information may become disconnected from the image and lost. In the former case, the storage is inherently less robust in that if a portion of the image is lost or damaged, it is very likely that all of the information will be lost or unrecoverable. Moreover, there is a general problem with digital information storage in file formats. The formats tend to be very media or computer dependent and change rapidly. Hence, information stored in obsolete formats quickly becomes unrecoverable as the software and computer systems to support the format become unavailable.
As images become increasingly pervasive and the variety of image applications grows, additional pixel-related information beyond the traditional red, green, and blue (or cyan, yellow, magenta, and black) is becoming extremely useful. For example, applications such as image scene modeling, image differences, image understanding, multi-image scenes, and three-dimensional images, display require per-pixel information such as scene distance from capture device, pixel differences from other images, color/exposure attributes, or object classification information. This information can be captured or calculated in a variety of ways and readily stored in a digital system but robust storage and recovery of the additional information is problematic when associated with hard-copy film or paper media.
There is a need therefore for an improved method for storing and retrieving additional information associated with each pixel in an image on hard-copy media.
This need is met according to the present invention by providing an image bearing article that includes a support; a visible image recorded on the support; and invisible information recorded on the support, the invisible information relating to and in registration with elements of the visible image.
The foregoing problems associated with the storage of additional pixel-related information on hard-copy image media, may be overcome by writing and then detecting pixel-related information stored in an invisible layer in the media at each pixel location. This invisible layer is created through conventional coating, sublimation, or deposition means. The information is written by any of various conventional digital means such as optical lasers, heating elements, light-emitting diodes, inkjets, or through the use of traditional optical means. The information must be relevant to the specific pixel at whose location the information is placed and must not be dependent on information stored at other pixel sites. The information is essentially stored in analog form in the medium at each pixel site in a data layer or data plane. Once written, the information in the layer is developed, laminated, or otherwise fixed and made robust in the medium. The information is read back by shining suitable invisible light (infrared or ultraviolet) onto the medium and measuring the light reflected back from the medium at each pixel location. The use of invisible light will not interfere with traditional viewing of the image on the media.
The system works as follows (see
In normal use, the image medium is used as a conventional reflection print or transparency. When a digital version of the image is required, the visible image can be read out using conventional scanning means by illuminating the image and detecting the light reflected from or transmitted through the image. If the invisible data is also required, the scanner illuminates the image with suitable invisible light and detects the light reflected from or transmitted through the invisible layer at each pixel location.
When the additional pixel data is required, the visible image is scanned with visible light to produce a digital image, and the invisible data image is scanned with invisible light 34 and the information recovered 36. Depending on the nature of the invisible dyes or inks, the exposure and reading may be a two-(or more) step process. For example, one might first read the red, then the green, then the blue, then the invisible layers. The image may then be processed using the additional data 38 and the processed image displayed 40.
The image written in step 28 of
Since the additional pixel-related information is physically stored in the same location on the medium as the pixels themselves and is not dependent on other pixels (as, for example, with run-length encoding), the information can be readily recovered under difficult circumstances. If, for example, a portion of the medium were damaged or destroyed by ripping, tearing, or defacing, the remainder of the information can be readily recovered with the pixels themselves. Moreover, the alignment of the hard-copy image and any readers will not be critical since the data information associated with each pixel is stored at the same physical pixel location.
Each medium on which the additional information layer is constructed may have a different mechanism for storage and retrieval of the information. The general requirements are that the exposure of the information may not interfere with the exposure or formation of the visible color image, that the information image is permanently fixed in the medium, and that the medium can then be illuminated with invisible light and reflect or transmit it in a detectable way that does not interfere with normal image viewing. The response of the invisible dyes or inks comprising the invisible image layer must be invisible as well as the illumination to support normal viewing, since it is difficult to guarantee that illumination with invisible light will not occur under normal image viewing circumstances.
A specific system requirement for the writing mechanism, reading mechanism, and dyes or inks is that they be capable of pixel-level resolution. The information may be written one pixel at a time, for example with a laser flying-spot technique or a movable LED or inkjet head, or written all at once with a light beam modulated on a per-pixel basis. Another requirement is that the additional information written within the additional layer be physically co-located (registered) with the traditional image information so that the data is associated with the relevant image pixel. This requirement is similar to traditional imaging systems in which the various color planes are registered together.
In all these cases, the amount of information stored at each pixel location is dependent on the ability of the recording medium and the reading mechanism to discriminate different amounts of invisible dye or material and is dependent on the amount of dye or material that can be present without affecting the normal viewing of the print. In the simplest case, only the presence or absence of the material may be detected, resulting in a one-data-bit per pixel system.
It is also possible to write and retrieve the invisible pixel-related information at different resolutions from the traditional image pixels. For example, an image might be printed at 600 pixels per inch in two dimensions but the invisible information printed and retrieved at 1200 pixels per inch. Thus, four data values might be associated with each pixel, thereby increasing the quantity and density of information in the print. Alternatively, if the material used for the invisible layers has a lower resolution, the invisible pixel-related data may be scanned at a lower resolution than the image data and associated with groups of pixels rather than individual pixels.
It is also possible to extend the concept of using invisible pixel information to create multiple invisible layers. Just as color images record more information than black-and-white images, multiple invisible layers could record more than one additional data point for each traditional pixel as long as they can be readily discriminated, for example, by frequency of illuminator or response. The technique can also be applied to sequential series of images such as motion pictures.
Various embodiments of the present invention are possible. There are many well-known ultraviolet or infrared dyes and inks with the necessary characteristics. Likewise, stabilizers and dispersants are known for coatings on inkjet, thermal sublimation, or silver halide media. Some preferred embodiments will be described here. In the case of traditional chemical color papers or films, an additional layer of light responsive material is coated onto the medium. The layer may or may not be the top layer, so long as it is capable of responding to the invisible light for both writing and reading and can fix that exposure in the layer. Any of the traditional coupler and dye methods known in the art may be used so long as they do not interfere with the normal imaging process, their development is complementary to the normal imaging process, and they meet the system requirements listed above. Note that false-color imaging techniques known in the art can be utilized for the image exposure step (responding to the invisible light) but not the generation of a false-color image plane. Rather, an invisible image plane must be generated in response to the exposure. This can be accomplished using inks, pigments, or dyes that absorb and/or fluoresce in the infrared or ultraviolet regions of the electromagnetic spectrum using means as exemplified in U.S. Pat. Nos. 5,542,971 and 5,514,860. U.S. Pat. No. 5,542,971 issued Aug. 6, 1996 to Auslander et al., entitled Bar Codes Using Luminescent Invisible Inks discloses bar codes using luminescent invisible inks, and U.S. Pat. No. 5,514,860 to Berson (cited above) discloses transparent materials imprinted with inks visible only in the infrared portion of the electromagnetic spectrum.
A second embodiment can utilize thermal media. Conventional image recording using thermal techniques utilizes a donor whose colored material is deposited on the receiver media when stimulated by heat. Each primary color is deposited using a separate, colored donor in a multi-pass system with a covering material laminated on top in a final pass. This technique is readily extended to include an additional, invisible layer encoding information at each pixel site prior to the final pass.
A third embodiment can utilize inkjet media. Conventional inkjet images are recorded by squirting tiny amounts of ink onto a paper medium at each pixel location. Different colors are obtained by squirting different amounts of colored inks, typically cyan, magenta, yellow, and black. Again, this technique is readily extended by squirting an appropriate amount of invisible dye at each pixel location and on top of the existing inks.
Although these systems can be both analog and digital, for many practical applications the invisible light exposure must be done separately from the conventional image information and this is done most easily with digital imaging systems that provide a greater degree of control over the information content and the image data itself. Digital to analog converters are needed to appropriately control the exposure of the medium. As with any hard copy image medium, the actual record will be an analog quantity, typically the amount of dye present.
Once the medium is properly exposed or written and both the visible and invisible image plane data is fixed in the medium, it can be read back and the information applied within the context of the desired imaging system. As with the exposure system, this is most easily accomplished within a digital system. Each pixel can be exposed sequentially or in parallel and the reflected (or transmitted, in the case of film) light measured at the various frequencies of interest. These are, of course, the traditional color planes plus the invisible data plane with the additional information. Traditional scanning techniques known in the art are suitable with the addition of the invisible light. For example, lasers and LED devices can illuminate the medium with an additional frequency while CCDs could respond to the reflected or transmitted invisible light by utilizing an additional light gathering element with a suitable filter. Note that, as with traditional red, green, or blue pixel information, the additional information stored at each pixel location will not be sensitive to minor differences in value when read back, since the actual information is, at this point, analog.
The additional data plane information stored at each pixel location can be used in a variety of applications. For example, additional means might be employed when an image is captured to record the distance to each pixel in the scene from the point of capture. This information is associated with each pixel and can be written onto the image as disclosed in this invention. In a second example, an image might be recorded onto a medium incapable of storing all of the image information (for example, a restricted dynamic range or color gamut). The additional pixel information might represent the difference between the recorded image and the original image. In a third example, an image might be analyzed and each pixel associated with an object in the image or otherwise classified such as foreground and background, flash highlighted, in gamut or out of gamut colors, original or later added special effect, etc. This object classification information might then be stored as additional information at each pixel site. (In this case it is important that the data values be readily differentiable without error since small changes in value may have a major impact on the classification.) In a fourth example, the additional data might represent additional interpolated pixel values. In a fifth example, two images (for example, a stereo pair) might be stored together by recording one image conventionally and the difference between the recorded and the second image stored as additional information at each pixel site. In a still further example, the image is a constrained image, and the data image is the difference between the constrained and an unconstrained version of the image. The constrained image can be constrained for example in resolution, dynamic range, or color gamut.
In general, the invisible data values associated with each pixel must encode some pertinent information. The meaning of the data value should not be susceptible to small changes in the value or pixel location. If this requirement is accommodated, many different kinds of information may be encoded as described above. In other cases, interpolated pixel values might be encoded or information relating to the calibration or absolute range of the associated pixel values.
An image recording medium with the addition of an invisible layer to record additional image information at each pixel, together with means for writing and reading back the invisible information provide a simple and robust mechanism for recording non-traditional image information on image hard copy. The invention utilizes conventional media, digital writing, and digital reading techniques.
The invention has been described in detail with particular reference to certain preferred embodiments thereof, but it will be understood that variations and modifications can be effected within the spirit and scope of the invention.
Number | Name | Date | Kind |
---|---|---|---|
3755935 | Annenberg | Sep 1973 | A |
4151667 | Idelson et al. | May 1979 | A |
4213038 | Silverman et al. | Jul 1980 | A |
4627819 | Burrows | Dec 1986 | A |
4745268 | Drexler | May 1988 | A |
4776013 | Kafri et al. | Oct 1988 | A |
4818852 | Haddock et al. | Apr 1989 | A |
4972476 | Nathans | Nov 1990 | A |
5229259 | Yokota | Jul 1993 | A |
5337361 | Wang et al. | Aug 1994 | A |
5490217 | Wang et al. | Feb 1996 | A |
5514860 | Berson | May 1996 | A |
5542971 | Auslander et al. | Aug 1996 | A |
5646388 | D'Entremont et al. | Jul 1997 | A |
5661506 | Lazzouni et al. | Aug 1997 | A |
5763891 | Yoshinaga et al. | Jun 1998 | A |
5841885 | Neff et al. | Nov 1998 | A |
5984366 | Priddy | Nov 1999 | A |
5995193 | Stephany et al. | Nov 1999 | A |
6049627 | Becker et al. | Apr 2000 | A |
6101039 | Blazey | Aug 2000 | A |
6104812 | Koltai et al. | Aug 2000 | A |
6109526 | Ohanian et al. | Aug 2000 | A |
6142380 | Sansone et al. | Nov 2000 | A |
6174400 | Krutak et al. | Jan 2001 | B1 |
6252963 | Rhoads | Jun 2001 | B1 |
6330976 | Dymetman et al. | Dec 2001 | B1 |
6466209 | Bantum | Oct 2002 | B1 |
6556690 | Nelson | Apr 2003 | B1 |
6572025 | Nishikado et al. | Jun 2003 | B1 |
6610386 | Williams et al. | Aug 2003 | B1 |
6734887 | Field | May 2004 | B1 |
6808118 | Field | Oct 2004 | B1 |
Number | Date | Country |
---|---|---|
0 627 720 | May 1994 | EP |
07266755 | Oct 1995 | JP |