Patent Application
20070052814
The spectral distribution of different light sources can vary significantly such that pictures taken of the same object under different lighting conditions may appear quite different from one another. An incandescent lamp light source, for example, will produce a picture that appears to be warmer (redder) than it actually is, because the fraction of blue and green photons coming out of the lamp are less than they would be for light from a more natural light source such as sun light.
In an electronic camera, color information is recorded separately for each primary color, i.e., there are separate sub-pixels for red, green and blue at each pixel location on a light sensor of the camera. In order to accurately depict the color of an object photographed with an electronic camera, it is necessary to balance the colors of each sub-pixel at a pixel location in the correct proportion to account for the particular light source that was used to illuminate the object. Because a picture of a white object such as a piece of white paper may appear yellow as a result of the colors of the sub-pixels being incorrectly balanced, the process of bringing the color of the paper back to white and, in general, of correcting the color of a picture, is referred to as “white-balancing”.
In an electronic camera, a picture can be white-balanced by electronically changing the colors corresponding to different sub-pixels. This can be done by taking a picture of a white paper, and determining the scale factors needed to make the white balance come out correctly from information gathered about the picture. Alternatively, a camera can be provided with a “built-in” color correction capability such that an operator can indicate the illumination source type being used prior to taking a picture.
In accordance with the invention, a method and apparatus for white-balancing an image of an object is provided. Light reflected from the object illuminated with light having known spectral characteristics is collected, and the spectral reflectivity characteristics of the object is determined from the collected light. Light reflected from the object illuminated with light having unknown spectral characteristics is collected to form an image of the object, and the formed image is white-balanced in accordance with the determined spectral reflectivity characteristics of the object.
Furthermore, the invention provides embodiments and other features and advantages in addition to or in lieu of those discussed above. Many of these features and advantages are apparent from the description below with reference to the following drawings.
Exemplary embodiments in accordance with the invention provide a method and apparatus for white-balancing an image of an object.
When using digital camera 100 to take a picture of an object, such as object 120 in
Since what is actually measured in digital camera 100 is the charge generated in each sub-pixel for its associated color, the color of a photographed scene can be altered by electronically changing the color corresponding to each sub-pixel at a pixel location using image processor 112. Image processor 112 may be built-in camera 100 as shown in
In particular, in order to accurately depict the color of an object photographed with an electronic camera, such as digital camera 100 in
White-balancing is necessary to accurately depict the color of a photographed object because the spectral characteristics of light from different light sources can differ significantly such that photographs taken of the same object under different lighting conditions may appear quite different from one another. The object being photographed may, for example, be illuminated by ambient light from an incandescent light source, a fluorescent light source, a candle, sunlight or another light source or combination of light sources. Each different light source type has different spectral characteristics, such that the color of an image of the object may not accurately reflect the actual color of the object.
According to exemplary embodiments in accordance with the invention, an image of an object is white-balanced by determining the spectral reflectivity characteristics of the object, and then white-balancing the image in accordance with the spectral reflectivity characteristics of the object.
Referring back to
In accordance with an exemplary embodiment in accordance with the invention, the light source emitting light having known spectral characteristics is flash unit 106 that is already provided in digital camera 100. Specifically, flash units that are currently built-in or otherwise linked to digital cameras produce a pulse of light having a very short duration of, for example, from less than about 1.0 to about 3.0 milliseconds, and is sufficiently bright to minimize the effect of any ambient light that may also illuminate the object. It should be understood, however, that the invention is not limited to use of any particular light source emitting light of known spectral characteristics, and other light sources can also be used, if desired.
The spectral characteristics of the light emitted by flash unit 106 is known by the manufacturer of the flash unit and/or of the camera, and can be stored in memory 114, for example, a RAM (Random Access Memory), provided in camera 100. When object 120 is illuminated with light from flash unit 106, the light reflected from the object is collected and focused onto light sensor 104 by lens 102 to form an image of object 120 on light sensor 104. The color of the image, as detected by individual pixels 212, 214 and 216 at each pixel location 210 is a function of both the spectral characteristics of light 134 from flash unit 106, and the spectral reflectivity characteristics of object 120. Since the spectral characteristics of the illuminating light 134 is known and stored in memory 114, its effects are removed from the received light signal by image processor 112 to permit the spectral reflectivity characteristics of the object to be determined. The determined spectral reflectivity characteristics of the object is also stored in memory 114.
A picture is then taken of object 120 under ambient light conditions wherein the object is illuminated by light having unknown spectral characteristics. Light reflected from the object is collected by lens 102 and an image of the object is formed on light sensor 104. Since the spectral reflectivity characteristics of the object is now known and stored in memory 114, image processor 112 is able to use this information to correctly balance the colors of each sub-pixel at each pixel location on light sensor 104 in the correct proportion to account for the spectral characteristics of the ambient light that illuminated the object during the photographing operation, to produce a picture, generally represented by arrow 116, that accurately represents the color of the object.
Typically, white-balancing is done by boosting and/or reducing the strength of one or more of the sub-pixels at one or more pixel locations. In particular, the picture taking process can be symbolically represented by the following equation:
[Rij]=Ired [Aij]+Igreen [Bij]+Iblue [Cij]
where Rij is the response of the composite pixel location “ij”, and Ired, Igreen and Iblue are the red, green and blue illumination source functions. Aij, Bij and Cij are the reflecting functions of the object that contribute to light arriving at pixel location “ij”.
Since the illumination source functions Ired, Igreen and Iblue are known, they can be deconvolved from the response Rij permitting the reflecting functions Aij, Bij and Cij to be determined. This information is then used to determine the fraction of the signal received at each sub-pixel that is due to the strength of the ambient light that illuminates the object and to white-balance the picture of the object by adjusting the strength of each pixel to remove the effect of the ambient light from the signal.
According to an exemplary embodiment in accordance with the invention, the actual picture of the object under ambient light conditions is taken substantially immediately after the picture is taken using the flash unit so that the object will not have moved (if it is a movable object) or otherwise changed in such a way that might alter its spectral reflectivity characteristics. It should be understood, however, that the invention is not limited to any particular timing between taking the picture under known lighting conditions and under ambient lighting conditions, or to any particular order in which the pictures are taken.
In accordance with exemplary embodiments in accordance with the invention, an image of an object can be white-balanced quickly and accurately without involvement of the operator of the camera. In addition, the method and apparatus according to exemplary embodiments of the invention make use of the light sensor and flash unit that are already present in most digital cameras, and, thus, the present invention adds very little to the overall cost of the camera.
As a flash unit ages, its spectral characteristics may change with time which may result in the white-balancing algorithm used by image processor 112 to white-balance an image degrading over time. This can be countered by providing a calibration mechanism, schematically designated by reference number 140 in
While what has been described constitute exemplary embodiments in accordance with the invention, it should be recognized that the invention can be varied in numerous ways without departing from the scope thereof. Because exemplary embodiments in accordance with the invention can be varied in numerous ways, it should be understood that the invention should be limited only insofar as is required by the scope of the following claims.
