The present invention generally relates to a method for enhancing digital photo images, and, more specially, to a method for enhancing digital photo images of non-uniform brightness that are affected by excessive light sources. Images affected by excessive light sources generally exhibit high contrast and cause backlighting or reflection problems. Such problems can consequently result in overexposing and obscure images. The present invention aims to improve the aforementioned problems.
In the fields of digital cameras and digital camcorders, backlighting refers to a situation that the sun or a strong light source is positioned behind the objects to shoot. Under backlighting situations, images become unsatisfactory as the objects appear dark or obscure. In contrast, reflection refers to a situation that a strong light source or a strong reflected glare is contained in an image, making the image extremely bright or overexposed. Digital images or digital video clips taken under these two kinds of situations are defined as digital images of non-uniform brightness.
Digital images of non-uniform brightness are commonly seen in digital photography, which represent a difficult problem that both the industry and academic are attempting to solve. The tradition techniques typically applied in the industry are explained in the following. The first technique is called “auto exposure control”, which has become an essential element in digital cameras. A good auto exposure control adjusts the shutter and aperture for the best quality of image. However, because the settings of the shutter and aperture in auto exposure control are achieved by using the light metering method to detect the intake of light, good results can be expected in scenes under normal lighting but non-uniform lighting.
The second technique is known as Gamma correction in digital image processing. It is also very commonly used in digital cameras. Gamma correction transforms digital images via a nonlinear mathematical function. Under backlighting conditions, Gamma correction brightens darker areas in the original digital image, but may further brighten those already bright areas and lose some color information. Similarly, under reflection conditions, Gamma correction decreases the brightness of brighter areas, but may further darken those already dark objects and lose some color information.
Based on the explanation above, the two previously proposed techniques cannot effectively enhance digital images of non-uniform brightness caused by backlighting or reflection, and such problems are a major topic to be solved in digital photography.
The main purpose of the present invention is to overcome backlighting or reflection problems in digital photography, caused by strong light sources. Strong light sources may produce high contrast in digital images, making objects to shoot too dark, too bright or overexposed and rendering the images non-uniform in color and brightness.
To achieve this goal, the present invention proposes a method for enhancing images of non-uniform brightness. In the invention, a surround function is developed to analyze relationship of individual pixel brightness to that of surrounding pixels in an image. This brightness information is then used to set a gain function to decide the required adjustments on the values of RGB (red, green and blue) color channels of each pixel. The final value of a pixel is the sum of the adjusted values of R, G and B channels. The present method can imitate human vision and adaptively adjust brightness in every region of an image while preserving color consistency.
The application file contains at least one drawing executed in color. Copies of this patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.
FIG. 3 shows a flow chart explaining the steps of the present invention for enhancing images of non-uniform brightness.
For persons of ordinary skill in the art to understand the purpose, features and effects of the present invention, the invention is described in detail as follows, along with examples and figures.
In order to circumvent the problems of non-uniform brightness caused by backlighting or reflection, the present invention proposes an enhancement technique for digital images. It imitates the adaptiveness of human eyes to improve the quality of digital images of non-uniform brightness. In human vision, the pupil adjusts to various lighting conditions by dilating or contracting to control the intake of light. With the concept of the adaptiveness, the proposed enhancement technique is able to process gray-scale and color digital images by adjusting the brightness in images.
FIG. 3 shows a flow chart explaining the steps of the present invention for enhancing images of non-uniform brightness. Method 20 proposed in the present invention comprises of step 201, step 203 and step 205, which are described as follows. Step 201 uses a surround function to analyze global image brightness and obtain an intermediate image, by calculating relationship of individual pixel brightness to that of surrounding pixels. This brightness relationship is an important piece of information obtained in step 201. Step 203 uses a given gain function to process both image 10 and the intermediate image in order to determine the RGB values that should be added to each pixel in image 10. In step 205, the determined enhancement RGB values are added to the original RGB values of their corresponding pixels in image 10 and a digital image with enhanced brightness is produced. As described above, the present invention is able to enhance image 10 via steps 201, step 203 and step 205.
The present method can be implemented on gray-scale and color digital images. Here we begin the description of the present invention with gray-scale digital images. The image processing in step 201 is defined as
where “*” represents convolution, I(x,y) represents gray-scale values in image 10, and F(x,y) is the surround function used to calculate the brightness relationship. A one-dimensional or two-dimensional low-pass filter or a vector function is chosen as F(x,y) to calculate brightness relationship of individual pixel brightness to that of surrounding pixels, followed by step 203 involving the gain function. Backlighting and reflection, two problems that the present invention aims to circumvent, are described as follows.
In a backlighting situation, the gain function in step 203 is defined as
where D(x,y) is the intermediate image obtained in step 201 and processed by the surround function, and CGain is a gain coefficient. The main role of CGain is to control the enhancement by using Equation (2) and adaptively adjusting brightness. Since D(x,y) represents the brightness of pixel I(x,y) relative to surrounding pixels, the brightness is increased if I(x,y) is darker than the surroundings; the brightness is decreased if I(x,y) is brighter than the surroundings. Therefore, the brightness of each pixel can be adjusted adaptively.
In a reflection situation, the gain function in step 203 is defined as
where Imax is the maximum value in image 10. The adaptive adjustment of brightness in a reflection situation can be achieved by using Equation (3).
In step 205, a final and enhanced image is calculated by summing the enhancement amount and the original value for each pixel in image 10. Substituting Equation (2) into Equation (1) obtains Equations (4), which represents the image processing algorithm in method 20 in a backlighting situation; and Equations (5) is obtained by substituting Equation (3) into Equation (1) and represents the image processing algorithm in method 20 in a reflection situation.
(A). In a backlighting situation:
(B). In a reflection situation:
where I is the enhanced pixel value.
Similarly,
where CAnti-noise is an anti-noise coefficient.
Image processing results by using Equation (6) are demonstrated in FIGS. 6A˜6D.
For situations under reflection, method 20 uses an image processing algorithm defined as
where C is an anti-noise coefficient. Results by using Equation (7) are demonstrated in
The procedure described above is designed for processing gray-scale images, and the following procedure is developed for processing color images of non- uniform brightness. For color images, brightness must be adaptively enhanced while color consistency must be retained. If a processing technique is directly applied on R (Red), G (Green) and B (Blue) color space, which are commonly used to present color images, a color shifting problem may occur. Therefore, HSI color space is used instead to process color images. HSI color space and the transformation of RGB color space to HIS color space are defined as:
where I is intensity, S is saturation and H is hue.
In HSI color space, the color-related information is controlled by hue H and saturation S. If color consistency is needed, according to Equation (9), hue H and saturation S must remain constant. Namely, keeping r, g and b constant can maintain color consistency and chaanging intensity I can adjust the brightness.
For color image 10 in a backlighting situation, RGB of image 10 are transformed to intensity I, followed by the processing algorithm defined as:
where R(x,y), G(x,y), B(x,y) are the color information, I(x,y) is the intensity, and {overscore (I)}(x,y) is the enhanced intensity of image 10.
Next, a final and enhanced image is presented as
where {overscore (R)}(x,y), {overscore (G)}(x,y), {overscore (B)}(x,y) are the RGB values after brightness enhancement. By substituting Equation (11) into Equation (10), the image processing algorithm of method 20 can be simplified as:
Image processing results by using Equation (12) are demonstrated in
Similarly, the proposed image processing algorithm for reflection problems can be simplified as
Image processing results by using Equation (13) are demonstrated in FIGS. 9A˜9C.
Moreover, method 20 in the present method can be realized in an electronic device, which can further be integrated into a digital camera, a monitor or any other image output device.
It should be noted that the functionality of the present invention is exemplified, but not limited to, the instance provided in this patent document. Definitions for certain words and phrases are provided throughout this patent document, and those of ordinary skill in the art should understand that this is by way of illustration and not of limitation, and the scope of the appended claims should be contained as broadly as the prior art will permit.
Number | Date | Country | Kind |
---|---|---|---|
93123985 | Aug 2004 | TW | national |