1. Field of the Invention
The present invention relates to a method for enhancing a perceptibility of an image under a dim backlight condition, and more particularly, to a method for enhancing the perceptibility of the image by boosting a background luminance layer of the image.
2. Description of the Prior Art
Multimedia devices, particularly portable devices, are designed to be used anywhere and anytime. To prolong the battery life of the portable devices, various techniques are utilized for saving the LCD (Liquid Crystal Displayer) power of the portable devices since the backlight of the LCD dominates the power consumption of the portable devices. However, as known by those skilled in this art, the image viewing quality is strongly related to the intensity of LCD backlight. The dimmer the backlight, the worse the image quality is. Therefore, maintaining image quality under various lighting conditions is critical.
Relevant techniques can be found in the image enhancement and tone mapping fields. The conventional methods are mainly designed to maintain a human vision system (HVS) response estimated by a specific HVS model exploited in the method. There are many choices of such models, ranging from the mean square difference to complex appearance models. Among these models, classical contrast and perceptual contrast are the most exploited ones due to the fact that contrast is the most important factor that affects overall image quality. Classical contrast is defined base on the signal processing knowledge, such as Michelson contrast, Weber fraction, logarithmic ration, and the signal to noise ratio. On the other hand, perceptual contrast, which is different from classical ones, exploits the psychological properties of HVS to estimate the HVS response. Most perceptual contrasts are designed based on a transducer function derived from just noticeable difference (JND) theory. The transducer function transfers the image signal from the original spatial domain to a domain which can better represents the response of the HVS. The perceptual contrasts are then defined in the domain with the definition mimic to the classical ones. To take both the local and global contrast into consideration, the conventional techniques are often applied in a multi-scale sense, where larger scales are corresponding to contrast of a border region. Furthermore, different kinds of sub-band architectures are developed to help the decomposition of the multi-scale techniques.
Though the conventional methods have good results for common viewing scenario (i.e., 50% or more LCD backlight), they do not work well for dim backlight scenario as low as 10% LCD backlight. The main reason is that the HVS has different characteristic between these scenarios and the HVS response estimators used in the conventional methods are no longer accurate for the dim backlight scenario.
Therefore, preserving the perceptibility of the original perceptible regions becomes an important issue for image enhancement under dim backlight.
Therefore, one of the objectives of the present invention is to provide a method for enhancing a perceptibility of an image by boosting a background luminance layer of the image.
According to an embodiment of the present invention, a method for enhancing a perceptibility of an image is disclosed. The method comprises the step of: processing the image in accordance with a first luminance characteristic and a second luminance characteristic of the image, wherein a plurality of pixels with the first luminance characteristic are brighter than a plurality of pixels with the second luminance characteristic; compressing the plurality of pixels with the first luminance characteristic; and adjusting the plurality of pixels with the second luminance characteristic.
These and other objectives of the present invention will no doubt become obvious to those of ordinary skill in the art after reading the following detailed description of the preferred embodiment that is illustrated in the various figures and drawings.
Certain terms are used throughout the description and following claims to refer to particular components. As one skilled in the art will appreciate, electronic equipment manufacturers may refer to a component by different names. This document does not intend to distinguish between components that differ in name but not function. In the following description and in the claims, the terms “include” and “comprise” are used in an open-ended fashion, and thus should be interpreted to mean “include, but not limited to . . . ”. Also, the term “couple” is intended to mean either an indirect or direct electrical connection. Accordingly, if one device is coupled to another device, that connection may be through a direct electrical connection, or through an indirect electrical connection via other devices and connections.
The main reason that the above-mentioned conventional techniques do not perform well is that the HVS has different characteristics under dim backlight scenario and original scenario the conventional techniques designed for. According to the present invention, there are two main features that are caused by the HVS characteristic for image enhancement under dim backlight. First, there is higher percentage of imperceptible luminance range for the image displayed under dim backlight than the original backlight. This indicated that most regions in the displayed image are laid in the imperceptible luminance range. Second, the degradation of color becomes a more significant artifact in the dim backlight scenario. Usually, the hue of a color tends to be darker when displayed with a dimmer backlight display and the dimmer the luminance of a pixel, the higher the degradation of color it has. Therefore, degradations of color are mainly occurred in the dark regions of the image and need to be compensated.
To combat the missing detail problem, an s-shape HVS response curve is exploited in the present invention to demonstrate how it happened. The main idea is that the sensitivity of HVS tends to be zero in the dark region and hence the luminance variation in the dark region cannot be perceived by HVS. In other words, the proposed luminance enhancement of the present invention can effectively enhance the perceptual contrast in the dim backlight scenario. Furthermore, the present invention also proposes a luminance enhancement idea base on the observation that the same perceptual contrast can be achieved with less contrast in a brighter region. General speaking, according to the present invention, the method for enhancing a perceptibility of an image comprises the following steps: a) processing the image in accordance with a first luminance characteristic and a second luminance characteristic of the image, wherein a plurality of pixels with the first luminance characteristic are brighter than a plurality of pixels with the second luminance characteristic; b) compressing the plurality of pixels with the first luminance characteristic; and c) boosting the plurality of pixels with the second luminance characteristic.
To demonstrate the dimming back light effects in the following description of the present invention, the dim backlight is assumed to be 10% backlight and the HVS response curves of an original image displayed with 100% and 10% backlight display are demonstrated in
As shown in
Please refer to
Therefore, a just noticeable decomposition (JND) method can be utilized to decompose the original image into a HVS response layer and a luminance layer. Then, the dark region of the HVS response layer can be boosted to the new dark region, and the HVS response layer preserves the image details of the original image.
Please refer to
Step 502: loading the original image 602;
Step 504: deriving an original luminance layer 604 of the original image 602, wherein the original luminance layer 604 has an original luminance range;
Step 506: performing a low-pass filtering operation upon the original luminance layer 604 to generate a first luminance layer 606, wherein the first luminance layer 606 has a first luminance range;
Step 508: dimming the first luminance layer 606 to generate a dim luminance layer 608;
Step 510: defining a second luminance range which is different from the first luminance range, wherein the second luminance range has an upper luminance threshold value and a lower luminance threshold value;
Step 512: boosting a relatively dark region of the dim luminance layer 608 to brighter than the lower luminance threshold value and compressing a relatively bright region of the dim luminance layer 608 to darker than the upper luminance threshold value to thereby generate a second luminance layer 610 fitted into the second luminance range;
Step 514: generating a human vision system (HVS) response layer 612 corresponding to the original luminance layer 604, wherein the HVS response layer has an HVS response range;
Step 516: clipping the HVS response range of the HVS response layer 612 into a predetermined HVS response range to generate a clipped HVS response layer 614;
Step 518: composing the second luminance layer 610 and the clipped HVS response layer 614 to generate an enhanced luminance layer 616;
Step 520: restoring the color of the original image 602 to the enhanced luminance layer 616 to generate an enhanced image 618.
In step 502, when the original image 602 is loaded, each pixel of the original image 602 comprises color information and luminance information. Therefore, the color information should be extracted from the original image 602 to obtain the original luminance layer 604 of the original image 602, wherein the original luminance layer 604 has the original luminance range, which is represented by the distribution lines 302 as shown in
Then, to obtain the first luminance layer 606, which is the background luminance layer of the original luminance layer 604, by the low-pass filtering operation in step 506, the background and foreground regions in the original luminance layer 604 have to be clearly defined. Consider the area inside the square 702 of
S=2*L*tan(5/2π). (1)
According to the embodiment of the present invention, the area of the background area is a square of 15 by 15 pixels as shown in
When each background luminance value of the pixels of the first luminance layer 606 (i.e., the background luminance layer) are obtained in step 506, each HVS response of the pixels of the original luminance layer 604 can also be derived by
In other words, by recording the HVS response and the background luminance value for each pixel, the original luminance layer 604 can be decomposed into two layers: the first luminance layer 606 (i.e., the background luminance layer) and the HVS response layer 612 (step 514). Please note that, in another embodiment of the present invention, the HVS response of the original luminance layer 604 can obtained by searching a predetermined HVS response table for the HVS response of the pixel according to the original luminance value and the first luminance value.
In step 508, since the embodiment of the present invention is utilized to enhance the perceptibility of the original image 602 under the 10% backlight condition, the first luminance layer 606 is dimmed to the 10% backlight condition to generate the dim luminance layer 608, which has the luminance range represented by the distribution line 304 as shown in
Then, a scaling operation is applied to boost the relatively dark region of the dim luminance layer 608 to brighter than the lower luminance threshold value and compressing the relatively bright region of the dim luminance layer 608 to darker than the upper luminance threshold value to thereby generate the second luminance layer 610 fitted into the second luminance range, wherein the second luminance layer 610 is the background luminance layer of the enhanced image 618 and the scaling operation is represented by the following equation (2):
where B and B′ are the luminance value of each pixel of the dim luminance layer 608 and the second luminance layer 610 respectively. BTH is the luminance threshold value chosen to preserve the maximum HVS response for a given upper bound of display luminance under the 10% backlight condition. The factor Scale in equation (2) is the dimming scale of the luminance. According to the equation (2), the second luminance layer 610, which is the background luminance layer of the enhanced image 618, can be obtained.
On the other hand, in step 516, a clipping is applied to the HVS response of each pixel on the HVS response layer 612 to compress the HVS response layer 612 by the following equation (3) and to generate the clipped HVS response layer 614:
where HVS′ is the HVS response of each pixel of the clipped HVS response layer 614, HVSmean is the mean of all pixels of the HVS response layer 612. Furthermore, HVSTH is a HVS response threshold and is chosen to preserve 80% of HVS response for the original image 602. According to the equation (3), the clipped HVS response layer 614, which is the HVS response layer of the enhanced image 618, can be obtained.
It should note that, the JND decomposition is reversible, thus the second luminance layer 610 and the clipped HVS response layer 614 is composed to generate the enhanced luminance layer 616 according to the relationships between the HVS response, the background luminance value and the foreground luminance value as shown in
Then, in step 520, the enhanced image 618 is restored according to the equation (4):
M′=M*(Lenh/Lori)1/Y, (4)
where Lori is the luminance value of the original image 602, Lenh is the luminance value of the enhanced image 618, M is the original pixel value of a color of the original image 602, and M′ is the enhanced pixel value of a color of the enhanced image 618.
It can be shown that the enhanced image with 100% backlight 620 has a better image quality under the same lighting condition as the original image 602. Therefore, the present invention preserves the perceptual quality of images displayed under extremely dim light since the present method preserves the detailed information of dark regions to be in an appropriate luminance range. Furthermore, experimental results show that the present method preserves the detail while reducing the shading effect. It should also be noted that the masking effect due to relatively strong ambient light helps the present method combat the halo effect that affects most two-layer decomposition methods.
Those skilled in the art will readily observe that numerous modifications and alterations of the device and method may be made while retaining the teachings of the invention. Accordingly, the above disclosure should be construed as limited only by the metes and bounds of the appended claims.
This application claims the benefit of U.S. provisional application No. 61/035,728, which was filed on Nov. 3, 2008 and is included herein by reference.
Number | Date | Country | |
---|---|---|---|
61035728 | Mar 2008 | US |