Patent Application
20090091660
This invention relates to image processing, and more particularly, to a method and apparatus of processing color components of a pixel (e.g., RGB channels of a pixel) through adding white light adaptively.
Image capturing is commonly used in many fields, such as digital cameras. In generally, each color channel, R, G or B, of a captured pixel has 10 bits originally; however, the data range of the following signal processing stage permits 8 bits only. Therefore, clipping the output data is implemented. Please refer to
It is therefore one of the objectives of the present invention to provide a method and apparatus of processing color components of a pixel (e.g., RGB channels of a pixel) by adding white light adaptively.
According to one embodiment of the present invention, a method of processing a plurality of color components of a pixel is disclosed. The method comprises: finding a maximum value of the color components; determining a scaling factor and an adjusting value according to the maximum value; and adjusting each of the color components according to a multiplication utilizing the scaling factor and an addition utilizing the adjusting value.
According to one embodiment of the present invention, an apparatus of processing a plurality of color components of a pixel is provided. The apparatus comprises: a maximum value determining module, for finding a maximum value of the color components; a first adjustment control module, coupled to the maximum value determining module, for determining a scaling factor and an adjusting value according to the maximum value; and an adjusting module, coupled to the first adjustment control module, for adjusting each of the color components according to a multiplication utilizing the scaling factor and an addition utilizing the adjusting value.
These and other objectives of the present invention will no doubt become obvious to those of ordinary skill in the art after reading the following detailed description of the preferred embodiment that is illustrated in the various figures and drawings.
Certain terms are used throughout the description and following claims to refer to particular components. As one skilled in the art will appreciate, manufacturers may refer to a component by different names. This document does not intend to distinguish between components that differ in name but not function. In the following description and in the claims, the terms “include” and “comprise” are used in an open-ended fashion, and thus should be interpreted to mean “include, but not limited to . . . ”. Also, the term “couple” is intended to mean either an indirect or direct electrical connection. Accordingly, if one device is coupled to another device, that connection may be through a direct electrical connection, or through an indirect electrical connection via other devices and connections.
Methods for processing color components of pixel through adding white light adaptively, or certain aspects or portions thereof, may take the form of program codes (i.e., instructions) embodied in tangible media, such as floppy diskettes, CD-ROMs, hard drives, or any other machine-readable storage medium, wherein, when the program codes are loaded into and executed by a machine, such as a mobile phone, a smart phone or the like, the machine becomes an apparatus for practicing the invention. The disclosed methods may also be embodied in the form of program codes transmitted over some transmission medium, such as electrical wiring or cabling, through fiber optics, or via any other form of transmission, wherein, when the program codes are received and loaded into and executed by a machine, such as a mobile phone, a smart phone or the like, the machine becomes an apparatus for practicing the invention. When implemented on a general-purpose processor, the program codes combine with the processor to provide a unique apparatus that operate analogously to specific logic circuits.
The adaptive neutral delta module 204 is used for adding a neutral delta value (i.e., white light) to each of the color components (i.e., RGB channels) according to an adaptive scheme. As shown in
Please refer to
I
in=MAX(R,G,B) (1)
The maximum value determining unit 212 then outputs the found maximum value Iin to the comparator 302 and the first adjustment control unit 214, respectively.
The third adjustment control unit 218 is responsible for determining a threshold value TP and a mapping function LUT associated with the threshold value TP. For example, the mapping function LUT describes relationship between the input data SIN′ and the output data SOUT′, in which a proportion of the output data SOUT′ to the input data SIN′ is lower than one when the input data SIN′ is greater than the threshold value TP, with a higher input data SIN′ corresponding to a lower or equal proportion. In a preferred embodiment, an adaptive scheme is applied to select the threshold value TP from a plurality of candidate threshold values and the mapping function LUT is selected from a plurality of candidate mapping functions. Please refer to
Exemplary results of determining the threshold value TP and the mapping function LUT in steps 412-416 are shown in
As shown in
M=TP/I
in (2)
D=LUT(Iin) (3)
The scaling factor M is generated by directly dividing the threshold value TP given by the third adjustment control unit 218 into the maximum value Iin found by the maximum value determining unit 212. The adjusting value D is determined according to the mapping function LUT selected by the third adjustment control unit 218. Examples of the adjusting value D corresponding to the maximum value Iin are depicted in
Next, the adjusting unit 220 uses the multipliers 312-316 and the adders 322-326 to adjust the gray values R, G, B of the input data SIN′ according to the scaling factor M and the adjusting value D. The computation of the adjusted gray values R′, G′, B′, as clearly indicated by following equations.
R′=R*M+D (4)
G′=G*M+D (5)
B′=B*M+D (6)
As stated above, the second adjustment control unit 216 controls if the output data SOUT′ is the original input data SIN′ directly or an adjusted version of the input data SIN′ produced by the adaptive neutral delta module 204. In this exemplary embodiment, the comparator 302 in the second adjustment control unit 216 compares the maximum value Iin outputted from the maximum value determining unit 212 and the threshold value TP outputted from the third adjustment control unit 218 to generate a selection signal SEL to the multiplexer 304. That is, the selection signal SEL controls if the original input data SIN′ is bypassed and serves as the output data SOUT′ directly. When the maximum value Iin reaches (exceeds or equals) the threshold value TP, meaning that the criterion of adding the neutral delta value (i.e., white light) to the input data SIN′ is met, the selection signal SEL then instructs the multiplexer 304 to output the adjusted gray values R′, G′, B′ as the gray values R″, G″, B″ of the output data SOUT′; otherwise, the selection signal SEL instructs the multiplexer 304 to output the original gray values R, G, B as the gray values R″, G″, B″ of the output data SOUT′ as the above criterion is not met.
Next, the color clipping module 206 processes the received output data SOUT′ and generate the resultant output data SOUT by clipping any gray values exceeding the upper limit of the data range. It should be noted that the same adjusting value (i.e., the neutral delta) is added to each color component by the adjusting unit 220. In this way, the aforementioned discontinuous hue caused by color clipping can be alleviated or eliminated, thereby achieving the greasy-highlight reduction and improving the image quality accordingly.
Additionally, if the mapping function LUT is well-defined to have a maximum output less than or equal to the upper limit of the permissible data range, the color clipping module 206 can be omitted. In short, the color clipping operation can be integrated into the mapping function LUT. In this way, when the adjusting value D is added to the color components of the input data SIN′, the adjusted color components are well-controlled to fall in the permissible range. As a result, the color clipping is needed no more.
Generally speaking, three color components of each pixel must exist simultaneously to precisely present an image. However, to simultaneously capture all three color components, the image system requires three individual image detectors for each pixel. This would be prohibitive due to the high cost and would cause the packaging to be very complex. To keep the size and cost of the image system to a minimum, an image sensor array must keep its size small as well. Therefore, the number of color samples must be kept low. An alternative approach is to make one pixel have a detector for gathering information for a single color component. Therefore, the image system typically uses a mosaic filter generally called a color filter array (CFA), and acquires a captured scene image by sampling one of the three different color components to obtain an array that stores only one color component per pixel. The image system gets a raw sensory data having less color samples per pixel because it ignores the other two color components for each pixel. Before the captured scene image is further processed or displayed, the missing color components of each pixel must be reconstructed so that each pixel contains all three color components as required. The process of reconstructing missing color components is commonly called demosaicking. When the disclosed color clipping control, realized by the combination of the adaptive neutral delta module 204 and the color clipping module 206, is applied to the image system mentioned above, it can be implemented to process pixel data in the raw data domain and/or RGB domain, depending upon design requirements. The pixels in the RGB domain have been processed by the above-mentioned demosaicking, and each pixel therefore contains three color components. Regarding the pixels in the raw data domain, they are not processed by the above-mentioned demosaicking yet. Therefore, each pixel in the raw data domain contains one color component only. In view of above description, applying the disclosed color clipping control to process the pixels in the RGB domain is self-explanatory. Further description is omitted here. Regarding the implementation of the disclosed color clipping control in the raw data domain, a slight modification is required. Taking a pixel in the raw data domain that has a gray value of the R channel for example, the adaptive neutral delta module 204 obtains missing gray values of the G channel and B channel through a simplified demosaicking scheme. Next, the aforementioned flow of processing the input data activated. Further description is omitted here for brevity. It should be noted that the output data SOUT of the color clipping module 206 includes one color component per pixel as the normal demosaicking operation is not executed yet. Preferably, the disclosed color clipping control is activated after the white balance correction has been applied to thereby guarantee that the adjustments made to color component(s) of each pixel will produce an adjusted image with correct hue as desired.
In the above exemplary embodiments, the adaptive neutral delta module 204 is dedicated to solving the problem caused by color clipping. However, this is not meant to be a limitation of the present invention. That is, any image processing applications using the adaptive neutral delta module and related method thereof to adjust the pixel data all fall in the scope of the present invention.
Briefly summarized, the disclosed adaptive neutral delta module adds “white light” to all of the color components of a pixel when there is one color component exceeding the permissible data range. As all of the color components are adjusted by the same amount of “white light,” the hue distortion is avoided or alleviated after the color clipping is applied. Additionally, an adaptive scheme is utilized for changing the setting of the threshold value used for judging if the saturation condition is met and the setting of the mapping function used for determining the amount of added “white light” according to image histogram information. In this way, the disclosed method of adding “white light” can meet image processing requirements of different scenes, for example, having different background luminance.
Furthermore, a multiplication using the same scaling factor is applied to each color channel, and an addition using the same adjusting value (i.e., the neutral delta value) is applied to each color channel, where the scaling factor is easily obtained through dividing the threshold value by the maximum value among the color components, and the adjusting value is easily obtained through referring to the mapping function. For pixels in the same image, the same mapping function and the same threshold value can be used repeatedly as the histogram of the image is not changed when each pixel in the image is processed. Therefore, implementation of the disclosed scheme of adding “white light” does not require many hardware resources, and can be easily integrated into any image processing systems.
Those skilled in the art will readily observe that numerous modifications and alterations of the device and method may be made while retaining the teachings of the invention.
