Patent Application
20080049274
a is diagram of a white area for a specific image sensor plotted in a two-dimension space.
b is a graphical representation of a white curve, within the white area identified in
a and 5b are graphical representations of an example of a white curve for a specific image sensor and a pixel considered for automatic white balance statistics in accordance with an embodiment of the invention.
In the following detailed description, reference is made to various specific embodiments in which the invention may be practiced. These embodiments are described with sufficient detail to enable those skilled in the art to practice them. It is to be understood that other embodiments may be employed, and that structural and logical changes may be made.
The term “pixel” refers to a picture element in an image. Digital data defining an image may, for example, include one or more color values for each pixel. For a color image, each pixel's values may include a value for each of a plurality of colors, such as red, green, and blue.
The image sensor 404 is a sensor that may be any type of solid state images including CMOS, CCD and others. The image sensor 404 receives image information in the form of photons, and converts that information to pixel analog electrical signals, which are subsequently provided to down stream processing circuits. In the imaging apparatus 400, the image sensor 404 provides electrical signals to the image processing circuit 408.
The image processing circuit 408 performs image processing on the digital signals received from analog-to-digital (A/D) converter 406. The image processing circuit can be implemented using logic circuits in hardware, or using a programmed processor, or by a combination of both. The image processing circuit 408 may include other circuits that perform pixel defect correction, demosaicing, image resizing, aperture correction, and correction for other effects or defects.
In an embodiment, the image processing circuit 408 outputs are pixels having RGB channels representational of the image data from red, blue and green pixels of image sensor 404. The image processing circuit 408 also converts the RGB image signals into a second set of data structures representing image pixel data in YUV format that are also representational of the RGB data. YUV stores image data in Y—luminance (“luma”), U—blue chrominance (“blue chroma” or “Cb”) and V—red chrominance (“red chroma” or “Cr”).
The auto-white balance circuit 414 receives image data, in the form of the YUV data structures and computes the correction values, if required, to perform white balancing. The auto-white balance circuit 414 provides the results of its computation to the image processing circuit 408. The white balance computation in the auto-white balance circuit 414 is performed using the YUV data structure because of the inherent properties of the YUV data structure. The YUV data structure breaks down an image into the Y, the luminance values, and UV, which is essentially a two-dimensional representation of color, where the two color components are U and V (i.e., Cb and Cr). Any color can be expressed in terms of the two color components, which can be plotted in an x and y coordinate space.
The image processing circuit 408 receives information from the auto-white balance circuit 414 and provides gain information to the component of the YUV data structure and thus makes appropriate adjustments to an image.
At step 104, a first pixel considered for automatic white balance statistics is selected from the captured image. Typically, the pixels considered for automatic white balance statistics are selected from the captured image in same order, row by row and pixel by pixel within a row. Next, at step 112, it is determined if the pixel is in the white area (described below in more detail). The acceptance of the pixel for automatic white balance statistics, depicted as a “Yes” response in Step 112, depends on the selecting criterion (described below in more detail) used at step 112.
At step 120, the data for a selected pixel that meets the selecting criterion is saved, and the method 100 continues at step 124. On the other hand, if the selected pixel does not meet the selecting criterion at step 112, then the method 100 continues at step 124 without saving the pixel data. At step 124, a determination is made to decide if the selected pixel is the last pixel to be tested. If it is determined that the pixel is not the last pixel, the method 100 continues at step 128. Otherwise, the method 100 continues at step 136.
Steps 112-128 are repeated until it is determined at step 124 that the last pixel from the captured image has been selected. Once it is determined at step 124 that a selected pixel is the last pixel considered for automatic white balance statistics; at step 136, saved pixel data is used to obtain the automatic white balance statistics. At step 140, the automatic white balance statistics are then used by the auto-white balance circuit 408 to perform a white balancing operation.
If it is determined at step 124 that the selected pixel is not the last pixel to be considered for automatic white balance statistics, then at step 128, the next pixel is selected, and the method 100 continues at step 112.
Determining if a pixel meets the selecting criterion used at step 112 includes estimating the distance from the selected pixel to a white curve, e.g.,
Once the first pixel has been selected at step 102 of
a illustrates the white curve L50 having a set of nodes P1, P2, P3, P4 with associated coordinates (X1,Y1), (X2,Y2), (X3,Y3), (X4,Y4). Interval L12 is between nodes P1 and P2. Interval L23 is between nodes P2 and P3. Interval L34 is between nodes P3 and P4. The selected pixel being considered for white balance statistics is defined by node P0 having associated coordinate (X0,Y0).
Referring to
Dis1=DIST(P0, P1);
Dis2=DIST(P0, P2);
Dis3=DIST(P0, P3); and
Dis4=DIST(P0, P4);
where DIST(A, B) estimates the distance between node A and B. If node A has coordinate [Xa,Ya] and node B has coordinate [Xb,Yb], the equation defining the distance between these two nodes is DIST(A, B)=√{square root over ((Xa−Xb)2+(Ya−Yb)2)}{square root over ((Xa−Xb)2+(Ya−Yb)2)}. Implementation of this equation might be difficult and costly; alternatively, the following estimation can be easily implemented. The following estimation provides an acceptable, less than 10% error, in distance detection.
Applying the estimation to the illustrated example in
(1) Performing an initial determination:
dX=|X P
0
−X P
1
|; dY=|Y P
0
−Y P
1|,
D=dX+dY;
(2) Calculating the Min and Max:
(3) Setting the distance based on these relationships:
The DIST(A, B) procedure discussed above is repeated to calculate the distance between selected pixel node P0, and each white curve node P1, P2, P3, P4. Accordingly, DIST(A, B) procedure continues, where Dis2=DIST(P0, P2); Dis3=DIST(P0, P3); and Dis4=DIST(P0, P4).
At step 208, the two nodes on the white curve closest to the selected pixel node P0 defines an interval on the white curve, which is selected for further evaluation. In the example shown in
b shows a blow-up of the two nodes P1 and P2 with associated coordinates (X1,Y1) and (X2,Y2). At step 210, the interval L12 between the two nodes P1 and P2 is divided into four equal intervals.
X
5
=X
1+deltaX/4;
X
6
=X
1+deltaX/2;
X
7
=X
1+deltaX/2+deltaX/4;
Y
5
=Y
2+deltaY/2+deltaY/4;
Y
6
=Y
2+deltaY/2; and
Y
7
=Y
2+deltaY/4,
where deltaX=X2−X1 and deltaY=Y1−Y2.
Next at step 212, the distance between the selected pixel node P0 and each sub-node P1, P5, P6, P7, and P2 is calculated.
d
1
=DIST(P0, P1);
d
5
,=DIST(P0, P5);
d
6
,=DIST(P0, P6);
d
7
=DIST(P0, P7); and
d
2
=DIST(P0, P2),
where the procedure DIST(A,B) calculates the distances d1, d5, d6, d7 and d2, applying the same estimation, as discussed above.
It should be appreciated to one skilled in the art, that steps 208 through 212 can be repeatedly performed to further narrow the intervals on the white curve and define the sub-node closest to the pixel; thereby determining the shortest distance possible from the pixel to the white curve.
At step 214, the shortest distance among the distances d1, d5, d6, d7 and d2 is selected for the next evaluation. At step 216, the threshold distance TH, shown in
The selecting process described above is independent of the type of coordinates used to define the white curve. Furthermore, the selecting process does not require a large amount of resources for implementation due to the estimations used.
System 700, for example, a camera system includes a lens 402 for focusing an image on a pixel array of the image sensor 404, central processing unit (CPU) 705, such as a microprocessor, which controls camera operation, and which communicates with one or more input/output (I/O) devices 710 over a bus 715. Imaging apparatus 400 also communicates with the CPU 705 over bus 715. The processor system 700 also includes random access memory (RAM) 720, and can include removable memory 725, such as flash memory, which also communicate with CPU 705 over the bus 715. Imaging apparatus 400 may be combined with the CPU, with or without memory storage on a single integrated circuit or on a different chip than the CPU.
The above description and drawings illustrate embodiments of the invention. Although certain advantages and embodiments have been described above, those skilled in the art will recognize that substitutions, additions, deletions, modifications and/or other changes may be made. Accordingly, the embodiments are not limited by the foregoing description but are only limited by the appended claims.
