Patent Grant
8203582
The present invention relates to subpixel rendering and gamut mapping in display devices.
Novel sub-pixel arrangements are disclosed for improving the cost/performance curves for image display devices in the following commonly owned United States Patents and Patent Applications including: (1) U.S. Pat. No. 6,903,754 (“the '754 Patent”) entitled “ARRANGEMENT OF COLOR PIXELS FOR FULL COLOR IMAGING DEVICES WITH SIMPLIFIED ADDRESSING;” (2) United States Patent Publication No. 2003/0128225 (“the '225 application”) having application Ser. No. 10/278,353 and entitled “IMPROVEMENTS TO COLOR FLAT PANEL DISPLAY SUB-PIXEL ARRANGEMENTS AND LAYOUTS FOR SUB-PIXEL RENDERING WITH INCREASED MODULATION TRANSFER FUNCTION RESPONSE,” filed Oct. 22, 2002; (3) United States Patent Publication No. 2003/0128179 (“the '179 application”) having application Ser. No. 10/278,352 and entitled “IMPROVEMENTS TO COLOR FLAT PANEL DISPLAY SUB-PIXEL ARRANGEMENTS AND LAYOUTS FOR SUB-PIXEL RENDERING WITH SPLIT BLUE SUB-PIXELS,” filed Oct. 22, 2002; (4) United States Patent Publication No. 2004/0051724 (“the '724 application”) having application Ser. No. 10/243,094 and entitled “IMPROVED FOUR COLOR ARRANGEMENTS AND EMITTERS FOR SUB-PIXEL RENDERING,” filed Sep. 13, 2002; (5) United States Patent Publication No. 2003/0117423 (“the '423 application”) having application Ser. No. 10/278,328 and entitled “IMPROVEMENTS TO COLOR FLAT PANEL DISPLAY SUB-PIXEL ARRANGEMENTS AND LAYOUTS WITH REDUCED BLUE LUMINANCE WELL VISIBILITY,” filed Oct. 22, 2002; (6) United States Patent Publication No. 2003/0090581 (“the '581 application”) having application Ser. No. 10/278,393 and entitled “COLOR DISPLAY HAVING HORIZONTAL SUB-PIXEL ARRANGEMENTS AND LAYOUTS,” filed Oct. 22, 2002; and (7) United States Patent Publication No. 2004/0080479 (“the '479 application”) having application Ser. No. 10/347,001 and entitled “IMPROVED SUB-PIXEL ARRANGEMENTS FOR STRIPED DISPLAYS AND METHODS AND SYSTEMS FOR SUB-PIXEL RENDERING SAME,” filed Jan. 16, 2003. Each of the aforementioned '225, '179, '724, '423, '581, and '479 published applications and U.S. Pat. No. 6,903,754 are hereby incorporated by reference herein in its entirety.
For certain subpixel repeating groups having an even number of subpixels in a horizontal direction, systems and techniques to affect improvements, e.g. polarity inversion schemes and other improvements, are disclosed in the following commonly owned United States patent documents: (1) United States Patent Publication No. 2004/0246280 (“the '280 application”) having application Ser. No. 10/456,839 and entitled “IMAGE DEGRADATION CORRECTION IN NOVEL LIQUID CRYSTAL DISPLAYS”; (2) United States Patent Publication No. 2004/0246213 (“the '213 application”) (U.S. patent application Ser. No. 10/455,925) entitled “DISPLAY PANEL HAVING CROSSOVER CONNECTIONS EFFECTING DOT INVERSION”; (3) United States Patent Publication No. 2004/0246381 (“the '381 application”) having application Ser. No. 10/455,931 and entitled “SYSTEM AND METHOD OF PERFORMING DOT INVERSION WITH STANDARD DRIVERS AND BACKPLANE ON NOVEL DISPLAY PANEL LAYOUTS”; (4) United States Patent Publication No. 2004/0246278 (“the '278 application”) having application Ser. No. 10/455,927 and entitled “SYSTEM AND METHOD FOR COMPENSATING FOR VISUAL EFFECTS UPON PANELS HAVING FIXED PATTERN NOISE WITH REDUCED QUANTIZATION ERROR”; (5) United States Patent Publication No. 2004/0246279 (“the '279 application”) having application Ser. No. 10/456,806 entitled “DOT INVERSION ON NOVEL DISPLAY PANEL LAYOUTS WITH EXTRA DRIVERS”; (6) United States Patent Publication No. 2004/0246404 (“the '404 application”) having application Ser. No. 10/456,838 and entitled “LIQUID CRYSTAL DISPLAY BACKPLANE LAYOUTS AND ADDRESSING FOR NON-STANDARD SUBPIXEL ARRANGEMENTS”; (7) United States Patent Publication No. 2005/0083277 (“the '277 application”) having application Ser. No. 10/696,236 entitled “IMAGE DEGRADATION CORRECTION IN NOVEL LIQUID CRYSTAL DISPLAYS WITH SPLIT BLUE SUBPIXELS”, filed Oct. 28, 2003; and (8) United States Patent Publication No. 2005/0212741 (“the '741 application”) having application Ser. No. 10/807,604 and entitled “IMPROVED TRANSISTOR BACKPLANES FOR LIQUID CRYSTAL DISPLAYS COMPRISING DIFFERENT SIZED SUBPIXELS”, filed Mar. 23, 2004. Each of the aforementioned '280, '213, '381, '278, '404, '277 and '741 published applications are hereby incorporated by reference herein in its entirety.
These improvements are particularly pronounced when coupled with sub-pixel rendering (SPR) systems and methods further disclosed in the above-referenced U.S. Patent documents and in commonly owned United States Patents and Patent Applications: (1) United States Patent Publication No. 2003/0034992 (“the '992 application”) having application Ser. No. 10/051,612 and entitled “CONVERSION OF A SUB-PIXEL FORMAT DATA TO ANOTHER SUB-PIXEL DATA FORMAT,” filed Jan. 16, 2002; (2) United States Patent Publication No. 2003/0103058 (“the '058 application”) having application Ser. No. 10/150,355 entitled “METHODS AND SYSTEMS FOR SUB-PIXEL RENDERING WITH GAMMA ADJUSTMENT,” filed May 17, 2002; (3) United States Patent Publication No. 2003/0085906 (“the '906 application”) having application Ser. No. 10/215,843 and entitled “METHODS AND SYSTEMS FOR SUB-PIXEL RENDERING WITH ADAPTIVE FILTERING,” filed Aug. 8, 2002; (4) United States Publication No. 2004/0196302 (“the '302 application”) having application Ser. No. 10/379,767 and entitled “SYSTEMS AND METHODS FOR TEMPORAL SUB-PIXEL RENDERING OF IMAGE DATA” filed Mar. 4, 2003; (5) United States Patent Publication No. 2004/0174380 (“the '380 application”) having application Ser. No. 10/379,765 and entitled “SYSTEMS AND METHODS FOR MOTION ADAPTIVE FILTERING,” filed Mar. 4, 2003; (6) U.S. Pat. No. 6,917,368 (“the '368 Patent”) entitled “SUB-PIXEL RENDERING SYSTEM AND METHOD FOR IMPROVED DISPLAY VIEWING ANGLES”; and (7) United States Patent Publication No. 2004/0196297 (“the '297 application”) having application Ser. No. 10/409,413 and entitled “IMAGE DATA SET WITH EMBEDDED PRE-SUBPIXEL RENDERED IMAGE” filed Apr. 7, 2003. Each of the aforementioned '992, '058, '906, '302, 380 and '297 applications and the '368 patent are hereby incorporated by reference herein in its entirety.
Improvements in gamut conversion and mapping are disclosed in commonly owned United States Patents and co-pending United States Patent Applications: (1) U.S. Pat. No. 6,980,219 (“the '219 Patent”) entitled “HUE ANGLE CALCULATION SYSTEM AND METHODS”; (2) United States Patent Publication No. 2005/0083341 (“the '341 application”) having application Ser. No. 10/691,377 and entitled “METHOD AND APPARATUS FOR CONVERTING FROM SOURCE COLOR SPACE TO TARGET COLOR SPACE”, filed Oct. 21, 2003; (3) United States Patent Publication No. 2005/0083352 (“the '352 application”) having application Ser. No. 10/691,396 and entitled “METHOD AND APPARATUS FOR CONVERTING FROM A SOURCE COLOR SPACE TO A TARGET COLOR SPACE”, filed Oct. 21, 2003; and (4) United States Patent Publication No. 2005/0083344 (“the '344 application”) having application Ser. No. 10/690,716 and entitled “GAMUT CONVERSION SYSTEM AND METHODS” filed Oct. 21, 2003. Each of the aforementioned '341, '352 and '344 applications and the '219 patent is hereby incorporated by reference herein in its entirety.
Additional advantages have been described in (1) United States Patent Publication No. 2005/0099540 (“the '540 application”) having application Ser. No. 10/696,235 and entitled “DISPLAY SYSTEM HAVING IMPROVED MULTIPLE MODES FOR DISPLAYING IMAGE DATA FROM MULTIPLE INPUT SOURCE FORMATS”, filed Oct. 28, 2003; and in (2) United States Patent Publication No. 2005/0088385 (“the '385 application”) having application Ser. No. 10/696,026 and entitled “SYSTEM AND METHOD FOR PERFORMING IMAGE RECONSTRUCTION AND SUBPIXEL RENDERING TO EFFECT SCALING FOR MULTI-MODE DISPLAY” filed Oct. 28, 2003, each of which is hereby incorporated herein by reference in its entirety.
Additionally, each of these co-owned and co-pending applications is herein incorporated by reference in its entirety: (1) United States Patent Publication No. 2005/0225548 (“the '548 application”) having application Ser. No. 10/821,387 and entitled “SYSTEM AND METHOD FOR IMPROVING SUB-PIXEL RENDERING OF IMAGE DATA IN NON-STRIPED DISPLAY SYSTEMS”; (2) United States Patent Publication No. 2005/0225561 (“the '561 application”) having application Ser. No. 10/821,386 and entitled “SYSTEMS AND METHODS FOR SELECTING A WHITE POINT FOR IMAGE DISPLAYS”; (3) United States Patent Publication No. 2005/0225574 (“the '574 application”) and United States Patent Publication No. 2005/0225575 (“the '575 application”) having application Ser. Nos. 10/821,353 and 10/961,506 respectively, and both entitled “NOVEL SUBPIXEL LAYOUTS AND ARRANGEMENTS FOR HIGH BRIGHTNESS DISPLAYS”; (4) United States Patent Publication No. 2005/0225562 (“the '562 application”) having application Ser. No. 10/821,306 and entitled “SYSTEMS AND METHODS FOR IMPROVED GAMUT MAPPING FROM ONE IMAGE DATA SET TO ANOTHER”; (5) United States Patent Publication No. 2005/0225563 (“the '563 application”) having application Ser. No. 10/821,388 and entitled “IMPROVED SUBPIXEL RENDERING FILTERS FOR HIGH BRIGHTNESS SUBPIXEL LAYOUTS”; and (6) United States Patent Publication No. 2005/0276502 (“the '502 application”) having application Ser. No. 10/866,447 and entitled “INCREASING GAMMA ACCURACY IN QUANTIZED DISPLAY SYSTEMS.”
Additional improvements to, and embodiments of, display systems and methods of operation thereof are described in: (1) Patent Cooperation Treaty (PCT) Application No. PCT/US 06/12768, entitled “EFFICIENT MEMORY STRUCTURE FOR DISPLAY SYSTEM WITH NOVEL SUBPIXEL STRUCTURES” filed Apr. 4, 2006; (2) Patent Cooperation Treaty (PCT) Application No. PCT/US 06/12766, entitled “SYSTEMS AND METHODS FOR IMPLEMENTING LOW-COST GAMUT MAPPING ALGORITHMS” filed Apr. 4, 2006; (3) U.S. patent application Ser. No. 11/278,675, entitled “SYSTEMS AND METHODS FOR IMPLEMENTING IMPROVED GAMUT MAPPING ALGORITHMS” filed Apr. 4, 2006, and published as United States Patent Application Publication 2006/0244686; (4) Patent Cooperation Treaty (PCT) Application No. PCT/US 06/12521, entitled “PRE-SUBPIXEL RENDERED IMAGE PROCESSING IN DISPLAY SYSTEMS” filed Apr. 4, 2006; and (5) Patent Cooperation Treaty (PCT) Application No. PCT/US 06/19657, entitled “MULTIPRIMARY COLOR SUBPIXEL RENDERING WITH METAMERIC FILTERING” filed on May 19, 2006. Each of these co-owned applications is also herein incorporated by reference in its entirety.
As explained in some of the above patent applications, an image 104 (
In some displays, each repeating set of subpixels includes a subpixel for each primary color. The subpixels are small, and are spaced closely together, to provide a desired resolution. This structure is not cost-effective however because it does not match the resolution of human vision. Humans are more perceptive to luminance differences than to chromatic differences. Therefore, some displays map an input pixel 106 into a subpixel repeating set that does not include only the subpixels of each primary color. The chromatic resolution is reduced, but the luminance resolution remains high.
One such display 110 is described in PCT application published as no. WO 2006/127555 A2 on 30 November 2006 and U.S. patent application Ser. No. 11/278,675 published as no. 2006/0244686 A1 on 2 Nov. 2006 and illustrated in
Each pixel 106 in column x and row y of the image (pixel “106x,y” below) is mapped into the subpixel pair 124 in column x and row y (“124x,y” below). In display 110, the consecutive indices x and y denote consecutive pairs, not consecutive subpixels. Each pair 124 has only two subpixels, and provides a high range and resolution in luminance but not in chrominance. Therefore, part of the input pixel's luminance may have to be shifted to adjacent pairs 124 in a “subpixel rendering” operation (SPR) described in some of the aforementioned patent applications and illustrated in
The color of each pixel 106 is expressed in a linear RGBW color coordinate system. For each RG pair 124x,y, the Rw value of the red subpixel is determined as a weighted sum of the R coordinates of all the pixels 106 which overlap with the sampling area 250 centered at the RG pair 124x,y. The weights are chosen to add up to 1, and are proportional to the areas of overlap of the respective pixels 106 with the sampling area 250. In particular, if the subpixel pair 124x,y is not at the edge of the display, then the red value Rw is:
Rw=½*Rx,y+⅛*Rx−1,y+⅛*Rx+1,y+⅛*Rx,y−1+⅛*Rx,y+1 (1)
In other words, the red subpixels 120R can be rendered by applying a 3×3 diamond filter to the R coordinates of the respective pixels 106 with the following filter kernel:
The same filter kernel can be used for the green, blue and white subpixels (except at the edges). Other filter kernels can also be used. See e.g. the aforementioned United States Patent Publication No. 2005/0225563.
The luminance shifts performed in the sub-pixel rendering may undesirably cause image degradation such as blurriness or loss of local contrast. The image can be improved by applying sharpening filters (e.g. DOG, i.e. Difference of the Gausians). See e.g. the aforementioned PCT application WO 2006/127555. Additional improvements in image quality are desirable.
Further, some of the operations described above may cause some subpixel values to be out of gamut, especially if the gamut is restricted in brightness to reduce power consumption. Forcing the subpixel values into the available gamut may distort the image, e.g. reduce local contrast, and such distortion should be minimized. It is desirable to improve gamut mapping operations, especially in low brightness environments.
This section summarizes some features of the invention. Other features may be described in the subsequent sections.
Block 454 performs subpixel rendering (e.g. as in
Another example of sharpening is Difference of the Gaussians. Other types of sharpening can also be applied.
The resulting subpixel values are provided to display 110 (possibly after gamma conversion if the subpixel luminances in display 110 are not linear functions of the subpixel values).
The display 110 of
In some embodiments of the present invention, the SPR operation is modified to shift more energy from D to one of the adjacent diagonals A and B than to the other one of A and B. The diagonal line D will appear sharper as a result.
Further, in a conventional LCD display, data are displayed in frames. A frame is a time interval needed to display a whole image 104. The data processing of
The invention includes embodiments that improve image quality at relatively low cost. More particularly, circuit 320 can be constructed to thoroughly analyze the image 104 and provide the best image quality for any type of image, and such circuits are within the scope of the invention, but such circuits can be large and/or complex and/or slow. In some embodiments, the image analysis is simplified to provide high image quality for many images at reasonable cost.
The invention is not limited to the features and advantages described above except as defined by the appended claims. For example, the invention is not limited to the displays 110 of
The embodiments described in this section illustrate but do not limit the invention.
Some embodiments of the present invention will now be described on the example of the display unit 110 of
Conversion to RGBW (step 420). For the sake of illustration, let us suppose that block 410 outputs, for each pixel 106, color coordinates r, g, b in a linear RGB color space. Each of the r, g, b coordinates is an integer allowed to vary from 0 to some maximum number MAXCOL inclusive. For example, if r, g, and b are represented in 8 bits, then MAXCOL=255. In some embodiments, the color coordinates are stored in more bits to avoid loss of precision. For example, if the pixel colors are initially represented in a non-linear color space (e.g. sRGB), with each coordinate being an 8-bit value, then conversion to the linear RGB color space (“gamma conversion”) may produce fractional values for r, g, and b. To reduce quantization errors, each of r, g, b can be represented in 11 bits, with MAXCOL=2047.
The color r=g=b=0 is absolute black, and the color r=g=b=MAXCOL is the brightest possible white. We will assume that RGBW is a linear representation in which each of R, G, B, W is an integer allowed to vary from 0 to MAXCOL inclusive. The brightest RGB white is converted to the brightest RGBW white, whose coordinates are R=G=B=W=MAXCOL. These assumptions are not limiting. MAXCOL can be different for different coordinates (r, g, b, R, G, B, W), and other variations are possible.
It is well known that under these assumptions, the conversion can be performed to satisfy the following equations:
r=M0R+M1W
g=M0G+M1W
b=M0B+M1W (3)
where M0 and M1 are constants corresponding to the luminance characteristics of pixels 120 as follows:
M0=(Yr+Yg+Yb)/(Yr+Yg+Yb+Yw)
M1=Yw/(Yr+Yg+Yb+Yw) (4)
where Yr, Yg, Yb, Yw are defined as follows. Yr is the luminance of display 110 when the backlight unit 310 is run at some reference output power (e.g. the maximum power), all the red subpixels 120R are maximally transmissive, and all the remaining subpixels are minimally transmissive. The values Yg, Yb, Yw are defined in a similar manner for the green, blue, and white subpixels.
If the W coordinate is known, then the R, G, and B coordinates can be computed from (3). The equations (3) clearly require that if r, g, or b is zero, then W must be zero. If r=g=b=MAXCOL, then W=MAXCOL. However, for many colors, W can be chosen in a number of ways (to define one or a number of metamers). In order for each of R, G, B, W to be in the range of 0 to MAXCOL, W must be in the following range:
minW≦W≦maxW
where
minW=[max(r,g,b)−M0*MAXCOL]/M1
maxW=min(r,g,b)/M1
To provide high image quality with minimum output power BL, the R, G, B and W coordinates of each pixel 106 should preferably be close to each other. In some embodiments, W is set to max(r,g,b). Other choices for W are also possible. See the aforementioned U.S. patent application 2006/0244686 (Higgins et al.). For example, W can be set to some representation of luminance. After being computed as described above, the W value can be hard-clamped to the range of minW to maxW. (As used herein, “hard-clamping” a value to a range of some numbers A to B means setting the value to the low bound A if the value is below A, and setting the value to the high bound B if the value is above B.)
Equations (3) may require the values R, G, B to exceed MAXCOL and be as high as MAXCOL/M0. For example, if b=0, then W=0; if r=g=MAXCOL, then R=G=MAXCOL/M0. For the sake of illustration, we will assume that M0=M1=½, i.e. the white subpixels are as bright as the red, green and blue subpixels. In this case, the R, G, and B values can be as high as 2*MAXCOL. The display 110 accepts only colors whose linear RGBW coordinates do not exceed MAXCOL. To display the other colors, the backlight unit's power BL can be multiplied by 1/M0 (i.e. doubled if M0=½), and the RGBW coordinates multiplied by M0 (divided by 2). However, to save power, some embodiments do not increase the backlight unit' power or they increase the backlight unit's power by a multiple less than 1/M0. The resulting loss of contrast may be severe as illustrated in
Suppose that the diagonal D is mapped into the RG pairs. Section II of
Further, let us assume that the backlight unit power is not increased, i.e. is kept at a level sufficient only for the pixel values not exceeding MAXCOL. Then the diagonals AA and BB will be out of gamut. Section III of
Meta luma sharpening operation exacerbates the contrast loss because on the diagonal D, the metamers will be selected to have a lower W value, and hence higher R and G values, thus possibly increasing the luminance on the diagonal.
In some embodiments of the present invention, at steps 444 (Scaler) and 450 (Gamut clamp) of
Loss of contrast may also occur if the diagonal D is bright saturated red mapped into BW pairs, and the surrounding pixels 106 are dark. See
If the answer is Yes, then a check is made (step 830) if the pixel 106x,y is on or near a diagonal line. If the answer is No, then (step 840) the diamond filter (1), (2) is applied. However, the meta luma sharpening is not performed because for the saturated colors W is near zero and therefore the choice of metamers is so limited that the meta luma sharpening is of little benefit. Instead, the image can be sharpened using, for example, same color sharpening on in some other way. Some embodiments perform same color sharpening using a DOG (Difference of the Gaussians) filter. An exemplary filter kernel for the DOG filter is:
This filter is applied to each subpixel 120 of the pixel pair 124x,y for the corresponding color plane. For example, if the pixel pair 124x,y is an RG pair, then the R subpixel is rendered by summing the output of diamond filter (1), (2) with the output of the DOG filter (6). Both filters operate on the red plane, i.e. on the R coordinates output by block 420. The green subpixel is similarly rendered. The processing of the BW pairs is similar.
In other embodiments, meta luma sharpening can be performed at step 840 and/or the DOG filter (6) can be applied at step 820. Other types of sharpening can also be used at these two steps.
If the answer is Yes at step 830, then box filtering is performed to shift the pixel energy to one but not both of the adjacent diagonals. An exemplary filter kernel is:
(0,½,½) (7)
Table 1 below illustrates simulation code for one embodiment of the SPR operation 454 of
In Table 1, “spr.band” is a bitwise-AND function, “spr.bor” is bitwise-OR, and spr.bxor is bitwise XOR.
In this implementation, the blue plane is shifted left or right by one pixel 106. This phase shift means that the blue subpixels in BW pairs 124x,y are treated as if they were located at the center of the adjacent RG pair 124x−1,y or 124x+1,y. For example, in the case of the left shift, the diamond filter (1), (2) calculates the blue subpixel value for the pair 124x,y as a weighted sum of the B coordinates of the pixel 106x−1,y and the four adjacent pixels. This is believed to provide more faithful hue display for some images. The direction of shift is to the left if FLIP_LEFT=0 (see line Spr5 in Table 1), and the direction is to the right if FLIP_LEFT=1. In the description below in this section, it is assumed for simplicity that the blue shift direction is to the left. The claims are not limited to the left shift unless indicated otherwise.
In this implementation, the step 830 checks for the patterns defined by the following 3×3 matrices as described in more detail below:
For each pixel 106x,y, each of these patterns D1-D15 can be checked separately on the pixel's R, G, and B coordinates, and possibly on the W coordinate. In some embodiments, if the pixel is mapped into an RG pair, then the patterns D1-D15 are checked on the R, G and B coordinates, and if the pixel is mapped into a BW pair, then the patterns are checked on the W coordinate. The check can be performed as follows. Each coordinate R, G, B, W is “thresholded” using some threshold value “BOBits”. See lines F22-F26 in Table 1. In some embodiments, MAXCOL=2047 and BOBits=is between 128 and 1920 inclusive, e.g. 256. For example, let the thresholded values for the red, green, blue and white coordinates be denoted rth, gth, bth and wth respectively. If R≧BOBits, the thresholded value “rth” is set to 1, and otherwise rth is set to 0. The thresholded values gth, bth, wth are obtained for the G, B, and W coordinates in the same manner. Then filters D1-D15 are used on the thresholded values for each coordinate. For example, for any i and j, let rthi,j denote the thresholded rth value for the pixel 106i,j. Then for the pixel 106x,y, the output of filter D7 is 1 (i.e. the D7 pattern is recognized on the red plane) if one of the following conditions (T1), (T2) is true:
rthx,y=rthx+1,y−1=rthx−1,y+1=1 and
rthx−1,y−1=rthx−1,y=rthx,y−1=rthx,y+1=rthx+1,y=rthx+1,y+1=0 (T1):
rthx,y=rthx+1,y−1=rthx−1,y+1=0 and
rthx−1,y−1=rthx−1,y=rthx,y−1=rthx,y+1=rthx+1,y=rthx+1,y+1=1 (T2):
Otherwise, the filter output is 0, i.e. the D7 pattern is not recognized in the red plane.
Patterns D1-D5 correspond to single dots. Loss of contrast can occur at a dot pattern, so these patterns are treated like diagonals. Patterns D8-D11 indicate that the pixel 106x,y is near a diagonal. Patterns D12-D15 indicate that the pixel may be at an end of a diagonal.
In this implementation, step 810 is performed using the following filter:
This filter is applied to a saturation threshold plane using an OR operation. More particularly, for each pixel 106x,y, a flag “sat” is computed which is equal to 1 if the saturation is high, and is equal to 0 otherwise. Possible “sat” computations are described below. Once the sat values have been computed, the Ortho filter is applied to a pixel 106x,y. The filter output “ortho” is zero if sat=0 for the pixel and the four adjacent pixels above, below, to the right, and to the left. Otherwise, ortho=1. In some other embodiments, ortho is also set to 1 if there is a saturated pixel (with sat=1) in two of the four diagonally adjacent pixels (i.e. 106x−1,y−1, 106x−1,y+1, 106x+1,y−1, 106x+1,y+1). See Table 1 lines Spr23-Spr30 and Spr73-Spr80; Table 2 lines Ps2, Ps9, Ps10.
The sat value can be computed as follows. In some embodiments, for each pixel 106x,y, the value sat is set to 0 if the following value “sin v” (saturation inverse) is above some threshold:
sin v=floor[min(r,g,b)/max(1,r,g,b)] (8)
where r, g, b are the input rgb coordinates. In other embodiments, the number formed by the upper bits of max(r,g,b), e.g. the four upper bits, is multiplied by some “saturation threshold”, “STH” (e.g. 0, 1, 2, or greater), and four most significant bits of the product are considered. If they form a number greater than min(r,g,b), then sat is set to 1, and otherwise to zero.
In other embodiments, “sat” is computed from the RGBW coordinates generated by step 420. An exemplary computation is as follows. If R, G, or B above MAXCOL, then sat is set to 1. If not, the upper four most significant bits within MAXCOL are extracted for each of R, G, and B (e.g. bits [10:7] if MAXCOL=2047). The maximum of these four-bit values is multiplied by STH. The four most significant bits of the product form a number. If this number is greater than the number formed by the upper four most significant bits [10:7] of W, then “sat” is set to 1, and otherwise to 0. See Table 1, lines F37-F45 (SATBITS=4 to implement the example above). The invention is not limited to the number bits or other particulars.
In Table 1, “ortho” is computed in line Spr6. In addition, for a BW pair, “bortho” is computed as the Ortho filter output on the adjacent pixel to the left, and is used in determining the blue subpixel value (lines Spr59, Spr89-Spr91).
At step 810, the answer is Yes if the Ortho filter's output “ortho” is zero, and the answer is No otherwise. See Table 1 lines Spr34 (for an RG pair), Spr108 (for a BW pair). In processing the blue subpixel, “bortho” is used in a similar manner (line Spr96).
If a pixel 106x,y is mapped into an RG pair, the pixel's processing is described in lines Spr9-Spr53 in Table, 1, lines PS1-PS7 in Table 2. The adjacent blue subpixel to the right can be processed at the same time. More particularly, if ortho is 0 (line Spr34 in Table 1, line PS3 in Table 2), then the R, G and B subpixel values (Rw, Gw, Bw) are set to the output of diagonal filter (2) plus the meta luma sharpening value “a” described in Addendum A below before the claims. See step 820 in
In line Spr39 of Table 1, line PS5 of Table 2, the value “diag” is 1 if, and only if, at least one pattern D1-D15 is recognized in at least one of the R and G coordinates of pixel 106x,y. In this case, step 850 is performed. In particular, the R and G subpixel values are set to the output of box filter (7).
If diag is not 1, then step 840 is performed (lines Spr44-Spr45 in Table 1, line PS6 in Table 2). The R and G subpixel values are set to the sum of the outputs of diagonal filter (2) and DOG filter (6).
In line Spr47 of Table 1, line PS7 of Table 2, the value “bdiag” is 1 if, and only if, at least one pattern D1-D15 is recognized in the B coordinate of pixel 106x,y. In this case (line Spr48 in Table 1, line PS7 in Table 2), at step 850, the B subpixel value is set to the output of box filter (7).
If bdiag is not 1, then at step 840 (lines Spr51, PS7), the B subpixel value is set to the sum of the outputs of diagonal filter (2) and DOG filter (6).
If the pixel 106x,y is mapped to a BW pair, it is processed as shown starting line Spr54 in Table 1, line PS8 in Table 2. In this case, the blue subpixel value is computed on the adjacent pixel to the left (i.e. with blue shift) as explained above. Thus, the blue subpixel processing is somewhat duplicative (although not entirely so), and is omitted in some embodiments. Alternatively, the blue subpixel processing in lines Spr9-Spr53 (for the RG pair) is omitted. In the simulation code of Table 1, the blue subpixel processing is performed twice, and the two results for the blue subpixel are stored in a memory (line Spr162). Subsequent processing can use either one of these two results.
The flags “ortho” and “bortho” are determined as described above.
In line Spr96 of Table 1, line PS11 of Table 2, if the Ortho filter output bortho is 0 on the adjacent pixel to the left, then the B subpixel value is set to the sum of the outputs of the diamond filter (2) and the meta sharpening filter value a (Addendum A). Both filters are computed on the pixel 106x−1,y. See line Spr97. Also, a flag “doedge” is set to 1 to perform special processing if the pixel 106x,y is adjacent to the left or right edge of the screen as shown in lines Spr120-Spr141 of Table 1, line PS19 of Table 2. This processing is performed to improve the hue if the image contains vertical white lines at the screen edges. More particularly, if certain conditions hold as shown in Table 1, each of the blue and white subpixel values is computed as the sum of the diamond filter (2) and the DOG filter (6). See lines Spr137-Spr138. The filters are computed on the pixel 106x,y.
If bortho is not zero, then (step 830) diag is checked on the blue plane (lines Spr70, Spr100 of Table 1, line PS13 of Table 2). If diag is 1, then (line Spr101) the box filter (7) is applied (step 850). The box filter is computed on the pixel 106x−1,y to output the average of the B coordinates of the pixels 106x,y and 106x−1,y. Thus, if bortho is one for pixel 106x,y, ortho is one for pixel 106x−1,y, and diag is one for both pixels 106x−1,y and 106x,y, then the box filter is applied so that the value of each of the corresponding subpixels 120R, 120G, 120B is the mean of the corresponding color coordinates R, G, B of the pixels 106x−1,y and 106x,y. In some embodiments, the value of the corresponding subpixel 120W is also the mean of the W coordinates of the same pixels 106x−1,y and 106x,y. In Tables 1 and 2 however the W subpixel value is computed differently as described below.
If diag is not 1 in lines Spr101 and PS13, then (step 840, lines Spr103, PS14) the B subpixel value is computed as the sum of the output of the diamond filter (2) and the DOG filter (6), both applied to the pixel 106x−1,y. (In Table 1, the variable blueshift is set to 1 if the blue shift is to the left as assumed in this discussion, or to −1 if the blue shift is to the right.) Also, doedge is set to 1 to perform the edge processing for the edge pixels as described above.
The W value is computed as shown starting in lines Spr108, PS15. If the Ortho filter output ortho on the pixel 106x,y is 0, then the W subpixel value is set to the sum of the outputs of the diamond filter (2) and the meta sharpening filter value ax,y, i.e. the value a (Addendum A). Both filters are computed on the pixel 106x,y. See line Spr109.
If ortho is not zero, then (step 830) diag is checked on the white plane (lines Spr111, Spr112, PS17). If diag is 1, then (line Spr113) the box filter (7) is applied (step 850). The box filter is computed on the pixel 106x,y to output the mean of the W values for the pixel 106x,y and the pixel 106x+1,y.
If diag is not 1, then (step 840, lines Spr115, PS18) the W subpixel value is computed as the sum of the output of the diamond filter (2) and the DOG filter (6). Both filters are applied to the pixel 106x,y in the white plane.
The processing starting lines Spr143, PS19 is performed for all the pixels, i.e. the pixels mapped into RG pairs and the pixels mapped into BW pairs. In lines Spr147-Spr155, the subpixel values for the red, green and blue subpixels are hard-clamped to the maximum range of 0 to MAXOOG, where MAXOOG=2*MAXCOL+1 is the maximum possible RGBW value when M0=½ (see equations (3)). The white subpixels' values are hard-clamped to the range of 0 to MAXCOL.
In lines Spr126-Spr134 and some other passages, the values HS and VS denote the starting horizontal and vertical coordinates when updating only a portion of the screen. The simulation code of Table 1 assumes HS=VS=0. Also, the variables xsiz and ysiz contain this width and height of the screen portion being updated.
Notes on the code in Table 1:
Note 1: brute force software implementation of blackjack type tests; requires a separate frame buffer named bin with pixels; thresholded to 0 or 1 already; returns 1 if pattern match or inversion of pattern match; hardware implements this with 9-bit bit-pattern tests.
Note 2: the test is performed for all patterns D1-D15. The code for the remaining patterns is omitted.
Scaling and Gamut Clamping
As stated above, at steps 444 (Scaler) and 450 (Gamut clamp) of
The presence of a black hole depends on the backlight unit's output power BL. More particularly, the input rgb data are assumed to define the image when the backlight unit generates some output power BL=BL0. As seen from equations (3), the R, G, and B subpixel values produced by the SPR block 454 are in the range of 0 to (MAXCOL/M0) inclusive. The W value Ww can be up to MAXCOL/M1, but it is typically chosen not to exceed max(r,g,b) and thus not to exceed MAXCOL. Therefore, Ww does not exceed MAXCOL/M0. The RwGwBwWw values produced by the SPR block 454 define the desired subpixel luminances when the backlight unit's output power is BL0. However, to provide a value input to display 110, the subpixel values must not exceed MAXCOL. If the subpixel values are multiplied by M0 to fit into the range of 0 to MAXCOL inclusive, then the backlight unit's output power BL0 needs to be divided by M0, i.e. to be set to
BL=BL0/M0.
In fact, a smaller BL value may suffice if the maximum subpixel value Pmax=max(Rw,Gw,Bw,Ww) is below MAXCOL/M0. More particularly, given the maximum value Pmax, the minimum BL value BLmin sufficient to display all the subpixels without distortion is
BLmin=BL0*Pmax/MAXCOL.
It may be desirable to set the output power BL to a value below BLmin. In any case, it is sometimes convenient to express the output power BL as a percentage of BL0, i.e.
BL=(1/INVy)*BL0
where INVy is the coefficient by which the subpixel values corresponding to BL0 need to be multiplied (in scaler 444) to correspond to BL. For example, if BL=BLmin, then INVy=MAXCOL/Pmax. If BL=BL0, then INVy=1.
If BL is below BLmin (i.e. INVy>MAXCOL/Pmax), then some subpixel values can be above MAXCOL, so scaling/gamut clamping may be needed. Some methods for determining BL in block 430 are described below in Addendum B.
Now suppose that quad 1010 corresponds to two adjacent pixels on diagonals D, B (
When the pixels on diagonals AA, A are processed (i.e. when quad 1010 corresponds to two pixels on diagonals AA, A), then blk_gain will be 1 because the pixels on diagonals AA, A are not in a black hole. However, in some embodiments described below, the normal gain XS_gain is a decreasing function of the maximum rgb coordinate (see equation (3)) of the two pixels. Therefore, XS_gain for diagonals AA, A could be lower than for diagonals D, B. This would result in a loss of contrast if the black hole gain were not used. Setting the black hole gain to a value below 1 for diagonals D, B acts to reduce the subpixel values for these two diagonals to regain contrast loss.
Table 3 below illustrates simulation code for a procedure “dopost” which simulates the method of
Step 910 is implemented in lines Sc46-Sc61 in Table 3. The initial (pre-clamping) subpixel values for quad 1010 are denoted Rw, Gw, Bw, Ww. The test of step 910 is as follows: If max (Rw,Bw,Gw,Ww) does not exceed MAXCOL in quad 1010 and the maximum subpixel value in each of quads 1020, 1030 exceeds MAXCOL, then the black hole is detected. Other tests can also be used. For example, a black hole may include an additional requirement that the maximum subpixel value in each of quads 1020, 1030 exceed MAXCOL by some factor (e.g. is at least 1.1*MAXCOL), and/or exceeds the maximum subpixel value in quad 1010 by some factor. Further, the test may check for the luminances in quads 1020, 1030 to be above the luminance in quad 1010 or above some value, or for the luminance in quad 1010 to be below some value. Other tests may also be used.
Of note, in this embodiment, the test does not depend on INVy. Thus, even if INVy=M0, a black hole is detected and blk_gain may be set to a value below 1. As is seen by comparing the sections I and II of
If the test fails (i.e. no black hole is detected), then blk_gain is set to 1 (step 914 in
If the test passes, then (see step 920 of
blk_gain=2*MAXCOL−1−(maximum subpixel value in quads 1020, 1030) (9)
In this example, MAXCOL=2047, and M0=M1=½. GAMBITS=11 in line Sc61. Alternatively, the following equation can be used:
blk_gain=ceiling[1/M0*MAXCOL]−1−(maximum subpixel value in quads 1020, 1030)
Then (line Sc65) blk_gain is increased by Ww/16. If the Ww value is large (i.e. the black hole is actually a white hole), then the black hole gain is increased by this operation. Then blk_gain is hard-clamped to the maximum value of 256 (i.e. to 1 after division by 256 in line Sc111).
At step 930, the “normal” gain XS_gain is determined as shown in lines Sc72-Sc109. The invention is not limited to a particular way of determining XS_gain. In some other embodiments, the normal gain is not used (or, equivalently, is set to 1). Some gamut clamping examples suitable for XS_gain determination are given in U.S. patent application published as no. 2007/0279372 A1 published on Dec. 6, 2007, filed by Brown Elliott et al., entitled “MULTIPRIMARY COLOR DISPLAY WITH DYNAMIC GAMUT MAPPING”, incorporated herein by reference.
In the particular example of Table 3, XS_gain depends on the saturation and the maximum of the r, g, b values which are defined as in equations (3). More particularly, as shown in line Sc91 of Table 2, XS_gain is calculated as the sum of the saturation-based gain sat_gain and a value “nl_off”. The sum is hard-clamped to the maximum value of INVy received from block 430.
The value sat_gain is determined in lines Sc72-Sc84 as a value between some predefined parameters GMIN and GMAX inclusive. In some embodiments, GMAX=1 (i.e. 256 before division by 256) and GMIN=½. The value sat_gain is a function of saturation, and more particularly of the saturation inverse sin v defined as follows:
sin v=Ww/max(1,Rw,Gw,Bw)
See lines Sc74-Sc83. If the saturation is at most some predefined threshold value (e.g. 50%), i.e. if sin v at least some threshold, then sat_gain is set to about GMAX. In line Sc84, the threshold is defined by REG_SLOPE (REG_SLOPE is an integer value corresponding to 1). If sin v is zero, then sat_gain is set to about GMIN. If sin v is between zero and its threshold, then sat_gain is obtained as a linear interpolation function equal to about GMIN at sin v=0 and to about GMAX at the threshold value. In addition, sat_gain is hard-clamped to the maximum value of 1 (to 256 in line Sc85).
The term nl_off (“non-linear offset”) is calculated in lines Sc87-Sc90 based on max(r,g,b) where r, g, b are as in (3). Equations (3) indicate that max(r,g,b)=M0*max(R,G,B)+M1*W. It is assumed for simplicity in Table 3 that the RGBW values are the subpixel values Rw, Gw, Bw, Ww. The value nl_off is calculated as a linear interpolation function equal to 0 when max(r,g,b)=MAXCOL, and equal to about N*INVy when max(r,g,b)=0, where N is a predefined parameter between 0 and 256 inclusive.
As stated above, XS_gain is the sum of sat_gain and nl_gain hard-clamped to INVy. The value XS_gain is then further adjusted to ensure that after being multiplied by XS_gain, the subpixel values Rw, Gw, Bw, Ww do not exceed MAXCOL (lines Sc97-Sc109).
Step 940 is performed at line Sc111.
At step 950, the Rw, Gw, Bw, Ww values are multiplied by XSC_gain (lines Sc115-Sc119).
Then, at lines Sc122-Sc128, the Ww value can be further adjusted so that the dopost process would not change the luminance of the quad 1010. More particularly, the luminance Lw can be calculated before and after the scaling and gamut clamping as:
Lw=(2*Rw+5*Gw+B2+8*Ww)/16 (see lines Sc44, Sc19).
The Ww value can be adjusted so that the post-scaling and pre-scaling luminances coincide.
Finally, the values Rw, Gw, Bw, Ww are hard-clamped to the range of 0 to MAXCOL inclusive (lines Sc129-Sc137).
Bit Blit Update As explained above with reference to
It will be assumed for ease of description that the new portion 1110 is rectangular. The invention is not limited to rectangular portions however.
In some other embodiments, the SPR operation is repeated over the whole image. More particularly, the display apparatus stores input data (rgb or RGBW) for each pixel of the image 104, and recalculates the pixel values in SPR operation 454 for the entire image when portion 1110 is received. The recalculation can be implemented as in
Some embodiments will now be described which are based on the SPR operation described above in connection with
In
In some embodiments, when performing the SPR operation 454 on the pixels 1110E, the pixels 1120 are replaced by mirror images of pixels 1110E. For example, suppose that the area 1110 is defined as x0≦x≦x1 and y0≦y≦y1 for some x0, x1, y0, y1. Then the border pixels 1120 are defined as follows for the SPR operation on the pixels 1110E:
106x
The corner pixels are also mirrored:
106x
Further challenge is presented if the SPR uses a blue shift. The case of the left shift will be described in detail. The right-shift embodiments are similar.
In the case of the left shift, if a pixel 106 in edge area 1110E at the left of portion 1110 is mapped into a BW pair, then the SPR filters may have to be applied to the adjacent pixel in border area 1120. In the example of
In some embodiments, if the new portion 1110 is one-column wide (and thus coincides with edge area 1110E), then all the Bw values corresponding to the new portion 1110 are unchanged.
In the case of the left shift, another challenge is presented at the right edge when a border pixel 1120 is mapped into a BW pair. This is illustrated in
For each quad 1404, the SPR operation 454 provides subpixel values Rw, Gw, Bw, Ww shown at 1410. In
Each subpixel corresponds to a memory location in frame buffer 610. The memory locations may be independently addressable, but this is not necessary. In the example of
As indicated above, the contents of memory location 610B can be lost (not updated) if the image is updated with a new portion 1110 mapped into a subpixel area located immediately to the left of the BW pair 124x+1,y. Therefore, in each quad, the memory location 610B stores only the least significant bits of some or all of the values Rw, Gw, Bw, Ww. In the embodiment of
Scrambling is performed when writing to frame buffer 610. When the frame buffer is read (e.g. by scaler 444 or block 430 in
For each BW pair at the left edge of the screen (i.e. each BW pair 1240,y), the MSB portion of the blue location 610B can be filled with a predefined value, e.g. 0. The BH value can be discarded. On descrambling, the BH value can be set to zero. The invention is not limited to this or other ways of handling the BW pairs at the edges.
For each RG pair at the right edge of the screen, in scrambling, the Bw value can be obtained by applying the suitable filters in the SPR operation to the pixel 106 corresponding to the RG pair. The BH portion of the Bw value can be written to the LSB portion of the location 610R. The BL and RL portions can be discarded. On descrambling, RL can be set to zero or some other value.
The invention is not limited to the embodiments described above. Other embodiments and variations are within the scope of the present teachings.
For example, some embodiments provide a method for processing image data for displaying an image in a display window of a display unit. The display window may for example be the whole or part of the pixel area of display unit 110 of
The display unit comprises subpixels each of which is to emit one of a plurality of primary colors and to have a luminance depending on the subpixel's state defined using a subpixel value of the subpixel. The primary colors may be RGBW or may be some other colors. The subpixels may or may not be laid out as in
The method comprises: (1) obtaining, by a circuit (e.g. circuit 320), pixel data (e.g. RGBW data obtained in block 420) which represent, for each pixel, the pixel's color in coordinates (e.g. RGBW) associated with the primary colors. The invention is not limited to RGBW however. The method further comprises: (2) performing, by the circuit (e.g. by block 454 of circuit 320), a subpixel rendering (SPR) operation to provide a subpixel value for each subpixel in a set of one or more subpixels of the display window, the subpixel value being used in defining the respective subpixel's state in displaying the image. For example, the SPR operation may provide a subpixel value for each subpixel in a display area corresponding to the new portion 1110 (
The SPR operation further comprises an operation (2B) of determining the subpixel values for the two subpixels of the primary color PC1. The operation (2B) is performed so that: (2B.1) if the first test fails (e.g. one or both of tests 810, 830 fail on each of pixels 106x−1,y, 106x+1,y), then in the SPR operation, the primary color PC1's coordinate of the pixel PX1 contributes to the subpixel values of the two subpixels with equal non-zero weights. For example, when step 820 or 840 is performed on each of the red subpixels in pairs 124x−1,y, 124x+1,y, and the diamond filter (1) is applied, the R coordinate of pixel 106x,y contributes to the Rw value of each of the red subpixels with the weight of ⅛.
The SPR operation is performed so that: (2B.2) if the first test passes (e.g. if the tests 810, 830 pass on at least one of subpixels 106x−1,y, 106x+1,y), then in the SPR operation, the primary color PC1's coordinate of the pixel PX1 contributes to the subpixel values of the two subpixels with different weights. For example, if the test 810 or 830 fails on pixel 106x−1,y, then the box filter is applied at step 850, and the R coordinate of the pixel 106x,y contributes with a weight of ½ to the subpixel value of the red subpixel in pair 124x−1,y. However, for the red subpixel in pair 124x+1,y, the R coordinate of pixel 106x,y contributes with a weight of zero (at step 850) or ⅛ (at step 820 or 840).
In some embodiments, in the SPR operation, each pixel (e.g. 106) is associated with a display unit's area (e.g. 124) in which the pixel is to be displayed. The area associated with the pixel PX1 does not contain an entire subpixel of the primary color PC1. For example, the area 124x,y may be a BW area, and thus without a red subpixel.
In some embodiments, the display unit's area associated with the pixel PX1 does not contain any portion of a subpixel of the primary color PC1.
In some embodiments, the two subpixels are located on opposite sides of the display unit's area associated with the pixel PX1.
In some embodiments, the first test comprises: (i) a test that at least one of the colors of one or more of the pixel PX1 and the one or more adjacent pixels is a saturated color, wherein a saturated color is defined by a saturation parameter (e.g. such as described above in connection equation (8)) being in a predefined range (the range may consist of a single number, i.e. the upper and lower bounds may or may not be different); and (ii) a test that one or more of the pixel PX1 and the one or more adjacent pixels belong, or is adjacent, to a feature defined by at least one of one or more predefined patterns. Examples are patterns D1-D15.
In some embodiments, each said pattern comprises one or more first pixels and one or more second pixels. For example, in patterns D1-D15, the first pixels may correspond to the 1 values and the second pixels to the 0 values. If the image contains the pattern then the one or more first pixels' associated areas do not contain a subpixel of a primary color present in at least one second pixel's associated area. For example, each pattern D1-D15 contains either only one value of 1 or multiple such values on a diagonal. Therefore, the associated pairs of the 1-value pixels are all RG or all BW, and hence contain a color not present in the pair associated with at least one 0-value pixel.
Some embodiments provide a method for processing image data for displaying an image in a display window. The method comprises: (1) obtaining, by a circuit (e.g. 320), pixel data (e.g. RGBW data obtained in block 420 of
Further: (2B.2) if the first test passes, then in the SPR operation, the primary color PC1's coordinates of said two pixels in the predefined position relative to the pixel PX1 contribute to the subpixel value of the subpixel SP1 with different weights. For example, in
In some embodiments, the display unit's areas associated with the two pixels do not include a subpixel of the primary color PC1. For example, if a pixel 106x,y is mapped into an RG pair, and the primary color PC1 is red, then the areas of pairs 124x−1,y and 124x+1,y do not have a red subpixel.
In some embodiments, the first test comprises a test (i) that at least one of the colors of the pixel PX1 and of the adjacent pixels whose associated areas do not have a subpixel of the color PC1 is a saturated color (e.g. as in step 810; the saturated colors are performed only on the pairs above, below, to the left and to the right of the pixel using the Ortho filter, so they do not have the color PC1 of the pair whose subpixel values are being computed). A saturated color is defined by a saturation parameter (e.g. sin v in (8)) being in a predefined range (e.g. the one-value range of [1,1], or the infinite range of all numbers greater than or equal to 1, or the range of all the positive numbers), wherein the adjacent pixels comprise said pixels on said two opposite sides of the pixel PX1 (e.g. on the right and left sides). In some embodiments, the first test also comprises a test (ii) that the pixel PX1 belongs, or is adjacent, to a feature defined by at least one of one or more predefined patterns. Examples are patterns D1-D15.
In some embodiments, each said pattern comprises one or more first pixels and one or more second pixels. For example, in the patterns D1-D15, the first pixels may correspond to the 1 values and the second pixels to the 0 values. If the image contains the pattern then the one or more first pixels' associated areas do not contain a subpixel of a primary color present in at least one second pixel's associated area. For example, each pattern D1-D15 contains either only one 1 value or multiple 1 values on a diagonal. Therefore, the associated pairs of the 1-value pixels are all RG or all BW, and hence contain a color not present in the pair associated with at least one 0-value pixel.
In some embodiments, the test (ii) passes if in a color plane of each of one or more selected primary colors at least one of which is not a sum of any other primary colors: the primary color's coordinate of each first pixel is above a predefined threshold (e.g. BOBits in Table 1 lines D39-D41) while the primary color's coordinate of each second pixel is at or below the predefined threshold, and/or the primary color's coordinate of each second pixel is above a predefined threshold while the primary color's coordinate of each first pixel is at or below the predefined threshold.
Circuits are provided for performing the methods described herein. Other operations (e.g. gamma conversion and image display) are performed as needed.
Addendum A: Meta Luma Sharpening
In some embodiments, the meta luma sharpening for a pixel 106x,y is performed as follows. The pixel's RGBW coordinates are determined according to equations (3). Also, values L representing the luminances of pixel 106x,y and adjacent pixels are computed in some way, for example:
L=(2R+5G+B+8W)/16 (A1)
Then if pixel 106x,y is mapped into a BW pair, then the following filter is applied to the luminance L to produce a value a:
where z is some positive constant, e.g. ½. In other words,
a=z*Lx,y−z/4*(Lx−1,y+Lx+1,y+Lx,y−1+Lx,y+1),
where Li,j is the luminance (A1) of pixel 106i,j. If pixel 106x,y is mapped into an RG pair, then the value a is set to the output of the following filter applied to the L values:
where z is some positive constant, e.g. ½. The z values may or may not be the same in the two filters. Then the value a is used to select a metamer for the pixel 106x,y by modifying the RGBW coordinates as follows:
W=W+a
R=R−mr*a
G=G−mg*a
B=B−mb*a (A2)
where mr, mg, mb are constants defined by the luminance emission properties of display 110 in such a way that the new RGBW values (i.e. the values on the left in equations (A2)) and the old values define the same color (i.e. are metamers). In some embodiments, mr=mg=mb=1. Additionally, the new RGWB values can be hard-clamped to the range of 0 to MAXCOL/M0 for R, G, and B, and to MAXCOL/M1 for W.
Addendum B: Determining Backlight Unit Output Power
Let us assume that RwGwBwWw are the subpixel values determined by the SPR block 454 of
BL=BL0/INVy
If the subpixel value P is the maximum value to be displayed without distortion, then
P*INVy=MAXCOL, and therefore
INVy=MAXCOL/P, i.e.
BL=BL(P)=BL0*P/MAXCOL (B1)
There are a number of ways to select P. In some embodiments, the Rw, Gw, Bw, Ww subpixel values generated by the SPR block 454 are multiplied by respective coefficients Rweight, Gweight, Bweight, Wweight (e.g. Rweight=84%, Gweight=75%, Bweight=65% or 75%, and Wweight=100%), and P is selected as the maximum, over the whole image, of the resulting values, i.e.
P=max(Rw*Rweight,Gw*Gweight,Bw*Bweight,Ww*Wweight) (B1-A)
In some embodiments, the coefficient Rweight is replaced by a variable coefficient Xweight computed as follows:
Xweight=Rweight+((Yweight−Rweight)*Gw/2SBITS) (B1-B)
wherein Rweight, Yweight, and SBITS are predefined constants.
The subpixel value P can be chosen in other ways to obtain a desired image quality.
In other embodiments, the BL value is computed as follows. First, for each subpixel 120, a value Psub is computed as in (B1-A) or (B1-B), i.e. the maximum in (B1-A) is taken over the Rw, Gw, Bw, Ww values for the subpixel and not over all the subpixels in the image. Then a BL value BL=BL(Psub) is initially computed in accordance with (B1) for each subpixel value 120 (with Psub replacing P). These initial BL values are accumulated into a histogram. The histogram's bins (counters) are traversed backwards (starting with the highest BL value), and an accumulate-error function E_sum is computed which is the sum of the BL values in the bins traversed. For example E_sum[i] can be the sum of the BL values in bins with bin numbers greater than or equal to i, where the index i increases with BL (i.e. higher BL values are placed in bins with higher i). The traversal stops when E_sum[i] reaches or exceeds a predefined threshold TH1. Suppose this happens at bin i=i0. In some embodiments, the backlight output power BL is set to some value in bin i0. For example, if each bin i counts the BL values between some numbers bi and bi+1 (all BL with bi≦BL<bi+1), then the output power BL can be set to bi0 or some other value at least bi0 and less than bi0+1.
In some embodiments, linear interpolation is performed to select the BL value in bin i0. For example, the output power BL can be defined as the sum:
BL=bi0+fine_adjust_offset (B2)
where
fine_adjust_offset=(Excess/Delta—E_sum[i0])*bin_size (B3)
where Excess=E_sum[i0]−TH1; Delta_E_sum[i0]=E_sum[i0]−E_sum[i0+1], where bin_size is the size of each bin, i.e. bin_size=bi+1−bi (this value is 16 in some embodiments).
An additional adjustment can be made by comparing Excess to another, upper threshold TH2. If Excess>TH2, then fine_adjust_offset can be set to:
fine_adjust_offset=(Excess/TH2)*bin_size
and then (B2) can be used to determine BL. These embodiments are not limiting.
In some embodiments, the BL and INVy values lag the RwGwBwWw data by one frame. More particularly, the INVy value determined from the RwGwBwWw data for one frame (“current frame”) is used by scaler 444 for scaling the next frame. The BL value determined from the RwGwBwWw data for the current frame is used to control backlight unit 310 when LCD panel 110 displays the next frame. The current frame is scaled and displayed using the BL and INVy values determined from the previous frame of data. This lag allows one to start displaying the current frame before the current-frame BL and INVy values have been determined. In fact, displaying the current frame may begin even before all the sRGB data for the current frame have been received. To reduce image errors, the BL value can be “decayed”, i.e. the BL value can be generated by block 430 as a weighted average of the BL value determined from the data for the current frame and the previous BL value. In some displays which display 30 frames per second, when the image brightness abruptly changes, it may take about 36 frames for the BL and INVy values to catch up with the image brightness. This delay is acceptable in many applications. Indeed, in the absence of abrupt changes of the image brightness, the BL and INVy values typically do not change much from frame to frame, and a one-frame lag does not cause significant degradation of the image. When abrupt changes of brightness do occur, it takes time for the viewer to visually adjust to the image, so image errors due to the lag of the BL and INVy values do not stand out. See also U.S. patent application published as US 2009/0102783 A1 on Apr. 23, 2009, filed by Hwang et al., incorporated herein by reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 7222306 | Kaasila et al. | May 2007 | B2 |
| 7248268 | Brown Elliott et al. | Jul 2007 | B2 |
| 7864194 | Credelle et al. | Jan 2011 | B2 |
| 20030034992 | Brown Elliott et al. | Feb 2003 | A1 |
| 20030085906 | Elliott et al. | May 2003 | A1 |
| 20040051724 | Elliott et al. | Mar 2004 | A1 |
| 20040196297 | Elliott et al. | Oct 2004 | A1 |
| 20050104908 | Brown Elliott | May 2005 | A1 |
| 20050225563 | Brown Elliott et al. | Oct 2005 | A1 |
| 20050225574 | Brown Elliott et al. | Oct 2005 | A1 |
| 20060244686 | Higgins et al. | Nov 2006 | A1 |
| 20070279372 | Brown Elliott et al. | Dec 2007 | A1 |
| 20080049048 | Credelle et al. | Feb 2008 | A1 |
| 20090102783 | Hwang et al. | Apr 2009 | A1 |
| Number | Date | Country |
|---|---|---|
| WO 2006127555 | Nov 2006 | WO |
| WO 2007047537 | Apr 2007 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20110043552 A1 | Feb 2011 | US |
