Embodiments of the present invention comprise methods and systems for generating, modifying and applying backlight driving values for an LED backlight array.
Some displays, such as LCD displays, have backlight arrays with individual elements that can be individually addressed and modulated. The displayed image characteristics can be improved by systematically addressing backlight array elements.
Some embodiments of the present invention comprise methods and systems for generating, modifying and applying backlight driving values for an LED backlight array.
The foregoing and other objectives, features, and advantages of the invention will be more readily understood upon consideration of the following detailed description of the invention taken in conjunction with the accompanying drawings.
Embodiments of the present invention will be best understood by reference to the drawings, wherein like parts are designated by like numerals throughout. The figures listed above are expressly incorporated as part of this detailed description.
It will be readily understood that the components of the present invention, as generally described and illustrated in the figures herein, could be arranged and designed in a wide variety of different configurations. Thus, the following more detailed description of the embodiments of the methods and systems of the present invention is not intended to limit the scope of the invention but it is merely representative of the presently preferred embodiments of the invention.
Elements of embodiments of the present invention may be embodied in hardware, firmware and/or software. While exemplary embodiments revealed herein may only describe one of these forms, it is to be understood that one skilled in the art would be able to effectuate these elements in any of these forms while resting within the scope of the present invention.
In a high dynamic range (HDR) display, comprising an LCD using an LED backlight, an algorithm may be used to convert the input image into a low resolution LED image, for modulating the backlight LED, and a high resolution LCD image. To achieve high contrast and save power, the backlight should contain as much contrast as possible. The higher contrast backlight image combined with the high resolution LCD image can produce much higher dynamic range image than a display using prior art methods. However, one issue with a high contrast backlight is motion-induced flickering. As a moving object crosses the LED boundaries, there is an abrupt change in the backlight: In this process, some LEDs reduce their light output and some increase their output; which causes the corresponding LCD to change rapidly to compensate for this abrupt change in the backlight. Due to the timing difference between the LED driving and LCD driving, or an error in compensation, fluctuation in the display output may occur causing noticeable flickering along the moving objects. The current solution is to use infinite impulse response (IIR) filtering to smooth the temporal transition, however, this is not accurate and also may cause highlight clipping.
An LCD has limited dynamic range due the extinction ratio of polarizers and imperfections in the LC material. In order to display high-dynamic-range images, a low resolution LED backlight system may be used to modulate the light that feeds into the LCD. By the combination of modulated LED backlight and LCD, a very high dynamic range (HDR) display can be achieved. For cost reasons, the LED typically has a much lower spatial resolution than the LCD. Due to the lower resolution LED, the HDR display, based on this technology, can not display high dynamic pattern of high spatial resolution. But, it can display an image with both very bright areas (>2000 cd/m2) and very dark areas (<0.5 cd/m2) simultaneously. Because the human eye has limited dynamic range in a local area, this is not a significant problem in normal use. And, with visual masking, the eye can hardly perceive the limited dynamic range of high spatial frequency content.
Another problem with modulated-LED-backlight LCDs is flickering along the motion trajectory, i.e. the fluctuation of display output. This can be due to the mismatch in LCD and LED temporal response as well as errors in the LED point spread function (PSF). Some embodiments may comprise temporal low-pass filtering to reduce the flickering artifact.
Some embodiments of the present invention may be described with reference to
In some embodiments, the backlight image is given by
bl(x,y)=LED(i,j)*psf(x,y) (1)
where LED(i,j) is the LED output level of each individual LED in the backlight array, psf(x,y) is the point spread function of the diffusion layer and * denotes a convolution operation. The backlight image may be further modulated by the LCD.
The displayed image is the product of the LED backlight and the transmittance of the LCD: TLCD(x,y).
img(x,y)=bl(x,y)TLCD(x,y)=(led(i,j)*psf(x,y))TLCD(x,y) (2)
By combining the LED and LCD, the dynamic range of the display is the product of the dynamic range of LED and LCD. For simplicity, in some embodiments, we use a normalized LCD and LED output between 0 and 1.
Some exemplary embodiments of the present invention may be described with reference to
In these embodiments, the input image may be low-pass filtered 11 using the point spread function of the diffusion screen of the display to create an LPF image. This LPF image may then be sub-sampled 14 to an intermediate resolution. In some embodiments, the intermediate resolution will be a multiple of the LED array size (aM×aN). In an exemplary embodiment, the intermediate resolution may be 2 times the LED resolution (2M×2N). In some embodiments, the extra resolution may be used to reduce flickering. This subsampled image may be referred to as an LEDlp image.
The HDR input image 10 may also be low pass filtered 12 with a smaller filter kernel, such as a 5×5 kernel, to simulate the size of a specular pattern. This smaller low-pass filtered image (SLPF image) may then be divided 13 into aM×aN blocks with each block corresponding to one LED with some overlap between each block. For example, in an exemplary embodiment, the block size may be (1+k)*(m/M×n/N), where k is the overlap factor. In an exemplary embodiment, k may be set to 0.25. A maximum value may then be determined 15 for each block. These maximum block values may be used to form an LEDmax image with a resolution of M×N.
In some embodiments, a combined LED1 image may be created 16 by selecting between variations of the LEDmax image and the LEDlp image. In an exemplary embodiment, the LED1 image may be determined by selecting the greater of two times the LEDlp image and the LEDmax image as expressed in the following equation:
LED1=max(LEDlp×2,LEDmax). (3)
In some embodiments, the values in the LED1 image may be constrained to be less than one, for example, through the use of equation 4:
LED1=min(max(LEDlp×2,LEDmax),1). (4)
By taking into account the local maximum, the specular highlight is preserved. Also, using twice the LED1 image values ensures that the maximum LCD operating range will be used. These embodiments better accommodate images with high dynamic range and high spatial frequency.
The resulting LED1 image will have a size of M×N and a range from 0 to 1. Since the PSF of the diffusion screen is larger than the LED spacing to provide for a more uniform backlight image, there is considerable crosstalk between the LED elements that are located close together.
Because of the PSF of the LEDs, any LED has contribution from each of its neighboring LEDs. Although Equation 2 can be used to calculate the backlight, given an LED driving signal, deriving the LED driving signal to achieve a target backlight image is an inverse problem. This is an ill-posed de-convolution problem. In one approach, a convolution kernel is used to derive the LED driving signal as shown in Equation 5. The crosstalk correction kernel coefficients (c1 and c2) are negative to compensate for the crosstalk from neighboring LEDs.
The crosstalk correction matrix does reduce the crosstalk effect from its immediate neighbors, but the resulting backlight image is still inaccurate with a too-low contrast. Another problem is that it produces many out of range driving values that have to be truncated and can result in more errors.
Since the LCD output can not be more than 1, the LED driving value must be derived 17 so that backlight is larger than target luminance, e.g.,
led(i,j):{led(i,j)*psf(x,y)≧I(x,y)} (6)
In Equation 6, “:” is used to denote the constraint to achieve the desired LED values of the function in the curly bracket. Because of the limited contrast ratio (CR), due to leakage, LCD(x,y) can no longer reach 0. The solution is that when a target value is smaller than LCD leakage, the led value may be reduced to reproduce the dark luminance.
led(i,j):{led(i,j){circle around (x)}psf(x,y)<I(x,y)·CR} (7)
In some embodiments, another goal may be a reduction in power consumption so that the total LED output is reduced or minimized.
Flickering may be due to the non-stationary response of the LED combined with the mismatch between the LCD and LED. The mismatch can be either spatial or temporal. Flickering can be reduced or minimized 18 by reducing the total and localized led output fluctuation between frames.
where x0 and y0 define the distance from the center of the LED. To achieve Equation 9, a series of non-LED grid points or virtual points are introduced to minimize the LED output fluctuation. In some embodiments, one or more virtual points are inserted between two LEDs. Without the virtual point, when an object (bright) moves from one LED to another LED, the first LED decreases and the second LED increases. This occurs suddenly and causes flickering. With the virtual point, the bright object first moves to the virtual point, and then to the second LED. The virtual point causes the first LED to slowly reduce its output and the second LED to increase its output. In some embodiments, the flickering can be further reduced by temporal IIR filtering. Combining Equations 6 and 9 yields Equation 10 below.
In some embodiments, the algorithm to derive 17 the backlight driving values that satisfy Eq. 10, or other constraints, comprises the following steps:
Finding an LED driving value from a target value is an ill-posed problem that requires an iterative algorithm, which is computationally expensive and difficult to implement in hardware. Some aspects of embodiments of the present invention may be described with reference to
In an iterative approach, the backlight may be calculated by multiplying an LED driving value, e.g., a 1D vector of length MN, where MN is the total number of LEDs, with the crosstalk matrix (MN×MN). This is very computationally expensive and not necessary since the crosstalk between LEDs that far apart is very small.
In some exemplary embodiments, the backlight may be approximated 61 by convolving the LED driving value, Led1, with a truncated PSF 67 of size 7×5. In some embodiments, an iterative method may then be used 62 for a fixed number of iterations. In an exemplary embodiment, four iterations provide good results. A new LED driving value, Ledi+1 may be increased or decreased 63 by the scaled difference between a target value and a predicted value. The scale factor may be 0.28 in an exemplary embodiment and may vary based on the PSF and other factors.
In some embodiments, the intermediate LED driving value, Ledi+1, may then be multiplied by the ledMask and the result may be constrained 64 to be greater than 0 and to be found only on those LED grid points defined by ledMask. The constrained intermediate LED driving value may then be convolved 65 with the truncated PSF 67. The process may repeat for a few iterations to achieve the desired LED driving value 66 and will typically converge after about 4 iterations.
Aspects of some embodiments of the present invention may be described with reference to
In an exemplary embodiment, illustrated in
In some embodiments, the steps above may be approximated for hardware implementation with the following:
In some embodiments, since the LED output is non-linear with respect to the driving value and the driving value is an integer, inverse gamma correction 19 and quantization may be performed to determine the LED driving value that will be sent to the LED driver circuit 20.
In some embodiments, the backlight image may now be predicted from the LED image. The LED image may be upsampled 26 to the LCD resolution (m×n) and convolved with the PSF of the diffusion screen 25 to yield an LED backlight image 24. The LCD transmittance may be calculated 23 with equation 11.
TLCD(x,y)=img(x,y)/bl(x,y) (11)
Again, inverse gamma correction 22 may be performed, to correct for the non-linear response of the LCD and the resulting LCD image may be sent to an LCD driver circuit 21.
In some embodiments, to reduce the flickering effect, temporal low-pass filtering 18 may be used to smooth sudden temporal fluctuations. Equation 12 describes an exemplary filtering process.
wherein kup is typically chosen to be higher than kdown to satisfy Equation 6. In an exemplary embodiment, kup may be set to 0.5 and kdown may be set to 0.75.
The terms and expressions which have been employed in the foregoing specification are used therein as terms of description and not of limitation, and there is no intention in the use of such terms and expressions of excluding equivalence of the features shown and described or portions thereof.
Number | Name | Date | Kind |
---|---|---|---|
5978142 | Blackham et al. | Nov 1999 | A |
6268843 | Arakawa | Jul 2001 | B1 |
6992279 | Okuwaki et al. | Jan 2006 | B2 |
7312779 | Blevins | Dec 2007 | B1 |
7525528 | Feng | Apr 2009 | B2 |
7635200 | Atsushi | Dec 2009 | B2 |
7737930 | Inuzuka et al. | Jun 2010 | B2 |
7762470 | Finn et al. | Jul 2010 | B2 |
20020021292 | Sakashita | Feb 2002 | A1 |
20020050987 | Sakashita | May 2002 | A1 |
20040095558 | Whitehead et al. | May 2004 | A1 |
20050007781 | Okuwaki et al. | Jan 2005 | A1 |
20050206511 | Heenan et al. | Sep 2005 | A1 |
20060103621 | Feng | May 2006 | A1 |
20060125771 | Inuzuka et al. | Jun 2006 | A1 |
20060202945 | Feng | Sep 2006 | A1 |
20060291185 | Atsushi | Dec 2006 | A1 |
20070285382 | Feng | Dec 2007 | A1 |
20070296886 | Inada et al. | Dec 2007 | A1 |
20080129677 | Li et al. | Jun 2008 | A1 |
20080150880 | Inuzuka et al. | Jun 2008 | A1 |
20090015720 | Yamamoto | Jan 2009 | A1 |
20090174638 | Brown Elliott et al. | Jul 2009 | A1 |
20090251401 | Gorog | Oct 2009 | A1 |
20090295706 | Feng | Dec 2009 | A1 |
20100277515 | Ward et al. | Nov 2010 | A1 |
Number | Date | Country |
---|---|---|
200851905 | Mar 2008 | JP |
200870558 | Mar 2008 | JP |
2008051905 | Mar 2008 | JP |
2008070558 | Mar 2008 | JP |
WO03077013 | Sep 2003 | WO |
Entry |
---|
International Search Report for PCT/JP2009/063450 dated Nov. 2, 2009 claiming priority to the present application. |
H. Seetzen, W. Heidrich, W. Stuerzlinger, G. Ward, L. Whitehead, M. Trentacoste, A. Ghosh, and A. Vorozcovs, High Dynamic Range Display Systems, ACM SIGGRAPH 2004. |
European Search Report for PCT/JP2009053450 dated Sep. 20, 2011 claiming priority ot the present application. |
Japanese Patent Grant for corresponding application—JP 2011-501839, Oct. 12, 2012. |
Chinese Office Action for corresponding application—CN 20090128332.5, Jun. 18, 2008. |
Number | Date | Country | |
---|---|---|---|
20100020003 A1 | Jan 2010 | US |