A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
1. Field of Invention
The present invention relates to artifact reduction and particularly to reduction of LCD flare. The present invention comprises an improvement to existing process of computing the LCD and LED images.
2. Discussion of Background
Dynamic range is the ratio of intensity of the highest luminance parts of a scene and the lowest luminance parts of a scene. For example, the image projected by a video projection system may have a maximum dynamic range of 300:1.
The human visual system is capable of recognizing features in scenes which have very high dynamic ranges. For example, a person can look into the shadows of an unlit garage on a brightly sunlit day and see details of objects in the shadows even though the luminance in adjacent sunlit areas may be thousands of times greater than the luminance in the shadow parts of the scene. To create a realistic rendering of such a scene can require a display having a dynamic range in excess of 1000:1. The term “high dynamic range” means dynamic ranges of 800:1 or more.
Modern digital imaging systems are capable of capturing and recording digital representations of scenes in which the dynamic range of the scene is preserved. Computer imaging systems are capable of synthesizing images having high dynamic ranges. However, current display technology is not capable of rendering images in a manner which faithfully reproduces high dynamic ranges.
Blackham et al., U.S. Pat. No. 5,978,142 discloses a system for projecting an image onto a screen. The system has first and second light modulators which both modulate light from a light source. Each of the light modulators modulates light from the source at the pixel level. Light modulated by both of the light modulators is projected onto the screen.
Gibbon et al., PCT application No. PCT/US01/21367 discloses a projection system which includes a pre modulator. The pre modulator controls the amount of light incident on a deformable mirror display device. A separate pre-modulator may be used to darken a selected area (e.g. a quadrant).
Whitehead at al., U.S. Pat. No. 6,891,672, and related patents and patent applications describe many techniques, including, among others, the implementation and refinement of dual modulated displays, wherein a modulated backlight (aka local dimming) projects onto a front modulator (e.g., LCD) of a display.
The present inventors have realized the need for improved processes for computing LCD and LED images. In one embodiment, the present invention provides a display, comprising a front modulator, a backlight configured to produce a modulated light illuminating the front modulator, and a controller configured to process an image signal into a backlight control signal and a front modulator control signal, wherein at least one of the backlight control signal and the front modulator control signal comprises a control signal having an artifact removed and an artificial effect introduced into an image produced by the signals. The artifact may comprise, for example, an LCD flare and the artificial effect may comprise, for example, a veiling glare. The veiling glare is configured, for example, to minimize effects caused by a geometry of the backlight.
In another embodiment, the invention may comprise a display, comprising a front modulator, a backlight configured to produce a modulated light illuminating the front modulator, and
a controller configured to produce a backlight control signal and a front modulator control signal from an image signal, wherein at least one of the backlight control signal and the front modulator control signal comprises an adjustment of values that minimize the occurrence of LCD flare. The adjustment of values may comprise, for example, a reduction of visible flare in an image to be displayed, and the introduction of a veiling glare may be configured, for example, to obscure artifacts related to the backlight.
The invention may also be embodied as a method, including a method of driving a dual modulation display, comprising the steps of, determining a flare that would be visible in an output of the display, adjusting drive levels of a backlight so that the flare is reduced, adding a simulated veiling glare, and adjusting a backlight simulation to produce a shape of the veiling glare so as to hide a geometry of the backlight. The backlight may comprise, for example, an LED array and the backlight simulation adjustment hides the geometry of the LED array.
In yet another embodiment, the invention may comprise a method of driving a display comprising a modulated backlight and a front modulator illuminated by the modulated backlight, comprising the steps of, computing a front modulator image and a simulated backlight image from image data, determining locations of at least one LED “skirt,” simulating a veiling glare, calculating a backlight suppression image configured to compensate regions where the “skirt” exceeds the simulated glare, re-computing the simulated backlight in light of the backlight suppression image, determining “missing” glare sources, calculating a veiling glare for each missing glare source, and constructing a new LCD image comprising the calculated veiling glares. The front modulator may comprise, for example, an LCD panel, and the backlight may comprise, for example, an LED array. The backlight may comprise any of an RGB, RGBW, or RGB plus an additional color(s) (or white) LED array.
The veiling glare may be simulated, for example, via convolution. The step of identifying regions may comprise, for example, subtracting a convolution image used to produce the simulated glare from an image of the “skirt.” The step of suppressing the identified regions may comprise, for example, using a multiplier at each pixel where the “skirt” exceeds a predetermined epsilon of the simulated glare. The step of re-computing may comprise, for example, applying the backlight suppression image to at least part of image data used to create the backlight simulation and then recomputing the backlight simulation.
Portions of both the device and method may be conveniently implemented in programming on a general purpose computer, or networked computers, and the results may be displayed on an output device connected to any of the general purpose, networked computers, or transmitted to a remote device for output or display. In addition, any components of the present invention represented in a computer program, data sequences, and/or control signals may be embodied as an electronic signal broadcast (or transmitted) at any frequency in any medium including, but not limited to, wireless broadcasts, and transmissions over copper wire(s), fiber optic cable(s), and co-ax cable(s), etc.
A more complete appreciation of the invention and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings, wherein:
The invention comprises an improvement to the existing process of computing the LCD and LED images. Although preferably applied on an HDR display, the principles and features of the invention are also applicable to any dual modulation display where one of the modulators is an LCD panel. The dynamic range of the display can be low, for example any of the currently known modulated backlight LCD panels.
The specific improvement of the invention addresses the issue of illuminating small bright features on a dark surround. In this case, the LCD panel cannot block all light from the backlight (e.g., LEDs) in the dark surround and thus the flare of these LEDs creates a skirt of light that diminishes the intended appearance of the display. Because the feature is small, the perceptual effect of veiling luminance is not sufficient to hide the LED flare. In a modulated backlight using LEDs, as the feature moves across the display, neighboring LEDs are turned on and off as necessary to illuminate the feature, and the flare from these LEDs is visible and thus the geometry of the LED array is exposed to the viewer.
Referring now to the drawings, wherein like reference numerals designate identical or corresponding parts, and more particularly to
In one embodiment, the invention is a process that computes where the flare from the LEDs would be visible, adjusting the LED drive levels until the flare should not be visible, and adding additional simulated veiling glare to the image to simulate a bright small feature. The added glare is then adjusted by the LED backlight simulation to produce a stable glare shape that hides the LED array geometry. An exemplary process, that is performed for example in a processor and/or controller of a display is illustrated in
Relying entirely on the ideal veiling luminance capability of the display is not preferred because HDR displays may have difficulty in achieving their peak brightness for all feature sizes. Instead, small features are quite dim compared to large features. Thus, for small features, the contrast ratio of the LCD panel provides high frequency (spatial) details.
As noted above, current LCDs do not block all light, thus when the LCD is set to black, light from the LED backlight is attenuated but not completely extinguished. Bright LEDs are used to illuminate small bright features (it is not sufficient to have only large bright features, small bright features are required too). Unfortunately, even when the LCD is set to full black, some light comes through.
Thus, when illuminating a small bright feature, such as circle, on a black (dark) background, three regions are generally observed:
The Walking LEDs problem is magnified by attempts to brightly illuminate small bright features. However, a significant component of the problem is the down-sample scheme used to compute LED drive values from the input image.
In the various implementations of algorithms designed for Blur Correction, the input image is scaled (averaged) with some amount of filtering from the resolution of the LCD to the resolution of the LED Back Light Unit (BLU) array. For example, down sampling scheme can be essentially a box filter (or any other filter that computes LED target values)—such an implementation results in a system where small changes in the input image, such as the movement by one pixel of a small bright feature on black, can cause LED “target values” to jump to or from zero (off).
Using the Brightside DR37-P display processor, it is possible to “over-drive” LEDs to sufficiently illuminate isolated small bright features. The reference implementation in Matlab, and the normal operation of the DR37-P display processor, uses the block average luminance level around an LED to determine the LED drive level. Thus small bright features are typically under illuminated and as larger brighter features move closer to small bright features the small features increase in brightness. This change in brightness is undesirable, and the skirt artifact is an unintentional side effect of attempting to fully illuminate small features.
Following the down sample, the LED drive values are computed by an “exchange” process which attempts to take in to account the amount of light contributed by the neighboring LEDs. The exchange step can be thought of as a sharpening filter which decreases LED drive values in regions of uniformity, and increases drive values at edges or isolated features. Because LED drive values are restricted to the range [0.0, 1.0] it is possible for a single LED to jump between off and fully on from one frame to the next.
In one embodiment, the present invention may be embodied, for example, in the following steps:
The result is a display with simulated flare in regions where viewers should have experienced real flare, sufficient to mask remaining LED skirts.
Representations:
The most expensive parts of this computation are in steps 4 and 8 where the veiling glare of the display is calculated. Rather than use a relatively large glare filter at the full resolution of the LCD panel, separate the glare filter into a low frequency and a high frequency components and
The next most expensive parts of this computation are in steps 1 and 6 where the backlight is simulated. One option is to use the results of step 1 and only adjust it where in step 6 LEDs have changed in value by a significant amount (or any amount). This restricts light field simulation computation for LED values that change, rather than for all LEDs of the display. However, enough processing power should be provided to compute the entire backlight for any frame of input.
Finally, rather than compute the initial LCD1 and B1 in step 1 using the standard method, one alternative is to start with a large error (e.g., turning on all/or many LEDs) and letting the algorithm dampen them down (steps 2-9).
The mitigation algorithm is very likely to be sensitive to the down sample algorithm used to initially set the value of the LEDs. Analysis of the performance of the algorithm versus various down sample schemes shows that LEDs will still make sudden transitions from off to on to off given a down sample scheme that is extremely sensitive to the position of the small bright features in an image.
Critical parameters are the veiling luminance function (although this is approximately the same function for a very wide class of observers and is not tied to a specific display).
A mitigation technique implementing the present invention includes a process for solving the problem of illuminating a small bright feature on black surround. The process first reviews/determines a predicted veiling glare for image features, and suppresses LED skirts that exceed it.
The process then adds in a simulation of the flare that should be present from the missing stimulus. The process has an added benefit of simulating sources much brighter than could normally be represented, such as the sun or other intense highlights.
An exemplary mitigation technique according to the invention comprises the steps of:
The convolution kernel of step (4) may be expressed, for example, as:
Another possible convolution would be similar to:
Eccentricity (angle) is expressed in degrees from each pixel, which is calculated based on an expected viewing distance. Max angle is typically between approximately 1 and 4 LED spacings and based on viewing distance, and is set, for example, to 7 degrees, or where the convolution formula drops to less than ½ of a percent of its maximum at angle=0.
The result of the process is a display with simulated flare in regions where viewers should have experienced real flare, sufficient to mask remaining LED skirts.
The processes or techniques described above may, for example, be implemented in a dual modulation display that comprises, for example, a structure 300 as illustrated in
A backlight interface 33C provides data for driving an LED array 350, and an LCD interface is configured to drive an LCD of a front panel 360. The LED array 330 and LCD of front panel 360 provide dual modulation as computed/adjusted according to one or more of the above described processes techniques.
In describing preferred embodiments of the present invention illustrated in the drawings, specific terminology is employed for the sake of clarity. However, the present invention is not intended to be limited to the specific terminology so selected, and it is to be understood that each specific element includes all technical equivalents which operate in a similar manner. For example, when describing an LED BLU, any other equivalent device, such as laser or silicon based light arrays, silicon reflective arrays (e.g., LCoS), laser on DLP, e-paper, organic light sources (e.g., OLED), or other light source devices having an equivalent function or capability, whether or not listed herein, may be substituted therewith. Furthermore, the inventors recognize that newly developed technologies not now known may also be substituted for the described parts and still not depart from the scope of the present invention. All other described items, including, but not limited to dual modulation display systems, samplers, filters, LCDs, LEDs, etc should also be considered in light of any and all available equivalents.
Portions of the present invention may be conveniently implemented using a conventional general purpose or a specialized digital computer or microprocessor programmed according to the teachings of the present disclosure, as will be apparent to those skilled in the computer art.
Appropriate software coding can readily be prepared by skilled programmers based on the teachings of the present disclosure, as will be apparent to those skilled in the software art. The invention may also be implemented by the preparation of application specific integrated circuits or by interconnecting an appropriate network of conventional component circuits, as will be readily apparent to those skilled in the art based on the present disclosure.
The present invention includes a computer program product which is a storage medium (media) having instructions stored thereon/in which can be used to control, or cause, a computer to perform any of the processes of the present invention. The storage medium can include, but is not limited to, any type of disk including floppy disks, mini disks (MD's), optical discs, DVD, HD-DVD, Blue-ray, CD-ROMS, CD or DVD RW+/−, micro-drive, and magneto-optical disks, ROMs, RAMS, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices (including flash cards, memory sticks), magnetic or optical cards, SIM cards, MEMS, nanosystems (including molecular memory ICs), RAID devices, remote data storage/archive/warehousing, or any type of media or device suitable for storing instructions and/or data.
Stored on any one of the computer readable medium (media), the present invention includes software for controlling both the hardware of the general purpose/specialized computer or microprocessor, and for enabling the computer or microprocessor to interact with a human user or other mechanism utilizing the results of the present invention. Such software may include, but is not limited to, device drivers, operating systems, and user applications. Ultimately, such computer readable media further includes software for performing the present invention, as described above.
Included in the programming (software) of the general/specialized computer or microprocessor are software modules for implementing the teachings of the present invention, including, but not limited to, computing/simulating image backlights and final displays, computations for identifying, adding, subtracting, convolving, and comparing any of images, image features, aberrations, flares, glares, skirts, veils and the display, storage, or communication of results according to the processes of the present invention.
The present invention may suitably comprise, consist of, or consist essentially of, any of element, part, or feature of the invention and their equivalents. Further, the present invention illustratively disclosed herein may be practiced in the absence of any element; whether or not specifically disclosed herein. Obviously, numerous modifications and variations of the present invention are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims, the invention may be practiced otherwise than as specifically described herein.
This application claims priority to U.S. Provisional Patent Application No. 61/020,104, filed Jan. 9, 2008, hereby incorporated by reference in its entirety.
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/US2009/030207 | 1/6/2009 | WO | 00 | 7/1/2010 |
Publishing Document | Publishing Date | Country | Kind |
---|---|---|---|
WO2009/089211 | 7/16/2009 | WO | A |
Number | Name | Date | Kind |
---|---|---|---|
4816876 | Pryor | Mar 1989 | A |
5978142 | Blackham et al. | Nov 1999 | A |
6891672 | Whitehead et al. | May 2005 | B2 |
7692622 | Kim et al. | Apr 2010 | B2 |
8111234 | Tokuyasu | Feb 2012 | B2 |
20050110742 | Arakawa | May 2005 | A1 |
20050237293 | Kim | Oct 2005 | A1 |
20060103621 | Feng | May 2006 | A1 |
20060125775 | Itoh | Jun 2006 | A1 |
20060239033 | Jung et al. | Oct 2006 | A1 |
20060261754 | Lee | Nov 2006 | A1 |
20070285379 | Jung | Dec 2007 | A1 |
20100289833 | Budzelaar et al. | Nov 2010 | A1 |
Number | Date | Country |
---|---|---|
101086573 | Dec 2007 | CN |
2058792 | May 2009 | EP |
2005-258403 | Sep 2005 | JP |
2005-309338 | Nov 2005 | JP |
2007-240858 | Sep 2007 | JP |
2007-322882 | Dec 2007 | JP |
2009-116012 | May 2009 | JP |
2139559 | Oct 1999 | RU |
2226708 | Apr 2004 | RU |
0203687 | Jan 2002 | WO |
03077013 | Sep 2003 | WO |
2005036870 | Apr 2005 | WO |
Entry |
---|
Seetzen H et al,“High Dynamic Range Display Systems”, Proceedings ACM Siggraph, Dec. 9, 2004, pp. 1-9. |
Feng Li Chester F Carlson Center for Imaging Science et al,“66.1: Distinguished Student Paper: Deriving LED Driving Signal for Area-Adaptive LED Backlight in High Dynamic Range LCD Displays”, SID 2007, 2007 SID International Symposium, Society for Information Display, Los Angeles, USA, vol. XXXVIII, May 20, 2007, pp. 1794-1797. |
Seetzen H et al,“54.2: A High Dynamic Range Display Using Low and High Resolution Modulators”, 2003 SID International Symposium Digest of Technical Papers, Baltimore, MD, May 20-22, 2003, San Jose, CA, USA, vol. XXXIV, No. 1, May 20, 2003, pp. 1450-1453. |
Fang-Cheng Lin et al:“39,4: Inverse of Mapping Function (IMF) Method for Image Quality Enhancement of High Dynamic Range LCD TVs”, SID 2007, 2007 SID International Symposium, Society for Information Display, Los Angeles, USA, vol. XXXVIII, May 20, 2007, pp. 1343-1345. |
Seetzen H et al,“Self-Calibrating Wide Color Gamut High Dynamic Range Display”, IS&T/SPIE's Annual Symposium on Electronic Imaging, XX, XX, vol. 6492, 12 Fe. 2007, pp. 649202/I-9. |
Number | Date | Country | |
---|---|---|---|
20100277515 A1 | Nov 2010 | US |
Number | Date | Country | |
---|---|---|---|
61020104 | Jan 2008 | US |