Patent Application
20180366050
This disclosure relates generally to reproducing images on electronic display devices. More particularly, but not by way of limitation, this disclosure relates to reproducing images on high dynamic range (HDR) electronic display devices that are often connected to and/or integrated within a variety of electronic devices that include, but are not limited to mobile phones, tablet computer systems, laptop computer systems, televisions, and display monitors.
Today's electronic display devices are typically connected to and/or embedded within a wide variety of electronic applications, such as computer monitors, televisions, instrument panels, signage, gaming devices, clocks, watches, and mobile electronic devices. One common type of electronic display device is a liquid crystal display (LCD) that typically displays visual images to a viewer by modulating the intensity of light emitted from a light source. Although the use of LCDs continues to spread in popularity, LCDs, however, suffer from to a variety of technological challenges. For instance, a typical single panel LCD may have a limited dynamic range (e.g., about 1000:1) that adversely affects a viewer's perceived image quality when viewing images on the LCD. In particular, the single panel LCD may suffer from a reduction in the number of gray levels the LCD is able to reproduce and may also impair the visibility of dark areas of an image.
One approach to improve a LCD's dynamic range is to implement a dual layer LCD. A dual layer LCD is able to improve the black level of an LCD by stacking two liquid crystal panels in a series configuration. In comparison to a single panel LCD, a dual layer LCD is able to produce a more accurate grayscale by modulating the light from a light source twice. Even though dual layer LCDs improve an electronic display device's grayscale, dual layer LCDs may also experience other implementation and technological challenges. For instance, the distances from the two liquid crystal panels can cause parallax problems when a viewer observes the LCD off-axis. Additionally, while a dual layer LCD permits darker black colors, the dual layer LCD may be unable to display a wide variety of other saturated dark colors (e.g., non-black dark colors) in a display's color gamut. As such, improving the gamut size and minimizing parallax errors may be beneficial in enhancing a viewer's perceived image quality when displaying images on electronic display devices.
In one embodiment, the disclosed subject matter provides a method to improve gamut size for a multi-layer display device. The method includes receiving a color input value indicative of a target display color associated with an input image. The method may then determine a color saturation value for the received color input value. The color saturation values may be used to compute drive values for a monochromatic modulation panel and a color modulation panel. Afterwards, the monochromatic modulation panel and the color modulation panel may be illuminated based on the drive values. The method drives the monochromatic modulation panel and the color modulation panel such that the monochromatic modulation panel is not modulated until the color modulation panel is fully illuminated.
In another embodiment, the method improves gamut size for a multi-layer display device by minimizing the modulation of the color modulation panel and maximizing modulation of the monochromatic modulation panel. A method in accordance with this approach includes receiving a color input value indicative of a target display color associated with an input image at a per-pixel or per-panel element basis. The method may then determine a color saturation value for the received color input value and compute a monochromatic drive value that is a linear weighted sum of color saturation value. The monochromatic modulation panel may be driven by the monochromatic drive value, where the monochromatic modulation panel is not modulated until the color modulation panel is fully illuminated. To minimize parallax issues, the method may also apply a radial distortion to an image generated from the monochromatic modulation panel.
In yet another embodiment, the method reproduces dark saturated colors and reduces parallax artifacts on a dual layer LCD. This approach involves using a neutral back liquid crystal panel to lower luminance and a front liquid crystal panel to produce color. The back liquid crystal panel may serve as a monochromatic light shutter that modulates light to produce a gray scale. The front liquid crystal panel may serve as a chroma light shutter with a relatively higher resolution than the back liquid crystal panel. To expand the available gamut, the dual layer LCD may minimize the use of the front liquid crystal panel while maximizing the use of the back liquid crystal panel by determining a saturation value to drive both the back liquid crystal panel and the front liquid crystal panel. To reduce halos or parallax experienced by a viewer, a radial distortion may be applied to a back panel image to align the back liquid crystal panel and the front liquid crystal panel local with respect to a viewer's view point.
In one or more embodiments, each of the above described methods, and variation thereof, may be implemented as a series of computer executable instructions. Such instructions may use any one or more convenient programming language. Such instructions may be collected into modules and/or programs and stored in any media that is readable and executable by a computer system or other programmable control device.
This disclosure includes various example embodiments that reproduce dark saturated colors and minimizes parallax problems associated with image splitting for electronic display devices. In one embodiment, a dual layer liquid crystal display (LCD) reproduces dark, non-black, saturated colors (e.g., dark red, blue, and/or green) using a neutral back liquid crystal panel to lower luminance and a front liquid crystal panel to produce color. The back liquid crystal panel may serve as a monochromatic light shutter that modulates light to produce a gray scale. The front liquid crystal panel may serve as a chroma light shutter with a relatively higher resolution than the back liquid crystal panel. Based on the color of an input image, the dual layer LCD may determine how much to drive the two different liquid crystal panels. The dual layer LCD may drive the front liquid crystal panel and the back liquid crystal panel on a per pixel basis and/or a per back liquid crystal panel element when the back liquid crystal panel and the front liquid crystal panel do not have the same resolution. To expand the available gamut, the dual layer LCD may minimize the use of the front liquid crystal panel while maximizing the use of the back liquid crystal panel. Additionally, to reduce halos or parallax experienced by a viewer, a radial distortion may be applied to the back panel image to align the back panel and the front panel with respect to a viewer's view point.
In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the inventive concept. As part of this description, some of this disclosure's drawings represent structures and devices in block diagram form in order to avoid obscuring the invention. In the interest of clarity, not all features of an actual implementation are described. Moreover, the language used in this disclosure has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter, resort to the claims being necessary to determine such inventive subject matter. Reference in this disclosure to “one embodiment” or to “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention, and multiple references to “one embodiment” or “an embodiment” should not be understood as necessarily all referring to the same embodiment.
It will be appreciated that in the development of any actual implementation (as in any development project), numerous decisions must be made to achieve the developers' specific goals (e.g., compliance with system- and business-related constraints), and that these goals may vary from one implementation to another. It will also be appreciated that such development efforts might be complex and time-consuming, but would nevertheless be a routine undertaking for those of ordinary skill in the design and implementation of detecting motion having the benefit of this disclosure.
The light source and diffuser 102 may include a backlight unit that emits light that travels along the optical path in a direction towards the color filter layer 108. The light from the backlight unit may illuminate images that the back modulation panel 104 and the front modulation panel 106 produce for the viewer. By way of example, the backlight unit may be a direct light source type located directly behind the back modulation panel 104 or an edge light source type located at the edges of a screen. The backlight unit may include light emitters, such as cathode fluorescent lamps, light emitting diodes (LEDs) (e.g., a row or full array of white LEDs) organic LEDs (e.g., RGB LEDs)), quantum dot structures, solid state lasers, and/or any other known type of light source that emits lights for electronic display devices. The light source and diffuser 102 may also include one or more diffuser layers that scatter the light from the backlight unit and assist in homogenizing light that the backlight unit emits in order to reduce hotspots.
In one embodiment, light source and diffuser 102 may generate light from a backlight unit using quantum dots. The quantum dots may have a spectrum with wavelength peaks that align with the colors corresponding to the color filter elements in the color filter layer 108. The backlight unit of the light source and diffuser 102 may include a light source and a quantum dot structure. The light source may include one or more blue light-emitting diodes that produce blue light. The quantum dot structure receives the blue light from the light source and outputs corresponding light at wavelengths associated with the R, G, and B color filter elements in the color filter layer 108. The diffuser, which may be interchangeable and also referred to within this disclosure as “a light guide plate,” may distribute and scatter the light emitted from the backlight unit into the optical path. By doing so, the diffuser is able to evenly distribute light that travel along the optical path.
The back modulation panel 104 may modulate the intensity of the backlight received in the optical path in order to display pixels of varying shades of gray that range from black to white. The back modulation panel 104 may be generally referred to as a monochromatic stage, shutter stage, and/or a localized dimming stage that enhances the dynamic range of the multi-layer display system 100. For example, the back modulation panel 104 may provide a local dimming effect for the dark areas of a projected image. The local dimming effect delivers additional darkening of pixels in dark areas generated with the front modulation panel 106. As a monochromatic stage, the back modulation panel 104 may be configured to not impart color information to a viewer, and may not include a color filter layer and/or other types of color filter elements.
The front modulation panel 106 may be referred to as a color stage with pixels that create color images. As shown in
In one embodiment, the back modulation panel 104 and the front modulation panel 106 may each include a liquid crystal layer, such as an active matrix liquid crystal layer, a trans-reflective liquid crystal layer, and/or window liquid crystal layer, to modulate the intensity of light, which can be measured in terms of transmittance and/or luminance. The liquid crystal layer may comprise liquid crystals (e.g., twisted nematic) that orient and twist based on applying varying levels of current to the liquid crystal layer. For example, electric power that supplies current to the liquid crystal layer may cause one or more of the liquid crystals to untwist and orient themselves to block light from passing through. In areas where the liquid crystal layer allows for light that passes through, the orientation of the liquid crystals cause the passing light to rotate or change polarization. The back modulation panel 104 and the front modulation panel 106 may each include one or more polarizer layers. For example, the back modulation panel 104 may include a polarizer layer positioned between the liquid crystal layer and the light source and diffuser 102 and a second polarizer layer positioned between the liquid crystal layer and the front modulation panel 106.
In one embodiment, the back modulation panel 104 may be set to have a relatively lower pixel resolution than the front modulation panel 106. For example, the front modulation panel 106 may be set to produce an overall display resolution desired for the multi-layer display system 100 while the back modulation panel 104 may be set to produce a resolution less than overall display resolution. In embodiments where the back modulation panel 104 has a lower pixel resolution than the front modulation panel 106, the back modulation panel 104 may include arrays of local dimming elements that have a pixel-to-pixel spacing, which may also be referred to as “pixel pitch,” greater than the pixel-to-pixel spacing of the front modulation panel 106. By doing so, each of the local dimming elements may overlap and correspond to multiple higher-resolution pixels of the front modulation panel 106. For example, each local dimming element of the back modulation panel 104 may overlap a 2×2 subarray of higher-resolution pixels within the front modulation panel 106. In this instance, the back modulation panel 104 may perform local dimming operations for a set of four higher-resolution pixels of the front modulation panel 106. Other embodiments of the multi-layer display system 100 may differ in the number of higher-resolution pixels of the front modulation panel 106 that overlap and correspond to the local dimming pixels of the back modulation panel 104.
As show in
When performing image splitting operations on input image data, the multi-layer display system 100 may generate at least two coupled images, one for the back modulation panel 104 and another for the front modulation panel 106. Specifically, the multi-layer display system 100 may split and process a given input image (e.g., HDR input image) into two complimentary images, where one of the images is sent as monochromatic data and the second image is sent as color data. The display controller 110 may activate the back modulation panel 104 based on the monochromatic data and activate the front modulation panel 106 based on the color data. The multi-layer display system 100 is able to accurately reproduce the input image data to a viewer by combining the two coupled images.
In one embodiment, rather than performing a square root image splitting algorithm that splits transmission of the back modulation panel 104 and the front modulation panel 106 evenly, the display controller 110 may perform an image split operation that minimizes the use of the back modulation panel 104 and relies on driving the front modulation panel 106 to avoid parallax artifacts. In particular, the display controller 110 may not leverage the back modulation panel 104 to generate neutral colors or bright colors. Neutral colors are colors that have low saturation levels that are below 0.1. Instead, the display controller 110 may use the back modulation panel 104 in situations to achieve a darker black color and drive the back modulation panel 104 to reach full illumination before driving the front modulation panel 106 to produce color. In some instances, to improve image splitting operations, a low-pass filter may be applied to the back modulation panel 104 to blur the back panel image in order to avoid parallax artifacts. While this approach improves obtaining darker black colors, the multi-layer display system 100 may be unable to increase its gamut size for other colors (e.g., dark red, dark blue, or dark green).
To improve gamut size, the display controller 110 may perform image splitting operations that activate the back modulation panel 104 to lower luminance and drive the front modulation panel 106 for color. Based on the input image data, display controller 110 may determine how much to drive the back modulation panel 104 and the front modulation panel 106 by minimizing the use of the front modulation panel 106 and maximizing the use of the back modulation panel 104. By leveraging the back modulation panel 104 and fully using the color gamut of the front modulation panel 106, the multi-layer display system 100 may expand the available gamut by reproducing more dark saturated colors other than black (e.g., dark red, dark blue, or dark green).
Although
Other embodiments of the multi-layer display system 100 may include more than two modulation stages and/or implement one or more of the modulation stages without using a LCD panel. In one embodiment, the multi-layer display system 100 may include three modulation stages, a two dimensional (2D) backlight unit, a back modulation panel 104, and a front modulation panel 106. The 2D backlight unit may be a matrix of LEDs that are locally dimmed to reduce power and the back modulation panel 104 may be selected based on the current 2D LED image. In another embodiment, the multi-layer display system 100 may not include a back modulation panel 104, and instead use the 2D backlight unit to implement local dimming and the front modulation panel 106 for color modulation. The use and discussion of
Image splitting operation 200 can start at block 202 by receiving color input data corresponding to a received input digital image. Operation 200 may receive the color input data on a per-pixel and/or on a per-panel element basis. In one embodiment, operation 200 may receive color input data on a per-panel element when at least one of the modulation panels has a resolution that differs from the resolution of another modulation panel. For example, a monochromatic modulation panel may be set to a lower resolution than the color modulation panel such that each monochromatic modulation panel element overlaps with four color modulation panel pixels. Operation 200 may receive the color input data on a per-pixel basis, when the modulation panels have about the same resolution. For example, the monochromatic modulation panel may be set to about the same resolution as a color modulation panel such that each monochromatic modulation panel pixel corresponds to one color modulation panel pixel.
Image splitting operation 200 may then move to block 204 and determine a saturation value for the received color input data. Similar to block 202, the image splitting operation 200 may determine the saturation value on a per-pixel or per-panel element basis. In one embodiment, the saturation value for a pixel or panel element may be a defined by the following equation:
color delta/color sum (1)
where color delta represents the difference between the maximum of the three RGB color values and the minimum of the three RGB colors values of the pixels, and the color sum is determined from adding the maximum of the three RGB color values and the minimum of the three RGB colors values of the pixels. In one embodiment, when determining the color delta and color sum, the luminance values of the different pixels may be normalized values. Other embodiments of block 204 may adopt other well-known method of determining saturation. For example, image splitting operation 200 may convert the three RGB color values to the International Commission on Illumination LCH (CIELCH) color space, and then use chroma as saturation.
Image splitting operation 200 may then move to block 206 to determine a drive value for the monochromatic modulation panel. In one embodiment, the image splitting operation 200 determines the drive of the monochromatic modulation panel using a linearly weighted sum determined from the saturation value obtained in block 204. In particular, the drive value for the monochromatic modulation panel may be defined by equation 2:
Drive=(1-SV) (mmp starting point min)+(SV) (mmp_starting_point max) (2)
where “SV” refers to the saturation value determined in block 204, the “mmp starting point min” refers to the maximum drive value to produce the lowest luminance for the monochromatic modulation panel, and the “mmp starting point max” refers to the minimum drive value to produce full luminance for the monochromatic modulation panel. Other embodiments of block 206 may perform other well-known linear and/or non-linear functions using the saturation value to determine a drive value for the monochromatic modulation panel. Examples of this includes replacing SV with SV̂n for some power of n or use a piecewise function where if SV is less than a threshold value, then the drive is set to the maximum drive value, and if SV is greater than equal to the threshold value, then the drive is set to a minimum drive value. Basing the drive value on the saturation levels may allow the monochromatic modulation panel to activate in dark regions and maintain color purity in order to obtain dark saturated colors other than black. For input image data that is scaled between 0 and 1000, dark regions are located between the 0-1 range.
The drive level for the color modulation panel may be based on the drive level of the monochromatic modulation level. For example, the image splitting operation 200 may determine the color modulation panel drive value on a pixel-by-pixel that involves a color transformation matrix. The color transformation matrix may be based on a target color in the XYZ space and the monochromatic modulation panel's luminance, which is defined by defined by equation 3:
where [Xin, Yin, Zin] represents the input or target XYZ signal, LB represents a linear monochromatic modulation panel luminance and/or transmittance signal, Ak represents the color modulation panel drive value, [Rf, Gf, Bf] represents the color modulation panel drive value, [Xk, Yk, Zk] represents the minimum leakage black level for the color modulation panel drive and
represents the color transformation matrix. Solving for the color modulation panel drive value, [Rf, Gf, Bf] produces equation 4:
where A′ equals (LB(1−Ak)+Ak)).
Afterwards, the image splitting operation 200 may move to block 208 and drive the monochromatic modulation panel and a color modulation panel according to the drive values. The image splitting operation 200 may directly drive the monochromatic modulation panel using the drive value computed in block 206. To drive the color modulation panel, the image splitting operation 200 may apply a color panel algorithm that uses the drive value found in block 206 to determine a drive value for the color modulation panel, such as equation 4 as shown above.
Image splitting operation 200 may then move to block 210 and apply a radial distortion, such as a radial translation filter, to the monochromatic modulation panel in order to align the images generated from the two modulation panels. Image splitting operation 200 may align the image locally with respect to a viewer's view point. To perform the local alignment, one or more embodiments of the image splitting operation 200 may perform eye-tracker algorithm known by persons or ordinary skill in the art. For example, a gaze tracker operation may utilize a camera to capture an image of the user's eye and an image processing algorithm that analyze the image, extract image features and applies the image features to minimize parallax error. In other words, the gaze tracker operation may locate the position of the user and shift the images on each modulation panel by applying the radial distortion to the monochromatic modulation panel. By tracking a variety of factors associated with the viewer that include, but are not limited to gaze points, pupil size, frequency of blinking, emotional state of the user, and/or position of the user relative to the screen, the images from the modulation panels may be locally aligned with respect to the viewer's view point. In one embodiment eye tracking could employ infra-red light source and/or cameras to track eye movement.
To obtain darkly saturated colors and improve gamut range, the multi-layer display system may maintain a neutral monochromatic modulation panel while activating the color modulation panel.
As shown in
Image data can then be split into two channels by image splitter 920. Image splitter 920 may, for example, provide a high resolution color image data component of the image data to color modulation panel 922 while simultaneously providing a low resolution monochromatic localized dimming component of the image data to monochromatic modulation panel 924. The local dimming channel of the image data may be derived from the square of luminance Y in the XYZ color space. The color image data channel of the image data may be formed by dividing the Y channel by the square of Y and using this new Y data with corresponding X and Z data to form the final color image data. When displaying the color component of the image on the color modulation panel 922 and the lower-resolution local diming component of the image on the monochromatic modulation panel 924, gamma look-up tables may be used to convert the data from image splitter 920 into WRGB data.
As used herein, the term “dynamic range” of an electronic display refers to the ratio of the highest luminance portion (i.e., brightest portion) of an output of the electronic display and the lowest luminance portion (i.e., darkest portion) of the output of the electronic display.
As used herein, the term “dual-layer” can be interchanged and be generally referred throughout this disclosure as “dual-cell.”
Referring to
Processor 1005 may execute instructions necessary to carry out or control the operation of many functions performed by device 1000. Processor 1005 may, for instance, drive display 1010 and receive user input from user interface 615. The display 1010 may be a multi-layer display system as described in
Sensor and camera circuitry 1050 may capture still and video images that may be processed, at least in part, in accordance with the disclosed techniques by video codec(s) 1055 and/or processor 1005 and/or graphics hardware 1020, and/or a dedicated image processing unit incorporated within circuitry 1050. Images so captured may be stored in memory 1060 and/or storage 1065. Memory 1060 may include one or more different types of media used by processor 1005 and graphics hardware 1020 to perform device functions. For example, memory 1060 may include memory cache, read-only memory (ROM), and/or random access memory (RAM). Storage 1065 may store media (e.g., audio, image and video files), computer program instructions or software, preference information, device profile information, and any other suitable data. Storage 1065 may include one or more non-transitory storage mediums including, for example, magnetic disks (fixed, floppy, and removable) and tape, optical media such as CD-ROMs and digital video disks (DVDs), and semiconductor memory devices such as Electrically Programmable Read-Only Memory (EPROM), and Electrically Erasable Programmable Read-Only Memory (EEPROM). Memory 1060 and storage 1065 may be used to tangibly retain computer program instructions or code organized into one or more modules and written in any desired computer programming language. When executed by, for example, processor 1005 such computer program code may implement one or more of the operations described herein.
It is to be understood that the above description is intended to be illustrative, and not restrictive. The material has been presented to enable any person skilled in the art to make and use the claimed subject matter as described herein, and is provided in the context of particular embodiments, variations of which will be readily apparent to those skilled in the art (e.g., some of the disclosed embodiments may be used in combination with each other). For example, while
At least one embodiment is disclosed and variations, combinations, and/or modifications of the embodiment(s) and/or features of the embodiment(s) made by a person having ordinary skill in the art are within the scope of the disclosure. Alternative embodiments that result from combining, integrating, and/or omitting features of the embodiment(s) are also within the scope of the disclosure. Where numerical ranges or limitations are expressly stated, such express ranges or limitations may be understood to include iterative ranges or limitations of like magnitude falling within the expressly stated ranges or limitations (e.g., from about 1 to about 10 includes, 2, 3, 4, etc.; greater than 0.10 includes 0.11, 0.12, 0.13, etc.). The use of the term “about” means ±10% of the subsequent number, unless otherwise stated.
Many other embodiments will be apparent to those of skill in the art upon reviewing the above description. The scope of the invention therefore should be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. In the appended claims, the terms “including” and “in which” are used as the plain-English equivalents of the respective terms “comprising” and “wherein.”
| Number | Date | Country | |
|---|---|---|---|
| Parent | 15179811 | Jun 2016 | US |
| Child | 16048803 | US |
