Patent Grant
10964240
Display calibration has historically required either complex circuitry to perform or resulted in large errors. As people rely more on the color accuracy of the display to make decisions, display manufacturers have implemented per unit display calibration on the factory floor. The current state-of-the-art for display calibration hardware is to use a three-dimensional look up table (3D LUT) to store the corrections. The display manufacturer measures several different colors, generates the entries for the 3D LUT and then programs it. The number of measurements can be as few as a couple of dozen and as high as thousands depending on the sophistication of the algorithm used to generate the entries in the 3D LUT. Other methods such as polynomial color space converters can be difficult to program and do not provide as accurate results.
Modern displays, whether they are LCD based or OLED require a conversion of the input values that represents the desired color to the voltages or currents required to generate the desired optical output at the pixel. The Display Driver IC (DDIC) typically handles this process, referred to as Electro-Optical Transfer Function (EOTF), as part of the digital to analog conversion process. This non-linear process is designed to provide the correct response at the individual sub pixel level and to create a gamma response to the input similar to a cathode-ray tube (CRT) which has historically been assumed in color standards. A sub pixel represents how the display reproduces just one of the color components that make up the color for a pixel. For example, a pixel usually has a red, green and blue color sub pixel for each pixel to produce the corresponding color components. However, this is not always the case. Sometimes a pixel has only two color components with adjacent pixels having different combinations. For example, OLED displays can have a red-green pixel and a blue-green pixel. Other displays might also provide a white sub pixel in addition to the red, green and blue sub pixels. Regardless of the pixel configuration, the DDIC provides the appropriate driving voltage.
In an ideal world, the output of a sub-pixel would be independent of the content of the other sub pixels in the pixel, and other pixels in the display. That is, the amount of red measured for a pixel would only depend on the value of the red component of the color being transmitted to the red sub pixel. Unfortunately, this is not the case, and this is why displays require complex circuitry to calibrate. For example, in an LCD display you might get leakage from the liquid crystal shutter behind the green subpixel on the color filter through the red portion of the color filter for that pixel. That would result in the amount of red being measured being larger when displaying yellow than just red. For OLED's, the circuitry that determines the brightness of the sub pixel can be influenced by the overall brightness of the screen. This results in a pure red display having a brighter red component than when white is being shown. These errors are referred to as cross-talk and requires a process referred to as cross-talk correction.
A method to calibrate displays with simpler circuitry and more accurate color space conversion and cross-talk correction without added circuitry to keep costs down.
Some of the embodiments here involve transforming an input signal with a non-linear function to apply color space conversion, then transforming that signal with a different non-linear function to apply cross-talk correction. Finally the data is transformed by an inverse of the panel's Electrical to Optical Transfer Function (EOTF) before being sent to the display.
Other embodiments only apply a cross-talk correction, but have the parameters of that correction change based upon a summary statistic of the panel referred to as the Adjusted Average Picture Level (AAPL). In some embodiments, measurements of the color components displayed on the panel are used to generate non-linear models that have the best fit of the parameters of the non-linear model to the measured data, instead of simply interpolating between the measured data points. The measured data may be transformed in addition to the input signal transformations to account for differences between the measurement environment and the final product environment.
Much of the below discussion will refer to “color components.” It should be noted that color components are not a single wavelength, but a distribution of wavelengths. Part of the color calibration process is to determine not only the main wavelength of the color component, but also the purity of the color component. This is summarized in the xy chromaticity coordinates of the color. The xy coordinates are normalized XYZ measurements of the displays such that x+y+z=1. The XYZ measurements are a device independent description of the color and can be converted to the display RGB values using a 3×3 color conversion matrix multiply. The coefficients for the conversion can be derived from the xy chromaticity coordinates when measuring the red, green and blue values from the display when only one color component is displayed, and white when all 3 color components are displayed.
The discussion below refers to measuring a color component, or the optical output of a subpixel or a group of sub-pixels. The actual measurement may be using the XYZ device independent color, the spectrum of the color or some other description of the color and that is then converted to a value that represents the measurement of the output of the individual color component. This conversion requires the chromaticities for the primary colors and white, values that can be obtained from the XYZ measurements. Therefore, when the discussion refers to measuring the color component, in reality the XYZ color is measured and the optical output of the subpixel, the color component measurement, is calculated.
Furthermore, when referring to a color component, the discussion refers to the less than pure color component of the display, and not a single wavelength, or a color standard defined color. This means that the crosstalk only measures the interaction among sub pixels and other pixels in the display and is not a measure of how pure the colors are or how accurate they are with respect to a color standard. It will be assumed that the input signal has already be converted from the standard RGB values to the display RGB values using the chromaticity measurements before applying the correction for the crosstalk as described in
The circuitry and process of
In the discussion that follows, a primary color is displayed when only one of the subpixels is illuminated. For an RGB pixel arrangement, the primary colors are Red, Green and Blue. Complementary colors are when only two of the sub pixels are illuminated. That is, red plus green equals yellow, red plus blue equals magenta, and blue plus green equals cyan. Finally, when all the sub pixels are illuminated, they display white or gray. In addition, display calibration is usually done by measuring colors with a fixed window size. The window size can be anything from 1% to 100% of the display area. When measuring with a fixed window size the amount of current used in the display varies based on the color being displayed. This is one reason why the peak amplitude and electro-optical transfer function (EOTF) of the panel changes based on the color. The first part of the discussion below will describe how calibration can be done using a fixed window size measurement, the second part will then modify the approach to dynamically change the parameters based on the amount of panel current. While the discussion describes the embodiments by using the term gamma to make the description easier to follow, any non-linear function could be used as long as it accurately describes the response of the display.
One important factor in simplifying the calibration is that even though the peak brightness and the response of the signal is changing, the signal can still be accurately described using a gamma function. A gamma function, or gamma correction function, maps luminance levels to compensate the non-linear luminance effect of display devices, where gamma, γ, is a constant and x in the input value. The signal can be described using the relationship p=axγ, where p is the output in terms of photons and a is an amplitude. The a's and γ's change based on the amount of content being displayed both at the sub pixel and the overall display. If one wants the same amount of a color component when the display displays a primary color versus white in the same size window, then the following relationship exists:
po=xwγw=acxcγc;
where the w subscript designates a white color and the c subscript designates a primary color component. In some of the below discussion, when more than one gammas for the color components are used, they will be identified with subscripts r, g, and b. For most OLED displays, ac>1 and γw<γc.
However, there is only one input regardless of what color is being displayed which should equal the input required for white, xw, therefore the above equation needs to be rewritten and ga and ya found where the following is true for all xw. The entity ga is a gain function.
po=xwγw=ac(gaxwγw·γa)γc/γw (3)
expanding the left hand side results in:
xwγw=ac(gaγc/γw)(xwγw·γa)γc/γw=ac(gaγc/γw)(xwγa·γc) (4)
which is only true if ya is equal to yw/yc, resulting in the following relationships:
1=ac(gaγc/γw), or ga=(1/ac)γw/γc (5)
po=xwγw=ac(gaxwγw·γw/γc)γc/γw (6)
Therefore by transforming from a linear color space (xwγw) to a slightly nonlinear color space (xwγw·γw/γc) based on the ratio of the gamma for a given color component, γc, when displaying white, versus the same color component when displaying a primary color, it is possible to correct for the crosstalk caused by changes in the gamma and peak amplitude that vary with the purity of the color by multiplying by a constant. While this applies the crosstalk correction with the correct non-linear response, it will still end up being displayed with the wrong gamma. If the gammas are similar, this is not that big of an error and can usually be ignored. Fixing the response completely requires blending between two different gammas after the conversion and those gammas need to be based on how close the color is to a primary color versus white as will be shown later.
This is the basic equation for the crosstalk calculation used to adjust between a pure color component being displayed and when white is displayed. However, other colors are also possible. If the gammas for the complementary colors that include that color component are measured one can do the same calculation. For example, if one wants the crosstalk for red when displaying both red and green one would need the gamma for yellow (red & green) in addition to red and white. Changing the a subscript to rg (yellow), rr (red) and rw (white) to distinguish from the generic example above we get
grr=(1/arr)(γrw/γrr) (7)
grg=(1/arg)(γrw/γrg) (8)
grb=(1/arb)(γrw/γrb) (9)
If one assumes that the crosstalk is additive, one can use these gains in a matrix multiplication. First the process needs to adjust the yellow component so that it becomes the delta between the yellow and the red crosstalk.
g′rg=grg−grr (10)
The process can do the same thing for magenta, which is red and blue components displayed together, and get
g′rb=grb−grr (11)
Finally, because the additive mixture is an approximation, and because the process needs to ensure that white value is unchanged because that is the reference point for all of the calculations, the process can normalize the grr, g′rg and g′rb values so that they add up to 1. This is accomplished by dividing each of crosstalk values by the sum of the values. That is
g″rr=grr/(grr+g′rg+g′rb) (12)
g″rg=g′rg/(grr+g′rg+g′rb) (13)
g″rb=g′rb/(grr+g′rg+g′rb) (13)
Then the CSC matrix multiplication implements the following relationships:
R′=R*g″rr+G*g″rg+B*g″rb
G′=R*g″gr+G*g″gg+B*g″gb
B′=R*g″br+G*g″bg+B*g″bb (14)
However, the amount of crosstalk is not always a linear relationship between the amounts of the output of the other color components. For example, in an OLED, the cross talk can arise as the amount of current in each subpixel and total current in the panel impacts the voltage setting accuracy. For a light emitting diode, the relationship is complex, but can be approximated by a gamma curve, γx. Therefore, the process is really interested in
p′cγx/p′oγx=((xcγw)γx)/p′oγx (15)
Equation 3 doesn't change in this case and substituting xc for that results in the cross talk gain:
p′cγx/p′oγx=((((po/ac)(1/γc))γw)γx)/p′oγx (16)
Again p′o=po(γw/γc) so that the cross talk is constant results in
ga=acγw·γx/γc (17)
po=ac(gxxwγw·γw·γx/γc)γc/(γw·γx) (18)
This then allows the hardware shown in
Because γrwγx/γr, γgwγx/γg and γbwγx/γb are typically close to the same value and the crosstalk coefficients g″rr, g″gg and g″bb have a much larger amplitude than the other coefficients, the hardware of
Because the assumption that the cross talk is an additive function is only approximate, sometimes better results can be obtained if the crosstalk gamma (γx) used to calculate the crosstalk gains denoted as (γx1) is slightly different than the crosstalk gamma (γx2) used to transform the color components before the 3×3 matrix multiply.
ga=acγw·γx1/γc (19)
po=ac(gxxwγw·γw·γx2/γc)γc/(γw·γx2) (20)
Finally, as mentioned above, the output gamma response is not quite correct because the process has done nothing to adjust the gamma in the equations above in response to changes in the color. The simplest correction is to do a linear interpolation between a gamma response with the primary color and the gamma response for a white color. Simplifying equation 20 results in
po=ac(g′axwγw),g′a=gaγc/(γw·γx2) (21)
where po represents the output if the panel had a gamma response of γw, but the response is actually between γc and γw. Therefore, po needs to be converted back to a non-linear response using a value between 1/γc and 1/γw before sending the data to the panel to get the correct response. Because gamma is measured using a pure color and white, and gamma also seems to be related to the average picture level, the function can be approximated by the following (the input signal range is 0-1):
γb=γc+(γw−γc)*(R+G+B−1)/2 (22)
Equation 22 can be implemented with an embodiment of circuitry shown in
Up to this point, the discussion has focused on using the fixed window (or average picture level) approach. In this approach, the data undergoes 2 non-linear transforms into and out of the non-linear crosstalk correction space, and the 3×3 matrix multiply with all of the parameters being fixed regardless of the content being displayed.
As mentioned previously, another approach would be to dynamically change the parameters based on an amount of panel current. The amount of current can be estimated by a process similar to calculating the Average Picture Level (APL). To reinforce that it is based on the average content of the scene, but is calculated slightly differently, it will be referred to as the Adjusted Average Picture Level (AAPL). This approach results from the practical impact that the values for ac, γc and γw are known to change with the average picture level. The optimal γx value also changes, but because this gamma value is always combined with another gamma value, it doesn't impact the complexity. If the calibration is to be accurate for all types of content, then the coefficients for the 3×3 matrix multiply need to change as a function of the AAPL along with the 1D LUT that converts the output to the measured gamma of the panel. This is shown in
In
Mout=h(AAPL)*M1+(1−h(AAPL))*M2, where M1 and M2 are the 3×3 matrix found using the AAPL 1 and 2 respectively. Note that k(AAPL) is not necessarily equal to h(AAPL). The blended gamma function may result from a blend of all of the pixels in an image frame, or may result from a majority of the pixels in an image frame. That is, the gamma function may be determined from a number of pixels less than the number of pixels in an entire image frame, but usually more than 50% of the pixels.
The adjusted average picture level is a proxy for the amount of current and the impact of that current on the crosstalk values. Therefore, when calculating the AAPL a more generalized function than used for the APL can be used for increased accuracy. This will model the impact of the different colors, the different colors requiring a different amount of current for a given luminance output, and the non-linear impact of the color on the cross talk. This is shown in
Y=f(x) represents mapping the input value to a sub pixel current value for each color component, where R′=f(R), as shown at 70. Whatever the input color value is for the entire pixel is broken into its sub pixel values for each component and those values result in some amount of current being applied to the display. This mapping connects the sub pixel color component value to the current applied to achieve that value.
Y=g(x) represents mapping the sub pixel current to a pixel current value at 72. This takes the sub pixel current for each color component and map that current to the overall pixel current value. This is usually just the sum of the individual sub pixel currents, but some cross talk is possible at this point. For example, inductive or capacitive coupling between the traces in the panel used for setting a current or voltage value at an individual sub pixel could result in the total current for white being different than what would be expected by the sum of the pixel currents when displaying the primary colors.
At 76, the pixel currents are summed together to get the total current in the panel. The crosstalk coefficients are blended at 76 at h(AAPL), and the gammas are blended at 78, k(AAPL).
Most of the hardware is not included with typical SOC (system on a chip) product in most displays and would require custom circuitry to implement. But a lot of products do have some type of average picture level (APL) function. Even if the included APL does not sum the color values based on a value proportional to the current required, it will be close enough to provide an improvement for most image content. The final functions h(APL) and k(APL) can be implemented in software, using the APL circuitry, because it only changes once/frame. For example, typical hardware may only calculate the average R (APLR), the average G (APLG) and the average B (APLB) values. The software would need to combine those values together and then calculate a blending percentage, something easily accomplished between frames.
Finally, in a lot of cases, the differences between the gammas for the primary color, complementary colors, and white is the simply the result of differences in the AAPL. If the AAPL is calculated correctly, the gamma for a given color component should be independent of what the other colors are.
In that case, it is possible to further simplify things as shown in
The basis of the non-linear transforms used above, and the adjustments in the AAPL dynamic adjustment lies in measuring the color components. Accurate color calibration requires the correction of both the color crosstalk and deviations from the ideal input to current (or voltage) transform in the DDIC. For the latter, one series of gray values is required to map those deviations. Then, for each AAPL level for which accurate color calibration is required, the 6 primary and complementary colors at max output and a mid-level value are required. This allows the calculation of the γc's and ac's. Next, for a given panel design, the γx value needs to be found that minimizes the errors after calibration. The panel design includes, but is not limited to: technology such as OLED or LCD, the sub pixel layout, the traces on the panel, the resolution or pixel density, etc. These are characteristics that are not expected to change on a per unit basis. Currently finding γx is done by trial and error.
For calibration at a single AAPL level, a total of 21 measurements provides calibration accuracy the equivalent of other methods making close to 100. Note, while using a gamma function to model the non-linearities appears to be the most efficient method because it only requires two measurements to find the gamma parameter, higher accuracy may be possible with more complex non-linear functions that more closely mimic the underlying cause of the crosstalk.
When finding values for the AAPL adjusted method, one embodiment uses a test pattern that keeps the AAPL level for each individual color the same. This means that if you are displaying a primary color in the window, then the background is the opposite complementary color. For example, if the window is red, then the background would be cyan (green plus blue). Furthermore, if the gamma values are the same for primary, complementary and white colors, then it is only necessary to measure the white values and the 3 max primary values to calculate the necessary parameters for the color space conversion and crosstalk correction. This is because only the CSC values and the gammas calculated for a gray scale will be changing as a function of the AAPL.
Depending on the panel technology and manufacturing tolerances it may be possible to do a subset of the measurements. The first reduction in the measurements would be to eliminate the measurement of the complementary colors and estimate those values based on the measurements of the primary colors and white. For example, the complementary color values could be derived as a weighted sum of white and the primary color measurements that are common to the two complementary colors used to calculate the crosstalk correction for the subpixel of the same primary color.
γrg=w1*γrr+w3*γrw (21)
γrb=w4*γrr+w6*γrw (22)
or it could the weighted sum of the two primary colors used to create a complementary color and white.
γrg=w1*γrr+w2*γgg+w3*γrw (23)
γrb=w4*γrr+w5*γbb+w6*γrw (24)
As shown above, if the gamma is adjusted based on the measured AAPL, it may be enough to measure the gamma for the white test pattern only.
In general, the complementary color values could be calculated based on a model using the three primary colors and white. The values in the model are found by measuring several panels and performing a regression analysis fit to the model to the data.
From a practical standpoint, the measurements on the factory floor do not match the conditions of the final product. For example, the factory floor measurements might be done with a protective film attached, before the touch screen is attached, and in a test jig instead of the final product. This causes the measurements to shift because of changes to the optical path, temperature and warm up time. Therefore, the measurements made above need to be transformed to what the measurements will be in the final product. This is done by measuring several panels both before and after assembly. A regression analysis is performed and the results used to transform the measured factory floor values to values that will be used for calibration.
Finally, while the display component is manufactured to a specification, the specification allows significantly more variation in the display panel measurements than what is typically found in production. The variation within a batch of panels is even less. Therefore, to save cost a manufacturer may measure a sample of panels and base the correction factors on those measurements.
In all cases, the measurements obtained are used to estimate the variation in peak luminance output and gamma (or other non-linear function) for the 3 primary colors, 3 complementary colors and white. From this data it is possible to calculate the crosstalk correction matrix that provides accurate calibration for all the colors.
It will be appreciated that variants of the above-disclosed and other features and functions, or alternatives thereof, may be combined into many other different systems or applications. Various presently unforeseen or unanticipated alternatives, modifications, variations, or improvements therein may be subsequently made by those skilled in the art which are also intended to be encompassed by the following claims.
| Number | Name | Date | Kind |
|---|---|---|---|
| 20050275668 | Feng | Dec 2005 | A1 |
| 20070052735 | Chou | Mar 2007 | A1 |
| 20080218508 | Pearman | Sep 2008 | A1 |
| 20100128050 | Chou | May 2010 | A1 |
| 20100265266 | Liu | Oct 2010 | A1 |
| 20140071177 | Chu | Mar 2014 | A1 |
| 20140333797 | Kim | Nov 2014 | A1 |
| 20150317944 | Safaee-Rad | Nov 2015 | A1 |
| 20170140556 | Safaee-Rad | May 2017 | A1 |
| 20170162144 | Chen | Jun 2017 | A1 |
| 20200035173 | Kim | Jan 2020 | A1 |
