TRUE COLOR IMAGE ENHANCEMENT COMPENSATION

Abstract

A true color image enhancement system includes an ambient light sensor, a circuit and a processor. The ambient light sensor measures an ambient light level. The circuit generates a histogram based on an input video signal; and generates an output video signal by converting gray shades in the input video signal based on input video luminance ratios and a look up table to determine new output gray shade values to maintain input video color coordinates. The processor develops the look up table based on the histogram and the ambient light level; dynamically remaps the gray shades in the look up table in response to the ambient light signal to compress one region and stretch another region; dynamically remaps the gray shades in the look up table to correct for ambient lighting conditions; and transfers the look up table to the circuit.

Description

TECHNICAL FIELD

The present disclosure generally relates to systems and methods for vehicle display image enhancement.

BACKGROUND

In automotive display applications, light sensors have been utilized to automatically control the display luminance as a function of the ambient lighting environment. As the ambient illumination increases, the display luminance increases to maintain visibility of the images. Automatic luminance control methods maintain a comfortable level of viewing brightness, and reduce a display power consumption as the ambient illumination decreases. Although automatic luminance control methods maintain visibility of the symbols for peak-white gray shades, the visibility of lower gray shades may be compromised.

SUMMARY

A true color image enhancement compensation system is provided herein. The true color image enhancement compensation system includes an ambient light sensor, a circuit and a processor. The ambient light sensor is operational to measure an ambient light level. The circuit is operational to generate a histogram based on an input video signal, export the histogram, receive a gray shade look up table, and generate an output video signal by converting a plurality of gray shades in the input video signal based on a plurality of input video luminance ratios and the gray shade look up table to determine a plurality of new output gray shade values to maintain input video color coordinates. The processor is operational to: receive the histogram from the circuit, develop the gray shade look up table based on the histogram and the ambient light level, dynamically remap the plurality of gray shades in the gray shade look up table in response to the ambient light signal to compress a first region of the plurality of gray shades and stretch a second region of the plurality of gray shades, dynamically remap the plurality of gray shades in the gray shade look up table to correct for ambient lighting conditions, and transfer the gray shade look up table to the circuit.

A true color image enhancement compensation system is provided herein. The true color image enhancement compensation system includes an ambient light sensor, a circuit, and a processor. The ambient light sensor is operational to measure an ambient light level. The circuit is operational to generate a histogram based on an input video signal, export the histogram, receive a gray shade look up table, dynamically remap a plurality of gray shades in the gray shade look up table using a highest remapped color to determine and maintain a plurality of input luminance ratios of a plurality of lower gray shade colors, and generate an output video signal by converting a plurality of gray shades in the input video signal based on a plurality of input video luminance ratios and the gray shade look up table to determine a plurality of new output gray shade values to maintain input video color coordinates. The processor is operational to receive the histogram from the circuit, develop the gray shade look up table based on the histogram and the ambient light level, and transfer the gray shade look up table to the circuit.

A true color image enhancement compensation system is provided herein. The true color image enhancement compensation system includes an ambient light sensor, a circuit, and a processor. The ambient light sensor is operational to measure an ambient light level. The circuit is operational to generate a histogram based on an input video signal, export the histogram, receive a gray shade look up table, dynamically remap a plurality of gray shades in the gray shade look up table using a tristimulus conversion matrix to determine a plurality of normalized gray shades based on a normalization of a highest color when an upper limit is reached, and generate an output video signal by converting a plurality of gray shades in the input video signal based on a plurality of input video luminance ratios and the gray shade look up table to determine a plurality of new output gray shade values to maintain input video color coordinates. The processor is operational to receive the histogram from the circuit, develop the gray shade look up table based on the histogram and the ambient light level, and transfer the gray shade look up table to the circuit.

The above features and advantages and other features and advantages of the present teachings are readily apparent from the following detailed description of the best modes for carrying out the teachings when taken in connection with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a context of a platform.

FIG. 2 illustrates a graph of an image enhancement gray shade transfer function.

FIG. 3 illustrates a graph of an image enhancement transfer function.

FIG. 4 illustrates a graph of a normalized luminance transfer function.

FIG. 5 illustrates a graph of the 1931 CIE color chart.

FIG. 6 illustrates a graph of an orange icon color shift after image enhancement.

FIG. 7 illustrates a flow chart of an implementation of a 3-step alternative color correction method.

FIG. 8 illustrates a flow chart of an implementation of an 8-step tristimulus color correction method.

FIG. 9 illustrates a graph of an image enhancement transfer function.

FIG. 10 illustrates a graph of a dynamic adaptive image enhancement principle.

FIG. 11 illustrates an intermediate functional block diagram of a system.

The present disclosure may have various modifications and alternative forms, and some representative embodiments are shown by way of example in the drawings and will be described in detail herein. Novel aspects of this disclosure are not limited to the particular forms illustrated in the above-enumerated drawings. Rather, the disclosure is to cover modifications, equivalents, and combinations falling within the scope of the disclosure as encompassed by the appended claims.

DETAILED DESCRIPTION

Embodiments of the disclosure generally provide for a true color image enhancement compensation system. Where a gray shade transfer table is used to remap gray shades for image enhancement, an original color may not be maintained for secondary and tertiary colors. In various embodiments of the disclosure, the original image input colors are maintained after utilization with an image enhancement gray shade transfer table.

FIG. 1 illustrates a context of a platform 90 in accordance with one or more exemplary embodiments. The platform 90 generally includes an instrument panel 92. The instrument panel 92 includes a control unit 94 and one or more display panels 100a-100c. The instrument panel 92 may be implemented as part of a vehicle 93. The vehicle 93 may include mobile vehicles such as automobiles, trucks, motorcycles, boats, trains and/or aircraft. In some embodiments, the instrument panel 92 may be part of a stationary object. The stationary objects may include, but are not limited to, billboards, kiosks, and/or marquees. Other types of platforms 90 may be implemented to meet the design criteria of a particular application.

The control unit 94 implements one or more display-drive circuits. The control unit 94 is generally operational to generate control signals that drive the display panels 100a-100c. In various embodiments, the control signals may be configured to provide instrumentation (e.g., speed, tachometer, fuel, temperature, etc.) to at least one display panel 100a-100c (e.g., 100a). In some embodiments, the control signals may also be configured to provide video (e.g., a rear-view camera video, a forward-view camera video, an onboard DVD player, etc.) to the display panels 100a-100c. In other embodiments, the control signals may be further configured to provide alphanumeric information shown on one or more of the display panels 100a-100c.

In various embodiments, the control unit 94 generally comprises at least one microcontroller. The at least one microcontroller may include one or more processors, each of which may be embodied as a separate processor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or a dedicated electronic control unit.

The at least one microcontroller may be any sort of electronic processor (implemented in hardware, software executing on hardware, or a combination of both). The at least one microcontroller may also include tangible, non-transitory memory, (e.g., read-only memory in the form of optical, magnetic, and/or flash memory). For example, the at least one microcontroller may include application-suitable amounts of random-access memory, read-only memory, flash memory and other types of electrically-erasable programmable read-only memory, as well as accompanying hardware in the form of a high-speed clock or timer, analog-to-digital and digital-to-analog circuitry, and input/output circuitry and devices, as well as appropriate signal conditioning and buffer circuitry. The at least one microcontroller may be embedded as part of an FPGA or ASIC device.

Computer-readable and executable instructions embodying the present method may be recorded (or stored) in the memory and executed as set forth herein. The executable instructions may be a series of instructions employed to run applications on the at least one microcontroller (either in the foreground or background). The at least one microcontroller may receive commands and information, in the form of one or more input signals from various controls or components in the platform 90 and communicate instructions to the display panels 100a-100c through one or more control signals to control the displays panels 100a-100c.

The display panels 100a-100c are generally mounted to the instrument panel 92. In various embodiments, one or more of the display panels 100a-100c may be disposed inside the platform 90 (e.g., vehicle 93). In other embodiments, one or more of the display panels 100a-100c may be disposed exterior to the platform 90. One or more display panels 100a-100c may implement an active public/privacy viewing modes. One or more display panels 100a-100c may also implement the privacy mode. As illustrated, the display panel 100a may be a cluster display positioned for use by a driver. The display panel 100b may be a console display positioned for use by the driver and a passenger. The display panel 100c may be a passenger display positioned for use by the passenger and the driver.

Introduction

Light sensors may be utilized to automatically control the display luminance as a function of the ambient lighting environment. As the ambient illumination increases as detected by the light sensor(s), the display luminance is increased to maintain visibility of the image. The automatic luminance control maintains a comfortable level of viewing brightness, and reduces the display power consumption as the ambient illumination decreases. Although automatic luminance control methods do maintain visibility of symbology for peak white video content, the visibility of lower video gray shade content may be compromised. More recently, image enhancement methods are becoming available that are capable of making the video content more visible to the user. However, these image enhancement methods may or may not improve the image in response to the ambient illumination level. The combination of automatic luminance control coupled with image enhancement adjustment as a function of the ambient illumination may be utilized provide a new product class that provides visibility of the entire image under all ambient lighting conditions, while at the same time reduces the overall power of the display system.

Background

FIG. 2 illustrates a graph 110 of an example image enhancement gray shade transfer function in accordance with one or more exemplary embodiments. An issue with automatic luminance control is that the lower video gray shade content becomes less visible under reflected ambient lighting conditions and increasing the display luminance does little for lower video content visibility. An image enhancement function is used to change the shape of the video transfer function dynamically as a function of the ambient lighting conditions. The video transfer function as depicted in FIG. 2 has the input video gray shades on the x-axis and the output video gray shades are on the ordinate axis. Therefore, the input video gray shades are mapped to the output video gray shades as determined by the image enhancement transfer function. For example, if sunlight is shining on the display, the user may see a reflected background from the display commonly known as glare. The glare may make the lower video gray shades difficult to see since the glare is higher in luminance than the lower video gray shades. To make the lower gray shades visible, the lower gray shades (e.g., to the left of the line 116) may be mapped (e.g., stretched region 114) to higher output gray shades thereby increasing the lower gray shade luminance values and making them visible. Higher gray shades (e.g., to the right of an N_Max line 116 be mapped differently (e.g., compressed region 118). Due to the non-linear shape of the transfer function, ratios of the color mixtures may result in a color shift as the shape of the curve changes as a function of the ambient illumination level.

Analysis

FIG. 3 illustrates a graph 120 of an example image enhancement transfer function in accordance with one or more exemplary embodiments. A curve 122 represents the output gray shades without image enhancement (IE). A curve 124 represents the output gray shades with the image enhancement.

FIG. 4 illustrates a graph 130 of an example normalized luminance transfer function in accordance with one or more exemplary embodiments. A curve 132 represents normalized luminance output without image enhancement (IE). A curve 134 represents normalized luminance output with the image enhancement. A point 136 may be a breakpoint.

Consider a test image that includes multiple colors and multiple icons. A color shift of an orange icon to yellow may be caused in an absence of color compensation (or correction) where a high degree of image enhancement is employed. Since the colors are determined by the mixture of the R, G, B luminance values, FIG. 3 may be converted to normalized luminance, LN, per FIG. 4 using Equation 1 with values of γ=2.2 and GS_Max=1023 (10 bit).

$\begin{array}{cc}{L}_N={\left(\frac{GS}{G{S}_{\mathrm{Max}}}\right)}^{\gamma }& \mathrm{Eq}.\left(1\right)\end{array}$

• • Where:
  • γ is a gamma value.
  • GS is a gray-shade value.
  • GS_MAX is a maximum gray shade value.

Table 1 shows a comparison of the normalized luminance values with and without image enhancement for the orange color.

	TABLE 1
	Color	Gray Shade	LN w/o IE	LN with IE
	Red	250	0.96	0.99
	Green	102	0.13	0.60
	Blue	0	0	0

FIG. 5 illustrates a graph 140 of the 1931 CIE color chart in accordance with one or more exemplary embodiments. In Table 1, the ratio of red to green changes from 7.4 (0.96/0.13) without IE to 1.7 (0.99/0.60) with IE. Although the amount of red luminance stays relatively constant (0.96 to 0.99), the amount of green luminance increases (0.13 to 0.60) with image enhancement causing the orange color to become yellowish. As may be observed from the 1931 CIE color chart in FIG. 5, where as more green is mixed in, the color shifts from orange to yellow.

Returning to FIG. 4, note that before the breakpoint 136 (N_Max) in the image enhanced luminance curve 134, the curve 134 is essentially a gamma curve that may be expressed by Equation 2, as follows, where the offset gamma function is utilized instead of the constant contrast ratio function.

$\begin{array}{cc}{L}_N={\left(\frac{GS}{G{S}_{\mathrm{Max}}}\right)}^{\gamma }& \mathrm{Eq}.\left(2\right)\end{array}$

Therefore, if there are two gray-shade levels GS₁ and GS₂ for two colors normalized luminances L_N1 and L_N2, the formulas may be expressed as Equations 3 and 4 as follows:

$\begin{array}{cc}{L}_{N1}={\left(\frac{G{S}_1}{G{S}_{\mathrm{Max}}}\right)}^{\gamma }& \mathrm{Eq}.\left(3\right)\end{array}$ $\begin{array}{cc}{L}_{N2}={\left(\frac{G{S}_2}{G{S}_{\mathrm{Max}}}\right)}^{\gamma }& \mathrm{Eq}.\left(4\right)\end{array}$

A luminance ratio for the two color may be expressed by Equation 5 as follows:

$\begin{array}{cc}\mathrm{Ratio}=\frac{L_{N2}}{L_{N1}}=\frac{{\left(\frac{{\mathrm{GS}}_2}{{\mathrm{GS}}_{\mathrm{Max}}}\right)}^{\gamma }}{{\left(\frac{{\mathrm{GS}}_1}{{\mathrm{GS}}_{\mathrm{Max}}}\right)}^{\gamma }}={\left(\frac{{\mathrm{GS}}_2}{{\mathrm{GS}}_1}\right)}^{\gamma }& \mathrm{Eq}.\left(5\right)\end{array}$

Therefore, as the GS_Max and the maximum luminance changes as a function of the endpoint function, the luminance ratios may remain constant and therefore the corresponding color may not change. Where one, both, or all of the gray shades are above the break point, the luminance ratios and therefore the corresponding color changes. In the example, the orange color may not change where the values are Red=113, Green=44 and Blue=0. Since the Red value of 113 is just beyond the break point, the color remains largely unchanged even though the transfer function is changing dramatically.

Further analysis of an orange circular icon starts with sample display tristimulus values as shown in Table 2. The methods used to calculate color coordinates using the tristimulus values is described in “Color Space Conversions,” by A. Ford and A. Roberts, 1998, which is hereby incorporated by reference in its entirety.

TABLE 2
Tristimulus Values
	Red	Green	Blue
	X	330.26	293.35	177.66
	Y	172.27	622.85	69.7
	Z	15.61	41.72	933.48

Using the tristimulus matrix, the resulting tristimulus X, Y and Z values may be determined from the normalized R, G, B values according to Equation 6 as follows:

$\begin{array}{cc}❘\begin{array}{c}X\\ Y\\ Z\end{array}❘=❘\begin{array}{ccc}330.26& 293.35& 177.66\\ 172.27& 622.85& 69.7\\ 15.61& 41.72& 933.48\end{array}❘*❘\begin{array}{c}R\\ G\\ B\end{array}❘& \mathrm{Eq}.\left(6\right)\end{array}$

The R, G, B values in Equation 6 are the normalized luminance values calculated according to Equations 7, 8 and 9 as follows:

$\begin{array}{cc}R={\left(\frac{{\mathrm{GS}}_{\mathrm{Red}}}{{\mathrm{GS}}_{\mathrm{Max}}}\right)}^{\gamma }& \mathrm{Eq}.\left(7\right)\end{array}$ $\begin{array}{cc}G={\left(\frac{{\mathrm{GS}}_{\mathrm{Green}}}{{\mathrm{GS}}_{\mathrm{Max}}}\right)}^{\gamma }& \mathrm{Eq}.\left(8\right)\end{array}$ $\begin{array}{cc}B={\left(\frac{{\mathrm{GS}}_{\mathrm{Blue}}}{{\mathrm{GS}}_{\mathrm{Max}}}\right)}^{\gamma }& \mathrm{Eq}.\left(9\right)\end{array}$

Where the orange icon values of GS_Red=250, GS_Green=102, and GS_Blue=0 and inserted into Equations 7, 8 and 9, the normalized luminance RGB values are determined per Equations 10, 11 and 12 as follows:

$\begin{array}{cc}R={\left(\frac{{\mathrm{GS}}_{\mathrm{Red}}}{{\mathrm{GS}}_{\mathrm{Max}}}\right)}^{\gamma }={\left(\frac{250}{255}\right)}^{2.2}=0.95737& \mathrm{Eq}.\left(10\right)\end{array}$ $\begin{array}{cc}G={\left(\frac{{\mathrm{GS}}_{\mathrm{Green}}}{{\mathrm{GS}}_{\mathrm{Max}}}\right)}^{\gamma }={\left(\frac{102}{255}\right)}^{2.2}=0.13321& \mathrm{Eq}.\left(11\right)\end{array}$ $\begin{array}{cc}B={\left(\frac{{\mathrm{GS}}_{\mathrm{Blue}}}{{\mathrm{GS}}_{\mathrm{Max}}}\right)}^{\gamma }={\left(\frac{0}{255}\right)}^{2.2}=0& \mathrm{Eq}.\left(12\right)\end{array}$

Using the tristimulus Equation 6, the X, Y, Z tristimulus values may be determined per Equation 13 as follows:

$\begin{array}{cc}❘\begin{array}{c}X\\ Y\\ Z\end{array}❘=❘\begin{array}{ccc}330.26& 293.35& 177.66\\ 172.27& 622.85& 69.7\\ 15.61& 41.72& 933.48\end{array}❘*❘\begin{array}{c}0.95737\\ 0.13321\\ 0\end{array}❘& \mathrm{Eq}.\left(13\right)\end{array}$ $❘\begin{array}{c}X\\ Y\\ Z\end{array}❘=❘\begin{array}{c}355.2576\\ 247.895\\ 20.502\end{array}❘$

The tristimulus values determined in Equation 13 may then be used to calculate the 1931 CIE x,y color coordinates per Equations 14 and 15, as follows:

$\begin{array}{cc}x=\frac{X}{X+Y+Z}=\frac{355.2576}{355.2576+247.895+20.502}& \mathrm{Eq}.\left(14\right)\end{array}$ $x=0.57$ $\begin{array}{cc}y=\frac{Y}{X+Y+Z}=\frac{247.895}{355.2576+247.895+20.502}& \mathrm{Eq}.\left(15\right)\end{array}$ $y=0.4$

Therefore, the input image color coordinates for the orange icon are (x,y)=(0.57,0.40).

After the image enhancement transfer function is applied to the input gray-shade values, the output values are:

• • GS_Red=1018.3 (10 bit)
  • GS_Green=810.862 (10 bit)
  • GS_Blue=0

Using these values, the new normalized luminance values may be determined per Equations 16, 17 and 18 as follows:

$\begin{array}{cc}{R}_N={\left(\frac{{\mathrm{GS}}_{\mathrm{Red}}}{{\mathrm{GS}}_{\mathrm{Max}}}\right)}^{\gamma }={\left(\frac{1018.3}{1023}\right)}^{2.2}=0.98992& \mathrm{Eq}.\left(16\right)\end{array}$ $\begin{array}{cc}{G}_N={\left(\frac{{\mathrm{GS}}_{\mathrm{Green}}}{{\mathrm{GS}}_{\mathrm{Max}}}\right)}^{\gamma }={\left(\frac{810.862}{1023}\right)}^{2.2}=0.59971& \mathrm{Eq}.\left(17\right)\end{array}$ $\begin{array}{cc}{B}_N={\left(\frac{{\mathrm{GS}}_{\mathrm{Blue}}}{{\mathrm{GS}}_{\mathrm{Max}}}\right)}^{\gamma }={\left(\frac{0}{1023}\right)}^{2.2}=0& \mathrm{Eq}.\left(18\right)\end{array}$

Using the tristimulus matrix, the X, Y, Z tristimulus values may be determined per Equation 19 as follows:

$\begin{array}{cc}❘\begin{array}{c}X\\ Y\\ Z\end{array}❘=❘\begin{array}{ccc}330.26& 293.35& 177.66\\ 172.27& 622.85& 69.7\\ 15.61& 41.72& 933.48\end{array}❘*❘\begin{array}{c}0.98992\\ 0.59971\\ 0\end{array}❘& \mathrm{Eq}.\left(19\right)\end{array}$ $❘\begin{array}{c}X\\ Y\\ Z\end{array}❘=❘\begin{array}{c}502.8552\\ 544.0612\\ 40.4724\end{array}❘$

As was done previously, the 1931 CIE x,y color coordinate values may be determined per Equations 20 and 21 as follows:

$\begin{array}{cc}x=\frac{X}{X+Y+Z}=\frac{502.8552}{502.8552+544.0612+40.4724}& \mathrm{Eq}.\left(20\right)\end{array}$ $x=0.46$ $\begin{array}{cc}y=\frac{Y}{X+Y+Z}=\frac{544.0612}{502.8552+544.0612+40.4724}& \mathrm{Eq}.\left(21\right)\end{array}$ $y=0.5$

FIG. 6 illustrates a graph 150 of an example orange icon color shift after image enhancement in accordance with one or more exemplary embodiments. After the image enhancement transfer function is applied the (x,y) values changes from (x,y)=(0.57,0.40) 152 to (0.46,0.50) 154. This change in color is shown in the CIE color chart per FIG. 6 as illustrated.

Image Recommendations

In certain display technologies, namely adaptive image processing, significant use of colors that are not primary colors or secondary colors (or close to) may present unusual effects when the image enhancement transfer function is applied. Use of such colors where the ratio of R, G, B is >1.5:1 may be limited (e.g., not used in a prominent way) if any of the RGB gray shades exceed 64. Color mixtures with RGB values less than 64 in any ratio generally do not present the unusual effects.

ISO 15008:2017 Table B.1 provides guidance for symbol/background color combinations.

TABLE B.1
Background	Symbol Color
Color	White	Yellow	Orange	Reda, Purple	Green, Cyan	Bluea, Violet	Black
White		−	0	+	+	++	++
Yellow	−		−	0	0	+	++
Orange	0	−		−	−	0	+
Reda, Purple	+	0	−		−	−	+
Green, Cyan	+	0	−	−		−	+
Bluea, Voilet	++	+	0	−	−		−
Black	++	++	+	+	+	−
++ Preferred
+ Recommended
0 Acceptable with high saturation differences
− Not recommended
aPure red and blue should be avoided in view of the fact that the eyes may have trouble focusing on these colors because of eye chromatic aberration.

Alternate Pixel-by-Pixel Image Enhancement Color Correction Method

To realize a more color robust image enhancement technique, a pixel-by-pixel color correction is performed. By operating on the video stream on a pixel-by-pixel basis, memory allocation sized to hold a full video frame may be avoided, resulting in a cost effective FPGA solution. An alternate color correction method has some small color shifts that may not be noticeable due to the way the image enhancement technique works by compressing the upper gray shades. White or colors with balanced gray shades may not be affected by using this alternate color correction method. The alternate color correction method uses less computations than tristimulus matrix methods and may be implemented with a current image enhancement FPGA architecture.

The alternate color correction method is based on the following proof and is fundamentally based on the “Power of a Fraction Rule” shown in Table 3.

TABLE 3
Rules of Exponents or Laws of Exponents
Multiplication Rule      ax × ay = ax+y
Division Rule            ax ÷ ay = ax+y
Power of a Power Rule    (ax)y = axy
Power of a Product Rule  (ab)x = axbx
Power of a Fraction Rule (a/b)x = ax/bx
Zero Exponent            a0 = 1

An aspect of the color correction method is understanding the concept that color is based on RGB luminance ratios. If the luminance ratios remain constant, the colors may remain constant.

A minimalistic proof starts with the luminance formulas for red, green, and blue as shown in Equations 22, 23 and 24, as follows, respectively, for 8-bit video.

$\begin{array}{cc}{L}_R={L_{\mathrm{maxR}}\left(\frac{{\mathrm{GS}}_R}{255}\right)}^{\gamma }& \mathrm{Eq}.\left(22\right)\end{array}$ $\begin{array}{cc}{L}_G={L_{\mathrm{maxG}}\left(\frac{{\mathrm{GS}}_G}{255}\right)}^{\gamma }& \mathrm{Eq}.\left(23\right)\end{array}$ $\begin{array}{cc}{L}_B={L_{\mathrm{maxB}}\left(\frac{{\mathrm{GS}}_B}{255}\right)}^{\gamma }& \mathrm{Eq}.\left(24\right)\end{array}$

The maximum input gray shades are transformed into the maximum output gray shades due the monotonic nature of the image enhancement transfer function. Consider a case where the input gray shades GS_Rin, GS_Gin, GS_Bin are transformed to output gray shades GS_Rout, GS_Gout, GS_Bout. Also suppose that the red input has the maximum gray-shade value, GS_Rinmax. Therefore, GS_Rinmax=GS_Rin. The color correction may be accomplished by using Equations 25, 26 and 27 as follows:

$\begin{array}{cc}{\mathrm{GS}}_{\mathrm{Rout}}={\mathrm{GS}}_{\mathrm{Rin}}\left(\frac{{\mathrm{GS}}_{\mathrm{Rout}}}{{\mathrm{GS}}_{\mathrm{Rinmax}}}\right)={\mathrm{GS}}_{\mathrm{Rout}}& \mathrm{Eq}.\left(25\right)\end{array}$ $\begin{array}{cc}{\mathrm{GS}}_{\mathrm{Gout}}={\mathrm{GS}}_{\mathrm{Gin}}\left(\frac{{\mathrm{GS}}_{\mathrm{Rout}}}{{\mathrm{GS}}_{\mathrm{Rinmax}}}\right)& \mathrm{Eq}.\left(26\right)\end{array}$ $\begin{array}{cc}{\mathrm{GS}}_{\mathrm{Bout}}={\mathrm{GS}}_{\mathrm{Bin}}\left(\frac{{\mathrm{GS}}_{\mathrm{Rout}}}{{\mathrm{GS}}_{\mathrm{Rinmax}}}\right)& \mathrm{Eq}.\left(27\right)\end{array}$

This approach disregards the original non-maximum GS_Gout and GS_Bout values, then replaces these values with the new values per Equations 26 and 27 above.

If the red to green luminance ratio is to be determined, the original input gray-shade luminance ratios may be determined using Equations 22 and 23 per Equation 28 as follows:

$\begin{array}{cc}\frac{L_{\mathrm{Rin}}}{L_{\mathrm{Gin}}}=\frac{{L_{\mathrm{maxR}}\left(\frac{{\mathrm{GS}}_{\mathrm{Rin}}}{255}\right)}^{\gamma }}{{L_{\mathrm{maxG}}\left(\frac{{\mathrm{GS}}_{\mathrm{Gin}}}{255}\right)}^{\gamma }}& \mathrm{Eq}.\left(28\right)\end{array}$

Using the “Power of a Fraction Rule” per Table 3, Equation 28 may be converted into Equation 29 as follows:

$\begin{array}{cc}\frac{L_{\mathrm{Rin}}}{L_{\mathrm{Gin}}}=\frac{L_{\mathrm{maxR}}\left[\frac{{\left({\mathrm{GS}}_{\mathrm{Rin}}\right)}^{\gamma }}{{\left(255\right)}^{\gamma }}\right]}{L_{\mathrm{maxG}}\left[\frac{{\left({\mathrm{GS}}_{\mathrm{Gin}}\right)}^{\gamma }}{{\left(255\right)}^{\gamma }}\right]}& \mathrm{Eq}.\left(29\right)\end{array}$

The “255” terms in Equation 29 may be cancelled out resulting in Equation 30 as follows:

$\begin{array}{cc}\frac{L_{\mathrm{Rin}}}{L_{\mathrm{Gin}}}=\frac{{L_{\mathrm{maxR}}\left({\mathrm{GS}}_{\mathrm{Rin}}\right)}^{\gamma }}{{L_{\mathrm{maxG}}\left({\mathrm{GS}}_{\mathrm{Gin}}\right)}^{\gamma }}=\frac{L_{\mathrm{maxR}}}{L_{\mathrm{maxG}}}{\left(\frac{{\mathrm{GS}}_{\mathrm{Rin}}}{{\mathrm{GS}}_{\mathrm{Gin}}}\right)}^{\gamma }& \mathrm{Eq}.\left(30\right)\end{array}$

In a similar fashion, the red to green luminance ratios may be determined using the output results from Equations 25 and 26. Starting with the gamma function, Equation 31 may be constructed as follows:

$\begin{array}{cc}\frac{L_{\mathrm{Rout}}}{L_{\mathrm{Gout}}}=\frac{{L_{\mathrm{maxR}}\left(\frac{{\mathrm{GS}}_{\mathrm{Rout}}}{255}\right)}^{\gamma }}{{L_{\mathrm{maxG}}\left(\frac{{\mathrm{GS}}_{\mathrm{Gout}}}{255}\right)}^{\gamma }}& \mathrm{Eq}.\left(31\right)\end{array}$

The “255” terms in Equation 31 may once again be cancelled out resulting in Equation 32 as follows:

$\begin{array}{cc}\frac{L_{\mathrm{Rout}}}{L_{\mathrm{Gout}}}=\frac{{L_{\mathrm{maxR}}\left({\mathrm{GS}}_{\mathrm{Rout}}\right)}^{ℽ}}{{L_{\mathrm{maxG}}\left({\mathrm{GS}}_{\mathrm{Gout}}\right)}^{ℽ}}=\frac{L_{\mathrm{maxR}}}{L_{\mathrm{maxG}}}{\left(\frac{{\mathrm{GS}}_{\mathrm{Rout}}}{{\mathrm{GS}}_{\mathrm{Gout}}}\right)}^{ℽ}& \mathrm{Eq}.\left(32\right)\end{array}$

Equations 25 and 26 may then be substituted into Equation 32 resulting in Equation 33 as follows:

$\begin{array}{cc}\frac{L_{\mathrm{Rout}}}{L_{\mathrm{Gout}}}=\frac{L_{\mathrm{maxR}}}{L_{\mathrm{maxG}}}{\left(\frac{{\mathrm{GS}}_{\mathrm{Rin}}\left(\frac{{\mathrm{GS}}_{\mathrm{Rout}}}{{\mathrm{GS}}_{\mathrm{Rinmax}}}\right)}{{\mathrm{GS}}_{\mathrm{Gin}}\left(\frac{{\mathrm{GS}}_{\mathrm{Rout}}}{{\mathrm{GS}}_{\mathrm{Rinmax}}}\right)}\right)}^{ℽ}& \mathrm{Eq}.\left(33\right)\end{array}$

Cancelling like terms in Equation 33 yields Equation 34, as follows, which matches the input luminance ratio in Equation 30.

$\begin{array}{cc}\frac{L_{\mathrm{Rout}}}{L_{\mathrm{Gout}}}=\frac{L_{\mathrm{maxR}}}{L_{\mathrm{maxG}}}{\left(\frac{{\mathrm{GS}}_{\mathrm{Rin}}}{{\mathrm{GS}}_{\mathrm{Gin}}}\right)}^{ℽ}& \mathrm{Eq}.\left(34\right)\end{array}$

Image Simulation

To confirm the pixel-by-pixel image enhancement color correction method, image processing was employed. The procedure used was: 1) extract the RGB gray-shade values from a test image; 2) implement the prescribed equations and method within a spreadsheet and 3) transformation of the data back to an image file to provide the graphical confirmation of the results. In order to operate with a reasonable spreadsheet file size, the input test image was cropped to 300×300 pixels. The cropping thus limits the processing within the spreadsheet to 300×900 RGB gray-shade values.

The RGB gray-shade values of the test image were extracted using an image editing program, ImageJ (National Institute of Health, Bethesda, Maryland) by saving the image as a “Text Image”. The text file containing the RGB gray-shade values was then imported into the spreadsheet for data processing. The following process was implemented as formula within spreadsheet taking into account the appropriate fixed-point mathematics and data truncation. This allows the pixel-by-pixel implementation and calculates the output RGB gray scale values for each of the 300×300 input pixels. After saving the results as a text file, the results were transformed back to a color corrected image file using the ImageJ software. The corrected image file showed perceptible gray-shade enhancements with imperceptible color shifting.

FPGA Implementation Recommendations

To simplify the color conversion mathematics for a FPGA, the basic premise is to use fixed-point numbers and to avoid fractional numbers. The use of lookup tables is encouraged.

The alternate color correction method may be summarized by starting with Equations 35, 36 and 37, as follows:

$\begin{array}{cc}{\mathrm{GS}}_{\mathrm{Rout}}={\mathrm{GS}}_{\mathrm{Rin}}\left(\frac{{\mathrm{GS}}_{\mathrm{maxout}}}{{\mathrm{GS}}_{\mathrm{maxin}}}\right)& \mathrm{Eq}.\left(35\right)\end{array}$ $\begin{array}{cc}{\mathrm{GS}}_{\mathrm{Gout}}={\mathrm{GS}}_{\mathrm{Gin}}\left(\frac{{\mathrm{GS}}_{\mathrm{maxout}}}{{\mathrm{GS}}_{\mathrm{maxin}}}\right)& \mathrm{Eq}.\left(36\right)\end{array}$ $\begin{array}{cc}{\mathrm{GS}}_{\mathrm{Bout}}={\mathrm{GS}}_{\mathrm{Bin}}\left(\frac{{\mathrm{GS}}_{\mathrm{maxout}}}{{\mathrm{GS}}_{\mathrm{maxin}}}\right)& \mathrm{Eq}.\left(37\right)\end{array}$

• • Where:
  • GS_maxin is the maximum gray shade of the RGB input gray shades.
  • GS_maxout is the maximum gray shade of the RGB output gray shades using the transfer table.

The maximum gray shade on the output may always be associated with the same color component for the maximum input gray shade.

Since GS_maxin may match one of the GS_RGBin gray-shade values, only two of the three equations could be determined. Since division is difficult in an FPGA, one method is to construct a table of 255 multiplier M_maxin values per Equation 38 as follows:

$\begin{array}{cc}{M}_{\mathrm{maxin}}=\frac{2^{16}}{{\mathrm{GS}}_{\mathrm{maxin}}}& \mathrm{Eq}.\left(38\right)\end{array}$

Since 2¹⁶ divided by 1 may be represented with 2 bytes, the table may take 3-bytes×256=6,144 bits of memory. For GS_maxin=0, the output from the lookup table gray shade may be set to zero to avoid division by zero singularity in Equation 38. In addition, the lookup table value for GS_maxin=1 may be set to 2¹⁶−1 (65,535) to fit into 2 bytes. The 2¹⁶ multiplication improves the accuracy of the results without resorting to floating point multiplication.

The multiplication may be done in 34-bit fixed-point mathematics. Using the multiplier table concept, Equations 35, 36 and 37, are transformed into Equations 39, 40, and 41 as follows:

$\begin{array}{cc}{\mathrm{GS}}_{\mathrm{Rout}}\times 2^{16}={\mathrm{GS}}_{\mathrm{Rin}}{\mathrm{GS}}_{\mathrm{maxout}}{M}_{\mathrm{maxin}}& \mathrm{Eq}.\left(39\right)\end{array}$ $\begin{array}{cc}{\mathrm{GS}}_{\mathrm{Gout}}\times 2^{16}={\mathrm{GS}}_{\mathrm{Gin}}{\mathrm{GS}}_{\mathrm{maxout}}{M}_{\mathrm{maxin}}& \mathrm{Eq}.\left(40\right)\end{array}$ $\begin{array}{cc}{\mathrm{GS}}_{\mathrm{Bout}}\times 2^{16}={\mathrm{GS}}_{\mathrm{Bin}}{\mathrm{GS}}_{\mathrm{maxout}}{M}_{\mathrm{maxin}}& \mathrm{Eq}.\left(41\right)\end{array}$

• • Alternative forms of Equations 39, 40, and 41 are shown by Equations 42, 43, and 44, respectively, as follows:

$\begin{array}{cc}{\mathrm{GS}}_{\mathrm{Rout}}=\frac{{\mathrm{GS}}_{\mathrm{Rin}}{\mathrm{GS}}_{\mathrm{maxout}}{M}_{\mathrm{maxin}}}{2^{16}}& \mathrm{Eq}.\left(42\right)\end{array}$ $\begin{array}{cc}{\mathrm{GS}}_{\mathrm{Gout}}=\frac{{\mathrm{GS}}_{\mathrm{Gin}}{\mathrm{GS}}_{\mathrm{maxout}}{M}_{\mathrm{maxin}}}{2^{16}}& \mathrm{Eq}.\left(43\right)\end{array}$ $\begin{array}{cc}{\mathrm{GS}}_{\mathrm{Bout}}=\frac{{\mathrm{GS}}_{\mathrm{Bin}}{\mathrm{GS}}_{\mathrm{maxout}}{M}_{\mathrm{maxin}}}{2^{16}}& \mathrm{Eq}.\left(44\right)\end{array}$

FIG. 7 illustrates a flow chart 160 of an example implementation of a 3-step alternative color correction method in accordance with one or more exemplary embodiments. The alternative color correction method may be implemented in an FPGA.

The data flow for the video pixel stream is to process each pixel in real time and send the modified RGB gray scale values for processing by the FRC function or directly to the TDMS converter depending on the options selected. There are 3 basic steps. The goal is to implement the mathematics in fixed 32-bit binary using multiplication and a 16-bit right shift. In addition, a lookup table is used to determine one of the three multiplier values. A mechanism is provided to ensure that the output video timing for the in-line pixel processing remains constant.

Step 1: Find the Gray Shade maximum input value, GS_maxin, from Input RI, GI, BI Values and Lookup Multiplier M_maxin using GS_maxin.

The first step consists of two parts. First, the maximum value of the red, green, and blue gray shades of the incoming pixel is determined as a gray scale maximum input value GS_maxin. This gray shade maximum input value is then used to lookup the multiplier value using a lookup table that consists of multiple (e.g., 256) entries according to the Equation 38. Each output from the table is 16 bits and therefore every table entry is 3 bytes for a total table size of 3×8×256=6144. Note that for GS_maxin=0, the output from the lookup table shade may be set to zero to avoid division by zero in Equation 38. In addition, the lookup table value for GS_maxin=1 is set to 216-1 (65535) to fit into 2 bytes.

Step 2: Use GS_maxin to Lookup GS_maxout using Transfer Function Table.

The step 2 uses GS_maxin from step 1 to lookup a gray scale maximum output value GS_maxout using the transfer function uploaded from a source (e.g., a processor external to the FPGA) via an SPI or another suitable digital interface. GS_maxout may be a 10-bit value ranging from 0 to 1023.

Step 3: Calculate GS_Rout, GS_Gout, GS_Bout

Calculate gray scale output values GS_Rout, GS_Gout, and GS_Bout using Equations 45, 46 and 47 as follows:

$\begin{array}{cc}{\mathrm{GS}}_{\mathrm{Rout}}=\frac{{\mathrm{GS}}_{\mathrm{Rin}}{\mathrm{GS}}_{\mathrm{maxout}}{M}_{\mathrm{maxin}}}{2^{16}}& \mathrm{Eq}.\left(45\right)\end{array}$ $\begin{array}{cc}{\mathrm{GS}}_{\mathrm{Gout}}=\frac{{\mathrm{GS}}_{\mathrm{Gin}}{\mathrm{GS}}_{\mathrm{maxout}}{M}_{\mathrm{maxin}}}{2^{16}}& \mathrm{Eq}.\left(46\right)\end{array}$ $\begin{array}{cc}{\mathrm{GS}}_{\mathrm{Bout}}=\frac{{\mathrm{GS}}_{\mathrm{Bin}}{\mathrm{GS}}_{\mathrm{maxout}}{M}_{\mathrm{maxin}}}{2^{16}}& \mathrm{Eq}.\left(47\right)\end{array}$

The numerator may be calculated prior to the 16-bit right shift division to maintain the accuracy of the results. Before the 16-bit right shift division, the product of the numerator may use a storage size of 34 bits.

FIG. 8 illustrates a flow chart 180 of an example implementation of an 8-step tristimulus color correction method in accordance with one or more exemplary embodiments. The tristimulus color correction method (or calculations) may be implemented in an FPGA. A comparison of example implementations of two methods shows that the 3-step alternative color correction method involves different computations and FPGA resources than the 8-step tristimulus color correction method.

Step 1: Convert the Input Pixel Gray Shade Values to R_I, G_I, B_I Values.

A lookup table is used to convert the input pixel gray shade values to R_I, G_I, and B_I values which are proportional to luminance values. An example equation may be

$R_I,G_I,B_I={\left(\frac{\mathrm{GS}}{255}\right)}^{2.2}.$

Step 2: Convert the R_I, G_I, B_I Values to X_I, Y_I, and Z_I Values Using a Tristimulus Conversion Matrix.

The R_I, G_I, B_I values are converted to the X_I, Y_I, and Z_I values using a proportional tristimulus conversion matrix. The conversion matrix is received from the microprocessor via the SPI or another suitable digital interface. An example of the conversion matrix is shown in Equation 48 as follows:

$\begin{array}{cc}\left[\begin{array}{c}{X}_I\\ Y_I\\ Z_I\end{array}\right]=\left[\begin{array}{ccc}0C38& 0\mathrm{ADB}& 0693\\ 0660& 170C& 0294\\ 0094& 018B& 228A\end{array}\right]*\left[\begin{array}{c}{R}_I\\ G_I\\ B_I\end{array}\right]& \mathrm{Eq}.\left(48\right)\end{array}$

The mathematics may be done in 32-bit fixed point format.

Step 3: Convert the Transfer Function Gray Shade Values into First Intermediate Values R_N, G_N, B_N.

The input pixel gray shade values are first converted to new gray shade values GS_NRed, GS_NGreen, and GS_NBlue using the transfer table that is uploaded from the microprocessor to the FPGA via the SPI port on a periodic (e.g., 60 Hz) update rate. The new gray shade values from the transfer table are then used to lookup the R_N, G_N, B_N values using another lookup table.

Step 4: Calculate Second Intermediate Value Y_N.

Calculate the second intermediate value Y_N using the same conversion matrix used in Step 2 as shown in Equation 49 as follows:

$\begin{array}{cc}\left[\begin{array}{c}{X}_N\\ Y_N\\ Z_N\end{array}\right]=\left[\begin{array}{ccc}0C38& 0\mathrm{ADB}& 0693\\ 0660& 170C& 0294\\ 0094& 018B& 228A\end{array}\right]*\left[\begin{array}{c}{R}_N\\ G_N\\ B_N\end{array}\right]& \mathrm{Eq}.\left(49\right)\end{array}$

The value of Y_N alone may be calculated. X_N and Z_N are calculated in Step 5. Shift the results (bit-shift right, e.g., divide by 2″) by the number of bits from the lookup table. Use the sum of the input pixel gray shade R, G, B (e.g., R+G+B) values to determine the number of bits to shift. Discard the bits that fall off the right side.

Step 5: Calculate Second Intermediate Values X_N and Z_N.

Calculate the second intermediate values X_N and Z_N using Equations 50 and 51 as follows. Multiply the numerators first and then divide by Y_I (discarding the remainder).

$\begin{array}{cc}{X}_N=\frac{Y_N{X}_I}{Y_I}& \mathrm{Eq}.\left(50\right)\end{array}$ $\begin{array}{cc}{Z}_N=\frac{Y_N{Z}_I}{Y_I}& \mathrm{Eq}.\left(51\right)\end{array}$

If Y_I=0, set X_N and Z_N to zero.

Step 6: Calculate Third Intermediate Value R_NC, G_NC, and B_NC Using Inverse Matrix.

Use the inverse matrix uploaded from the microprocessor via the SPI port to calculate the third intermediate values R_NC, G_NC, and B_NC using Equation 52 as follows:

$\begin{array}{cc}\left[\begin{array}{c}{R}_{\mathrm{NC}}\\ G_{\mathrm{NC}}\\ B_{\mathrm{NC}}\end{array}\right]=\left[\begin{array}{ccc}6F5F& \mathrm{CCB}6& \mathrm{EEA}2\\ E140& 3\mathrm{AD}1& 0176\\ \mathrm{FF}83& \mathrm{FE}3B& 1\mathrm{DDF}\end{array}\right]*\left[\begin{array}{c}{X}_N\\ Y_N\\ Z_N\end{array}\right]& \mathrm{Eq}.\left(52\right)\end{array}$

Inverse Matrix values may be represented as 2's complement values to account for negative numbers.

Shift the results (bit-shift right, e.g., divide by 2″) by the number of bits from the lookup table. Use the sum of the input pixel gray shade R, G, B (e.g., R+G+B) values to determine the number of bits to shift. First, implement the additions and then perform the subtractions. If a resultant value is negative, set the result to zero.

Step 7: Normalize R_NC, G_NC, B_NC Values to R_Norm, G_Norm, and B_Norm.

If any of the R_NC, G_NC, and/or B_NC values are >196965, use Equations 53, 54 and 55, as follows, to calculate the normalized values R_Norm, G_Norm, and B_Norm. First divide by 16, discarding the remainders. Next multiply the numerators. Finally divide by the denominator, discarding the remainders. Note that the highest R_NC, G_NC, B_NC value will result in 196965 and therefore that calculation shall be avoided.

$\begin{array}{cc}{R}_{\mathrm{Norm}}=\frac{\left(\frac{R_{\mathrm{NC}}}{16}\right)*196965}{\left\{\left(\frac{\mathrm{Highest}\mathrm{of}{R}_{\mathrm{NC}},G_{\mathrm{NC}},B_{\mathrm{NC}}}{16}\right)\right\}}& \mathrm{Eq}.\left(53\right)\end{array}$ $\begin{array}{cc}{G}_{\mathrm{Norm}}=\frac{\left(\frac{G_{\mathrm{NC}}}{16}\right)*196965}{\left\{\left(\frac{\mathrm{Highest}\mathrm{of}{R}_{\mathrm{NC}},G_{\mathrm{NC}},B_{\mathrm{NC}}}{16}\right)\right\}}& \mathrm{Eq}.\left(54\right)\end{array}$ $\begin{array}{cc}{B}_{\mathrm{Norm}}=\frac{\left(\frac{B_{\mathrm{NC}}}{16}\right)*196965}{\left\{\left(\frac{\mathrm{Highest}\mathrm{of}{R}_{\mathrm{NC}},G_{\mathrm{NC}},B_{\mathrm{NC}}}{16}\right)\right\}}& \mathrm{Eq}.\left(55\right)\end{array}$

If none of the results for R_NC, G_NC, or B_NC>196965, Equations 56, 57, and 58 may be utilized. Otherwise:

$\begin{array}{cc}{R}_{\mathrm{Norm}}=R_{\mathrm{NC}}& \mathrm{Eq}.\left(56\right)\end{array}$ $\begin{array}{cc}{G}_{\mathrm{Norm}}=G_{\mathrm{NC}}& \mathrm{Eq}.\left(57\right)\end{array}$ $\begin{array}{cc}{B}_{\mathrm{Norm}}=B_{\mathrm{NC}}& \mathrm{Eq}.\left(58\right)\end{array}$

Step 8: Convert R_Norm, G_Norm, and B_Norm to GS_NRed, GS_NGreen, GS_NBlue

Use an additional table in a reverse order to look up the GS_NRed, GS_NGreen, and GS_NBlue values that are then sent to a frame rate control (FRC) function or directly to the TMDS converter (bypass mode).

As a numeric example, the original orange icon case had input gray-shade values according to Equations 39, 40, and 41 with:

• • GS_Rin=250;
  • GS_Gin=102; and
  • GS_Bin=0.

The output values after the image enhancement transfer function were:

• • GS_Rout=1018;
  • GS_Gout=811; and
  • GS_Bout=0.

Using the alternate method, the multiplier, M_maxin, may be determined from the lookup table with a value per Equation 59 as follows:

$\begin{array}{cc}{M}_{\mathrm{maxin}}=\frac{2^{16}}{{\mathrm{GS}}_{\mathrm{maxin}}}=\frac{2^{16}}{250}=262& \mathrm{Eq}.\left(59\right)\end{array}$

Using Equations 39, 40, and 41 results in Equations 60, 61, and 62 as

$\begin{array}{cc}{\mathrm{GS}}_{\mathrm{Rout}}\times 2^{16}=250\times 1018\times 262=66679000& \mathrm{Eq}.\left(60\right)\end{array}$ $\begin{array}{cc}{\mathrm{GS}}_{\mathrm{Gout}}\times 2^{16}=102\times 1018\times 262=27205032& \mathrm{Eq}.\left(61\right)\end{array}$ $\begin{array}{cc}{\mathrm{GS}}_{\mathrm{Bout}}\times 2^{16}=0\times 1018\times 262=0& \mathrm{Eq}.\left(62\right)\end{array}$

Due to the 2¹⁶ multiplication embedded in the M_maxin multiplier, the values in Equation 59, 60, and 61 are divided by 216 to produce the following results:

• • GS_Rout=1017;
  • GS_Gout=415; and
  • GS_Bout=0.

The tristimulus color correction method produces the following results:

• • GS_Rout=1023;
  • GS_Gout=417; and
  • GS_Bout=0.

An implementation of the FPGA may include six multiplications and two 16-bit right shifts. The equation associated with the maximum value may not be computed because two terms cancel each other and the original value is obtained.

Although the output luminance is a bit lower by this alternate method, the output color is correct. This method is simple to implement, and the resulting luminance variations are imperceptible.

From a luminance aspect, the current transfer function methods may not produce the correct luminance for any color that does not have equal RGB gray shades. Luminance for white is correctly produced, and luminance is also correctly produced for any secondary color where two of the gray shades are matched and one gray shade is zero. All other color mixtures generally do not follow the intended Fechner curves. Other more complex methods may be implemented to accomplish this goal. As an example, the orange icon has input gray shades according to GS_Rin=250, GS_Gin=102, and GS_Bin=0. To study the luminance effects, one method is to convert pixel color gray shades to equivalent monochrome gray shade. The use of equivalent monochrome gray shades allows observation of what the image enhancement transfer function should have produced from a luminance aspect. To convert the orange icon to a monochrome gray-shade value, Equation 63, as follows, may be utilized where the coefficients are based on the sample tristimulus luminance data as recorded in Table 2.

$\begin{array}{cc}{\mathrm{GS}}_M={\left[\frac{L_R}{L_{m\mathrm{ax}}}{\left({\mathrm{GS}}_R\right)}^{\gamma }+\frac{L_G}{L_{m\mathrm{ax}}}{\left({\mathrm{GS}}_G\right)}^{\gamma }+\frac{L_B}{L_{m\mathrm{ax}}}{\left({\mathrm{GS}}_B\right)}^{\gamma }\right]}^{\left(1/\gamma \right)}& \mathrm{Eq}.\left(63\right)\end{array}$

GS_M in Equation 60 is the equivalent monochrome gray shade that provides the display luminance when used with Equation 64 as follows:

$\begin{array}{cc}L={L_{m\mathrm{ax}}\left(\frac{{\mathrm{GS}}_M}{{\mathrm{GS}}_{m\mathrm{ax}}}\right)}^{\gamma }& \mathrm{Eq}.\left(64\right)\end{array}$

By substituting Equation 63 into Equation 64, Equation 65, as follows, may be used to prove that Equation 62 is correct.

$\begin{array}{cc}L={L_{m\mathrm{ax}}\left(\frac{{\left[\frac{{L_R\left({\mathrm{GS}}_R\right)}^{\gamma }}{L_{m\mathrm{ax}}}+\frac{{L_G\left({\mathrm{GS}}_G\right)}^{\gamma }}{L_{m\mathrm{ax}}}+\frac{{L_B\left({\mathrm{GS}}_B\right)}^{\gamma }}{L_{m\mathrm{ax}}}\right]}^{\left(1/\gamma \right)}}{{\mathrm{GS}}_{m\mathrm{ax}}}\right)}^{\gamma }& \mathrm{Eq}.\left(65\right)\end{array}$

Using the “Power of a Fraction Rule” per Table 3, Equation 65 may be manipulated to reveal Equation 66 as follows:

$\begin{array}{cc}L=L_{\mathrm{ma}x}\left(\frac{{\left({\left[\frac{{L_R\left({\mathrm{GS}}_R\right)}^{\gamma }}{L_{m\mathrm{ax}}}+\frac{{L_G\left({\mathrm{GS}}_G\right)}^{\gamma }}{L_{m\mathrm{ax}}}+\frac{{L_B\left({\mathrm{GS}}_B\right)}^{\gamma }}{L_{m\mathrm{ax}}}\right]}^{\left(1/\gamma \right)}\right)}^{\gamma }}{{\left({\mathrm{GS}}_{\mathrm{ma}x}\right)}^{\gamma }}\right)& \mathrm{Eq}.\left(66\right)\end{array}$

Next the “Power of a Power Rule” may be employed to simplify Equation 66 into Equation 67 as follows:

$\begin{array}{cc}L=L_{m\mathrm{ax}}\left(\frac{\frac{{L_R\left({\mathrm{GS}}_R\right)}^{\gamma }}{L_{m\mathrm{ax}}}+\frac{{L_G\left({\mathrm{GS}}_G\right)}^{\gamma }}{L_{m\mathrm{ax}}}+\frac{{L_B\left({\mathrm{GS}}_B\right)}^{\gamma }}{L_{m\mathrm{ax}}}}{{\left({\mathrm{GS}}_{m\mathrm{ax}}\right)}^{\gamma }}\right)& \mathrm{Eq}.\left(67\right)\end{array}$

The L_max terms in Equation 67 may be cancelled thereby yielding Equation 68 as follows:

$\begin{array}{cc}L=\frac{{L_R\left({\mathrm{GS}}_R\right)}^{\gamma }+{L_G\left({\mathrm{GS}}_G\right)}^{\gamma }+{L_B\left({\mathrm{GS}}_B\right)}^{\gamma }}{{\left({\mathrm{GS}}_{m\mathrm{ax}}\right)}^{\gamma }}& \mathrm{Eq}.\left(68\right)\end{array}$

Using the distributive law, Equation 68 may be rewritten as Equation 69 as follows:

$\begin{array}{cc}L=\frac{{L_R\left({\mathrm{GS}}_R\right)}^{\gamma }}{{\left({\mathrm{GS}}_{m\mathrm{ax}}\right)}^{\gamma }}+\frac{{L_G\left({\mathrm{GS}}_G\right)}^{\gamma }}{{\left({\mathrm{GS}}_{m\mathrm{ax}}\right)}^{\gamma }}+\frac{{L_B\left({\mathrm{GS}}_B\right)}^{\gamma }}{{\left({\mathrm{GS}}_{m\mathrm{ax}}\right)}^{\gamma }}& \mathrm{Eq}.\left(69\right)\end{array}$

Finally, the “Power of a Fraction Rule” per Table 3 may be used to yield Equation 70 as follows:

$\begin{array}{cc}L={L_R\left(\frac{{\mathrm{GS}}_R}{{\mathrm{GS}}_{m\mathrm{ax}}}\right)}^{\gamma }+{L_G\left(\frac{{\mathrm{GS}}_G}{{\mathrm{GS}}_{m\mathrm{ax}}}\right)}^{\gamma }+{L_B\left(\frac{{\mathrm{GS}}_B}{{\mathrm{GS}}_{m\mathrm{ax}}}\right)}^{\gamma }& \mathrm{Eq}.\left(70\right)\end{array}$

The coefficients in Equation 63 are derived by using the “Y” row values of Table 2 per Equations 71, 72, and 73 as follows:

$\begin{array}{cc}{R}_C=\frac{172.27}{172.27+622.85+69.7}=0\text{.199}& \mathrm{Eq}.\left(71\right)\end{array}$ $\begin{array}{cc}{G}_C=\frac{622.85}{172.27+622.85+69.7}=0.720& \mathrm{Eq}.\left(72\right)\end{array}$ $\begin{array}{cc}{B}_C=\frac{69.7}{172.27+622.85+69.7}=0.081& \mathrm{Eq}.\left(73\right)\end{array}$

Inserting these coefficients into Equation 63, results in Equation 74 as follows:

$\begin{array}{cc}{\mathrm{GS}}_M={\left[0.199{\left({\mathrm{GS}}_R\right)}^{\gamma }+0.72{\left({\mathrm{GS}}_G\right)}^{\gamma }+0.081{\left({\mathrm{GS}}_B\right)}^{\gamma }\right]}^{\left(1/\gamma \right)}& \mathrm{Eq}.\left(74\right)\end{array}$

Substituting the icon input RGB gray-shade values GS_Rin=250, GS_Gin=102, and GS_Bin=0 in Equation 74 produces Equation 75 as follows:

$\begin{array}{cc}{\mathrm{GS}}_M={\left[0.199{\left(250\right)}^{2.2}+0.72{\left(102\right)}^{2.2}+0.081{\left(0\right)}^{2.2}\right]}^{\left(1/2.2\right)}& \mathrm{Eq}.\left(75\right)\end{array}$

Performing the calculation in Equation 75 shows that the equivalent monochrome gray shade is 144.

The luminance associated with gray shade 144 may be determined using Equation 64 with L_max=864.82, as shown in Equation 76 as follows:

$\begin{array}{cc}L=864.82{\left(\frac{144}{255}\right)}^{2.2}=246\mathrm{nits}& \mathrm{Eq}.\left(76\right)\end{array}$

Note that L_max is the summation of the “Y” row terms in the tristimulus matrix Table 2.

In the uncompensated image enhancement lookup table method, where the icon color transitioned from orange to yellow, the new gray-shade values where GS_Rout=1018, GS_Gout=811 and GS_Bout=0. The equivalent monochrome 10-bit gray shade for the yellow icon may be determined using Equation 74 as shown with numeric values per Equation 77 as follows:

$\begin{array}{cc}\begin{array}{c}{\mathrm{GS}}_M={\left[0.199{\left(1018\right)}^{2.2}+0.72{\left(811\right)}^{2.2}+0.081{\left(0\right)}^{2.2}\right]}^{\left(1/2.2\right)}\\ {\mathrm{GS}}_M=829\end{array}& \mathrm{Eq}.\left(77\right)\end{array}$

The luminance associated with 10-bit monochrome gray shade 829 may be determined according to Equation 78 as follows:

$\begin{array}{cc}L=864.82{\left(\frac{829}{1023}\right)}^{2.2}=545\mathrm{nits}& \mathrm{Eq}.\left(78\right)\end{array}$

The equivalent monochrome gray shade (GS_M) for the tristimulus based gray shades GS_Rout=1023, GS_Gout=417, and GS_Bout=0 is calculated per Equation 79 as follows:

$$\begin{gathered} \begin{array}{cc}G{S}_M={\left[0.199{\left(1023\right)}^{2.2}+0.72{\left(417\right)}^{2.2}+0.081{\left(0\right)}^{2.2}\right]}^{\left(1/2.2\right)}\text{ \\ G⁢SM=5⁢9⁢1}& \mathrm{Eq}.\left(79\right)\end{array} \end{gathered}$$

The luminance associated with 10-bit monochrome gray shade 591 may be determined according to Equation 80 as follows:

$\begin{array}{cc}L=864.82{\left(\frac{591}{1023}\right)}^{2.2}=259\mathrm{nits}& \mathrm{Eq}.\left(80\right)\end{array}$

The alternate color correction method produced gray-shade values according to Equations 60, 61, and 62 with the resulting monochrome gray shade per Equation 81.

$$\begin{gathered} \begin{array}{cc}G{S}_M={\left[0.199{\left(1017\right)}^{2.2}+0.72{\left(415\right)}^{2.2}+0.081{\left(0\right)}^{2.2}\right]}^{\left(1/2.2\right)}\text{ \\ G⁢SM=5⁢8⁢8}& \mathrm{Eq}.\left(81\right)\end{array} \end{gathered}$$

The luminance associated with 10-bit monochrome gray shade 588 may be determined according to Equation 82 as follows:

$\begin{array}{cc}L=864.82{\left(\frac{588}{1023}\right)}^{2.2}=256\mathrm{nits}& \mathrm{Eq}.\left(82\right)\end{array}$

FIG. 9 illustrates a graph 190 of an example image enhancement transfer function 192 in accordance with one or more exemplary embodiments. Notice that although the alternate color correction method may not provide the maximum possible orange luminance, the value is close due to the fact that the red color is in the upper compression zone of the transfer function 192 as depicted in FIG. 9.

The original transfer function method is based on obtaining the desired luminance based on the Burnette Fechner curves. For the color white, which has equal RGB gray-shade values, the desired output luminance is obtained using the transfer function. However, for secondary colors like orange that do not have equal gray-shade values, the correct luminance may not be obtained. As an example, a monochrome input gray shade for the orange icon was determined to be a value of 144 per Equation 61 above. If the value of 144 is used in conjunction with the transfer function per FIG. 9, the desired output luminance would be gray shade 894 that would translate into 642 nits using Equation 83 as follows:

$\begin{array}{cc}L=864.82{\left(\frac{894}{1023}\right)}^{2.2}=642\mathrm{nits}& \mathrm{Eq}.\left(83\right)\end{array}$

However, this luminance may not be realizable because the red luminance suitable to obtain overall luminance of 642 nits may be beyond what the display is capable of supplying and therefore the luminance is scaled back to maintain the orange color. The reason that the orange icon turns yellow is because the input red gray shade of 250 is in the upper compression zone, while the green gray shade of 102 is just coming out of the “linear” zone with a different slope. Therefore, since one of the gray shades is in the upper compression gray shades, it is already near the maximum value and therefore this is one of the reasons that the alternate color method may be utilized and causes little difference in the results when compared to the more computation intensive tristimulus color correction method. Table 4 is a summary of the results.

TABLE 4
Summary for Input GS (R, G, B) = (250, 102, 0)
Configuration	GSM	Luminance (Nits)	Color
Input	144 (8-bit)	246	Orange
	(578 10-bit)
No Color	829 (10-bit)	545	Yellow
Correction
Tristimulus Color	591 (10-bit)	259 (bounded	Orange
Correction		by max red GS)
Alternate Color	588 (10-bit)	256	Orange
Correction

Table 4 shows that there is a negligible difference in the final luminance value for the alternate color correction method versus the tristimulus color correction method. Therefore, the use of the alternate color correction method has minimal effect of the results due to the nature of the monotonic compression of the upper gray shades.

The second aspect of the alternate color correction method is that the curve below the compression zone is mostly linear in nature while the offset gamma function is used. For example, suppose the color orange has a red gray-shade value of 75 below the compression break point. The corresponding green gray-shade value may be determined according to Equation 84 as follows:

$\begin{array}{cc}G{S}_G=\frac{102}{250}75=31& \mathrm{Eq}.\left(84\right)\end{array}$

These three values may represent the input orange gray-shade value of (75,31,0).

The image enhancement transfer function before color correction would convert the 8-bit values of (75,31,0) to 10-bit values of (639,302,0). This results in an equivalent monochrome gray shade of 389 according to Equation 85 as follows:

$$\begin{gathered} \begin{array}{cc}G{S}_M={\left[0.199{\left(639\right)}^{2.2}+0.72{\left(302\right)}^{2.2}+0.081{\left(0\right)}^{2.2}\right]}^{\left(1/2.2\right)}\text{ \\ G⁢SM=3⁢8⁢9}& \mathrm{Eq}.\left(85\right)\end{array} \end{gathered}$$

The monochromic gray shade of 389 results in a luminance value of 103 nits according to Equation 86 as follows:

$\begin{array}{cc}L=864.82{\left(\frac{389}{1023}\right)}^{2.2}=103\mathrm{nits}& \mathrm{Eq}.\left(86\right)\end{array}$

Using the alternate color correction method, the new output gray shades may be determined according to Equations 87, 88, and 89 as follows:

$\begin{array}{cc}G{S}_{Rout}=75\left(\frac{639}{75}\right)=639& \mathrm{Eq}.\left(87\right)\end{array}$ $\begin{array}{cc}G{S}_{Gout}=31\left(\frac{639}{75}\right)=264& \mathrm{Eq}.\left(88\right)\end{array}$ $\begin{array}{cc}G{S}_{Bout}=0\left(\frac{639}{75}\right)=0& \mathrm{Eq}.\left(89\right)\end{array}$

The equivalent monochrome gray-shade value is calculated per Equation 90 as follows:

$$\begin{gathered} \begin{array}{cc}G{S}_M={\left[0.199{\left(639\right)}^{2.2}+0.72{\left(264\right)}^{2.2}+0.081{\left(0\right)}^{2.2}\right]}^{\left(1/2.2\right)}\text{ \\ G⁢SM=3⁢7⁢1}& \mathrm{Eq}.\left(90\right)\end{array} \end{gathered}$$

The gray shade of 371 results in a luminance value of 93 nits according to Equation 91 as follows:

$\begin{array}{cc}L=864.82{\left(\frac{371}{1023}\right)}^{2.2}=93\mathrm{nits}& \mathrm{Eq}.\left(91\right)\end{array}$

In summary, the tristimulus color correction method may have corrected the color with an output luminance of 103 nits compared to the alternate color correction method that yield 93 nits. Both methods produce the correct orange color coordinate. The difference of 93 nits compared with 103 nits is generally not noticeable. In addition, the correct Burnette transfer function may have taken the equivalent monochrome gray calculated by Equation 92 as follows:

$$\begin{gathered} \begin{array}{cc}G{S}_M={\left[0.199{\left(75\right)}^{2.2}+0.72{\left(31\right)}^{2.2}+0.081{\left(0\right)}^{2.2}\right]}^{\left(1/2.2\right)}\text{ \\ G⁢SM=4⁢4}& \mathrm{Eq}.\left(92\right)\end{array} \end{gathered}$$

The input luminance may be determined according to Equation 93 as follows:

$\begin{array}{cc}L=864.82{\left(\frac{44}{255}\right)}^{2.2}=18\mathrm{nits}& \mathrm{Eq}.\left(93\right)\end{array}$

The image enhancement transfer function converts the gray shade 44 to a 10-bit gray-shade value of 404 resulting in a luminance of 112 nits per Equation 94 as follows:

$\begin{array}{cc}L=864.82{\left(\frac{404}{1023}\right)}^{2.2}=112\mathrm{nits}& \mathrm{Eq}.\left(94\right)\end{array}$

The original color correction method and alternate color correction method produce reasonably close values to the intended luminance, and the differences may not be noticeable. A summary is presented in Table 5. Although there is a 20% decrease in luminance for the alternate color correction method from the ideal case, the decrease may not be noticed due to the counter measure of the Helmholtz effect.

TABLE 5
Summary for Input GS (R, G, B) = (75, 31, 0)
Configuration	GSM	Luminance (Nits)	Color
Input	44 (8-bit)	18	Orange
	(177 10-bit)
No Color	389 (10-bit)	103
Correction
Tristimulus Color	389 (10-bit)	103	Orange
Correction
Alternate Color	371 (10-bit)	93	Orange
Correction
Exact Solution	404 (10-bit)	112	Orange

The tristimulus color correction method may be improved by converting the input gray shades to a monochrome gray shade and then applying the transfer function to the input monochrome gray shade to obtain the desired output luminance value. The desired output luminance value may then be used to derive the correct color coordinates (purest mind set). By implementing the alternate color correction method, the FPGA calculations and resources are reduced, and a 90% solution is obtained that may not be noticeable by the user from the exact solution.

Test Image Simulation Comparison

FIG. 10 illustrates a graph 200 of an example dynamic Adaptive Image Enhancement (AIE) principle in accordance with one or more exemplary embodiments. An original luminance output L_O (curve 202) may be shifted to a new luminance output L_ONew (curve 204). An original breakpoint N_Max (line 206) may be shifted to a new breakpoint N_MaxNew (line 208).

FIG. 11 illustrates an example intermediate functional block diagram 230 of a system in accordance with one or more exemplary embodiments.

In various embodiments, the diagram 230 may be implemented with a field programmable gate array (FPGA) or application specific integrated circuit (ASIC) 232 connected to a vehicle interface processor (VIP) 234 or other processor. Each of the blocks in the diagram are described as follows:

Pixel Luminance Converter Block 236:

The pixel luminance converter block 236 converts every RGB pixel into a monochrome gray shade value based on Equation 95 where the RGB coefficients are (0.19, 0.68, 0.13).

$\begin{array}{cc}{\left(\frac{G{S}_M}{G{S}_{\mathrm{Max}}}\right)}^{\gamma }=0.19{\left(\frac{G{S}_R}{G{S}_{\mathrm{Max}}}\right)}^{\gamma }+0.68{\left(\frac{G{S}_G}{G{S}_{\mathrm{Max}}}\right)}^{\gamma }+0.13{\left(\frac{G{S}_B}{G{S}_{\mathrm{Max}}}\right)}^{\gamma }& \mathrm{Eq}.\left(95\right)\end{array}$

Cancelling terms and solving for the equivalent monochrome gray shade results in Equation 96.

$\begin{array}{cc}G{S}_M={\left[0.19{\left(G{S}_R\right)}^{\gamma }+0.68{\left(G{S}_G\right)}^{\gamma }+0.13{\left(G{S}_B\right)}^{\gamma }\right]}^{\left(\frac{1}{\gamma }\right)}& \mathrm{Eq}.\left(96\right)\end{array}$

Gray Scale Data Accumulators Block 238:

Each pixel is converted into a monochrome gray scale value per Equation 101 and a value of 1 is added for each pixel to each corresponding gray scale accumulator consisting of GS_Max accumulators for each frame. For 8-bit video, this would use 256 accumulators. Histogram data is sent to the vehicle interface processor 234 at the end of each video frame. In addition, a Frame End Trigger (FET) is sent to the vehicle interface processor 234 indicating that the accumulators are ready to be down loaded to the vehicle interface processor 234. In addition, the frame end trigger is used to trigger the uploading of the AIE-LVE assignment table in preparation for the next frame of video conversion. Finally, the frame end trigger is used to signal that the new light sensor data is to be sampled. The light sensor sampling generally occurs once per frame cycle such that all functions used the same filtered data LBG light sensor data for each frame.

Metadata AIE-LVE Bypass Selector Block 240:

The function of bypass selector block 240 is to use the metadata embedded in the video stream to determine safety related or other video symbols to which the AIE-LVE enhancement is not to be applied. In this case, the AIE-LVE assignment table is not utilized and the video for the areas are sent directly to the display.

Video Gray Shade Assignment Table Block 242:

The gray shade assignment table block 242 utilizes the AIE-LVE assignment table to convert the incoming video gray shade values into the enhanced video gray shade values. Table 1 is an example of a video look up table that is uploaded by the FPGA or ASIC 232 during the video vertical blanking time in preparation for the next active video time period. The output from the gray scale assignment table may be either 10-bit video or 11-bit video.

10-Bit or 11-Bit Frame Rate Control Block 244:

The frame rate control block 244 applies frame rate control (FRC) methods to convert the 10-bit (or 11-bit) video data (high-dynamic range video) to dithered 8-bit video.

Gray Shade Detection Threshold Determination Block 246:

The gray shade detection threshold determination block 246 determines the maximum gray scale number such that a specific percentage of the pixels are above a threshold percentage according to Equation 97. Therefore, the method starts at a highest accumulator and keeps adding the next lower accumulator value until the threshold percentage is determined for the corresponding gray shade value. This is the GS_Max N value used for stretching the dynamic video range after being filtered into variable N_Max by the rate limiting filter per block 248.

$\begin{array}{cc}\mathrm{If}\left[\frac{\mathrm{SUM}\left(N:N_{\mathrm{Max}}\right)}{\mathrm{SUM}\left(0:N_{\mathrm{Max}}\right)}>\frac{T_{0/0}}{100},{\mathrm{GS}}_{\mathrm{Max}}=N\right]& \mathrm{Eq}.\left(97\right)\end{array}$

Max Gray Shade Filter Block 248:

The filter block 248 is a rate limiter whereby the N_Max value in incremented or decremented by one gray shade value (8 bit) towards the GS_Max value each TL time increment. The rate of the filter is set by the TL time increment input.

Backlight Level Block 250:

A commanded backlight pulse width modulation (PWM) level is obtained and used to determine the display luminance, L_DMax, per Equation 98 as follows, where LD is the display luminance at 100% backlight drive, % BL.

$\begin{array}{cc}{L}_{\mathrm{DMax}}={\%}_{BL}\times L_D& \mathrm{Eq}.\left(98\right)\end{array}$

Light Sensor Pre-Filter Block 252:

In response to the Frame End Trigger (FET), N samples of the light sensor are successively measured and averaged resulting in the variable L_BGNew per Equation 99 as follows:

$\begin{array}{cc}{L}_{BGNew}=R_F\times AVG\left(N_{S\mathrm{amples}}\right)& \mathrm{Eq}.\left(99\right)\end{array}$

Note that the reflectance factor, R_F, is used to determine how much of the light sensor value is reflected to the user and also to do any conversion to user observed nits.

Light Sensor Filter Block 254:

The light sensor value obtained in block 252 is further filtered according to Equation 100 to obtain a faster exponential rise time of about 1 second than the fall time of about 60 seconds to mimic the eye adaptation times of the human visual system and to provide a peak detector function for the picket fence effects.

$\begin{array}{cc}{L}_{BG}=\frac{L_{BGNew}+\left(F-1\right)L_{B\mathrm{GOld}}}{F}& \mathrm{Eq}.\left(100\right)\end{array}$

The F values are selected to be F_Up when the new light sensor value, L_BGNew is larger than the previous filtered value, L_BGOld, and F_Down where the new light sensor value, L_BGNew is smaller than the previous filtered value, L_BGOld. The time constant for the filter is selected for the 1 second rise time and 60 second fall time goal.

L_Max Determination Block 256:

The user observed maximum luminance is determined per Equation 101, as follows, and is the summation of the end point display luminance (reference block 270) and the reflected background luminance, LBG, that the user sees.

$\begin{array}{cc}{L}_{\mathrm{Max}}=L_{\mathrm{DLimit}}+L_{BG}& \mathrm{Eq}.\left(101\right)\end{array}$

L_Max is the higher end point for Equation 109 or Equation 110.

Gray Shade 0 Calculator Block 258:

Gray shade 0 (GS₀) needs to be handled as a special case. If there is no active privacy function on the display, GS₀ is set to 0 nits to obtain the appearance of a black display where no information is present. If the display does have the privacy function active, the GS₀ background luminance may be raised to reduce the contrast ratio of the image. In various embodiments, the Weber Fraction level may be less than 25 for low black luminance values.

The Weber Fraction is the amount of white crosstalk that the driver sees when observing the passenger display. The “Fraction” is the amount of white luminance seen divided by the black luminance that the driver sees as formulated per Equation 102 as follows:

$\begin{array}{cc}{W}_F=\frac{W_{Leakage}}{L_{GS0}+L_{BG}}<25& \mathrm{Eq}.\left(102\right)\end{array}$

• • Where:
  • W_Leakage=the amount of white leakage that the user sees;
  • L_GS0=the amount of display black luminance; and
  • L_BG=the amount of reflected background luminance that the user sees at night.

Equation 102 may be rearranged to yield Equation 103, as follows, that determines how much the display black luminance value may be increased to obtain a Weber Fraction of less than 25.

$\begin{array}{cc}{L}_{GS0}>\frac{W_{Leakage}}{25}{L}_{BG}& \mathrm{Eq}.\left(103\right)\end{array}$

For proper operation in the formula for L_Min, L_BG is added to the display luminance to obtain L_Min. Therefore, to meet the Weber Fraction criteria, L_Min may be determined per Equation 104 as follows:

$\begin{array}{cc}{L}_{\mathrm{Min}}>\frac{W_{Leakage}}{25}& \mathrm{Eq}.\left(104\right)\end{array}$

Such a AIE-LVE implementation provides that the black level luminance, L_GS0, may be increased to meet the Weber Fraction requirement.

L_Min Determination Block 260:

A minimum luminance level for the assignment table calculation is based on the larger of the Weber Fraction black luminance level calculated per Equation 105 or the minimum luminance level calculated for display visibility per Equation 106 as follows:

$\begin{array}{cc}{L}_{\mathrm{Min}}=\frac{W_{Leakage}}{25}& \mathrm{Eq}.\left(105\right)\end{array}$ $\begin{array}{cc}{L}_{\mathrm{Min}}={B_O\left(L_{BG}\right)}^C+L_{BG}& \mathrm{Eq}.\left(106\right)\end{array}$

The Fechner offset B_O and slope c constants are used together with the background luminance, L_BG, to determine the minimum luminance, L_Min, that the user needs to barely perceive the lowest gray shade information values. L_Min is the lower end point for Equation 109 or Equation 110. The minimum Fechner constant value may be calculated per Equation 107 as follows:

$\begin{array}{cc}{L}_{\mathrm{MinFechner}}={B_O\left(L_{BG}\right)}^C+L_{BG}& \mathrm{Eq}.\left(107\right)\end{array}$

Assignment Table Calculator Block 262:

The assignment table calculation may have two parts. A first part is the calculation for the “black” level GS0 per Equation 108 as follows:

$\begin{array}{cc}G{S}_0=\left(2^{N_{New}}-1\right){L_{\mathrm{DMax}}^{\left(-1/{\gamma }_D\right)}\left[\left[L_{GS0}\right]\right]}^{\left(\frac{1}{{\gamma }_D}\right)}& \mathrm{Eq}.\left(108\right)\end{array}$

A second calculation involves first choosing which assignment table calculation method may be utilized based on the “Table Select” command.

• • 1. Constant Contrast Ratio Assignment Table 264; or
  • 2. Gamma Assignment Table 266.

Constant Contrast Ratio Assignment Table 264:

The process of calculating the intermediate input video gray shade values GS₁ through GS_NMax between L_Min and L_Max such that each successive gray shade has the same contrast ratio taking the reflected ambient into account the reflected ambient per Equation 109 as follows:

$\begin{array}{cc}G{S}_N=\left(2^{N_{New}}-1\right){L_{\mathrm{DMax}}^{\left(-1/{\gamma }_D\right)}\left[{{\left[L_{\mathrm{Max}}\right]}^{\left(\frac{N-1}{N_{\mathrm{Max}}-1}\right)}\left[L_{\mathrm{Min}}\right]}^{\left(\frac{N_{\mathrm{Max}}-N}{N_{\mathrm{Max}}-1}\right)}-L_{BG}\right]}^{\left(\frac{1}{{\gamma }_D}\right)}& \mathrm{Eq}.\left(109\right)\end{array}$

Gamma Assignment Table 266: The process of calculating the intermediate input video gray shade values

GS₁ through GS_NMax between L_Min and L_Max such that each of the gray shades follow an offset gamma function are determined according to Equation 110 as follows:

$\begin{array}{cc}{\mathrm{GS}}_N=\left(2^{N_{New}}-1\right){L_{\mathrm{DMax}}^{\left(-1/{\gamma }_D\right)}\left[\left(L_{\mathrm{Max}}-L_{\mathrm{Min}}\right){\left(\frac{N}{N_{\mathrm{Max}}}\right)}^{{\gamma }_G}+\left(L_{\mathrm{Min}}-L_{BG}\right)\right]}^{\left(\frac{1}{{\gamma }_D}\right)}& \mathrm{Eq}.\left(110\right)\end{array}$

Assignment Table Modifiers Block 268:

Block 268 provides an assignment table developed per Block 262 and is modified by multiplying other shaping functions.

End Point Backlight Level Block 270:

Block 270 provides some overhead luminance that may be used in conjunction with AIE. If AIE is not used, or no overhead is desired, the input variable “% End Point Overhead” is set to zero. The end point modifier function in Block 270 may be activated if the “% End Point Overhead” is not zero.

Selector S1, Metadata Control:

The selector S1 block 272 has two functions. If the metadata control is activated, the metadata is used to either bypass the AIE-LVE video assignment table or use the assignment table as a function of the metadata. If the metadata control is deactivated, the assignment table is used for all video data.

Selector S2, FRC Bypass Control:

The selector S2 block 274 controls whether the Frame Rate Control function is utilized. If the display has a native 10-bit or 11-bit video input, the FRC function may be bypassed.

Selector S3, Assignment Table Upload Control:

The selector S3 block 276 is triggered to upload the video assignment table as a function of the Frame End Trigger (FET) emanating from the FPGA (or ASIC) when the last video pixel for the current frame has been received. The video assignment table is to be uploaded during the video vertical blanking time in preparation for the next frame.

Selector S4, AIE Activation Control:

The selector S4 block 278 control activates the AIE stretching function. If AIE stretching is not activated, N_Max is set to 255 and therefore all input gray shades (e.g., 0-255) are formulated to have a constant contrast ratio or gamma offset between L_Min and L_Max. If however the AIE stretching function is activated, N_Max is set according to the filtered video accumulator threshold per Blocks 246 and 248.

Selector S5, Active Privacy Control:

The selector S5 block 280 control is utilized to activate raising the black gray shade 0 luminance, L_GS0, to a level required to meet the Weber Fraction criteria. This has the effect of raising the black level for both the driver and the passenger. Since the black level luminance, L_GS0, is a function of the (ambient) light sensor determined background luminance, L_BG, per Equation 108, as the reflected background luminance increases towards daylight conditions, the black level luminance, L_GS0, is decreased such that under daytime conditions the black gray shade luminance is not increased.

VIP and FPGA Input List and Description:

Table 4 is a tabulation of the various inputs to the AIE-LVE software (or code). The variables may be stored in a Variable Control Register (VCR) that may be changed via a controller area network (CAN) bus or other communication methods.

TABLE 4
VIP Input List Description
I.D.	Name	Description
0	Threshold %	% of upper gray shade
		pixels that are not stretched
		by AIE. The range for this
		variable is 0% to 30% with a
		typical value of 5%.
1	TL	GSMax rate limiter ΔT.
		This is the time between
		Incrementing or decrementing
		the NMax towards GSMax.
		This time will be in the range
		of 10 ms to 100 ms. The
		default value should be 10 ms
2	AE Sel	Enables or disables AIE
		stretching function.
		0 = Disable and NMax is set to 255.
		1 = Enable. The default
		value should be 1.
3	γD	This is the gamma value
		for the display which is
		normally 2.2. The range for
		this gamma will be from 1
		to 3 with two decimal points.
		The default value should be 2.2.
4	NNew	This is the number of bits
		for the output of the assignment
		table calculator (normally 10 or
		11). The selection should be 8,
		10, 11 or 12 bits. The default
		value should be 8 bits.
5	LDisplay	Specifies the target display
		luminance to the algorithm.
		The range of this variable
		will be 500 to 1500 nits with
		1000 nits as the default value.
6	Backlight %	Specifies the backlight PWM
		% commanded to the display.
		This variable will range from
		0% to 100% with 100% being the
		default value.
7	FUp	Filter Up weighting number. For a
		16.67 ms frame loop time, the
		factor will be 32 for a 10% to
		90% time of 1.2 seconds.
		The range should be 16 to 64
		with increments of 16 with the
		default value of 32.
8	FDown	Filter Down weighting number. For a
		16.67 ms frame loop time, the
		factor will be 2048 for a 90%
		to 10% time of 75 seconds.
		The range should be 256
		to 2304 with increments of 256
		with the default value of 2048.
9	Reflectance	Display reflectance factor used
	Factor	with light sensor input to
		determine the reflected
		luminance that the user sees in
		nit units. The range of the
		reflectance should be 0.005
		to 0.100 with 3 decimal places
		and a default value of 0.01.
10	Light Sensor	Light sensor input in Lux
		units. This input will range
		from 1 lux to 100K lux with
		a default value of 5K lux.
11	NSamples	Number of samples to be
		averaged by the pre-filter. This
		number can range from 4 to
		32 by increments of 4
		and a default value of 16.
12	WLeakage	The amount of white leakage
		that the driver sees in privacy
		mode. The amount can range from
		0.1 to 10 nits with a default
		value of 0.75 nits.
13	Active	Conveys active privacy mode
	Privacy	to algorithm. 0 = No Privacy.
		1 = Active Privacy. Default is 0.
14	Fechner	Offset BO constant in nits.
	Constants	Slope Constant c. These constants
		are used to determine the
		minimum viewing luminance as a
		function of the light sensor
		input and reflectance factor.
		The offset constant BO will
		range from 1 to 100 with a
		default value of 11. Use 1
		decimal place. The slope
		constant “c” will range from
		0.1 to 0.5 with a default value
		of 0.273 (3 decimal places).
15	Table Mod	Selects which table modification
	Selection	functions are activated.
16	Table Mod	(not shown in Block diagram)
	Upload	Allows uploading of tables.
17	% End Point	This is the percentage of
	Overhead	the maximum display
		luminance that is specified to
		use for input gray shade
		levels that are above the
		threshold. The percentage
		should range from 0 to 50%.
18	Table Select	Selects either between the
		Constant Contrast Ratio
		Assignment Table (Table
		Select = 1, default) or the
		Gamma Assignment Table
		(Table Select = 1)
s19	γG	This is the gamma value
		for the gamma function
		assignment table which is
		normally 2.2. The range for
		this gamma will be from
		0.5 to 3 with two decimal
		points. The default value
		should be 2.2.
20	Modifiers	This will be an 8 bit word that
		controls invocation of modifier
		functions.
		Bit 0: M0 = 0 (No Modifiers),
		1 (Modifiers)
		Bit 1: M1 = 0 (No Mod), 1 (End
		Modifier)
		Bit 2: M2 = 0 (No Mod), 1 (IPM
		Modifier)
		Bit 3: M3 = 0 (No Mod), 1
		(Gamma Modifier)
		Bits 4-7: M4 = 0 (No Mod), 1
		(Reserved)
21	M1 Inputs	End Point Modifier Function:
		“% End Point Overhead”
		input is used to allocate how
		much luminance is allocated for
		the input gray shades that are
		greater than the threshold.
22	M2 Inputs	Inflection Point Modifier (IPM)
		Function:
		A1 = Gain of 1st phase which
		can range from −0.5 to +0.5
		with a default value of 0.
		A2 = Gain of 2nd phase which
		can range from −0.5 to +0.5
		with a default value of 0.
		NI = Input gray shade
		inflection number.
23	M3 Inputs	Gamma Modifier Function:
		γM = −0.9 to 3 with a default
		value of 0.5.

The methodology of how much the stretching should be decreased as the ambient light level is lowered, is complex and many possible solutions exist. In various embodiments, the stretching is based on the premise that most of the image is stretched below the N_Max input gray shade level. Since most of the mage luminance is controlled to be highest at the N_Max input gray shade, the luminance at the N_Max location may be lowered as a function of the ambient light level by increasing the N_Max gray shade to a new N_MaxNew gray shade level. As shown in FIG. 10, the ideal is to move the transfer function such that the luminance is decreased to the level appropriate to maintain display visibility at the N_Max gray shade. By taking such an approach, the image is less stretched under lower ambient lighting levels and a new N_MaxNew threshold point is utilized. The idea of this approach is to calculate a new threshold, N_MaxNew, based on the light sensor data and to use the new threshold value instead of N_Max value in the method.

Those having ordinary skill in the art will recognize that terms such as “above,” “below,” “front,” “back,” “upward,” “downward,” “top,” “bottom,” etc., may be used descriptively herein without representing limitations on the scope of the disclosure. Furthermore, the present teachings may be described in terms of functional and/or logical block components and/or various processing steps. Such block components may be comprised of various hardware components, software components executing on hardware, and/or firmware components executing on hardware.

The foregoing detailed description and the drawings are supportive and descriptive of the disclosure, but the scope of the disclosure is defined solely by the claims. As will be appreciated by those of ordinary skill in the art, various alternative designs and embodiments may exist for practicing the disclosure defined in the appended claims.

Claims

1. A true color image enhancement compensation system comprising: an ambient light sensor operational to measure an ambient light level;a circuit operational to: generate a histogram based on an input video signal;export the histogram;receive a gray shade look up table; andgenerate an output video signal by converting a plurality of gray shades in the input video signal based on a plurality of input video luminance ratios and the gray shade look up table to determine a plurality of new output gray shade values to maintain input video color coordinates; anda processor operational to: receive the histogram from the circuit;develop the gray shade look up table based on the histogram and the ambient light level;dynamically remap the plurality of gray shades in the gray shade look up table in response to the ambient light signal to compress a first region of the plurality of gray shades and stretch a second region of the plurality of gray shades;dynamically remap the plurality of gray shades in the gray shade look up table to correct for ambient lighting conditions; andtransfer the gray shade look up table to the circuit.

2. The true color image enhancement compensation system according to claim 1, wherein the processor includes: a constant contrast table operational to adjust each successive gray shade of the plurality of gray shades to a constant contrast ratio.

3. The true color image enhancement compensation system according to claim 1, wherein the processor includes: a gamma table operational to adjust each successive gray shade of the plurality of gray shades to follow an offset gamma function.

4. The true color image enhancement compensation system according to claim 1, wherein the processor includes: a plurality of table modifiers operational to modify one of a plurality of shaping functions for the plurality of gray shades.

5. The true color image enhancement compensation system according to claim 1, wherein the processor includes: a gray shade detection threshold operational to determine a maximum gray scale number such that a specific percentage of a plurality of pixels are above a threshold percentage.

6. The true color image enhancement compensation system according to claim 1, wherein the processor includes: a maximum gray shade rate limiter operational to control a rate at which a breakpoint value in adjusted to a single gray shade value each time increment.

7. The true color image enhancement compensation system according to claim 1, wherein the circuit includes: a frame rate controller operational to convert high-dynamic range video data to dithered video.

8. A true color image enhancement compensation system comprising: an ambient light sensor operational to measure an ambient light level;a circuit operational to: generate a histogram based on an input video signal;export the histogram;receive a gray shade look up table;dynamically remap a plurality of gray shades in the gray shade look up table using a highest remapped color to determine and maintain a plurality of input luminance ratios of a plurality of lower gray shade colors; andgenerate an output video signal by converting a plurality of gray shades in the input video signal based on a plurality of input video luminance ratios and the gray shade look up table to determine a plurality of new output gray shade values to maintain input video color coordinates; anda processor operational to: receive the histogram from the circuit;develop the gray shade look up table based on the histogram and the ambient light level; andtransfer the gray shade look up table to the circuit.

9. The true color image enhancement compensation system according to claim 8, wherein the circuit is further configured to: find a gray shade maximum input value among three color values of an incoming pixel in the input video signal.

10. The true color image enhancement compensation system according to claim 9, wherein the circuit is further configured to: lookup a multiplier value using a lookup table based on the gray shade maximum input value.

11. The true color image enhancement compensation system according to claim 10, wherein the circuit is further configured to: lookup the highest remapped color using a transfer function based on the gray shade maximum input value.

12. The true color image enhancement compensation system according to claim 11, wherein the circuit is further configured to: remap the plurality of gray shades based on the multiplier value and the gray shade maximum input value.

13. The true color image enhancement compensation system according to claim 12, wherein the transfer function is loadable from a source external to the circuit.

14. The true color image enhancement compensation system according to claim 12, wherein a plurality of numerators of the plurality of gray shades as remapped may be calculated prior to a right shift division to maintain an accuracy of results.

15. A true color image enhancement compensation system comprising: an ambient light sensor operational to measure an ambient light level;a circuit operational to: generate a histogram based on an input video signal;export the histogram;receive a gray shade look up table;dynamically remap a plurality of gray shades in the gray shade look up table using a tristimulus conversion matrix to determine a plurality of normalized gray shades based on a normalization of a highest color when an upper limit is reached; andgenerate an output video signal by converting a plurality of gray shades in the input video signal based on a plurality of input video luminance ratios and the gray shade look up table to determine a plurality of new output gray shade values to maintain input video color coordinates; anda processor operational to: receive the histogram from the circuit;develop the gray shade look up table based on the histogram and the ambient light level; andtransfer the gray shade look up table to the circuit.

16. The true color image enhancement compensation system according to claim 15, wherein the circuit is further operational to perform a tristimulus calculation that includes: first convert the plurality of gray shades to a plurality of new gray shades using a first transfer table; andsecond convert the plurality of new gray shades to a plurality of first intermediate gray shades using a lookup table.

17. The true color image enhancement compensation system according to claim 16, wherein the tristimulus calculation further includes: convert the plurality of first intermediate gray shades to a second intermediate value Y using a lookup table and the tristimulus conversion matrix.

18. The true color image enhancement compensation system according to claim 17, wherein the tristimulus calculation further includes: convert the plurality of first intermediate gray shades to a second intermediate value X and a second intermediate value Y based on the second intermediate value Y.

19. The true color image enhancement compensation system according to claim 18, wherein the tristimulus calculation further includes: convert the second intermediate values into a plurality of third intermediate values using an inverse matrix.

20. The true color image enhancement compensation system according to claim 19, wherein the tristimulus calculation further includes: normalize the plurality of third intermediate values.