Patent Application
20090167782
Embodiments of the invention relate to the calibration of display devices, and more particularly, to the automated monitoring and calibration of one or more display devices to obtain image uniformity.
Modern electronic display devices utilize different technologies such as cathode ray tube (CRT), liquid crystal display (LCD), plasma, digital light processing (DLP), and the like. Given the same input signal, each of these displays may produce a different “color temperature,” white point and color balance (color characteristics) due to fundamental differences in how the displayed image is generated. In addition, assembly tolerances, material variations, environmental effects (e.g. temperature, humidity), and component aging can result in different image color characteristics on two different display devices of the same technology, even if they were produced from the same batch of manufactured displays.
Because of these differences in color characteristics, obtaining uniform and desired color characteristics has always been a challenge to overcome. While the problem of adjusting a display device to obtain a desired color characteristic is applicable to a single display device, the problem becomes even more acute when multiple display devices are used together to form a larger display, where accurate color matching is desired.
Multi-screen display solutions are becoming more and more common for large image and video presentations. For example, multiple displays may be used to form a large display intended to be seen by passersby, each display showing only a portion of an overall image. Ultra-thin bezels make it possible to join multiple displays (image stitching) with only a very small, almost seamless gap between them. Multiple display devices may also be used to surround a viewer to create an “immersive” effect, such as in flight simulators, “virtual” meeting rooms, and “surround”-type games and entertainment systems. Additionally, multiple displays may be used together where each display shows the same image for artistic effect, or completely different images for a functional and/or aesthetic purpose (e.g. multiple television channels being displayed simultaneously in the background of a television newscast).
Any display device can be viewed as a nonlinear system, which can be difficult to model. Therefore, the exact color behavior of a display device with respect to a given input signal can be difficult to predict. The nonlinearity of each display device additionally emphasizes the difference in colors between two display devices.
It has been shown that the human visual system (HVS) is very sensitive to color and intensity differences. Even an untrained observer can easily notice the color difference between adjacent monitors showing the same image, such as in a consumer electronics store where numerous televisions may be on display and showing the same image. Thus, the equalization of color differences is important to both display device manufacturers and those who set up, maintain, and utilize one or more display devices.
In order to compensate for the differences in the transfer functions between individual display devices, a precise transfer function of each display device must be known. However, determining a transfer function for each display device is complex and often impractical. In addition, because the characteristics of a display device change with various parameters (e.g., time, temperature), the transfer function of the display device is not constant, but rather is a function of those parameters.
One conventional methodology for performing color correction and/or equalization involves constantly monitoring and manually adjusting one or more display devices until the desired color temperature is achieved, or in the case of multiple displays, until an observer cannot perceive the color difference between the displays. However, this is a slow, tedious and daunting task. To perform manual correction, a person may have to attach a sensor to the display device, connect the sensor to a measurement device, take a reading, attempt to manually correct the color, and then take another reading to verify the correction had its intended effect.
Another conventional methodology for adjusting the color characteristics of a single display device is to insert a compensating device at the input to the display device. The compensating device, such as an image processor, compensates for the differences in the transfer functions between individual display devices.
One conventional processing step utilized within image processors is the use of so-called three-dimensional (3D) look-up tables (LUTs). Input video image data can be applied to a 3D LUT to generate output video image data having video image characteristics specific to that particular 3D LUT. For example, a 3D LUT can be used to apply a certain amount of color correction to a digital video signal.
When the actual 10-bit digital output signal values x, y, and z for each color R, G and B, respectively, are applied to the 3D LUT, where x, y and z range from 0 to 1023, the 3D LUT generates modified digital output signal values x′, y′ and z′. Note, however, that in embodiments in which the 3D LUT is a lower resolution table (e.g. 17×17×17 instead of 1024×102×1024), the image processor may perform extrapolation on entries in the 3D LUT to obtain accurate x′, y′ and z′ values.
Next, the digital output signal may be gamma-converted in gamma (gain) processing block 406, and then fed into a matrix 408 which can perform intentional cross-contamination of one color with another (i.e. mixing of colors), adjust gain, saturation, and the like. The digital output signal may then be fed into a saturation processing block 410, to change the saturation of the image, and then to another one-dimensional (1D) LUT 412 to perform additional color conversion. The result of all image processing steps is a modified digital output signal values x″, y″ and z″ (see reference character 414).
While the image processor described above is suitable for adjusting the color characteristics of a single display device, any adjustments to the image processing steps described above are performed without benefit of any automated feedback from the output of the display device itself. Moreover, any color correction performed by the image processing steps described above is performed without consideration for any other display devices, or any preferred colorimetric reference standard.
Therefore, there is a need to perform calibration of one or more display devices in an automated fashion using feedback obtained directly from the display devices, without requiring any manual or subjective evaluation.
Embodiments of the invention are directed to performing calibration (e.g. color correction and/or equalization) of one or more display devices in an automated fashion using feedback obtained directly from the display devices, without requiring any manual or subjective evaluation. A sensor associated with a particular display device automatically measures certain characteristics of the display device, and feeds back calibration information to an image processor at the input to the display device. Based on the feedback, the image processor adjusts the characteristics of the display device to match a reference characteristic. When multiple sensors are used with multiple display devices and image processors, substantially uniform display characteristics and matching of the multiple display devices is possible.
The calibration that may be achieved includes color and brightness correction of digital video signals and other types of digital images (e.g. images from digital photography). Calibration may be achieved over multiple display devices, with each display device either showing (1) only a portion of an overall image, (2) the same image for artistic effect, or (3) completely different images for a functional and/or aesthetic purpose (e.g. multiple television channels being displayed simultaneously in the background of a television newscast). In addition, a single display to be maintained at a particular reference display characteristic can be calibrated.
In a multi-display device system including devices displaying different portions of the input image, a digital input image is first divided into N bitstreams by an image splitter. Each bitstream is fed into a different image processor, which generates a modified bitstream. Each modified bitstream, which represents at least a portion of the complete digital input image, is fed into a different display device. A different sensor associated with each display device measures certain display characteristics of the display device and sends feedback information back to the image processor associated with that display device. The image processor then modifies its 3D LUT in accordance with the feedback signal.
The image processor has a digital image input for receiving digital image data when the image processor is being used in its normal mode, which is to perform image processing on incoming digital image data. Within image processor is a test signal generator which is capable of automatically generating digital test patterns when the image processor is being used for calibration. One or more test patterns may be presented in one or more pixels over a number of frames. Either the digital image input or the test signal generator or combinations thereof can be processed by an image processing block. The output of the internal image processing block is sent out of the image processor via a digital image output. A feedback input port connectable to an external sensor can be used to provide a representation of the display characteristics of a portion of the display device to the internal image processing logic. The internal image processing logic can compare the display characteristics from the feedback input to the test signal from the test signal generator to compute a “correction” 3D LUT that compensates for the changes produced by the display device.
The sensor may be located in front of a portion of the display device, either in the visible area of the display device or hidden in the bezel of the display device. If the sensor is located in the visible area, it may be located in an extreme corner of the display device where the sensor causes that corner to have a rounded appearance. The sensor may be discrete, attached to the front of the display and connected via wiring, or it may be integrated into the bezel or behind the bezel. Alternatively, the sensor may be a remote sensor that focuses on a portion of the display using a telescoping lens, for example. This embodiment may be useful in large, stadium-style display devices where it is convenient to locate all sensors at a location remote from the display devices.
a illustrates an exemplary image processor according to embodiments of the invention.
b illustrates an image processor in an exemplary system environment according to embodiments of the invention.
In the following description of preferred embodiments, reference is made to the accompanying drawings which form a part hereof, and in which it is shown by way of illustration specific embodiments in which the invention may be practiced. It is to be understood that other embodiments may be used and structural changes may be made without departing from the scope of the preferred embodiments of the present invention.
Embodiments of the invention are directed to performing calibration (e.g. color correction and/or equalization) of one or more display devices in an automated fashion using feedback obtained directly from the display devices, without requiring any manual or subjective evaluation. A sensor associated with a particular display device automatically measures certain characteristics of the display device, and feeds back calibration information to an image processor at the input to the display device. Based on the feedback, the image processor adjusts the characteristics of the display device to match a reference characteristic. When multiple sensors are used with multiple display devices and image processors, substantially uniform display characteristics and matching of the multiple display devices is possible.
Although some embodiments of the invention may be described herein in terms of color correction of digital video signals, embodiments of the invention are also applicable to other types of correction (e.g. brightness) and other types of digital images (e.g. images from digital photography).
Furthermore, although some embodiments of the invention may be described herein in terms of multiple display devices with each display device showing only a portion of an overall image, embodiments of the invention are applicable to multiple display devices with each display showing the same image for artistic effect, or completely different images for a functional and/or aesthetic purpose (e.g. multiple television channels being displayed simultaneously in the background of a television newscast).
In addition, embodiments of the invention are also applicable to a single display that is to be maintained at a particular reference display characteristic. For example, a display calibration system can be built into a single display device to maintain a particular reference color characteristic over time as the display ages. In another example, because DVDs are generally mastered for a particular type of display (e.g. plasma), in order to ensure that the DVD will produce desired color characteristics when played back on another type of display (e.g. LCD), the calibration system according to embodiments of the invention could be used to automatically calibrate color from a DVD. The DVD itself may have test patterns for calibrating the color. In such embodiments, no test signal generator may be needed.
As mentioned above, image processor 510 calibrates displayed images based on measurements from the sensor 514. Image processor 510 can be functionally similar to the image processor disclosed in commonly owned U.S. patent application Ser. No. 11/715,772, entitled “Generation of 3D Look-Up Tables For Image Processing Devices,” filed on Mar. 7, 2007, the contents of which are incorporated by reference herein. The image processor 510 has a digital input and output e.g. a DVI or SMPTE292 interface and enough memory to store one or more 3D LUTs. The digital interface allows for manipulation of digital images such as inserting test color signals at predetermined spatial and temporal positions. These test signals can be displayed as part of the displayed image and measured by the sensor 514. The image processor 510 can insert these test patterns continuously or periodically. The temporal frequency of the test signals can be based on the predicted change of a display's color characteristic over time.
a illustrates an exemplary image processor 600 according to embodiments of the invention. In
Within image processor 600 is a test signal generator 604 which is capable of automatically generating digital test patterns when the image processor is being used for calibration. For example, if a 17×17×17 3D LUT is to be generated within internal image processing block 606, the test signal generator 604 may generate 17×17×17=4913 different x, y, z RGB combinations, each representing a different test pattern. One or more test patterns may be presented in one or more pixels over a number of frames. Either the digital image input 602 or the test signal generator 604 or combinations thereof can be processed by the image processing block 606. In embodiments of the invention, the internal image processing logic 606 inserts test patterns from the test signal generator 604 into specific parts (e.g. pixels) of the digital video stream 620, either continuously or at specific times.
The output of the internal image processing block 606 is sent out of the image processor 600 via a digital image output 612. A feedback input port 624 connectable to an external sensor can be used to provide a representation of the display characteristics of a portion of the display device to the internal image processing logic 606. The feedback input port 624 may include any suitable interface circuitry for receiving signals from the external sensor. The internal image processing logic 606 can compare the display characteristics from the feedback input 622 to the test signal from the test signal generator 604 to compute a “correction” 3D LUT that compensates for the changes produced by the display device.
The image processor 600 may be housed in a single enclosure suitable for connection to a single display device. Alternatively, multiple image processors may be housed in a single enclosure suitable for connection to multiple display devices. In such embodiments, a single test signal generator may generate test patterns for each of the multiple image processors. The enclosure could also include the image splitter or distribution amplifier, as needed. Embodiments of the invention could also be contained a circuit board built into a display device itself
In some embodiments of the invention, the sensor may be a small CMOS or CCD sensor that reads and quantifies the output of a least one three-color pixel and transmits results back to the image processor. The sensor according to embodiments of the invention may be small in size (e.g. a line array several millimeters square), occupying only one or more pixels to minimize obstruction of the image being displayed. Such sensors are well-understood by those skilled in the art and will not be discussed in further detail herein.
The sensor may be located in front of a portion of the display device, either in the visible area of the display device or hidden in the bezel of the display device. If the sensor is located in the visible area, it may be located in an extreme corner of the display device where the sensor causes that corner to have a rounded appearance. The sensor may be discrete, attached to the front of the display and connected via wiring, or it may be integrated into the bezel or behind the bezel. Alternatively, the sensor may be a remote sensor that focuses on a portion of the display using a telescoping lens, for example. This embodiment may be useful in large, stadium-style display devices where it is convenient to locate all sensors at a location remote from the display devices.
In some embodiments, the sensor may function as a spectrometer and measure wavelengths of light, and in particular measure R, G and B. Other types of measurements could include brightness and grey scale. In alternative embodiments, the sensor may measure the characteristics of a particular type of display device (e.g. characteristics unique to LCD, plasma, etc.) for matching the spectrograph characteristics of those types of display devices. In other embodiments, the sensor may only measure the intensity of a color pixel, with the spectral characteristic of the sensor being flat or having some known response. Intensity measurements alone can be sufficient because during the measuring process, the image processor has prior knowledge of which color (or combination of colors) is being sent to the one or more test pixels.
The spatial position of the test patterns within a frame is dependent on the position of the sensor. For example, if the sensor is placed over a group of pixels at the top left corner of a frame, then the test patterns should be inserted into those pixels. A method for determining the spatial position of the test patterns is needed because the sensor may be placed on the display device at any location, or may be only generally placed in a particular area of the display device. The spatial position can be determined with an automatic test pattern detection process prior displaying any actual image data. In one exemplary embodiment, the image processor can insert a white pixel at various positions within an otherwise black frame until the sensor detects the white pixel.
b illustrates the image processor 600 in an exemplary system environment according to embodiments of the invention. In the example of
When the display correction process is being performed, the image processing logic 606 automatically inserts a test pattern from test signal generator 604 into the digital image at a particular location, either continuously or periodically, and generates digital image output data including the test pattern for display on a display device. This test pattern appears at one or more selected pixels on display device 622 that are being monitored by the sensor 626. The sensor 626 then provides a representation of a particular characteristic (e.g. color) detected at those pixels to the image processing block 606. The image processing block 606 compares the display characteristics at the feedback input 624 to the display characteristics of the test pattern from the test signal generator 604, and based on any difference between the two, generates a new entry for a “correction 3D LUT” being maintained in the image processing block. As different test patterns are automatically inserted into the digital image over time, a complete correction 3D LUT is generated without human intervention. This correction 3D LUT thereafter results in the display device 622 producing substantially the same display characteristics as the reference signals generated by the test signal generator 604. In multiple display device systems, each display device will produce a different correction 3D LUT corresponding to the particular characteristics of that display device. However, because each correction 3D LUT is based on the same test patterns, each display device will generate display characteristics that are substantially similar. Multiple display devices with adjusted displays will result in seamless stitching of separate images into one larger image with substantially uniform display characteristics.
As mentioned above, during this display correction process, test patterns are inserted into the digital image. One color may be used in the test pattern at a time, or multiple colors could be used for different pixels if multiple sensors are employed for each display device. The calibration or correction process need not be performed on a per-frame basis. Rather, one test pattern could be inserted into the digital image for one or more consecutive frames, followed by a number of frames for which no test pattern is inserted. Alternatively, the changing test patterns could be continuously inserted into the digital image, but because the number of pixels reserved for display correction processing are small and at the periphery of the displayed image, and may even be hidden from view, this insertion of test patterns should not produce any significant degradation in a user's viewing experience. Depending on the frequency and number of the test patterns, the entire display correction process may take minutes or even hours. In one embodiment, 4096 different colors could be used in the test pattern, one color per frame, and the color correction process could take on the order of a couple of minutes to complete. The display correction process may be performed periodically to account for gradual atmospheric and aging effects, or may be performed only once, or at irregular intervals, primarily to account for manufacturing differences.
In some embodiments of the invention, the calibration process may only set contrast and brightness. For this type of calibration, only black and white pixels are needed. In other embodiments, the calibration process may adjust the amount of R, G and B for any number of combinations. In still other embodiments, the calibration process may adjust grayscale.
In the example of
Although the present invention has been fully described in connection with embodiments thereof with reference to the accompanying drawings, it is to be noted that various changes and modifications will become apparent to those skilled in the art. Such changes and modifications are to be understood as being included within the scope of the present invention as defined by the appended claims.
