Patent Application
20150228219
The present invention relates to displays systems and, more particularly, to display systems having Enhanced Dynamic Range (EDR) capability.
In a dual and/or multi-modulator display system, it may be the case that one modulator may be refreshed with image and/or control data from a controller at a faster rate than another modulator with the same display system. When this occurs, it is possible to have undesirable visual artifacts. One conventional display system may comprise two modulators: (1) an array of Light Emitting Diodes (LEDs) that provide a locally dimmable backlight that illuminates (2) a Liquid Crystal Display (LCD) that further modulates the light to produce the final viewable image.
One prior art reference describes the visual artifact and the desirability of eliminating and/or mitigating such artifacts: United States Patent Application Publication No. 2010/0295879 to Tanaka et al., published on Nov. 25, 2010 and entitled “IMAGE DISPLAY APPARTUS”—which is hereby incorporated by reference.
Several embodiments of display systems and methods of their manufacture and use are herein disclosed. A dual and/or multi modulator display systems is disclosed comprising at least a first modulator and a second modulator wherein one of modulators has a faster response time that the other modulator. The response of the slower modulator may be characterized according to various image data inputs and this characterized data may then be used by the display system to derive control and/or data signals to the faster modulator. These control/data signals may represent a fitted set of data matched to one or more characteristics of the slower modulator in order to reduce light produced during frame or other transition times of the modulators. One or more characteristics may be employed to reduce such undesirable visual effects.
In one embodiment, a display system is disclosed comprising: a light source, said light source comprising an array of LED backlights; an LCD modulator, said LCD modulator illuminated by said light source and modulating said light source to render an image; a controller, said controller further comprising: a processor; a memory, said memory associated with said processor and said memory further comprising processor-readable instructions, such that when said processor reads the processor-readable instructions, causes the processor to perform the following instructions: receiving image data, said image data to be rendered by said display system; receiving LCD characterization data, said LCD characterization data based upon a characterization of the one or more LCD behaviors; deriving LED control signals, said LED control signals providing a fitted LED response based upon one or more LCD behaviors; and sending LED control signals to said light source and control signals to said LCD to form the desired screen image.
In another embodiment, a method for providing LED signals to match LCD characteristics is provided, said method comprising: receiving input image data to be rendered on a display system, said display system further comprising an array of LED backlights illuminating an LCD modulator; receiving data characterizing the raster scanning of the LCD; calculating timing offsets to be applied to control signals to LEDs in proximity to the LCD pixels such that the LED illumination matches the raster scanning of LCD; and sending the LED control signals to the LEDs.
Other features and advantages of the present system are presented below in the Detailed Description when read in connection with the drawings presented within this application.
Exemplary embodiments are illustrated in referenced figures of the drawings. It is intended that the embodiments and figures disclosed herein are to be considered illustrative rather than restrictive.
Throughout the following description, specific details are set forth in order to provide a more thorough understanding to persons skilled in the art. However, well known elements may not have been shown or described in detail to avoid unnecessarily obscuring the disclosure. Accordingly, the description and drawings are to be regarded in an illustrative, rather than a restrictive, sense.
As utilized herein, terms “component,” “system,” “interface,” “controller” and the like are intended to refer to a computer-related entity, either hardware, software (e.g., in execution), and/or firmware. For example, any of these terms can be a process running on a processor, a processor, an object, an executable, a program, and/or a computer. By way of illustration, both an application running on a server and the server can be a component and/or controller. One or more components/controllers can reside within a process and a component/controller can be localized on one computer and/or distributed between two or more computers.
The claimed subject matter is described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the subject innovation. It may be evident, however, that the claimed subject matter may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to facilitate describing the subject innovation.
Dual/Multi-Modulator EDR Display System Embodiments
EDR display systems comprising dual/multi-modulators have been described in commonly-owned patents and patent applications, including:
(1) United States Patent Application Publication US 2010/0277515 to Ward et al., published on Nov. 4, 2010 and entitled “MITIGATION OF LCD FLARE”;
(2) United States Patent Application 20100328537 to Davies et al., published on Dec. 30, 2010 and entitled “SYSTEM AND METHOD FOR BACKLIGHT AND LCD ADJUSTMENT”;
(3) United States Patent Application 20120075360 to Messmer, published on Mar. 29, 2012 and entitled “SYSTEMS AND METHODS FOR CONTROLLING DRIVE SIGNALS IN SPATIAL LIGHT MODULATOR DISPLAYS”;
(4) United States Patent Application 20110193895 to Johnson et al., published on Aug. 11, 2011 and entitled “HIGH DYNAMIC RANGE DISPLAY WITH REAR MODULATOR CONTROL”; and
(5) United States Patent Application 20090322800 to Atkins, published on Dec. 31, 2009 and entitled “METHOD AND APPARATUS IN VARIOUS EMBODIMENTS FOR HDR IMPLEMENTATION IN DISPLAY DEVICES”.
which are hereby incorporated by reference in their entirety.
It will be appreciated that
Staying with the embodiments involving an LED backlight disposed behind an LCD, there is likely a significant difference in response times, with the LED being orders of magnitude quicker than the LCD. Given the more rapid response time of the LEDs, there may be an opportunity to modulate one modulator advantageously with respect to the other modulator to reduce and/or mitigate any visual artifact.
In one embodiment, it may be possible to modulate the LEDs such that the LEDs may be slowly transitioned through intermediate states to better match the response and alignment of the LCD. In this way, the displayed light at a pixel (which is the product of the LED and LCD transmittance) may be substantially constant throughout the LCD transition. As LEDs have very high speed drive signals, this embodiment represents one possible option.
In another embodiment, there may be an alternative option to affect in order to minimize undesirable visual artifacts. This alternative employs another imbalance that tends to exist in LED/LCD dual modulation displays. That is that the number of individually addressable LCD elements tends to be many orders of magnitude higher than the number of LED elements. As a result, each LED creates a backlight that passes through hundreds or thousands of LCD elements.
Thus, as a practical matter, it may be difficult to directly determine any given transition (e.g., skewed transition or the like) of the LEDs between states, as there are thousands of possible LCD responses to match. It may be possible to affect an analysis of typical LCD transition curves (along with a focus on correcting the most visually harmful artifacts) that may yield transition curves that may improve the visual performance of these systems—despite the daunting numbers of potentially unique LCD transitions near each LED. In fact, it may be possible that the actual LCD starting and ending transmittances need not be known in determining the best LED slew transition, thus, creating a simplified real time solution.
Examples of Visual Artifact Introduction
To better understand the undesirable visual artifacts that may arise during the course of displaying images with this display system,
In
Once the LCD has ramped up to its desired level for a given frame, then the illumination through the LCD pixel enters a steady state at 208. At the end of this frame period, the controller may order the LED backlight OFF at 210 and the LCD to reduce its transmittance (through curve 212). Again, there is a period 214 during which the mismatch in transition speed of the LEDs and the LCD may give rise to an undesirable visual artifact.
In
Embodiments Involving Vertical Phasing
One technique for addressing the transition time mismatch between a first modulator and a second modulator—e.g., such as in a LED/LCD display system or a LCD/MEMS projector system—may involving a proper phasing of the data/control signals between the two modulators.
To better understand this technique, it should be appreciated that many of today's LCD technologies use a raster scan to update the display. This is typically done in a vertically descending region order. The time it takes to update the entire raster is directly related to the LCD's refresh rate. For example, a LCD driven at a refresh rate of 100 Hz takes approximately 10 ms to scan the raster. Let fLCD be the LCD's refresh rate. If a LCD is divided into N vertical regions, then the nth ε[1, N] vertical region starts updating after
seconds.
This behavior may be leveraged to good advantage in this embodiment—while the LCD updates as described, the LED back light display may be changed along a different, desired time interval. In fact, when using LED elements to construct a backlight, the typical time interval to change between one value to another can be as short as tens to hundreds of μs. As mentioned, this temporal mismatch between the two modulators may cause an objectionable visual artifact such as a flash during a transition in the source signal.
For merely one example, without a proper synchronization of the two modulators, there may be one arbitrary region that is “optimal” in terms of synchronization. In this case, artifacts may be minimal around this “optimal” region; however, as considering the displayed image moving vertically further (in both directions) away from this region, the output may tend to become increasingly un-synchronized. The outcome of this behavior under certain input signal patterns may yield a non-uniform flashing output signal which is more objectionable to the human eye.
In the case of the LCD being paired with a faster modulator (e.g., LEDs, MEMS and the like),
In the case of the LED backlight/LCD second modulator display systems,
In the example of
Each vertical region starts transitioning approximately T/N seconds after the region above it.
Underneath the time axis, one embodiment of a suitable timing scheme is given which is essentially a description of when to transition a backlight row in order to achieve synchronization with the LCD.
In another embodiment, it may be desirable to apply an offset parameter to this timing scheme that may be derived offline, as is further described below.
Embodiments Employing Offline Analysis for Improved Offset
While there may be improvement with the mitigation of visual artifacts by applying the vertical phasing to the faster modulator as described above, there are other embodiments that may be applied that may tend to further improve the situation. In these embodiments, a time offset between the two modulator's (e.g. LED and LCD) start of transition time may be selected in a way that may further minimizes visual artifacts.
In one embodiment, an offline processing of the slow modulator's (e.g. LCD) characteristics may be discerned and applied. For example, in the case of an LCD, a LCD response may be used to synthesize a LCD output signal as shown in
The integral of the multiplication (LCD×LED) is the observed light in front of screen. In this example, the LED and LCD are switching in opposite directions and in an ideal system the output light will remain constant. However, due to the different temporal behavior of the responses an excess light (flash) may result. It may be desirable to minimize this excess light by finding the optimal offset setting. It will be appreciated that the offline processing may also determine the excess light when the LEDs and the LCD are switching in the same direction and the scope of the present application encompasses such same directional switching. However, it was noted in repeated runs that switching in the opposite direction may be preferred.
As may be seen, the offset setting tends to have a discernable effect on the AUC result. In this exemplary set of plots, it may be seen that the AUC is minimized when the AUC shape is of two balanced lobes such as with offset 0.4, as depicted in
Embodiments Employing LED Slewing
Yet another system/method/technique for improving the visual experience may arise from the occasional image processing situations. For example, in a dual modulation display, the change in the source video signal may cause the two modulators to transition in opposite direction. For merely one example, abrupt appearance of text (e.g., subtitles) on the screen may cause the overall light field of the backlight behind it to increase. In order for other regions of the screen that did not change to maintain to same output, the LCD will have to ramp down (e.g., close) such that the combined (dual modulated) output is substantially maintaining the same luminance level as prior to the appearance of the text. This represents yet another opportunity to create undesired visual artifacts.
In addition, the following should also be noted:
LED Slewing
As with other techniques, this proposed solution is to minimize the amount of excess light during the opposite LED vs. LCD transitions. This is achieved by changing the shape of the signal driving the LED backlight. In one embodiment, as the amount of displayed light is the product of the LED and the LCD transmittance, the LED response would be an inverse of the LCD response, resulting in a constant amount of light (e.g., the product of the LED output and LCD transmittance) displayed throughout the LCD transition to its final transmittance.
LCD Characterization (Offline Analysis)
Similar to the discussion above, it may be desirable to determine the characteristics of a typical LCD—as well as the characteristics of dual modulation systems. Generally, LCD transition curves differ in shape as a function of the starting and ending transmittances, as well as the direction of the change (towards higher or lower transmittance). For generality then, it may be assumed that every pair of starting and ending transmittance values follows a unique curve.
If either end point transmittance level is fully open or closed, the slope of the inverse response (ideally realized by the LED) can be prohibitively steep and does not represent a good candidate for an LED response curve.
Mid-level to mid-level transmittance changes may be typically somewhat similar in shape (“mid-level” may be defined substantially as the 20% to 80% transmittance range). The most commonly encountered and objectionable flicker artifacts may occur in mid-level luminance areas that have bright objects coming or going nearby. For typical dual modulation systems, this often results in mid-level to mid-level LCD transmittance transitions. Therefore, the transitions that typically result in the most significant visual artifacts may behave somewhat similarly.
For dual modulation systems, hundreds or thousands of LCD elements are lit by the same LED and are likely to have unique transition curves as each experiences a slightly different backlight and may have a different target level (image pixel level). Each LED may have only one slew response between states, which affects each of the hundreds or thousands of pixels in the same way. One embodiment may attempt to minimize the most objectionable artifacts. In such an embodiment, this may mean targeting mid-level to mid-level transitions.
With suitable signal processing, an ‘average’ mid-level to mid-level transition curve may be created. Since this curve represents what has been identified as the potential solution for visible artifacts and does not look at any individual LCD elements, it can be applied without knowledge of the LCD element transitions. This represents a potential simplification in a real time algorithm—as the volume of LCD data may not need to be examined, and no real time decisions may be needed to be made to determine the ‘optimal’ solution.
LED Based on Inverse LCD (Offline Analysis)
With a reduced set of curves consisting of mid-level to mid-level transitions, it may be possible to find a representative curve that may improve most situations. From a group of mid-level transition curves experimentally collected (or calculated or otherwise derived), a level of precision may be desired in aligning the curves, thus, potentially avoiding noise at either endpoint and allowing a true average curve to be formed. This alignment and averaging may be used to create an inverse process to generate the LED slewing.
The curve generated from the fitting/averaging process of the greatly reduced set of transition curves represents an improved overall LED-LCD product regardless of the actual LCD transitions. It also represents the largest improvement possible by targeting the most common and most artifact inducing LCD transitions.
LED Slewing with LED/LCD Offset Embodiments
Real Time LED Slewing Embodiments
In one embodiment, these techniques described herein may be those that present a desired LED response that is derived from the LCD characteristics. This may be achieved in a system that is constructed to meet this goal. For example, if a DAC (digital to analog converter) was driving each individual backlight element, such a system may be possible; if not costly to realize.
In other embodiments, there may be systems where backlight elements are driven in a typical PWM (pulse width modulation) and other methods and techniques may be used. For example, a new LED response can be realized based on approximating the optimal LED response with a step function. The number of steps depends of the specifics of a given system's limitations. In general, the more steps that can be realized, the better the results will be.
Embodiments Combining LED Phasing & LED Slewing
In the previous discussion above, at least two problems and their relevant solutions were detailed. It may be possible to create many other embodiments by combining the various techniques described herein.
For one example, it may be possible (and possibly desirable) to combine the techniques of vertical phasing, LED slewing and/or offsetting to realize even further improvement in visual quality.
One Possible Embodiment
Having said that, in this embodiment, system/method 1400, the processing may therefore be parsed between real time processing 1402 and offline processing 1404. The controller of the display system may be suitable to process the real time processing (or some other controller, such as on a set top box, codec, or the like). The controller may receive input image data to render upon the display system and then determine the LED backlight values based upon the input image data at 1406. From offline processing 1404, the LCD may be characterized and potential responses to input image data may be derived at 1410.
At 1412, the LED response to the particular input image at issue may be derived, estimated or otherwise computed—e.g., as an inverse response to the LCD characterization and/or fitted to the input image data. This LED response may be an input into 1408 to determine the control/data signals to send to the LED backlight to effect LED slewing.
In addition to that, the system/method may compute the LED vertical phasing scheme to match the LCD's raster scanning at 1416. This may be input into 1414 to determine the control/data signals to send the LED rows to match the LCD scanning. The results of the control/data signals so derived and/or calculated may be combined at 1418—together with any possible offset computed at 1420 to send out a final control/data signal to the LEDs at 1422 in order to improve the visual quality and experience of the rendered images.
It should be appreciated that many other possible systems/methods that perform any one of the techniques, either singly or in combination, to affect different embodiments of such an improved dual/multi modulator display system. In addition, the results of the offline processing may be stored in computer readable memory residing in the display system (e.g., on a LUT, ROM, RAM or the like) and accessed by the display system in real time.
In one embodiment, the invention comprises a controller for a display system configured to receive image data. The data may be received, for example, over a computer network, Internet, private network (e.g., Virtual Private Network), from storage (e.g., optical disk, flash drive, hard drive, cloud based storage, or other storage systems), via or incorporating direct communications such as wireless, cell, Wi-Fi, Bluetooth, Near Field Communications (NFC) said image data to be rendered by the display system. In one embodiment, the image data is received from a user's account in a copyright controlled storage system such as Ultraviolet. The controller may be further configured to receive modulator characterization data based upon modulator behavior(s). The controller is, for example, embedded in the display system, and may be, for example, a mobile device, LED backlit LCD display, or a projector utilizing first and second modulation systems (e.g., dual LCD projection system). The projector may be a cinema projector that includes a DMD modulator that controls an illumination of a second modulator in a manner that incorporates one or both of vertical synching and slewing and/or matching any other characteristic of the second modulator. Such illumination control may be implemented via control of individual pixels of the DMD or groups of pixels forming tiles that are energized together (e.g., energized in predetermined patterns). Regardless of the mechanism for providing the illumination, the end result is an illumination that matches or corresponds to the characteristics of the second (downstream) modulator.
In one embodiment, the characteristics matching the vertical synching and slewing are either hardwired or programmed into the controller. In another embodiment, the characteristics of the second modulator are downloaded and stored in memory accessible to the controller and used by the controller to produce corresponding control signals. In one embodiment, a projector according to the present invention may be upgraded or repaired by the installation of a new second modulator and the characteristics of the new modulator are downloaded and installed in the controller accessible memory. The characteristics stored in the controller may be updated or modified based on recent updates or changes due to aging of one or both of the modulators. All of the above applies to projector, mobile device, computer monitor, and home entertainment type displays regardless of the types of modulators utilized in the displays.
In one embodiment, the backlight of a display is an OLED or other lighting system that exhibits measureable performance changes over its lifetime. The invention includes monitoring backlight performance and changing the particulars of how the backlight performs or maintains vertical synching and/or slewing based on the changes in backlight performance. The vertical synching and/or slewing may be changed according to measured changes in performance or may be made via programming on a predetermined time and/or usage schedule. In cases where aging issues are not known when the display is put into use (or even if they are), those aging issues may be addressed via an update to an energization algorithm that controls the backlight and does so in the context of the second modulator's known characteristics. Such updates may be linked to a user's content account, such as Ultraviolet, along with other display related data such as extended dynamic range or color gamut tables that may be generic to the user's display device or tailored for specific images or content.
In the above embodiments, the controller is configured to derive control signals from the image data for the backlight (e.g., LEDs, OLEDs, Laser light sources, combination of Laser light sources and modulator such as DMD, LCD, MEMS based modulators, etc.), the backlight control signals configured to provide a fitted response (illumination) based upon one or more behaviors (e.g., characteristics of an LCD or other modulator that are programmed into the controller, or based on data or programming such as data and/or programming downloaded to the controller). Ultimately, the backlight control signals are intended to be communicated to a backlight which is energized accordingly and producing the fitted response.
The controller is further configured to prepare and communicate 2nd modulator control signals to a second modulator illuminated by the backlight and configured to form the desired screen image.
A display system according to the invention may comprise a modulated light source, a second modulator illuminated by the modulated light source, and a controller that controls the modulated light source according to a combination of desired image data and characteristics of the second modulator. The characteristics may include for example, any characteristic related to performance or energization of the second modulator, or any combination of characteristics. The characteristics may include, for example, any combination of vertical synching and slewing of the second modulator or patterns of synching and/or slewing produced by the modulated light source that illuminate the second modulator.
A detailed description of one or more embodiments of the invention, read along with accompanying figures, that illustrate the principles of the invention has now been given. It is to be appreciated that the invention is described in connection with such embodiments, but the invention is not limited to any embodiment. The scope of the invention is limited only by the claims and the invention encompasses numerous alternatives, modifications and equivalents. Numerous specific details have been set forth in this description in order to provide a thorough understanding of the invention. These details are provided for the purpose of example and the invention may be practiced according to the claims without some or all of these specific details. For the purpose of clarity, technical material that is known in the technical fields related to the invention has not been described in detail so that the invention is not unnecessarily obscured.
