Patent Grant
7224335
This invention relates to image display systems that employ so-called reflective spatial light modulators (SLMs) such as those in the form of digital micro-mirror devices (DMDs). In particular it relates to techniques for using multiple SLM devices arranged serially.
U.S. Pat. No. 5,612,753 to Poradish, et al., discusses a system where the images of two SLMs are projected onto a screen. The two images are the same size and are fully overlapped. In this arrangement there is a one to one correspondence between the pixels of each SLM. The Poradish patent makes reference primarily to the use of this parallel SLM technique for the improvement in light output and mitigation of color artifacts produced by the use of a field sequential color system that usually employs a color filter wheel and a single SLM.
Poradish does mention the use of parallel SLM devices for the mitigation of gray level contouring artifacts or increasing the dynamic range of the display, but the patent does not teach a method by which this may be accomplished.
When SLM based projection systems are to be used for a cinematic presentation, it is desirable for the image quality of the system to approach that of the film based projection systems commonly in use. This requires that the system has a high spatial resolution and that the system possesses a wide dynamic range. While film based systems easily produce contrast ranges of 1000:1 or greater, SLM based systems such as those based on DMDs are typically limited to contrast ranges of 500:1 or less.
In particular SLM devices have a finite black level. That is, the minimum displayed intensity in the off state for any SLM is greater than zero. In circumstances found in motion picture theatres this minimum black level is often found to be too high and the “gray” image visible when the picture is “black” is distracting and serves to emphasize that the system is different from a film based projection system.
SLM devices consist of a number of separately addressable picture elements or pixels in a x-y array. Different types of SLMs employ different methods for achieving a gray scale intensity control for each pixel. Some devices employ a continuously varying analog signal (such as reflective liquid crystal devices) while DMD devices employ pulse width modulation of pixel sized mirrors that switch between only two states, “on” and “off”.
A DMD is essentially an array of microscopic-sized mirrors on a computer chip. A post that allows the mirror to tilt supports each mirror. The chip includes electronic circuitry for moving each mirror between on and off positions using an electrostatic field. When the DMD mirror is in the on position light is reflected from the DMD onto a screen or other image plane. When the DMD is off the light is reflected to a “light dump” and does not reach the screen or image plane.
The mirrors are individually addressable so that which mirror is on and which is off at any given time determines the image. A memory cell associated with each mirror stores a bit of data that determines the on or off state of the address signal. Each mirror is addressed with a signal that indicates whether or not the mirror is to be on or off and therefore whether or not light is to be reflected to the image plane or screen.
A grayscale is obtained through a technique called binary pulse width modulation (PWM).
Under PWM, a frame interval T is divided into n time durations of
each represents a bit in a n-bit binary word x={bnbn−1bn−2 . . . b1}. As a result, the value of binary word x is represented by y, the time duration when the mirror is on, and y can be calculated by:
The shortest time duration
represents the least significant bit (LSB) of the binary word, and it is called the LSB time, expressed as:
The LSB time is mainly limited by the mechanical switching time of a DMD chip. Typically, the LSB time is about 20 μsec. Therefore, the bit-depth, or the number of bits in a binary word representing the intensity of each pixel in the image formed by the DMD, is limited by both the LSB time and the frame rate. For a frame rate of 24 fps, the maximum bit-depth that a DMD can support is n=11.
Each single micro mirror on a DMD chip is a reflective device, and the total amount of output light for a binary word x is proportional to the “on-time” Tx if the optical switching time can be ignored. In fact, the optical switching time of a DMD is around 2 μs, which is about one-tenth of the LSB time, and can be neglected in the following analysis. If the input binary word x is simply encoded to be linear to scene brightness, then a DMD chip can be considered a linear device with respect to brightness.
A simplified cross section of a single DMD mirror array is shown in
P=(αy+δ)L=αyL+Ldark
The second item Ldark=δL represents the “dark level” of a DMD. This corresponds to the light reflected by a pixel of the array when the associated mirror is in the off state. This dark level is the combined result of light diffraction from mirror edges, reflection from the underlying substrate and scattering from the mirror surface, particularly around the dimple formed at the support post location. This combined effect is modeled by a factor δ and is typically:
The “dark level” determines the contrast ratio of a DMD device. It can be reduced by architectural improvement to the DMD pixels, but it may not be completed eliminated.
The simplest implementation of a PWM scheme makes the period of the most significant bit ½ of the total frame time. The period of the next most significant bit is ½ of this or ¼ of the total and so on.
At certain bit transitions, such as for example in an 8 bit system where the msb switches off, and the remaining bits switch on, that is the displayed code value switches from 1000000 binary to 01111111 binary, the temporal position of the associated light reflected by a pixel making this code transition changes from the first half of the frame interval to the second half. As a result temporal artifacts are produced that are found to be problematic when displaying moving images.
These artifacts can be reduced by avoiding such transitions, and by spreading the time that a bit is on during the frame more uniformly throughout the frame duration. This is obtained for example by dividing the most significant bit into say 4 separate parts, making four equal on-off transitions for the bit during the frame time, yielding the same intensity, but splitting the on time into four parts equally spaced throughout the frame duration. Additional complexity may be introduced to further smooth the transitions that occur during a gray scale ramp and to avoid the creation of low frequencies that are more noticeable as flicker in the image.
Where full-color images are required, three DMDs may be used, one for each primary color (RGB). The light from the DMDs converges so that the viewer perceives color. Another technique is to use a single DMD and a so-called “color wheel” having sections in the primary colors so that light incident on the DMD is sequentially colored RGB. The viewer's eye then integrates the sequential images into a continuous color image.
Other types of SLMs may be similarly arranged to obtain color images.
The performance of currently available SLMs is not sufficient for creating a visual sensation equivalent to a film-based system. There are three issues: dynamic range, contrast ratio and frame rate.
The present invention is directed towards a projection system comprising a controller, a first reflective SLM and a second reflective SLM. The first reflective SLM is operatively coupled to the controller to receive first image data from the controller and is aligned to receive light from a light source and to reflect imaging light correlated to the first image The second reflective SLM is operatively coupled to the controller to receive second image data from the controller and is aligned to receive imaging light from the first reflective SLM and to reflect modified imaging light. Typically, the projection system will also include a light source and optical components such as a projection lens.
The invention is also directed towards a projection system comprising a controller and a plurality of reflective SLMs operatively coupled to the controller to receive image data from the controller. The reflective SLMs are aligned in series to receive light from a light source and to reflect imaging light correlated to the image data.
The present invention will now be described, by way of example only, with reference to the following drawings, in which like reference numerals refer to like parts and in which:
For SLMs such as DMDs where mechanical switching is used, the maximum number of bits possible with a binary PWM scheme is determined by the period of the LSB. For current DMD devices the minimum period of the LSB is equal to the mechanical switching time of the mirrors, which is about 20 μsec. For a frame rate of 24 fps, the period of the LSB means that a total of 11 bits can be represented in the 41.67 msec. available for each frame. The number of possible bits determines the total number of steps available to represent a grayscale image, and this corresponds to the dynamic range of the resulting image.
It should also be clear that increasing the frame rate reduces the time available for the creation of various bit durations using binary PWM, and as a result the number of bits available is reduced, as is the corresponding dynamic range.
The contrast ratio of a system is limited by the maximum difference between the dark level of the SLM and the maximum light level that the SLM can deliver that corresponds to a full white input signal level.
The available bit depth of 11 bits, or a dynamic range of 2048 steps, is inferior to a film-based system which can reach an equivalent bit-depth of nearly 14 (16,383 steps) in shadow areas. The current DMD contrast ratio is about 500:1, which is lower than the 1000:1 ratio required by film-based system. Finally, for certain applications, such as 48 fps Imax HD projection system or Imax 3D system using LCD alternating shutter glasses, there is a need to increase the frame rate of a projection system. In the case of the DMD, any increase in frame rate will further reduce effective bit-depth.
By placing two reflective SLMs in series the limitations of a single DMD can be overcome through the multiplicative effect of the two SLMs when displaying a grayscale. This can be better understood with reference to the projection system shown generally as 2 in
As illustrated by light ray vector 4 shown in dotted line, light source and reflector 10 illuminate integrator rod 12 via cold mirror 11. Relay 14 directs the light 4 from 12 onto the surface of a first DMD device 16 at a suitable angle so that the on state light is directed through relay 18 which images the light from 16 onto the surface of a second DMD device 20 so that there is a one to one correspondence between the pixels of 16 and 20. On state light from the second DMD 20 then passes onto the projection lens (not shown) or other imaging system and then to the projection screen.
As will be understood, the system 2, also includes a controller 22, operatively coupled to the first and second DMDs 16, 20. The controller 22 is suitably programmed to provide image data to the DMDs 16, 20. As discussed in greater detail below, the image data comprises pulse width modulation data corresponding to each micro mirror in the array of each DMD 16, 20.
The system 2 may optionally include a color wheel 9 (shown in dotted outline) operatively coupled to the controller 22 for color display. In such a configuration, the image data generated by the controller 22 will include pulse width modulation signals correlated to and synchronized with each color on the color wheel (typically, red, green and blue).
As will be understood by one skilled in the art, if the optical configuration of the system 2 results in an inverted or reverse image, the image data provided by the controller 22 to the first DMD 16, will correspondingly be reversed or inverted.
While the system 2 has been shown as incorporating two DMDs 16, 20 in series, it should be understood that the system 2 may include more than two DMDs in series, providing additional improvements in contrast levels in the projected images. Typically the loss of light as a result of optical transmission inefficiencies limit the number of DMDs which may be aligned in series and produce acceptable results.
A color version of the serial SLM system of the present invention may be understood with reference to the color projection system shown generally as 3 in
As illustrated by light ray vector 4 shown in dotted line, white light source and reflector 10 illuminate integrator rod 12 via cold mirror 11. By using mirrors 21 and 23, which are dichroic mirrors that selectively reflect some wavelengths of light and transmit others, the white light from the source 10 is divided into three wavelength bands, corresponding to the red, green and blue portions of the spectrum. Wavelength selectively reflecting mirror 21 reflects the red portion of the light from the integrator rod 12 towards relay 14A and allows the green and blue portion of the light to pass on towards mirrors 23 and 24.
Relay 14A directs the red light from mirror 21 onto the surface of a first DMD device 16A at a suitable angle so that the on state light is directed through relay 18A which images the light from 16A onto the surface of a second DMD device 20A so that there is a one to one correspondence between the pixels of 16A and 20A. On state light from the second DMD 20A then reaches plain mirror 25 and is reflected towards wavelength selective mirrors 26 and 27 which pass the red light onto the projection lens (not shown) or other imaging system and then to the projection screen.
In a similar manner, wavelength selectively reflecting mirror 23 directs the green portion of the light towards relay 14B and the components following while allowing the blue portion of the light to continue on towards mirror 24. Plain mirror 24 directs the blue light that remains towards relay 14C and the components following.
The output of the second serial DMD system consisting of components 14B-20B is directed towards the projection lens by wavelength selective mirror 26 which passes the red light from mirror 24 and reflects the green light from 20B towards the projection lens. In a similar fashion, the output of the third serial DMD system 14C-20C is directed towards the projection lens by wavelength selective mirror 27 which passes the red light from mirror 24 and green light from mirror 25 while reflecting the blue light from 20C towards the projection lens.
It should also be clear that other systems of dividing the input light into color bands and combining the output of the serial DMD systems may be used without departing from the spirit of the invention. It should also be understood that three colored light sources with reflectors, cold mirrors and integrating rods could be used to separately illuminate the three serial DMD systems. It should also be understood that the system 3 is a schematic representation of a three color system, and those skilled in the art will recognize that additional optical elements may be required to allow combining the output of each color channel into a single projection lens or image plane.
As will be understood, the system 3 also includes a controller 22, operatively coupled to the three sets of first and second DMDs 16A-C and 20A-C. For clarity,
The operation of a serial DMD system may be further understood as follows. Assume that the optical efficiencies of the first and second DMDs are represented by α1,α2, the “dark level” factors by δ1,δ2, the output pixel brightness of the two serial DMDs can be described by (the serial DMD equation):
Where P1 is the output brightness of the first DMD 16 and α1, δ1 and y1 are the optical efficiency, dark level factor and bit duration of the first DMD 16 respectively. Similarly P2, α2, δ2 and y2 are the optical efficiency, dark level factor and bit duration of the second DMD 20.
Compare the serial DMD equation with the single DMD equation shown previously in the background section. Here the two serial DMDs have a total optical efficiency of αs=α1α2 and a dark level factor of δs=δ1δ2. Since the optical efficiency of a single DMD is about 60% with current technology, the combined optical efficiency will be reduced to αs∪0.36. However, the combined dark level factor δs will also be much lower than that of a single DMD. The second item εsL=α1δ2y1+α2δ1y2 in the serial DMD equation represents the cross talk between two DMDs and should be considered signal dependent noise. When n is large, both δ1, and δ2 should be small compared to the signals in the serial DMD equation and can be ignored.
If we ignore the cross talk item εs then the output brightness from two serial DMDs can be considered linear to the product of bit time durations y1y2. In order for the two PWM sequences of the serial DMDs to function in this manner the following requirements must be met:
In practice, requirement 2, above, typically limits the bit sequences of the first DMD to simple PWM schemes, while more complex bit splitting schemes may be employed with the second DMD.
In this embodiment one method for allocating the bits of a larger binary word between the two DMD devices is to allocate the top k most significant bits of each m-bit binary word to the first DMD and the balance to the second. This can be explained by a simple example.
Assume each DMD in a serial DMD configuration has 100% optical efficiency and supports n=4 bit depth. When the first DMD produces a brightness value that is exactly half of the total brightness range, the second DMD will represent the half brightness range with 2n=24=16 brightness levels. The result is equivalent to a bit depth of m=5, effectively adding one bit to the bit-depth within the lower half of the brightness range. When the first DMD output increases to maximum brightness, the DMD will produce 24=16 levels for the entire brightness range. However, since the lower half already has a representation of 16 levels, then only the top half needs additional 8 levels. The resulting coding will have total of 24 levels, with top half represented by 4 bit-depth and lower half by 5 bit-depth. Considering human brightness perception actually has a near logarithmic response, this coding scheme may actually be more efficient than the linear coding used in a conventional single DMD based system.
In general, if the first DMD maintains 2k equally spaced brightness levels, a serial DMD device should have a dynamic range performance equivalent to k+n bit depth, at least within the lowest level. In practice, due to the binary pulse width modulation (PWM) scheme utilized, a frame duration is not divided into 2n but 2n−1 equal segments, which makes actual code design more complex. In this example we select 2k levels for the first DMD, and run the second DMD at the full pixel depth available based on the frame rate.
The actual code value represented by each code word will be selected based on the following two criteria:
The table in
While this invention has been described with reference to DMD devices, other reflective SLMs may be employed, particularly ones which do not rely on PWM techniques may be used for one or both SLMs, freeing the design from the code design restrictions and complexity imposed by the PWM scheme.
It should also be clear to those skilled in the art that alternative arrangements of optical elements and SLMs other than those shown in
This application claims priority to U.S. Provisional Application No. 60/189,480 filed on Mar. 15, 2000 and International Application No. PCT/CA01/00339 filed on Mar. 15, 2001 and published in English as International Publication No. WO 01/69941 A2 on Sep. 20, 2001.
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/CA01/00339 | 3/15/2001 | WO | 00 | 12/9/2002 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO01/69940 | 9/20/2001 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 4638309 | Ott | Jan 1987 | A |
| 5157523 | Yamagishi et al. | Oct 1992 | A |
| 5272473 | Thompson et al. | Dec 1993 | A |
| 5278652 | Urbanus et al. | Jan 1994 | A |
| 5280277 | Hornbeck | Jan 1994 | A |
| 5300966 | Uehira et al. | Apr 1994 | A |
| 5386253 | Fielding | Jan 1995 | A |
| 5400093 | Timmers | Mar 1995 | A |
| 5490009 | Venkateswar et al. | Feb 1996 | A |
| 5555035 | Mead et al. | Sep 1996 | A |
| 5589852 | Thompson et al. | Dec 1996 | A |
| 5592239 | Hara et al. | Jan 1997 | A |
| 5612753 | Poradish et al. | Mar 1997 | A |
| 5626411 | Takahashi et al. | May 1997 | A |
| 5673060 | Blaxtan et al. | Sep 1997 | A |
| 5686939 | Millward et al. | Nov 1997 | A |
| 5699130 | Taylor | Dec 1997 | A |
| 5757348 | Handschy et al. | May 1998 | A |
| 5796442 | Gove et al. | Aug 1998 | A |
| 5809182 | Ward et al. | Sep 1998 | A |
| 5844663 | Holley et al. | Dec 1998 | A |
| 5865520 | Kavanagh et al. | Feb 1999 | A |
| 5902030 | Blanchard | May 1999 | A |
| 5933588 | Easwar et al. | Aug 1999 | A |
| 5956000 | Kreitman et al. | Sep 1999 | A |
| 5978142 | Blackham et al. | Nov 1999 | A |
| 5986640 | Baldwin et al. | Nov 1999 | A |
| 5990982 | Gove et al. | Nov 1999 | A |
| 6017123 | Bleha et al. | Jan 2000 | A |
| 6034660 | Millward et al. | Mar 2000 | A |
| 6057816 | Eckersley | May 2000 | A |
| 6064366 | Millward et al. | May 2000 | A |
| 6215547 | Ramanujan et al. | Apr 2001 | B1 |
| 6224217 | Tanaka | May 2001 | B1 |
| 6276801 | Fielding | Aug 2001 | B1 |
| 6285349 | Smith | Sep 2001 | B1 |
| 6317112 | Handschy et al. | Nov 2001 | B1 |
| 6348907 | Wood | Feb 2002 | B1 |
| 6373603 | Popovich et al. | Apr 2002 | B1 |
| 6456301 | Huang | Sep 2002 | B1 |
| 6795221 | Urey | Sep 2004 | B1 |
| 20030063226 | Gibbon et al. | Apr 2003 | A1 |
| 20040001184 | Gibbon et al. | Jan 2004 | A1 |
| Number | Date | Country |
|---|---|---|
| 2227920 | Jul 1999 | CA |
| 0 139 991 | May 1985 | EP |
| 0 606 162 | Jul 1994 | EP |
| 0 751 683 | Jan 1997 | EP |
| 0 786 687 | Jul 1997 | EP |
| 0 961 502 | Dec 1999 | EP |
| 0 755 556 | May 2000 | EP |
| H6-102484 | Apr 1994 | JP |
| WO 9410675 | May 1994 | WO |
| WO 9525292 | Sep 1995 | WO |
| WO 9604582 | Feb 1996 | WO |
| WO 9960557 | Nov 1999 | WO |
| WO 0141455 | Jun 2001 | WO |
| WO 0196907 | Dec 2001 | WO |
| WO 0196907 | Dec 2001 | WO |
| WO 0203687 | Jan 2002 | WO |
| WO 0203687 | Jan 2002 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20030142274 A1 | Jul 2003 | US |
| Number | Date | Country | |
|---|---|---|---|
| 60189480 | Mar 2000 | US |
