Patent Grant
7312765
The invention relates generally to image display/projection systems, and more particularly to a display system, such as a retinal scanning display (RSD) system, and a method for reducing the magnitude of or eliminating a visual artifact that a viewer may otherwise perceive when viewer shifts the direction of his gaze by moving his eye pupil.
A display such as a retinal scanning display (RSD) system typically generates images for viewing, and such images are typically graphical or video. A graphical image, i.e., graphic, typically changes infrequently or not at all. For example, a flight-instrument graphic of cockpit instruments may overlay a pilot's view. Typically, there is little change in this graphic other than the movement of the instrument pointers or numbers. Conversely, video images are a series of frames that typically change frequently to show movement of an object or the panning of a scene. For example, a television displays video images.
One aspect of a typical display system is the system's exit pupil, which defines the “window” through which the viewer can perceive an image when the pupil of the viewer's eye is aligned with the exit pupil. In a simplistic analogy, the exit pupil is much like a keyhole in a door. If the viewer's eye pupil is aligned with the keyhole, then the light which defines the image passes through the keyhole and enters the eye through the eye pupil such that the viewer perceives the image. However, if the viewer's eye pupil moves relative to the keyhole such that the light from the keyhole does not enter the eye pupil, then the viewer will not perceive the image. In some applications the display system generates two images, each via a respective exit pupil for each eye. This can allow the viewer to see a composite image stereoscopically.
As the viewer shifts his gaze within his field of view, the physical rotation of the viewer's eye may cause the viewer's eye pupil to move relative to the exit pupil. If the viewer's eye pupil moves sufficiently far, it can move out of alignment with the exit pupil. More specifically, the viewer will perceive an image as long a portion of the eye pupil is aligned with a portion of the exit pupil. That is, the viewer will still perceive the image as long as light from the exit pupil enters the eye pupil (although the viewer's perception of the image may vary depending on the degree of alignment between the eye pupil and the exit pupil). Assuming that the diameter of a human viewer's eye pupil typically ranges from about 2 millimeters (mm) in bright light to about 7 mm in dim light and that the width and height of the exit pupil are always smaller (e.g., 1 mm) than the diameter of the viewer's eye pupil, the viewer can move his eye over a range approximately equal to the sum of the diameter of his eye pupil and the width/height of the exit pupil (e.g., 3-8 mm) without losing sight of the image. But if the viewer shifts his gaze such that his eye pupil moves beyond this range—which he often does—then he typically loses sight of the image. While this example assumes, for simplicity of explanation, that the exit pupil is smaller than the eye pupil, this is not always the case. However, the basic concepts can still apply even where the eye pupil is larger than the exit pupil.
To prevent the viewer from losing sight of an image as he shifts his gaze, an RSD system may include a tracking display system to track the movement of the viewer's eyes and to move the exit pupils to keep them aligned with the respective eye pupils. An example of a tracking RSD is disclosed in commonly assigned International Publication WO 01/33282, filed Oct. 29, 1999, which is incorporated herein by reference.
A common problem with a tracking display system is that it may allow a viewer to perceive visual artifacts when he shifts his gaze. A visual artifact is an undesired phenomenon that a viewer perceives in an image. For example, flicker, which is a rapid fluctuation in brightness, is a visual artifact that a viewer may perceive in an image, particularly a raster-scanned image. Because a viewer's eyes can typically move faster than the display system can track the movement—a viewer's eyes can typically rotate at angular velocities up to 500°/second—there is often a slight delay between the time when the eye pupils attain their new positions and the time when the respective exit pupils become realigned with the eye pupils. During this period of misalignment, the viewer may perceive that a composite image is flickering, particularly if the display system is a raster-scanning type of display. Specifically, during the period of misalignment, light from the composite image does not enter the eye pupils, and thus the image can “disappear” until the display system realigns each exit pupil with its corresponding eye pupil. Thus, the viewer may perceive this momentary “disappearance” and the subsequent “reappearance” of the image as an artifact such as flicker.
Moreover, the viewer's perception of flicker may be exacerbated if the display produces the perceived image with raster-scanned, modulated beams of light. Because the peripheral rods and cones (responsible for peripheral vision) of the human eye have relatively fast response times, they, unlike the straight-ahead rods and cones (responsible for straight-ahead vision), may detect the flicker inherent in a scanned image if the scanning frequency is too low. Consequently, as the eye pupil and the exit pupil come into alignment, light from the exit pupil may initially strike the peripheral rods and cones. Consequently, the increased flicker sensitivity of the peripheral rods and cones may increase the viewer's perception of flicker.
Unfortunately, visual artifacts such as flicker may annoy or distract the viewer. For example, if the display system generates a flight-instrument overlay graphic, such visual artifacts may distract or irritate the pilot or slow the pilot's response time.
An expanded-exit-pupil display system can eliminate the need to track eye movements by generating one or more arrays of multiple identical exit pupils—typically an equal number of arrays for each eye—such that at least one exit pupil is always aligned with each pupil as the viewer shifts his gaze. Each array, often called an expanded exit pupil, is the region within which the individual exit pupils are located. The effective size of the expanded exit pupil is defined by the region of the eye's field of view (FOV) over which the exit pupils are distributed. This is often called the “eye box.”
Unlike the single exit pupil of the tracking RSD system discussed above, which has a single exit pupil for each eye, the cross-sectional dimensions of the expanded exit pupil are significantly larger than the diameter of the corresponding eye pupil. And ideally, the gaps between the exit pupils with the expanded exit pupil will be less than the diameter of the viewer's eye pupil. Consequently, as long as the viewer's gaze remains within the ideal expanded exit pupil, he will see the composite image because at least one of the exit pupils within each expanded exit pupil will always be aligned with the respective pupils of the viewer's eyes. Examples of a RSD system that generates an expanded exit pupil are disclosed in U.S. Pat. Nos. 5,701,132 and 6,157,352, which are incorporated by reference.
Unfortunately, generating and maintaining an ideal and sufficiently large and uniform expanded exit pupil may be difficult in many applications. Additionally, a large exit pupil may require more optical energy for a given perceived brightness. Consequently, it may be desirable to combine tracking with an expanded-exit-pupil display. An example of such a combination display system is disclosed in commonly assigned International Publication WO 01/33282, filed Oct. 29, 1999, which is incorporated by reference. Such a system may suffer from the same problems as the tracker and expanded-exit-pupil display systems discussed above.
Consequently, with many expanded exit pupils, the viewer may perceive visual artifacts even when he shifts his gaze such that it remains aligned with the expanded exit pupil. For example, if the gaps between the individual exit pupils within the expanded exit pupil are greater than the diameter of the viewer's eye pupil, then the viewer may perceive flicker as his pupil moves from a starting exit pupil to a destination exit pupil. This flicker is caused by the viewer's eye pupil losing alignment with the starting exit pupil before becoming aligned with the destination exit pupil (i.e., like moving from “keyhole” to “keyhole”). The flicker may be especially noticeable when the intensity or other characteristics of the respective images viewed through the starting and destination exit pupils are different. Furthermore, if the system is a tracking system, then the viewer may perceive additional flicker as the expanded exit pupil follows the movement of the viewer's eyes.
In one aspect according to the invention, a display system includes an image generator that generates an image and a circuit that reduces or eliminates the viewer's perception of a visual artifact when the viewer's gaze shifts with respect to the image.
For example, such a display system may generate a fill-in light to reduce or eliminate the viewer's perception of flicker or other visual artifacts when the viewer shifts his gaze in a manner that moves his eye pupil with respect to an exit pupil. In addition, the system may match the fill-in light's brightness, color, or both the brightness and color, to the brightness and/or color of the image.
At an initial time T1, a viewer's eye 10 is in a steady-state orientation and is aligned with an exit pupil 12, which is generated by a tracking RSD system such as the system of
At a time T2, the eye 10 begins rotating (as indicated by the arrow A) about an axis 28, which is parallel to the Z axis, toward a path 30 that has the same dimensions as the exit pupil 12. Of course the pupil 14, and thus the eye's FOV 20, follow the rotation of the eye 10.
At a time T3, the eye 10 stops rotating at another steady-state position where the pupil 14 is aligned with the path 30. Consequently, the center axis 24T3 of the FOV 20T3 is aligned with the path 30, and the center axis and the path both intersect the center region 16T3 of the retina 18.
At time T2 or shortly thereafter, the RSD system (
At a time T4 after the eye 10 stops rotating—the eye stops rotating at time T3—the exit pupil 12 “catches up” to the center axis 24T3, and thus overlaps the path 30. Consequently, the RSD system has effectively moved the exit pupil 12 from outside of the eye's zone of vision back into the eye's straight-ahead zone of vision. Although in this example, the exit pupil 12 effectively moved outside of the eye's FOV 20, the analysis is similar where the exit pupil moves to the eye's peripheral zone of vision.
As discussed in the Background section of this application, the viewer (not shown) may perceive flicker or other visual artifacts during the period between times T2 and T4 when the exit pupil 12 effectively moves from the viewer's straight-ahead zone of vision, through the viewer's peripheral zone of vision, to outside the viewer's zone of vision, and then back to the straight-ahead zone.
Furthermore, the viewer's other eye and the exit pupil through which that eye views the image or composite image (neither the other exit pupil nor the other eye is shown) act the same or approximately the same way as the eye 10 and exit pupil 12 during the period from T1-T4.
Still referring to
The operation of the RSD system 40 is now discussed in conjunction with
At time T2, the eye 10 begins rotating, thus causing the pupil 14 and the center axis 24 to rotate away from the exit pupil 12 and toward the path 30.
Also at time T2 or shortly thereafter, the eye-position mechanism 50 detects the rotation of the eye 10. In one embodiment, the image generator 46 directs an infrared tracking beam 60 onto the eye 10, which reflects the beam back to the eye-position mechanism 50 via the partially transmissive mirror 54 and the mirror 58. The eye's cornea 62 has a central region 64 that is aligned with the pupil 14. Because the central region 64 has a different reflectivity than the other regions of the cornea 62 and the eye's white part 66, the mechanism 50 can detect when the beam 60 is being reflected from a region of the eye 10 other than the central corneal region 64. In response to such detection, the control circuit 42 causes the mechanism 50 to move the assembly 44 so that beam 60 tracks the central corneal region 64, and, consequently, so that the exit pupil 12 tracks the center axis 24 of the FOV 20. This tracking operation is further discussed in U.S. patent application Ser. No. 09/128,954, entitled “PERSONAL DISPLAY WITH VISION TRACKING”, which is incorporated by reference. But as stated above in conjunction with
As discussed above in conjunction with
Sometime between times T2 and T4, to reduce or eliminate potential flicker and/or other visual artifacts, the control circuit 42 activates the light source 48 in response to the mechanism 50 sensing misalignment of the eye 10 with the exit pupil 12. In some embodiments, some or all of the duration, brightness, and color of the fill-in light and the direction and aperture (not shown) of the light source 48 are preprogrammed into the control circuit 42—the aperture is the opening that determines the spread angle 68, and thus the size, of the flash field 59. In other embodiments, however, the control circuit 42 calculates one or more of these quantities based on the brightness and/or color of the exit-pupil image and on the length of the arc through which the eye 10 has rotated. For example, the image sensor 52 can sense the exit-pupil image via the partially transmissive mirror 58 and the mirror 56, and can determine the average brightness, average color, or both the average brightness and color of the exit-pupil image. The sensor 52 or control circuit 42 can add a scaling factor to account for the fact that the mirrors 56 and 58 direct only a portion of the image energy to the sensor 52. Alternatively, the control circuit 42 can determine the average brightness and/or average color from the electronic or optical signals (not shown) that the image generator 46 uses to generate the exit-pupil image. The control circuit 42 can then activate the light source 48 to generate the fill-in light having the same or approximately same average brightness, color, or both brightness and color as the exit-pupil image 12. Furthermore, the eye-position mechanism 50 can determine the relative position of the pupil 14 with respect to the exit pupil 12, and the control circuit 42 can set the aperture and/or direction of the light source 48 such that the fill-in light illuminates the desired portion of the eye's composite FOV.
Still referring to
In another embodiment, the RSD system 40 includes single assembly 44 for both of the viewer's eyes. In this embodiment, the image generator 46 generates two exit pupils 12, one for each eye. For example, the assembly 44 may include optics, similar to the optics in a periscope or stereo microscope, that split a source image from the generator 46 into two exit-pupil images. The sensor 52 senses the brightness and/or color of the source image (either before or after the split), or the control circuit 42 determines the color and brightness of the source image from the signals (not shown) used to generate the source image. The mechanism 50 senses movement of the viewer's eyes. Because a viewer's eyes typically move in tandem, particularly when viewing a far-field object, the mechanism 50 may track the movement of only one eye. The light source 48 generates the fill-in light within at least a portion of the viewer's composite FOV.
Referring to
First, a circuit—such as the control circuit 42 or the image generator 46—respectively sums the red, green, and blue components of the pixels of an image before the image is displayed via the exit pupil 12. For example, where the image is stored in a buffer—the buffer may be part of the control circuit 42 or the image generator 46—three adders (not shown), one for each color, can respectively sum the red, green, and blue pixel values for the image.
Next, the sums of the red, green, and blue pixel values are provided to the respective inputs of three digital-to-analog (D/A) converters—the D/A converters can be part of the control circuit 42 or the fill-in light source 48—that respectively generate red, green, and blue driving signals for the LEDs 80, 82, and 84.
When the eye-position mechanism 50 detects that the eye 10 has become misaligned with the exit pupil 12 as discussed above in conjunction with
When the eye-position mechanism 50 detects that the eye 10 is or is almost realigned with the exit pupil 12, then the controller circuit 42 deactivates the LEDs 80, 82, and 84. The controller circuit 42 may deactivate the LEDs abruptly, or may decrease their intensities gradually to reduce the visual impact of an abrupt deactivation.
Alternatively, the summing circuit may sum only the red, green, and blue values for some but not all of the pixels in the image depending on the image-display rate, the image resolution, and other display parameters. Or the summing circuit may sum one or two color values for all the pixels, and the remaining color value(s) for only some of the pixels.
At time T1, the viewer's eye 10 is in a steady-state position and is gazing straight ahead such that the center axis 24T1 of the viewer's FOV 20T1 is aligned with the exit pupil 104a and the lens (not shown) of the eye focuses the corresponding exit-pupil image onto the center region 16T1 of the retina 18.
At time T2, the eye 10 begins rotating toward an exit pupil 104b, and the eye's pupil 14 and FOV 20 follow the rotation of the eye.
At time T3, the eye 10 stops rotating and enters into a steady-state position where the FOV center axis 24T3 is aligned with the exit pupil 104b and intersects the center region 16T3 of the retina 18.
At time T2 or shortly thereafter, the RSD system (not shown in
At a time T4 after the eye 10 stops rotating—the eye stops rotating at time T3—the FOV center axis 24T3 is aligned with the exit pupil 104b.
As discussed in the Background section of this application, the viewer (not shown) may perceive flicker or other visual artifacts during the period between times T2 and T4 when the exit pupil 104a, and thus the image, effectively moves from the viewer's straight-ahead zone of vision to (and maybe beyond) his peripheral zone of vision, and the exit pupil 104b, and thus the image, effectively moves from the viewer's peripheral zone of vision (or beyond) to his straight-ahead zone of vision.
Furthermore, the viewer's other eye (not shown) shifts its alignment from one exit pupil to another (not shown) in the same way as the eye 10 shifts its alignment from the exit pupil 104a to the exit pupil 104b during the period between T2-T4.
Still referring to
Still referring to
Moreover, in some instances, the pupil 14 may not be perfectly aligned with one of the exit pupils 104 at steady-state times T1 and T3. But the exit pupils 104 are typically packed densely enough so that the viewer (not shown) can focus on at least one of the exit pupils 104 regardless of the direction in which he is gazing.
At time T1, the viewer's eye 10 is in a steady-state position and is gazing straight ahead such that the center axis 24T1 of the FOV 20T1 is aligned with the exit pupil 104a such that lens (not shown) of the eye 10 focuses the exit-pupil image onto the center region 16T1 of the retina 18.
At time T2, the eye 10 begins rotating toward a path 106a, and the eye's pupil 14 and FOV 20 follow the rotation of the eye.
At time T3, the eye 10 stops rotating and enters into a steady-state position where the center axis 24T3 of the FOV 20T3 is aligned with the path 106a, which, like the FOV center axis 24T3, intersects the center region 16T3 of the retina 18.
At time T2 or shortly thereafter, the eye-position mechanism (such as the mechanism 50 of
At time T4 after the eye 10 stops rotating, the exit pupil 104a “catches up” to the center axis 24T3 by moving into alignment with the path 106a, and thus by moving back into the eye's straight-ahead zone of vision.
As discussed in the Background section of this application, the viewer (not shown) may perceive flicker or other visual artifacts during the period between times T2 and T4 when the exit pupil 104a (and possibly the exit pupil 104c) effectively moves from the eye's straight-ahead zone of vision, to (and maybe beyond) its peripheral zone of vision, and back into its straight-ahead zone of vision.
Furthermore, the viewer's other eye (not shown) and the exit pupil(s) (not shown) viewed by that eye act the same way as the eye 10 does with respect to the exit pupil 104a (and possibly the exit pupil 104c) during the period from T1-T4.
Still referring to
Still referring to
The foregoing discussion is presented to enable a person skilled in the art to make and use the invention. Various modifications to the embodiments will be readily apparent to those skilled in the art, and the generic principles herein may be applied to other embodiments and applications without departing from the spirit and scope of the present invention as defined by the appended claims. Thus, the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.
This is a continuation-in-part of Ser. No. 09/129,739 that claims benefit of U.S. Pat. No. 6,583,772, filed on Aug. 5, 1998, and co-pending at the time this application was made.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5701132 | Kollin et al. | Dec 1997 | A |
| 6120461 | Smyth | Sep 2000 | A |
| 6157352 | Kollin et al. | Dec 2000 | A |
| 6245590 | Wine et al. | Jun 2001 | B1 |
| 6315412 | Snodderly et al. | Nov 2001 | B1 |
| 6583772 | Lewis et al. | Jun 2003 | B1 |
| 20020181733 | Peck | Dec 2002 | A1 |
| 20040075914 | Yamamoto | Apr 2004 | A1 |
| Number | Date | Country |
|---|---|---|
| WO 0133282 | May 2001 | WO |
| WO 0133298 | May 2001 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20040155834 A1 | Aug 2004 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 09129739 | Aug 1998 | US |
| Child | 10361095 | US |
