Patent Application
20170255020
The invention has application within the field of wearable displays. It is used for achieving a light weight design in head mounted displays.
Head-Mounted-Displays (HMD) is a type of device with increasing popularity within the consumer electronics industry. HMDs, along with similar devices such as helmet-mounted displays, smart glasses, and virtual reality headsets, allow users to wear a display device such that the hardware remains fixed to their heads regardless of the person's movement.
When combined with environmental sensors such as cameras, accelerometers, gyroscopes, compasses, and light meters, HMDs can provide users with experiences in virtual reality and augmented reality. Virtual reality allows a user to be completely submerged into a virtual world where everything the user sees comes from the display device. On the other hand, devices that provide augmented reality allow users to optically see the environment. Images generated by the display device are added to the scene and may blend in with the environment.
One of the primary elements of HMDs is a display module mounted onto the head. However, since the unaided human eye cannot accommodate for images closer than a certain distance from the eye, eye piece lenses are required to re-image the display module such that the display appears to be at a comfortable viewing distance from the user. Such optical configuration requires lots of space between the eye piece and the display module. Furthermore, complex lenses are needed if the HMD needs to display images with high quality and wide field of view. These lenses often make the device very bulky to wear.
A number of methods had been invented to eliminate the need of heavy lenses in HMDs. Light field displays use a high resolution image panel with a microlens array to integrate subsets of images onto different parts of the retina. This method leads to images with low effective resolution. Retinal scanning displays are capable of producing images with resolution equivalent to the native resolution of the laser scanner. However, the stringent requirement to align the scanning mirror through the eye's pupil means that it is very difficult to fabricate an HMD that fits different anthropometric variations.
Holographic HMDs typically suffer from several problems. Firstly, image quality is typically poor as spatial light modulators (SLMs) are only available for either phase or amplitude modulation but not both. Computational holograms often suffer from what is known as the zero order which consists of light appearing in unwanted regions on the retina. Secondly, speckle is usually visible in holographic displays which use laser sources. Thirdly, an ideal holographic image requires using an SLM with very high resolution or small pixel size comparable to optical wavelengths. This also means holographic images would typically require very high computational load.
WO9409472A1 (Furness et al., published Apr. 28, 1994), WO2015132775A1 (Greenberg, published Sep. 11, 2015), U.S. Pat. No. 8,540,373B2 (Sakakibara et al., issued Mar. 31, 2011), JP2013148609A (Pioneer, published Jan. 8, 2013, and JP5237267B2 (Yamamoto, issued Jul. 17, 2013) describe representative retinal scanning displays where a collimated beam and scanning mirrors are used to directly rasterize an image onto the retina. These devices include a gaze tracker which determines the gaze direction of the eye. Apart from the scanning mirrors that rasterizes the image, additional mechanical mirrors are used to move the single eye point of the optical system depending on eye position obtained from a gaze tracker. High accuracy/low latency gaze trackers are crucial if these systems are to work as designed. However, currently no gaze trackers can achieve desirable reliability. Also, scanning beam systems are often expensive.
U.S. Pat. No. 6,751,026B2 (Tomono, issued Jun. 15, 2004) describes a retinal direct projection display where a “light-speed controlling element” is used to make divergent light from the planar backlight into light parallel to an optical axis. However, having parallel light emerging from the image forming element may lead to poor energy efficiency as well as poor brightness uniformity in the HMD system, as the pupil size of the eye is typically much smaller than the image panel. In addition, the system requires a Fresnel lens after the image forming element to focus the image onto the retina. Large space will be required either between the Fresnel lens and the eye and/or between the image forming element and the Fresnel lens. This will make the device bulky.
JP4319028B2 (Dietrich, issued Dec. 24, 2004) describes a retinal projection system where an image is directly projected onto the retina. However, a conventional projection lens is used in the system. These lenses will make the device very large and heavy.
This invention concerns a design of a wearable display which enables the device to have reduced weight relative to known configurations without compromising other technical performances. The design is particularly suitable for head mounted displays or smart glasses with applications in virtual reality (VR) and augmented reality (AR).
The design involves the use of a special panel illumination unit which produces light converging towards the eye with high directionality. This converging light passes through an image panel, which attenuates light selectively at different pixels. The image panel is physically located close to the eye at a distance much closer than a normal unaided eye can accommodate. However, the high directionality of the panel illumination unit allows light emerging from each pixel to have a small divergence angle. Because of this, the eye can focus on the pixels, allowing a clear image to be visible without the need of large spaces or large optical elements between the image panel and the eye.
Unlike the prior art, in this invention optical elements with a single (or a small number of) optical axis is not necessary for the eye to accommodate for the image. Furthermore, the system does not need to employ “light field/integral imaging” type algorithms which may reduce the display's effective resolution. This means the user could potentially see the image at its full resolution equivalent to the panel's resolution.
The image panel may be a transparent display panel such as a liquid crystal display or a Microelectromechanical system shutter array. The image panel may also be a reflective display such as a liquid crystal on silicon (LCoS) or a Digital micro-mirror device (DMD), in which case the optics may have to be folded up with additional optics such as beam splitters.
The panel illumination unit may include a curved surface, such as a free form mirror, an approximately ellipsoid surface, or a mirror array with elements small enough that the surface can be considered curved at large scales. Alternatively, the panel illumination unit may include known diffractive and refractive features capable of performing similar functions.
At the scale of the HMD, light emerging from the image panel remains convergent towards the eye. However, at individual pixel scale, light emerging from each pixel is slightly divergent due to diffraction from the effective aperture of each pixel. Light emerging from each pixel has a divergence that is sufficiently small such that, at the pupil of the eye, the beam diameter is comparable to or smaller than the pupil diameter. The divergence can be changed to some extent by varying the shape parameters of the panel illumination unit.
However, the divergence of light emerging from pixels can also be improved by adding a lens array element after the image panel and decreasing the fill factor of the pixels. This lens array element is a thin element which could be a simple refractive lens array. Alternatively, the lens array may be an array of phase and/or amplitude diffractive lenses. It is also possible to design each lenslet such that they overlap spatially, with each lenslet only diffracting light emerging from specific pixels. This will allow light emerging from each pixel to be expanded and have a smaller divergence
To the accomplishment of the foregoing and related ends, the invention, then, comprises the features hereinafter fully described and particularly pointed out in the claims. The following description and the annexed drawings set forth in detail certain illustrative embodiments of the invention. These embodiments are indicative, however, of but a few of the various ways in which the principles of the invention may be employed. Other objects, advantages and novel features of the invention will become apparent from the following detailed description of the invention when considered in conjunction with the drawings.
In the annexed drawings, like references indicate like parts or features:
Figure (a): Where the micro-lenses are an array of spatially overlapping phase and/or amplitude diffractive lenses.
Figure (b): Showing one possibility of overlapping diffractive lenses by using adjacent lenses with different color filters such that each diffractive lens is only visible to its matching pixel.
Figure (c): Showing one possibility of maximizing the distance between two diffractive lenses visible to the same wavelengths.
Figure (d): Showing one possibility of effectively increasing the number of color filters by using different permutations of color filter pairs.
Figure (e): Showing diffractive lenses fabricated onto a non-planar substrate such that each diffractive lens will be visible to a different spectrum.
An aspect of this invention is a head mount display or similar display devices that are fixed to the head. In exemplary embodiments, the display device includes a light source, a panel illumination unit, an image panel, and an eye monitor. The panel illumination unit is characterized by its geometry, being structured such that light emerging from the panel illumination unit would converge to a small area towards the eye. The image panel displays a pattern that attenuates the illumination light at individual pixels. The image panel is physically located close to the eye at a distance much closer than an unaided eye could accommodate.
The first embodiment of this invention is shown in
The converging beam 8 illuminates an image panel 3 as shown in
The image panel is located at a distance 201 from the eye 4, and the image panel selectively transmits light at different pixels to generate the image content, with light emerging through each pixel being directed towards the eye 4. When only one small area light source is switched on, the light 8 incident onto the image panel 3 will be highly directional. Provided that the pixel size is not too small, light emerging through each pixel would substantially preserve the directionality of the incident light with only weak diffractive effects. The panel illumination unit is configured to limit divergence of light that is emitted from the image panel, such that when the display device is worn by the user, the image panel is located at a distance to the user's eye closer than a distance where the eye can effectively focus the pixels.
A diffraction effect of each pixel will lead to a weak divergence for light emerging from each pixel. Just before reaching the pupil 5 of the eye, the diverging beam from each pixel could have a diameter comparable to or smaller than the pupil. This divergence angle also needs to be small such that the eye's focusing mechanisms can accommodate for the beam to form a small pixel image on the retina 6. If the pixels in the image panel are too small, light emerging from a pixel will be diverging at a large angle due to diffraction, creating a large defocused pixel on the retina. On the other hand, if the pixels are too large, there would be too few pixels in the image panel within a given field of view (FoV) and given panel-to-eye distance. The effective resolution of the display could be maximized by optimizing the pixel size of the image panel such that both effects are balanced. The pixel apertures may also be circular or in other known shapes where the divergence angle of the diffracted beam is minimized.
The eye monitor 11 is a device used to monitor or measure information pertaining to an eye configuration of a user wearing the head mounted display device. In a preferred embodiment, the eye monitor includes a camera which is pointed at the eye, and a light source (which could be infrared such that it will be invisible for the user) for illuminating the eye. The camera may tracks the gaze direction, position, and pupil diameter of the eye to generate the eye configuration information. The system then switches on a light source according to the position of the eye to create a viewing zone that matches the pupil position based on the eye configuration information measured by the eye monitor, allowing light to be seen by the user. However, other technologies that gather information of the eye can also be used instead of the camera based eye tracker. One such example is a gaze tracker based on electrooculography. Information from the eye monitor can be used to control which light source (hence viewing zone) is to be switched on.
The panel illumination unit 10 in a preferred embodiment is configured as a curved mirror where one curved side is coated with a high reflectivity material. One possible shape of the mirror is shown in
Generally, the beam converging element could be a general free form element of any shape optimized for focusing one small area (the cluster of multiple light sources) to another (the eye). Due to the finite area of the pupil 4 and the segmentation of the mirror leading to offset in centers of curvature in each Fresnel zone, the optimal shape of the panel illumination unit 10 would not be an exact ellipsoid, but could be a shape perturbed from it. Such surface could be designed by numerical optimization in optical modelling software.
Although the panel illumination unit is drawn to include a single reflective surface, such unit, without a loss of generality, can also be configured as a flat element utilizing a waveguide/light guide type backlight with the use of known extraction methods to produce a converging/directional/collimated beam. The flat element can be illuminated with a fixed laser or LED light source or projection system for time sequential operation. The backlight and SLM panels can form the basis of a flat modular arrangement, in which each component includes a layer of a stack. The advantage of this approach is that the display is then thin and lightweight and could be included into an eye unit no larger than a pair of spectacles.
Subsequent embodiments in this description will be made in reference to the first embodiment and only the differences between the subsequent embodiments and the first embodiments will be discussed.
The second embodiment is shown in
The pitch of the lens array could, but not necessarily, have to exactly match the pixel pitch. For example, a device where the lens array's pitch matches a whole number multiple of the pixels' pitch could be easier to assemble; whereas a lens array 32 with a pitch slightly smaller than an integer multiple of the pixel's pitch (as drawn in
The lens array 32 in this embodiment is a refractive lens array where each micro-lens has a curved geometry. However, other known methods of lens array design, such as a holographic or diffractive lens array may also be used instead of this refractive lens array.
Poor efficiency arising from small pixel fill ratio could be improved with known methods such as using reflective pixel black masks.
In this embodiment, the panel illumination unit can also be planar as in the first embodiment (as a backlight for example, which includes a known type of collimated backlight interacting with other optical elements, for example an aperture & lens array to create a converging beam structure) which produces light converging towards the eye after the micro lens array by variation of the lens array's pitch.
The pitch of this second lens array 41 could, but not necessarily, have to match the pixel pitch. For example, a device where the lens array's pitch matches the pixels' pitch could be easier to be aligned; whereas a lens array 41 with a pitch slightly larger than the pixel's pitch could allow the optical axis of the micro-lens to be offset slightly from the corresponding pixel, allowing more efficient coupling of light from the panel illumination unit into each pixel. The pitch of this lens array can be non-linear and its alignment will be important for defining the convergence of light emerging through the pixel.
The second lens array 41 in this embodiment is a refractive lens array where each micro-lens has a curved geometry. However, other known methods of lens array design, such as a holographic or diffractive lens array may also be used instead of this refractive lens array.
In this embodiment, the panel illumination unit 40 can include a reflector or planar light guide that produces collimated/parallel light, but the lens array 41 has a pitch and alignment such that light emerging from this lens array through the pixel apertures are converging towards the eye.
As an example of achieving spatially overlapping zone plates, adjacent zone plates are made from a filter that absorbs/reflects different wavelengths. Instead of alternating between black and clear like common Fresnel zone plates, the zone plate of the micro-lens 52 (
In this case, light of a different wavelength will not be modulated by the adjacent zone plates (e.g. light emitted from a green pixel will not be modulated by the zone plate designed for a red pixel that alternates between clear and cyan because both filters are transparent to green light). In addition, if the spectral width absorbed/reflected by a zone plate is narrow, it may be possible to have multiple filters for each color, each transmitting a slightly different spectrum, with colors indistinguishable to regular users. {R, G, B}, y
{a-h}). The reflected/absorbed spectra of adjacent Fresnel zone plates do not overlap. Fresnel zone plates transmitting the same spectra can be kept furthest apart at a closest distance 202 between micro-lenses transmitting the same wavelength. Such structure can be designed by the patterning multiple layers of interference filters.
Another exemplary method to avoid cross talk in an overlapping diffractive lens could also involve exploiting the incident angle dependent properties of the interference filter. In this method, the diffractive lenses are also made from a patterned interference filter. However in this configuration, the diffractive lens is designed in a way such that a wrong pixel transmitting the same wavelength as the matching pixel could not contribute to crosstalk due to the large blue shift of the interference filter from the perspective of the wrong pixel.
Yet another method to achieve Fresnel zone plates responsive to a large number of different spectra would involve the use of grayscale lithography fabrication methods to produce Nano-structures of different thicknesses or photonics crystal with different periodicity. Such structures may be fabricated with reasonable production time with grayscale lithography.
The SLM is capable of steering a light beam based on information obtained from the eye monitor. For example, if the eye monitor detects a change in gaze direction of the eye, the SLM can change the position of the converging point (viewing zone) of the HMD accordingly such that the image remains visible to the user. Depending on the exact beam steering angle requirements and panel size requirements, the SLM can also be placed in other locations to achieve the same purpose. For example, the SLM can be placed between the image panel 3 and panel illumination unit 10, or between the light source 1 and the panel illumination unit 10.
An aspect of the invention is a head mounted display device. In exemplary embodiments, the head mounted display device includes a switchable light source that is switched to emit light for generating at least one viewing zone, a panel illumination unit that is illuminated by the light source, and an image panel. The panel illumination unit converges the light onto the image panel and the image panel selectively transmits light at different pixels to generate image content, wherein the image content is visible to the user when the eye is aligned with respect to the viewing zone. The panel illumination unit converges the light such that light emitted by the image panel converges into the viewing zone at a direction based on the eye configuration. The head mounted display device may include one or more of the following features, either individually or in combination.
In an exemplary embodiment of the head mounted display device, the panel illumination unit is configured to limit divergence of light that is emitted from the image panel, wherein when the display device is worn by the user, the image panel is located at a distance from the user's eye closer than a distance where the eye can focus the pixels.
In an exemplary embodiment of the head mounted display device, the switchable light source comprises a plurality of independently switchable light source units that each emits light converging towards different points in space for generating multiple viewing zones at different directions, and the light source units are selectively switched on or off to generate the viewing zone based on the eye configuration information measured by the eye monitor.
In an exemplary embodiment of the head mounted display device, the image panel is a transparent pixellated liquid crystal display panel.
In an exemplary embodiment of the head mounted display device, the device further includes an eye monitor that measures eye configuration information, wherein the panel illumination unit converges the light such that light emitted by the image panel converges into the viewing zone at a direction based on the eye configuration information measured by the eye monitor. The eye monitor may be configured to measure pupil size of the user, and the image panel is configured to minimize divergence of light such that the viewing zone is smaller in at least one dimension than twice the measured pupil size.
In an exemplary embodiment of the head mounted display device, the eye monitor comprises a gaze tracker that is configured to measure gaze direction, pupil position, and pupil diameter as included in the eye configuration information.
In an exemplary embodiment of the head mounted display device, the panel illumination unit comprises a curved mirror with a reflective coating on one curved side.
In an exemplary embodiment of the head mounted display device, the panel illumination unit is configured as a Fresnel lens.
In an exemplary embodiment of the head mounted display device, the panel illumination unit comprises a transparent substrate and a curved element with a reflective coating.
In an exemplary embodiment of the head mounted display device, the image panel is curved.
In an exemplary embodiment of the head mounted display device, the image panel comprises a lens array including a plurality of lenslets located adjacent a pixel panel, wherein the lenslets collimate light emerging from the pixels.
In an exemplary embodiment of the head mounted display device, a pitch of the lenslets matches a pixel pitch such that the lenslets selectively collimate light emerging from respective pixels.
In an exemplary embodiment of the head mounted display device, the lens array comprises overlapping holographic diffractive lenslets.
In an exemplary embodiment of the head mounted display device, the overlapping holographic diffractive lenslets comprise a plurality of different color filters.
In an exemplary embodiment of the head mounted display device, the overlapping holographic diffractive lenslets are formed on spatially overlapping zone plates.
In an exemplary embodiment of the head mounted display device, the zone plates are deposited on a non-flat dielectric substrate.
In an exemplary embodiment of the head mounted display device, the pixel illumination unit comprises a reflector and a lens array located immediately adjacent to the image panel and between the reflector and the image panel.
In an exemplary embodiment of the head mounted display device, the device further includes a spatial light modulator (SLM) configured to steer light emitted from the image panel into the viewing zone based on the eye configuration information measured by the eye monitor.
In an exemplary embodiment of the head mounted display device, the SLM is one of a liquid crystal panel, liquid crystal on silicon panel, electro-wetting panel, or pixellated micro-electro-mechanical systems (MEMS) mirror array.
In an exemplary embodiment of the head mounted display device, the image panel is a reflective image panel.
Although the invention has been shown and described with respect to a certain embodiment or embodiments, equivalent alterations and modifications may occur to others skilled in the art upon the reading and understanding of this specification and the annexed drawings. In particular regard to the various functions performed by the above described elements (components, assemblies, devices, compositions, etc.), the terms (including a reference to a “means”) used to describe such elements are intended to correspond, unless otherwise indicated, to any element which performs the specified function of the described element (i.e., that is functionally equivalent), even though not structurally equivalent to the disclosed structure which performs the function in the herein exemplary embodiment or embodiments of the invention. In addition, while a particular feature of the invention may have been described above with respect to only one or more of several embodiments, such feature may be combined with one or more other features of the other embodiments, as may be desired and advantageous
Industrial application will be mainly for wearable displays, in particular for achieving light weight Head Mounted Displays (HMD). The principal advantage of the invention allows HMD to be designed light weight as no large eyepiece lenses are required. Hardware manufactured using this invention may be useful in the fields of virtual reality (VR) and augmented reality (AR) for both consumer and professional markets. HMD manufactured by this invention could have applications including everyday use, gaming, entertainment, task support, medical, industrial design, navigation, transport, translation, education, and training.
