Patent Application
20200081254
The invention has application within the field of wearable displays. It is used for achieving a compact and light weight design in head mounted displays.
A Head-Mounted-Display (HMD) is a type of device with increasing popularity within the consumer electronics industry. HMDs, along with similar devices such as helmet-mounted displays, smart glasses, and virtual reality headsets, allow users to wear a display device such that the hardware remains fixed to their heads regardless of the person's movement.
When combined with environmental sensors such as cameras, accelerometers, gyroscopes, compasses, and light meters, HMDs can provide users with experiences in virtual reality and augmented reality. Virtual reality allows a user to be completely submerged into a virtual world where everything the user sees comes from the display device. On the other hand, devices that provide augmented reality allow users to optically see the environment. Images generated by the display device are added to the scene and may blend in with the environment.
One of the primary elements of HMDs is a display module mounted onto the head. However, since the unaided human eye cannot accommodate (that is, change its optical power to provide a focused image) for images closer than a certain distance from the eye, eye piece lenses are required to re-image the display module such that the display appears to be at a comfortable viewing distance from the user. Such optical configuration requires lots of space between the eye piece and the display module. Furthermore, complex lenses are needed if the HMD needs to display images with high quality and wide field of view (FoV). These lenses often make the device very bulky to wear.
A number of methods have been proposed to eliminate the need of heavy lenses in HMDs. Light field displays use a high resolution image panel with a microlens array to integrate subsets of images onto different parts of the retina. This method leads to images with low effective resolution. Retinal scanning displays are capable of producing images with resolution equivalent to the native resolution of the laser scanner. However, the stringent requirement to align the scanning mirror through the eye's pupil means that it is very difficult to fabricate an HMD that fits different anthropometric variations.
U.S. Pat. No. 8,508,851B2 (Miao et al., published 13 Aug. 2013) proposes a see-through display system that reflects light from a display panel using a beam splitter. Although the system is made compact by using a concave mirror, the field of view (FoV) of such system is limited as curved reflective surfaces can only provide low aberrations across very small FoV. In addition, the concave mirror means that the design is only suitable for small display panels. This limits the total resolution of the display system.
US20140168783A1 (Luebke et al., published 19 Jun. 2014) proposes the use of an optical system which uses a plurality of microlenses located in the near-eye range of an observer to simulate an object that is in focus to the observer. This method inevitably trades-off the resolution of the display for thickness. Furthermore, because the pixels of the panel are subjected to high magnification by a single curved optical surface, the image of the pixels will be severely aberrated, leading to very poor image quality.
US20020181115A1 (Massof et al., published 5 Dec. 2002) proposes a head mounted display that is comprised of a plurality of lenses and plurality of displays arranged tangent to a sphere. This design would result in a head mounted display that is heavy and expensive due to the large number of components present.
U.S. Pat. No. 9,244,277B2 (Cheng et al., published 26 Jan. 2016) and WO2016118643A1 (Benitez et al., published 28 Jul. 2016) propose head mounted displays that comprise of a plurality of tiled-up display channels each made from a prism with free-form surfaces and a micro-display. However, such system may be heavy as the solid volumes of the prisms need to be thick enough to accommodate for the microdisplays. Furthermore, the micro-display modules, which may be mounted on driver electronics, are located at a significant distance away from the users' face. This would shift the centre of mass of the whole system forward beyond the nose of the user, contributing to a downward tilting moment which may be required to be balanced with an uncomfortable head strap.
WO2016118647A1 (Benitez et al., published 28 Jul. 2016) proposes a display device which is comprised of one or more lenslet magnifying a cylindrical display surface. To achieve a virtual image that the eye can focus on, the display surface will need to be physically located at least several centimetres from the users' face. On the other hand, U.S. Pat. No. 6,008,778A (Takahashi et al., published 28 Dec. 1999) proposes a display apparatus that has two display units per eye wherein a surface which is partially transmitting and partially reflecting is being disposed to face the observer's eyeball with the panel further away from the face of the user. Again, both of these two designs will lead to a system with centre of mass far away from the user and contribute to discomfort.
Other examples of display devices are proposed in EP 2253989 (Canon KK), JP-H07-79393 (Olympus Co), EP 2565700 (Canon KK), US 2010/290125 (Canon KK) and JP-2012-247480 (Canon KK).
A first aspect of the present invention provides an optical system comprising: a first display panel for displaying a first image; a second display panel for displaying a second image; first and second eyepiece elements, the first eyepiece element having a first optical axis and the second eyepiece element having a second optical axis, the first optical axis extending generally parallel to, but offset from, the second optical axis; a first reflective element for directing light from the first display panel along a first optical path through the first eyepiece element to a viewing zone of the optical system for forming a first virtual image of the first image; and a second reflective element for directing light from the second display panel along a second optical path through the second eyepiece element to the viewing zone of the optical system for forming a second virtual image of the second image, the first optical path being different from the second optical path, and the first and second virtual images overlapping one another.
The optical system may be a wearable optical system, arranged such that, when the optical system is being worn by a user, the user's eye(s) is/are in the viewing zone of the optical system so that the user can perceive an image derived from the first and second images as displayed on the first and second display panels. The use of reflective elements allows the display panels to be physically located closer to the user's face than in, for example, U.S. Pat. No. 6,008,778. This allows most of the weight of the apparatus to be maintained closer to the user's face, making the optical system more compact and providing a more comfortable weight distribution. At the same time, the eyepiece elements (which have at least one surface with positive optical power) ensure that, despite the compact size of the optical system, the virtual images are located at a distance at which the user's eyes can comfortably focus.
The first display panel may be on the same side of the first reflective element as the viewing zone; and/or the first eyepiece element may be on the same side of the first reflective element as the viewing zone. Again, this allows the weight of the optical system to be maintained closer to the user's face.
Additionally or alternatively the second display panel may be on the same side of the second reflective element as the viewing zone, and/or the second eyepiece element may be on the same side of the second reflective element as the viewing zone. Again, this allows the weight of the optical system to be maintained closer to the user's face.
The first display panel may be laterally spaced apart from the second display panel, and optionally the first display panel and the second display panel may lie generally in a common plane. Further optionally, the first and second eyepiece elements may lie generally in a common plane with the first display panel and the second display panel.
Further optionally, the first and second eyepiece elements may be provided between the first display panel and the second display panel. This allows the optical system to be made physically compact, and again allows the weight of the optical system to be maintained closer to the user's face so making the optical system, when embodied as a wearable optical system, more comfortable for the user to wear.
In any aspect or embodiment the first eyepiece element and the second eyepiece element may be separate components. Alternatively, the first eyepiece element and the second eyepiece element may be formed as a single component.
Similarly, in any aspect or embodiment the first reflective element and the second reflective element may be separate components. Alternatively, the first reflective element and the second reflective element may be formed as a single component. As a further alternative, the first reflective element may be formed of two or more components and/or the second reflective element may be formed of two or more components.
It should be understood that specifying that optical system comprises first and second display panels does not mean that the optical system is limited to only two display panels. In other embodiments the optical system may have three, or even more, display panels. Similarly, specifying that optical system comprises first and second eyepiece elements does not mean that the optical system is limited to only two eyepiece elements, and the optical system could comprises more than two eyepiece elements. Preferably, the number of eyepiece elements is equal to, or greater than, the number of display panels so that the optical path from each display panel to the viewing zone passes through a respective, different eyepiece element.
In embodiments an optical system of the invention may have any (that is, may have one, more than one, or all) of the following optional features. It should be noted that, to avoid repetition, some of these features are defined with reference to only the first display panel, the first eyepiece element and/or the first reflective element. However, any feature defined with reference to the first display panel, the first eyepiece element and/or the first reflective element may alternatively or additionally be applied to the second display panel, the second eyepiece element and/or the second reflective element.
The first display panel may be on the same side of the first reflective element as the viewing zone.
The first eyepiece element may be on the same side of the first reflective element as the viewing zone.
For light emitted by at least a part of the first display panel, the first optical path may comprise multiple reflections by the first reflective element.
The first display panel may have a luminance profile that varies over the area of the first display panel.
The luminance profile of a region of the first display panel closer to the first eyepiece element may be narrower than the luminance profile of a region of the first display panel further from the first eyepiece element.
The first display panel may comprise an image display panel disposed in the path of light from a directional backlight. Additionally or alternatively the first display panel may comprise a light directing means disposed in the path of light from an image display panel.
The first eyepiece element may comprise at least one Fresnel lens.
The first eyepiece element may comprise a first segment and a second segment, the first eyepiece element and the first reflective element being arranged such that, for light emitted by at least a part of the first display panel, the first optical path comprises, in sequence, a first reflection by the first reflective element, refraction by the first segment of the first eyepiece element, a second reflection by the first reflective element, and refraction by the second segment of the first eyepiece element.
The second segment of the first eyepiece element may extend in a direction crossed with the first segment. It may for example extend substantially perpendicular to the first segment.
The first segment of the first eyepiece element may comprise a first Fresnel lens and/or the second segment of the first eyepiece element may comprise a second Fresnel lens.
The first reflective element may comprise a light guide arranged such that, for light emitted by a first part of the first display panel, the first optical path does not pass through the light guide, and such that, for light emitted by a second part of the first display panel, the first optical path passes through the light guide.
The second part of the first display panel may be closer to the viewing zone than is the first part of the first display panel.
The optical system may comprise an optical blocker for blocking light emitted by the first part of the first display panel from entering the light guide. The optical blocker may for example comprise a polarising element. In one implementation light from the first part of the first display panel may have a polarisation that is blocked or substantially blocked by the polarising element and light from the second part of the first display panel may have a polarisation that is transmitted substantially transmitted by the polarising element.
The optical system may comprise a further reflective element arranged such that the first light path further includes a reflection by the further reflective element.
The first display panel may be arranged to emit light of a first polarisation and the second display panel may be arranged to emit light of a second polarisation orthogonal or substantially orthogonal to the first polarisation; the first reflective element may be arranged to substantially reflect light of the first polarisation and to substantially transmit light of the second polarisation, and the second reflective element may be arranged to substantially reflect light of the second polarisation and to substantially transmit light of the first polarisation.
The optical system may comprise a lens array provided in the first optical path.
The optical system may comprise a gaze tracker for determining the position and/or orientation of an eye of a user of the optical system.
The optical system may be adapted to control the first image display panel and/or the second image display panel based on an output of the gaze tracker.
The first image display panel and/or the second image display panel may be non-flat.
Alternatively the first image display panel and the second image display panel may both be flat image display panels.
A second aspect of the invention provides a head mounted display comprising an optical system of the first aspect.
This invention concerns a design of a wearable display which enables the device to have reduced weight and increased compactness relative to known configurations without compromising other technical performances. The design is particularly suitable for a head mounted display or smart glasses with applications in virtual reality (VR) and augmented reality (AR).
The principle of the present design involves the creation of a wide field of view (FoV) image by conglomerating multiple display segments onto the user's retina. The system includes two or more image panels per eye, two or more planar reflective surfaces per image panel, and a refractive eyepiece element. Light from each image panel is reflected by a distinct set of optical surfaces before reaching the eye, forming at least two overlapping virtual images per eye. In addition, light from each virtual image is refracted by at least one optical surface with positive power such that their virtual images are located at a distance that the user's eye can comfortably focus on.
The present invention allows the image panel to be physically located at a position closer to the user's face than the centre of mass of other optical components. This geometry allows most of the weight of the apparatus to be maintained close to the user's face, providing a comfortable weight distribution.
For example, if the apparatus is fitted into a housing that resemble the shape of regular prescription eyeglasses, a system with a centre of mass close to the face would allow its torque about the nose pad to be minimised. This could potentially enable the apparatus to be secured relative to the head by resting on the ears and nose of the user, rather than requiring a strap around the head.
In a preferred embodiment, the apparatus includes two image panels, an eyepiece element, and a primary reflective element. The two image panels are positioned on two opposite ends of the apparatus with respect to the eyepiece element, as the panels are designed to be panels are located above and below the eye. Light emitted from each image panel is then transferred to the eye through a number of optical surfaces. Light is firstly reflected by a first and a second surface in the primary reflective element, followed by refraction from a positive Fresnel lens surface before reaching the eye.
The preferred image panels are standard liquid crystal panels, but could also be other known image panel technologies such as Organic Light Emitting Diode (OLED). In order to reduce crosstalk between unwanted optical surfaces, the panels may be designed to have a spatially varying luminance profile. For example, in liquid crystal displays such luminance profile can be achieved using known directional backlight designs or by using a louvre sheet after the image panel.
Although the preferred embodiment described the use of Fresnel lens surfaces reduce the system's total weight, it is also possible that at least one of these surfaces is substituted by other known lens surface types. For example, the refractive Fresnel lens surface may be substituted by a lens surface with continuous curvature, holographic, or diffractive elements.
Subsequent embodiments describe alternative schemes that also include similar arrangement of the reflective element and the use of an eyepiece element with at least two optical axes. In one embodiment, the apparatus has at least two positive power surfaces per optical path. In another embodiment, light from the image panel that contributes to a larger field of view are transmitted through a different light path than light that contributes to a smaller field of view. This could allow the centre of the image to have higher image quality than peripheral field of view while keeping the apparatus compacted.
To the accomplishment of the foregoing and related ends, the invention, then, comprises the features hereinafter fully described and particularly pointed out in the claims. The following description and the annexed drawings set forth in detail certain illustrative embodiments of the invention. These embodiments are indicative, however, of but a few of the various ways in which the principles of the invention may be employed. Other objects, advantages and novel features of the invention will become apparent from the following detailed description of the invention when considered in conjunction with the drawings.
In the annexed drawings, like references indicate like parts or features:
An aspect of this invention is a head mounted display or similar display devices that are fixed to the head. In exemplary embodiments, the display device includes two or more image panels per eye, two or more planar reflective surfaces per image panel, and a refractive eyepiece element.
The first embodiment of this invention is shown in
The embodiment of
The embodiment of
In the embodiment of
In this embodiment, the system includes two image panels per eye (2a-2b), a reflective optical element 3, and an eyepiece element 4. The image panels 2a,2b, the reflective optical element 3, and the eyepiece element 4 extend generally perpendicular to the plane of the paper. Where the invention is implemented in, for example, a head mounted display, the image panels 2a,2b, the reflective optical element 3, and the eyepiece element 4 may be provided separately for each eye of the user; alternatively one or more, and possibly all, of the image panels 2a,2b, the reflective optical element 3, and the eyepiece element 4 may be common to both eyes.
It should be noted that
These two image panels 2a, 2b are positioned on the opposite sides of the eyepiece element 4 respectively, such that they are positioned above and below the eye 1 when the apparatus is viewed by a user. For convenience the image panels will be described as being “above” and “below” the eye, but it should be understood that the terms “above” and “below” relate to the apparatus in use in its intended orientation. Light emitted from each panel is reflected by a first reflective surface of the primary reflective element 3, reflected by a second reflective surface of the primary reflective element, followed by refraction from a positive power eyepiece element 4, before leaving the apparatus towards the user's eye 1. (Details of the surface curvature of the eyepiece element are omitted from
In a preferred embodiment the image panels 2a, 2b are liquid crystal display panels, but similar function may also be achieved using other known image panel technologies, such as organic light emitting diodes (OLEDs).
A preferred primary reflective element 3 includes four reflective surfaces segments per eye, where one pair of reflective surfaces is used to reflect light from each of the image panels. In this embodiment, as shown in
A schematic representation of the surface curvature of eyepiece element 4 is shown in
In practice, in the preferred embodiment the curvature of the eyepiece element can be preserved by converting such element into a Fresnel lens as shown in
However, due to inevitable mis-positioning between the eye and the apparatus, in practice the centre of the eye will generally be at some distance from the ideal viewing position. In this case, the reflective element 3 will impose different FoV restrictions (106a, 106b) over the top and bottom images. Therefore, a design with a larger overlap 103 will allow the system to have larger eye box size.
It is worth mentioning that, although the optical elements in this embodiment have been drawn to be symmetrical above and below the eye, this does not have to be the case. The profile of the various elements above and below the eye could be asymmetrical to be optimised based on field of view versus resolution requirement of the human eye and weight distribution of the apparatus. As an example, because the fovea of the human eye is located above the optical axis of the eye's refractive elements, images located slightly below the human eye's line of sight will generally have greater importance than images above the line of sight. These considerations may be taken into account of such factors such that the overlapping regions of the top and bottom images, which could be susceptible to artefacts, are located some angle away from the fovea.
Subsequent embodiments in this description will be made in reference to the first embodiment and only the differences between the subsequent embodiments and the first embodiments will be discussed.
Similar results to reduce crosstalk may also be achieved with other known methods. For example, a light directing means, such as a louvre sheet (or privacy screen), may be placed after the light modulating unit 7. The louvre sheet may have grid size and shape that varies along different areas of the image panel. If a louvre sheet is used after an LCD image panel, then the element 6 may be an ordinary LCD backlight (or another non-directional backlight) instead of a directional backlight. However, if the image panel is based on OLED rather than LCD, a backlight would not be needed, meaning that in this case a louvre sheet would be the preferred solution to achieve low crosstalk.
The third embodiment of this invention is shown in
Details of the surface curvature of the eyepiece element are omitted from
Because the optical axis of the individual lens surfaces may be offset from the main optical axis of the eye, aberrations such as oblique astigmatism may have a significant effect on image quality of the apparatus. To overcome this, the surface curvature on both the primary and secondary segments 11-12 will need to be free form surfaces described in the form of generalised polynomials. Such surfaces can be cheaply mass manufactured using techniques such as injection moulding.
Although the eyepiece element may be manufactured as multiple pieces, to save assembling costs, the whole eyepiece element may also be injection moulded as a single piece. If the design of the master mould proves to be costly, the eyepiece element can also be designed to be moulded as a single piece with flexible/living hinges located at the joint between the primary and secondary segments of the eyepiece element 15. In such case the curved surfaces of all the segments can be formed by a single master, where the hinge may be bent to wrap the eyepiece element into a “U-shape” in the assembly process.
The curvature of this eyepiece element can also be preserved by converting one or more, and optionally all, of the primary and secondary segments of such element into a Fresnel lens as shown in
The fourth embodiment is shown in
The principles of the fourth embodiment are illustrated in
The function of the light guiding component 22, when combined with the magnifying power of the eyepiece 21, is to produce a virtual image of larger FoV 28 that merges with the virtual image of smaller field of view 27.
This embodiment may comprise a blocking means for blocking light emitted by the first part of the first display panel from entering the light guide. An example of a possible light guiding component is shown in
If light that contributes to the larger FoV image 28 does not pass through the secondary segment of the eyepiece element, it may be necessary for the light guiding element 22 to have positive power or curvature to correct for optical aberrations. For example in
As the aim of the light guiding element 22 is to replace the apex of the reflective element from the original embodiment, its thickness should be thin enough such that it extends to no more than the projected intersection between the reflective surfaces of element 20.
If the element 30 is a reflective polariser, then an additional quarter phase retarder may be added to the optical path between 30 and 31 such that the polarisation of light is rotated by 90 degree after the light passes through the phase retarder twice. This would allow the system to have higher optical efficiency.
This embodiment would allow the use of an eyepiece element with longer effective focal length or smaller magnification, allowing an image with high quality to be seen by the user. The advantages of this embodiment can also be achieved using other known arrangements for increasing the optical path length between the display panel 2a and the eyepiece element, which may include partially reflective mirrors and reflective polarisers to extend optical paths without increasing physical dimensions of the apparatus.
The reflective surfaces 42a-b may be reflective polarisers which transmits light at one state of polarisation and reflects light at an orthogonal state of polarisation. That is, the reflective surface 42a may substantially or completely reflect light of the polarisation state emitted by the image panel 40a and may substantially or completely transmit light of the polarisation state emitted by the display panel 40b, whereas the reflective surface 42b may substantially or completely reflect light of the polarisation state emitted by the image panel 40b and may substantially or completely transmit light of the polarisation state emitted by the display panel 40b. Additionally or alternatively, the primary reflective elements 41a, 41b may be reflective polarisers which transmit light at one state of polarisation and reflect light at an orthogonal state of polarisation. Hence light from the top half of the panel will not be affected by the surface 42b and light from the bottom half will not be affected by 42a. Such configuration may allow the system to have a larger overlap between the top image and the bottom image, which allows larger movement of the eye without seeing a gap between the two images, resulting in a larger “eye box” size, smoother overlapping between the top and bottom halves of the image, and more compact physical size.
In order for the crossed reflective polarisers 42a, 42b to be held in position, the reflective polarisers 42a, 42b may be embedded in glass or a transparent polymer in a geometry similar to “X-cube” light combiners commonly found in projectors. In an embodiment where this is done, one or more other elements such as the eyepiece element 43, the primary reflective element 41a and/or the primary reflective element 41b may also be embedded in the glass or polymer to hold it/them in a desired orientation relative to the reflective polarisers 42a, 42b.
This embodiment offers several advantages. Firstly, if the width of the lenslet used is smaller than 6 mm or the diameter of a human pupil (after accounting for the magnification of the eyepiece element), the system would be capable of displaying light field information. This means the system will be able to display 3D images that exhibit focus/accommodation depth cues to the user.
Secondly, using a lens array could reduce the power required by the eyepiece lens and the physical distances between the optical elements, thereby making the apparatus more compact.
Thirdly, if the lenslets in the lens array have focal lengths and widths that can be chosen independently for each lenslet, each of the lenslets could be individually optimised to correct for field-angle dependent aberrations such as field curvature, thereby improving the image quality of the system.
Although the position of the lens array 50 has been drawn to be located between the partially reflective element 30 and the reflective element 31, it is also possible for the lens array to be placed between other optical components as long as the lens array remains out of focus to the eye after the images have been magnified by the eyepiece element 51.
Additionally or alternatively, a lens array may be provided in the optical path from the image panel 2b to the viewing zone.
Such configuration could serve to lower the assembly cost of the system, and would also be particularly useful if the system offers a wide horizontal field of view.
This configuration may reduce the alignment cost/requirement between the two curved surfaces of the eyepiece element.
In a system without gaze trackers, the top and bottom image panels may appear misaligned if the eye is moved beyond a certain distance from the intended viewing position. Image transformation based on the gaze tracker could help to reverse this mis-alignment, thereby increasing the eye box size of the system and improving the system's image quality.
The preferred gaze tracker is based on an infrared camera and could include an infrared LED light source which illuminates the eye. However, the gaze tracker could also be based on other known technologies including other camera based or electrooculography based eye trackers.
Moreover, the gaze tracker could also be placed in a position that is complementary to the shape of the rest of the apparatus. For example,
Additional functions may also arise if the gaze tracker is positioned such that it images the eye through the primary segment of the eyepiece element 10. Because the eyepiece element has split optical axis, a single gaze tracker would be able to obtain stereoscopic information of the eye. This would allow the 3D position of the eye to be located using a single gaze tracker.
In addition, when the eye rotates to view the edge of the curved panel, the angle between the normal of the panel surface and the eye's gaze direction will be reduced. This could improve the luminance performance of the image panel at large FoV.
On the other hand, if the image panels are bent in the opposite direction, different advantages will arise (
Although the invention has been shown and described with respect to a certain embodiment or embodiments, equivalent alterations and modifications may occur to others skilled in the art upon the reading and understanding of this specification and the annexed drawings. In particular regard to the various functions performed by the above described elements (components, assemblies, devices, compositions, etc.), the terms (including a reference to a “means”) used to describe such elements are intended to correspond, unless otherwise indicated, to any element which performs the specified function of the described element (i.e., that is functionally equivalent), even though not structurally equivalent to the disclosed structure which performs the function in the herein exemplary embodiment or embodiments of the invention. In addition, while a particular feature of the invention may have been described above with respect to only one or more of several embodiments, such feature may be combined with one or more other features of the other embodiments, as may be desired and advantageous for any given or particular application.
Industrial application will be mainly for wearable displays, in particular for achieving small and light weight Head Mounted Displays (HMD). The principal advantage of the invention allows the apparatus to have a compacted form factor with weight distribution close to the user's face without compromising on image quality and resolution. The primary invention uses simple plain reflectors and an eyepiece element that creates at least two overlapping virtual images from two image panels per eye, allowing the user to see a single, continuous, wide field of view (FoV) virtual image.
Hardware manufactured using this invention may be useful in the fields of virtual reality (VR) and augmented reality (AR) for both consumer and professional markets. HMD manufactured by this invention could have applications including everyday use, gaming, entertainment, task support, medical, industrial design, navigation, transport, translation, education, and training.
| Number | Date | Country | Kind |
|---|---|---|---|
| 1621621.0 | Dec 2016 | GB | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2017/044854 | 12/14/2017 | WO | 00 |
