Patent Application
20170115489
Field of the Disclosure
The present disclosure relates generally to display devices and, more particularly, to head mounted display devices.
Description of the Related Art
Immersive virtual reality (VR) and augmented reality (AR) systems typically utilize a head mounted display (HMD) device that presents stereoscopic imagery to the user so as to give a sense of presence in a three-dimensional (3D) scene. Most conventional HMD devices implement either a single flat display that is separated into two independent display regions, one for the left eye and one for the right eye of the user, or a pair of independent flat displays, one for each eye of the user. Such devices also typically include a single lens for each eye so as to focus the entire image of the display into the user's eye. However, the use of flat displays and a single lens for each eye often results in a bulky HMD form factor, which in turn imparts a high moment of inertia when in use. Moreover, the flat displays and lenses constrain the total lateral field of view, often to 110 degrees or less. The bulky size and limited field of view of these conventional HMD devices can deleteriously impact the user's sense of presence in the displayed image and thus inhibit the feeling of being immersed in the presented scene.
Various solutions have been proposed to address these shortcomings. In some approaches, the display system is separated into a set of separate display panels that are tiled together to obtain a larger field of view. However, under this approach, the physical seams between the optics and between the display panels are often noticeable and thus detract from the experience. Moreover, the characteristics of each display panel may differ, and thus render it difficult to achieve uniform color and brightness across the entire field of view. Additionally, the design and fabrication of such systems is complex and thus can be cost-prohibitive.
The present disclosure may be better understood, and its numerous features and advantages made apparent to those skilled in the art by referencing the accompanying drawings. The use of the same reference symbols in different drawings indicates similar or identical items.
The HMD device 100 includes a pair of display panels 104, 105 mounted in a frame 106. The HMD device 100 further includes an optical assembly 108 mounted to the frame 106 (e.g., via the bridge of the frame 106). The optical assembly 108 includes a pair of optical subassemblies 110, 111, one for each eye of a user. The optical assembly 108 further includes a bridge structure 112 that includes a standoff structure that extends from a posterior surface of the frame 106, and thus offsetting the optical subassembly 110 from the face of the display panel 104 and offsetting the optical subassembly 111 from the face of the display panel 105. For example, the standoff structure may implement a vertical structure extending from the bridge of the frame 106 (as shown in
As shown in
To illustrate, in the depicted embodiment the display panel 104 includes two lateral sections: a left central field of view (FOV) section 116 and a left peripheral FOV section 117, and the optical subassembly 110 is implemented with two lenses: a left central lens 118 focused on the left central FOV section 116 and a left peripheral lens 119 focused on the left peripheral FOV section 117. Similarly, in the depicted embodiment the display panel 105 includes two lateral sections: a right central field of view (FOV) section 120 and a right peripheral FOV section 121, and the optical subassembly 111 is implemented with two lenses: a right central lens 122 focused on the right central FOV section 120 and a right peripheral lens 123 focused on the right peripheral FOV section 121. The lenses 118, 119, 122, 123 are illustrated as convex substantially circular lenses. However, the lenses may be implemented in any of a variety of suitable shapes, such as rotationally symmetric or non-rotational symmetric (e.g., toroidal or freeform) lenses, Fresnel lenses, and the like. Further, while embodiments wherein lenses 118, 119, 122, 123 each comprise a single larger lens, in other embodiments one or more of lenses 118, 119, 122, and 123 may be implemented as a plurality of lenses. The lenses may be composed of any of a variety of materials or combinations of materials suitable for fabricating laterally-curved lenses, such as plastic, glass, crystal, and the like.
Through the use of display panel sections with different curvatures and/or orientations relative to the user's eye and optics subassemblies with separate lens elements focused on separate display panel sections accordingly, the HMD device 100 may be fabricated with a form factor that maintains the bulk of the HMD device 100 closer to the user's head, thereby reducing its moment of inertia as well as providing a wider lateral field of view and a more aesthetically pleasing appearance. Moreover, as each display panel section is not a separate display panel but rather is a logical sectioning of a larger display panel, a more uniform brightness and coloration is maintained between the display panel sections.
Although not shown in
As explained above, the display panel 104 is mounted or otherwise disposed to the left of the medial plane 202 in the HMD device 100 such that the face of the display panel 104 forms a left central FOV section 116 and the left peripheral FOV section 117 and the display panel 105 likewise is mounted or otherwise disposed to the right of the medial plane such that the face of the display panel 105 forms the right central FOV section 120 and the right peripheral FOV section 121. Further, the cross-section view of
In the example implementation of
Due to the dimensions and orientations of the FOV sections 316, 317 of the display panel 104, it may not be practical to use axi-symmetric or rotationally-symmetric (that is, “complete”) convex lenses for both lenses 318, 319. Accordingly, in some embodiments, one or both of the lenses 318, 319 may be laterally truncated (that is, rotationally or axially asymmetric) so as to facilitate a more compact lens subassembly configuration. Thus, as illustrated in
Turning to the example implementation of
For this configuration, the optical subassembly 110 includes a convex lens 418 (one example of optical element 118) having an optical axis 402 normal to the facing surface of the central FOV section 416 and a convex lens 419 (one example of optical element 119) having an optical axis 404 normal to the facing surface of the peripheral FOV section 417. As with the implementation of
In the example implementation depicted in
For this configuration, the optical subassembly 110 includes a convex lens 518 (one example of optical element 118) having an optical axis 502 normal to the facing surface of the central FOV section 516 and a convex lens 519 (one example of optical element 119) having an optical axis 504 normal to the facing surface of the peripheral FOV section 517. One or both of the lenses 518, 519 may be laterally truncated so as to facilitate a more compact lens subassembly configuration. To illustrate, the proximal side of the lens 519 is truncated and shaped so as to conform with the curvature of the lens 518 in the region 506 of their contact, and then fused together or held in that arrangement using a mechanical assembly.
In the depicted example, the electronic display system 600 includes an application processor 604, a system memory 606, a sensor hub 608, and an inertial management unit 610. In some embodiments, the HMD device 100 may incorporate image capture for purposes of visual localization or visual telemetry, or for real-time display of imagery captured of the local environment in support of AR functionality. In such embodiments, the electronic display system 600 further may include, for example, one or more image sensors 612, 614 and a structured-light or time-of-flight (ToF) depth sensor 616.
The electronic display system 600 further includes display hardware 622 including a compositor 624, the left-eye display panel 104, the right-eye display panel 105, and a display memory 626. The compositor 624 is a hardware device that may be implemented as, for example, an ASIC, programmable logic, or a combination thereof, and includes a left display controller 628 for driving the left eye display panel 104 and a right display controller 630 for driving the right eye display panel 105.
In operation, the application processor 604 executes the VR/AR application 602 (stored in, for example, the system memory 606) to provide VR/AR functionality for a user. As part of this process, the VR/AR application 602 manipulates the application processor 604 to render a sequence of textures (or pictures) for each eye at a render rate X. Each texture contains visual content that is either entirely computer generated or visual content that is a combination of captured imagery (via the imaging sensors 612, 614) and a computer-generated overlay. The visual content of each texture represents a scene from a corresponding pose of the user's head (or pose of the HMD device 100) at the time that the texture is determined.
Optical lenses, such as those of the optical assembly 108, typically introduce some form of spatial distortion, such as barrel distortion, pincushion distortion, or complex distortion (also referred to as “moustache distortion”). Conventionally, display systems can at least partially correct for these spatial distortions by performing one or more warp transforms on each buffered image so as to compensate for the spatial distortion either present in the buffered image or that will be introduced when the buffered image is viewed through the lenses in an eyepiece.
Accordingly, in some embodiments, the electronic display system 600 may operate to introduce a complementary spatial distortion into the textures as they are displayed (that is, “pre-warp” the textures) so as to correct or compensate for the spatial distortion introduced by the lenses of the optical assembly 108, and thus the imagery presented to the user's eyes is perceived as substantially rectilinear. In some embodiments, this pre-warp process may be performed by the compositor 624 (with each of the left side and right side textures receiving separate pre-warping). In other embodiments, the pre-warp process may be implemented by the rendering algorithm of the VR/AR application 602. Because each of the display panels 104, 105 implements two or more different “sections” and the optical assembly 108 implements a different optical element for each section, in at least one embodiment the electronic display system 600 is configured to implement a different spatial distortion map for each lateral section of a display panel. This process is illustrated in greater detail with reference to
With the generation of the raw image 702, the electronic display system 600 then pre-distorts the raw image 702 to compensate for the complementary distortion that will be introduced by the lenses 118, 119 when the image is displayed and viewed through the lenses 118, 119. However, the lenses 118, 119 typically are not of the same configuration and thus typically do not introduce the same spatial distortion. To illustrate, the lenses 118, 119, may be of a different magnification or prescription, a different focal length, and the like. As such, the degree and type of distortion introduced by each lens may differ. Further, as noted above, one or both of the lenses 118, 119 may be laterally truncated (that is, rotationally or axially asymmetric) so as to permit a more compact assembly for the lenses 118, 119. This truncated configuration for a lens also may be a factor in the particular pre-warping to be applied to the corresponding image content.
As such, in at least one embodiment, the electronic display system 600 employs different spatial distortion maps for each section, with each spatial distortion map being configured for the particular arrangement of lateral display panel section and lens. For the lateral section 704, the electronic display system 600 employs a spatial distortion map 714 that is configured based on the curvature of the central FOV section 116, the magnification and anticipated distortion introduced by the lens 118, and the like. To illustrate, if the lens 118 is expected to introduce a pin-cushion distortion, the spatial distortion map 714 may introduce a compensatory barrel distortion, with the particular parameters of the barrel distortion determined from the parameters of the lens 118, the central FOV section 116, and the like. Likewise, for the lateral section 705, the electronic display system 600 employs a different spatial distortion map 715 that is configured based on the curvature (or lack thereof) of the peripheral FOV section 117, the magnification and anticipated distortion introduced by the lens 119, and the like. To illustrate, because the lens 119 is a laterally-truncated or laterally-asymmetrical lens, the spatial distortion map 715 configured for the lateral section 705 may be similarly truncated or asymmetrical.
The electronic display system 600 applies the spatial distortion maps 714, 715 to the lateral sections 704, 705, respectively, of the raw image 702 to generate a pre-warped image 706. The pre-warped image 706 is then used by the left display controller 628 to drive the left display panel 104 such that the pre-warped image 706 is displayed at the display panel 104 with the image content of a lateral section 708 of the pre-warped image 706 displayed in the region represented by the central FOV section 116 and the image content of a lateral section 709 of the pre-warped image 706 displayed in the region represented by the peripheral FOV section 117. With the pre-warped image 706 so displayed, when viewed by the user through the corresponding optical elements of the HMD 100, the spatial distortion introduced into the displayed image 706 via the spatial distortion maps 714, 715 partially or completely counteracts, or complements, the spatial distortion introduced by the optical elements of the HMD 100, thereby presenting a continuously undistorted, substantially rectilinear image to the user's eye.
Much of the inventive functionality and many of the inventive principles described above are well suited for implementation with or in integrated circuits (ICs) such as application specific ICs (ASICs). It is expected that one of ordinary skill, notwithstanding possibly significant effort and many design choices motivated by, for example, available time, current technology, and economic considerations, when guided by the concepts and principles disclosed herein will be readily capable of generating such ICs with minimal experimentation. Therefore, in the interest of brevity and minimization of any risk of obscuring the principles and concepts according to the present disclosure, further discussion of such software and ICs, if any, will be limited to the essentials with respect to the principles and concepts within the preferred embodiments.
Note that not all of the activities or elements described above in the general description are required, that a portion of a specific activity or device may not be required, and that one or more further activities may be performed, or elements included, in addition to those described. Still further, the order in which activities are listed are not necessarily the order in which they are performed. Also, the concepts have been described with reference to specific embodiments. However, one of ordinary skill in the art appreciates that various modifications and changes can be made without departing from the scope of the present disclosure as set forth in the claims below. Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the present disclosure.
Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and any feature(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential feature of any or all the claims. Moreover, the particular embodiments disclosed above are illustrative only, as the disclosed subject matter may be modified and practiced in different but equivalent manners apparent to those skilled in the art having the benefit of the teachings herein. No limitations are intended to the details of construction or design herein shown, other than as described in the claims below. It is therefore evident that the particular embodiments disclosed above may be altered or modified and all such variations are considered within the scope of the disclosed subject matter. Accordingly, the protection sought herein is as set forth in the claims below.
