Patent Grant
10031338
The present systems, devices, and methods generally relate to scanning laser-based display technologies and particularly relate to expanding the eyebox of a scanning laser-based wearable heads-up display.
Wearable Heads-UP Displays
A head-mounted display is an electronic device that is worn on a user's head and, when so worn, secures at least one electronic display within a viewable field of at least one of the user's eyes, regardless of the position or orientation of the user's head. A wearable heads-up display is a head-mounted display that enables the user to see displayed content but also does not prevent the user from being able to see their external environment. The “display” component of a wearable heads-up display is either transparent or at a periphery of the user's field of view so that it does not completely block the user from being able to see their external environment. Examples of wearable heads-up displays include: the Google Glass®, the Optinvent Ora®, the Epson Moverio®, and the Sony Glasstron®, just to name a few.
The optical performance of a wearable heads-up display is an important factor in its design. When it comes to face-worn devices, however, users also care a lot about aesthetics. This is clearly highlighted by the immensity of the eyeglass (including sunglass) frame industry. Independent of their performance limitations, many of the aforementioned examples of wearable heads-up displays have struggled to find traction in consumer markets because, at least in part, they lack fashion appeal. Most wearable heads-up displays presented to date employ large display components and, as a result, most wearable heads-up displays presented to date are considerably bulkier and less stylish than conventional eyeglass frames.
A challenge in the design of wearable heads-up displays is to minimize the bulk of the face-worn apparatus while still providing displayed content with sufficient visual quality. There is a need in the art for wearable heads-up displays of more aesthetically-appealing design that are capable of providing high-quality images to the user without limiting the user's ability to see their external environment.
Eyebox
In near-eye optical devices such as rifle scopes and wearable heads-up displays, the range of eye positions (relative to the device itself) over which specific content/imagery provided by the device is visible to the user is generally referred to as the “eyebox.” An application in which content/imagery is only visible from a single or small range of eye positions has a “small eyebox” and an application in which content/imagery is visible from a wider range of eye positions has a “large eyebox.” The eyebox may be thought of as a volume in space positioned near the optical device. When the eye of the user (and more particularly, the pupil of the eye of the user) is positioned inside this volume and facing the device, the user is able to see all of the content/imagery provided by the device. When the eye of the user is positioned outside of this volume, the user is not able to see at least some of the content/imagery provided by the device.
The geometry (i.e., size and shape) of the eyebox is an important property that can greatly affect the user experience for a wearable heads-up display. For example, if the wearable heads-up display has a small eyebox that centers on the user's pupil when the user is gazing directly ahead, some or all content displayed by the wearable heads-up display may disappear for the user when the user gazes even slightly off-center, such as slightly to the left, slightly to the right, slightly up, or slightly down. Furthermore, if a wearable heads-up display that has a small eyebox is designed to align that eyebox on the pupil for some users, the eyebox will inevitably be misaligned relative to the pupil of other users because not all users have the same facial structure. Unless a wearable heads-up display is deliberately designed to provide a glanceable display (i.e., a display that is not always visible but rather is only visible when the user gazes in a certain direction), it is generally advantageous for a wearable heads-up display to have a large eyebox.
Demonstrated techniques for providing a wearable heads-up display with a large eyebox generally necessitate adding more bulky optical components to the display. Technologies that enable a wearable heads-up display of minimal bulk (relative to conventional eyeglass frames) to provide a large eyebox are generally lacking in the art.
A wearable heads-up display may be summarized as including: a support structure that in use is worn on a head of a user; a scanning laser projector carried by the support structure; a holographic combiner carried by the support structure, wherein the holographic combiner is positioned within a field of view of an eye of the user when the support structure is worn on the head of the user; and an optical replicator carried by the support structure and positioned in an optical path between the scanning laser projector and the holographic combiner, the optical replicator comprising at least one optical element arranged to receive a light signal generated by the scanning laser projector and redirect respective ones of N instances of the light signal towards the holographic combiner, where N is an integer greater than 1, and wherein the holographic combiner comprises at least one hologram positioned and oriented to redirect each one of the N instances of the light signal towards the eye of the user. The at least one hologram of the holographic combiner may redirect the N instances of the light signal all spatially in parallel with one another towards respective regions of the eye of the user. At least one optical element of the optical replicator may be arranged to redirect respective ones of the N instances of the light signal towards the holographic combiner effectively from respective ones of N spatially-separated virtual positions for the scanning laser projector.
The support structure may have a general shape and appearance of an eyeglasses frame. The wearable heads-up display may further include a prescription eyeglass lens, wherein the holographic combiner is carried by the prescription eyeglass lens.
The at least one hologram of the holographic combiner may converge each one of the N instances of the light signal to a respective one of N exit pupils at or proximate the eye of the user. The holographic combiner may include at least N multiplexed holograms, and each one of the at least N multiplexed holograms may converge a respective one of the N instances of the light signal to a respective one of the N exit pupils at or proximate the eye of the user. The scanning laser projector may include a red laser diode, a green laser diode, and a blue laser diode, and the holographic combiner may include a wavelength-multiplexed holographic combiner that includes at least one red hologram, at least one green hologram, and at least one blue hologram. In this case, the at least one red hologram may converge a respective red component of each one of the N instances of the light signal to a respective one of the N exit pupils at or proximate the eye of the user, the at least one green hologram may converge a respective green component of each one of the N instances of the light signal to a respective one of the N exit pupils at or proximate the eye of the user, and the at least one blue hologram may converge a respective blue component of each one of the N instances of the light signal to a respective one of the N exit pupils at or proximate the eye of the user. The holographic combiner may include a wavelength-multiplexed and angle-multiplexed holographic combiner that includes at least N angle-multiplexed red holograms, at least N angle-multiplexed green holograms, and at least N angle-multiplexed blue holograms, and each one of the at least N angle-multiplexed red holograms may converge a respective red component of each one of the N instances of the light signal to a respective one of the N exit pupils at or proximate the eye of the user, each one of the at least N angle-multiplexed green holograms may converge a respective green component of each one of the N instances of the light signal to a respective one of the N exit pupils at or proximate the eye of the user, and each one of the at least N angle-multiplexed blue holograms may converge a respective blue component of each one of the N instances of the light signal to a respective one of the N exit pupils at or proximate the eye of the user.
The optical path between the scanning laser projector and the holographic combiner may include a total scan range θ of the scanning laser projector, and at least one optical element of the optical replicator may be arranged to receive all light signals corresponding to a sweep of the total scan range θ by the scanning laser projector and redirect respective ones of N instances of all light signals corresponding to the sweep of the total scan range θ of the scanning laser projector towards the holographic combiner.
The light signal may include an image comprising at least two pixels, and each one of the N instances of the light signal may include a respective instance of the image. Alternatively, each one of the N instances of the light signal includes a respective instance of a same pixel in a different instance of a same image.
The wearable heads-up display may further include: an eye tracker carried by the support structure, positioned and oriented to determine a gaze direction of the eye of the user; and at least one controllable shutter carried by the support structure and positioned in at least one optical path between the optical replicator and the holographic combiner, the at least one controllable shutter controllable to selectively block all but at least one of the N instances of the light signal redirected towards the holographic combiner by the optical replicator, the at least one of the N instances of the light signal that is not blocked by the at least one controllable shutter corresponding to the at least one of the N instances of the light signal that, when redirected by the holographic combiner, is redirected by the holographic combiner towards a region of the eye of the user that contains a pupil of the eye of the user based on the gaze direction of the eye of the user determined by the eye tracker.
A method of operating a wearable heads-up display, the wearable heads-up display including a scanning laser projector, an optical replicator, and a holographic combiner positioned within a field of view of an eye of a user when the wearable heads-up display is worn on a head of the user, may be summarized as including: generating a first light signal by the scanning laser projector; redirecting respective ones of N instances of the first light signal towards the holographic combiner by the optical replicator, where N is an integer greater than 1; and redirecting each instance of the first light signal that is received from the optical replicator towards the eye of the user by the holographic combiner. The method may further include: receiving the first light signal from the scanning laser projector by the optical replicator; and replicating the first light signal into the N instances of the first light signal by the optical replicator. Redirecting each instance of the first light signal that is received from the optical replicator towards the eye of the user by the holographic combiner may include redirecting each instance of the first light signal that is received from the optical replicator spatially in parallel with one another towards respective regions of the eye of the user by the holographic combiner. Redirecting respective ones of N instances of the first light signal towards the holographic combiner by the optical replicator may include redirecting respective ones of N instances of the first light signal towards the holographic combiner by the optical replicator effectively from respective ones of N spatially-separated virtual positions for the scanning laser projector.
Redirecting each instance of the first light signal that is received from the optical replicator towards the eye of the user by the holographic combiner may include converging each instance of the first light signal that is received from the optical replicator to a respective exit pupil at or proximate the eye of the user by the holographic combiner. The holographic combiner may include at least two multiplexed holograms, and converging each instance of the first light signal that is received from the optical replicator to a respective exit pupil at or proximate the eye of the user by the holographic combiner may include converging each instance of the first light signal that is received from the optical replicator to a respective exit pupil at or proximate the eye of the user by a respective multiplexed hologram.
The scanning laser projector may include a red laser diode, a green laser diode, and a blue laser diode; the first light signal generated by the scanning laser projector may include a red component, a green component, and a blue component; and the holographic combiner may include a wavelength-multiplexed holographic combiner that includes at least one red hologram, at least one green hologram, and at least one blue hologram. In this case, converging each instance of the first light signal that is received from the optical replicator to a respective exit pupil at or proximate the eye of the user by a respective multiplexed hologram may include: converging a respective red component of each instance of the first light signal that is received from the optical replicator to a respective exit pupil at or proximate the eye of the user by the at least one red hologram; converging a respective green component of each instance of the first light signal that is received from the optical replicator to a respective exit pupil at or proximate the eye of the user by the at least one green hologram; and converging a respective blue component of each instance of the first light signal that is received from the optical replicator to a respective exit pupil at or proximate the eye of the user by the at least one blue hologram. The holographic combiner may include a wavelength-multiplexed and angle-multiplexed holographic combiner that includes at least two angle-multiplexed red holograms, at least two angle-multiplexed green holograms, and at least two angle-multiplexed blue holograms. In this case, converging a respective red component of each instance of the first light signal that is received from the optical replicator to a respective exit pupil at or proximate the eye of the user by the at least one red hologram may include converging a respective red component of each instance of the first light signal that is received from the optical replicator to a respective exit pupil at or proximate the eye of the user by a respective angle-multiplexed red hologram; converging a respective green component of each instance of the first light signal that is received from the optical replicator to a respective exit pupil at or proximate the eye of the user by the at least one green hologram may include converging a respective green component of each instance of the first light signal that is received from the optical replicator to a respective exit pupil at or proximate the eye of the user by a respective angle-multiplexed green hologram; and converging a respective blue component of each instance of the first light signal that is received from the optical replicator to a respective exit pupil at or proximate the eye of the user by the at least one blue hologram may include converging a respective blue component of each instance of the first light signal that is received from the optical replicator to a respective exit pupil at or proximate the eye of the user by a respective angle-multiplexed blue hologram.
The method may further include: generating at least a second light signal by the scanning laser projector; redirecting respective ones of N instances of the at least a second light signal towards the holographic combiner by the optical replicator; and converging each instance of the at least a second light signal that is received from the optical replicator to a respective exit pupil at or proximate the eye of the user by the holographic combiner.
The method may further include: generating light signals corresponding to a sweep of a total scan range θ by the scanning laser projector; receiving the light signals corresponding to the total scan range θ of the scanning laser projector by the optical replicator; redirecting respective ones of N instances of the total scan range θ of the scanning laser projector towards the holographic combiner by the optical replicator; and converging each instance of the total scan range θ of the scanning laser projector that is received from the optical replicator to a respective exit pupil at or proximate the eye of the user by the holographic combiner.
The wearable heads-up display may further include an eye tracker and at least one controllable shutter and the method further include: determining a gaze direction of the eye of the user by the eye tracker; and selectively blocking all but at least one of the N instances of the light signal redirected towards the holographic combiner from the optical replicator by the at least one controllable shutter. In this case, redirecting each instance of the first light signal that is received from the optical replicator towards the eye of the user by the holographic combiner may include, for the at least one of the N instances of the first light signal that is not blocked by the at least one controllable shutter, redirecting, by the holographic combiner, the at least one of the N instances of the first light signal towards a region of the eye of the user that contains a pupil of the eye of the user based on the gaze direction of the eye of the user determined by the eye tracker.
The first light signal may include an image comprising at least two pixels and redirecting respective ones of N instances of the first light signal towards the holographic combiner by the optical replicator may include redirecting N respective instances of a same image towards the holographic combiner by the optical replicator.
Redirecting respective ones of N instances of the first light signal towards the holographic combiner by the optical replicator may include redirecting N respective instances of a same pixel in a different instance of a same image towards the holographic combiner by the optical replicator.
In the drawings, identical reference numbers identify similar elements or acts. The sizes and relative positions of elements in the drawings are not necessarily drawn to scale. For example, the shapes of various elements and angles are not necessarily drawn to scale, and some of these elements are arbitrarily enlarged and positioned to improve drawing legibility. Further, the particular shapes of the elements as drawn are not necessarily intended to convey any information regarding the actual shape of the particular elements, and have been solely selected for ease of recognition in the drawings.
In the following description, certain specific details are set forth in order to provide a thorough understanding of various disclosed embodiments. However, one skilled in the relevant art will recognize that embodiments may be practiced without one or more of these specific details, or with other methods, components, materials, etc. In other instances, well-known structures associated with portable electronic devices and head-worn devices, have not been shown or described in detail to avoid unnecessarily obscuring descriptions of the embodiments.
Unless the context requires otherwise, throughout the specification and claims which follow, the word “comprise” and variations thereof, such as, “comprises” and “comprising” are to be construed in an open, inclusive sense, that is as “including, but not limited to.”
Reference throughout this specification to “one embodiment” or “an embodiment” means that a particular feature, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” include plural referents unless the content clearly dictates otherwise. It should also be noted that the term “or” is generally employed in its broadest sense, that is as meaning “and/or” unless the content clearly dictates otherwise.
The headings and Abstract of the Disclosure provided herein are for convenience only and do not interpret the scope or meaning of the embodiments.
The various embodiments described herein provide systems, devices, and methods for eyebox expansion in scanning laser-based wearable heads-up displays (“WHUDs”). Generally, a scanning laser-based WHUD is a form of virtual retina display in which a scanning laser projector (“SLP”) draws a raster scan onto the eye of the user. In the absence of any further measure the SLP projects light over a fixed area called the exit pupil of the display. In order for the user to see displayed content the exit pupil typically needs to align with, be encompassed by, or overlap with the entrance pupil of the user's eye. The full scan range of the SLP (i.e., the full resolution and/or field of view of the display) is visible to the user when the exit pupil of the display is completely contained within the entrance pupil of the eye. For this reason, a scanning laser-based WHUD typically employs a relatively small exit pupil that is equal to or smaller than the expected size of the entrance pupil of the user's eye (e.g., less than or equal to about 4 mm in diameter).
The eyebox of a scanning laser-based WHUD is defined by the geometry of the exit pupil of the display at or proximate the eye of the user. A scanning laser-based WHUD that employs a small exit pupil in order to achieve maximum display resolution and/or field of view typically has the drawback of having a relatively small eyebox. For example, the exit pupil may be aligned with the center of the user's eye so that the eye's pupil is located “within the eyebox” when the user is gazing directly ahead, but the eye's pupil may quickly leave the eyebox if and when the user glances anywhere off-center. A larger eyebox may be achieved by increasing the size of the exit pupil but this typically comes at the cost of reducing the display resolution and/or field of view. In accordance with the present systems, devices, and methods, the eyebox of a scanning laser-based WHUD may be expanded by optically replicating a relatively small exit pupil and spatially distributing multiple copies or instances of the exit pupil over a relatively larger area of the user's eye, compared to the area of the single exit pupil on its own. In this way, at least one complete instance of the display exit pupil (either as a single instance in its entirety or as a combination of respective portions of multiple instances) may be contained within the perimeter of the eye's pupil for each of a range of eye positions corresponding to a range of gaze directions of the user. In other words, the present systems, devices, and methods describe eyebox expansion by exit pupil replication in scanning laser-based WHUDs.
Throughout this specification and the appended claims, the term “replication” is used (e.g., in the context of “exit pupil replication”) to generally refer to situations where multiple instances of substantially the same thing (e.g., an exit pupil) are produced. The term “exit pupil replication” is intended to generally encompass approaches that produce concurrent (e.g., temporally parallel) instances of an exit pupil as well as approaches that produce sequential (e.g., temporally serial or “repeated”) instances of an exit pupil.
Throughout this specification and the appended claims, the term “carries” and variants such as “carried by” are generally used to refer to a physical coupling between two objects. The physical coupling may be direct physical coupling (i.e., with direct physical contact between the two objects) or indirect physical coupling that may be mediated by one or more additional objects. Thus, the term carries and variants such as “carried by” are meant to generally encompass all manner of direct and indirect physical coupling, including without limitation: carried on, carried within, physically coupled to, and/or supported by, with or without any number of intermediary physical objects therebetween.
SLP 120 may include multiple laser diodes (e.g., a red laser diode, a green laser diode, and/or a blue laser diode) and at least one scan mirror (e.g., a single two-dimensional scan mirror or two one-dimensional scan mirrors, which may be, e.g., MEMS-based or piezo-based). SLP 120 may be communicatively coupled to (and support structure 110 may further carry) a processor and a non-transitory processor-readable storage medium or memory storing processor-executable data and/or instructions that, when executed by the processor, cause the processor to control the operation of SLP 120. For ease of illustration,
Holographic combiner 130 is positioned within a field of view of at least one eye of the user when support structure 110 is worn on the head of the user. Holographic combiner 130 is sufficiently optically transparent to permit light from the user's environment (i.e., “environmental light”) to pass through to the user's eye. In the illustrated example of
Optical replicator 150 is positioned in an optical path between SLP 120 and holographic combiner 130. Optical replicator 150 comprises at least one optical element (e.g., at least one lens, reflector, partial reflector, prism, diffractor, diffraction grating, mirror, or other optical element, or at least one configuration, combination, and/or arrangement of such) that is arranged to receive light signals generated and output by SLP 120, produce multiple (e.g., N, where N is an integer greater than 1) copies or instances of the light signals, and redirect respective ones of the N instances of the light signals towards holographic combiner 130. Advantageously, optical replicator 150 may be a static and passive component that, without power consumption or any moving parts, receives as an input a light signal generated by SLP 120 and provides as outputs N replicated instances of that light signal, the N replicated instances of the light signal temporally substantially in parallel with one another and, as will be described in more detail later on, spatially separated from one another so that each one of the N instances appears to originate (i.e., “effectively” originates) from a respective one of N different spatially-separated “virtual positions” for SLP 120 as opposed to appearing to originate from the real position for SLP 120.
Throughout this specification and the appended claims, reference is often made to one or more “virtual position(s)” such as “N spatially-separated virtual positions for a SLP.” The “real position” of an object is its actual position in real, three dimensional space. A “virtual position” of an object is a position in real space at which the optics of a system cause light from the object to effectively originate even though the real position of the object may be elsewhere. In other words, the optics of the system cause light from the object to follow optical paths that would trace back, if the optics of the system were ignored during the trace back, to a “virtual position” in space that is spatially-separated from the object's “real position” in space. As a simple example, an object in front of a planar mirror has a “virtual position” on the other side of the planar mirror. A “virtual position” may be a result of one or more intervening optical element(s) in an optical path. When one or more optical element(s) redirects light signals from a SLP, a virtual position for the SLP refers to the position in real space at which the SLP would need to be located in order to provide light signals having that same trajectory without any intervening optics. The optics of the system cause the light signals to follow a trajectory that would correspond to a different point of origin if there were no such optics in the system. The light signals appear to have “effectively” originated from a different, or “virtual,” position for the SLP.
SLP 220 is located at a first position 260 (i.e., a “real” position) relative to holographic combiner 230 and is shown generating (e.g., projecting) a first light signal 270. Optical replicator 250 is positioned in an optical path between SLP 220 and holographic combiner 230 such that optical replicator 250 interrupts (e.g., receives) light signal 270 en route from SLP 220 to holographic combiner 230. As previously described, optical replicator 250 includes at least one optical element (e.g., at least one lens, reflector, partial reflector, prism, diffractor, diffraction grating, mirror, or other optical element, or at least one configuration, combination, and/or arrangement of such) that is arranged to receive light signal 270 from SLP 220 and redirect respective ones of N instances of light signal 270 towards holographic combiner 230. In the illustrated example of
SLP 220 is positioned at a first real position 260 in real space relative to holographic combiner 230. Optical replicator 250 (e.g., at least one optical element thereof) is arranged to redirect respective ones of the N=4 instances 271, 272, 273, and 274 of light signal 270 towards holographic combiner 230 effectively from respective ones of N=4 spatially-separated virtual positions 261, 262, 263, and 264 for SLP 220. Specifically, optical replicator redirects (e.g., is arranged to redirect) first instance 271 of light signal 270 towards holographic combiner 230 effectively from first virtual position 261 for SLP 220, second instance 272 of light signal 270 towards holographic combiner 230 effectively from second virtual position 262 for SLP 220, third instance 273 of light signal 270 towards holographic combiner 230 effectively from third virtual position 263 for SLP 220, and fourth instance 274 of light signal 270 towards holographic combiner 230 effectively from fourth virtual position 264 for SLP 220. Each respective one of the N=4 virtual positions 261, 262, 263, and 264 for SLP 220 effectively established by optical replicator 250 is spatially-separated from the other ones of the N=4 virtual positions 261, 262, 263, and 264 for SLP 220 so that each respective instance 271, 272, 273, and 274 of light signal 270 effectively impinges on holographic combiner 230 from a different position in space. Advantageously, each of the N=4 virtual positions 261, 262, 263, and 264 for SLP 220 may correspond to a respective position and orientation of SLP 220. In other words, relative to the other ones of the N=4 virtual positions 261, 262, 263, and 264 for SLP 220, each one of the virtual positions 261, 262, 263, and 264 may correspond to a respective displacement and rotation of SLP 220. Such is the case in WHUD 200 for which, as would be apparent to one of ordinary skill in the art, a line connecting each of the N=4 virtual positions 261, 262, 263, and 264 for SLP 220 in
Each one of the N=4 instances 271, 272, 273, and 274 of light signal 270 is output by optical replicator 250 and received by holographic combiner 230. As previously described, holographic combiner 230 includes at least one hologram that is operative (e.g., designed, crafted, encoded, recorded, and/or generally positioned and oriented) to redirect each one of the N=4 instances 271, 272, 273, and 274 of light signal 270 towards eye 290 of the user. Advantageously, the at least one hologram of holographic combiner 230 may converge each one of the N=4 instances 271, 272, 273, and 274 of light signal 270 to a respective one of N=4 exit pupils 281, 282, 283, and 284 at or proximate eye 290 of the user. Specifically: optical replicator 250 directs first instance 271 of light signal 270 towards holographic combiner 230 effectively from first virtual position 261 for SLP 220 and holographic combiner 230 converges first instance 271 of light signal 270 to first exit pupil 281 at or proximate eye 290; optical replicator 250 directs second instance 272 of light signal 270 towards holographic combiner 230 effectively from second virtual position 262 for SLP 220 and holographic combiner 230 converges second instance 272 of light signal 270 to second exit pupil 282 at or proximate eye 290; optical replicator 250 directs third instance 273 of light signal 270 towards holographic combiner 230 effectively from third virtual position 263 for SLP 220 and holographic combiner 230 converges third instance 273 of light signal 270 to third exit pupil 283 at or proximate eye 290; and optical replicator 250 directs fourth instance 274 of light signal 270 towards holographic combiner 230 effectively from fourth virtual position 264 for SLP 220 and holographic combiner 230 converges fourth instance 274 of light signal 270 to fourth exit pupil 284 at or proximate eye 290. The eyebox 280 of WHUD 200 is given by the total range of pupil positions (or gaze directions) for eye 290 in which at least one of exit pupils 281282, 283, and/or 284 aligns with and/or impinges on the pupil of eye 290. Without optical replicator 250, a single instance of light signal 270 (e.g., second instance 272) would impinge on eye 290 at a single exit pupil (e.g., second exit pupil 282) to provide a relatively small eyebox. In such a configuration, displayed content would disappear from the user's point of view if and when the user gazed in a direction that caused the pupil of eye 290 to move away from the single exit pupil (e.g., second exit pupil 282). In accordance with the present systems, devices, and methods, optical replicator 250 replicates light signal 270 to produce N=4 (where 4 is an illustrative example) instances 271, 272, 273, and 274 of light signal 270 and because each of these four instances 271, 272, 273, and 274 effectively originates from a different spatially-separated virtual position 261, 262, 263, and 264, respectively, for SLP 220, holographic combiner 230 converges each of these four instances 271, 272, 273, and 274 to a respective spatially-separated exit pupil 281, 282, 283, and 284 at or proximate eye 290. Spatially-separated exit pupils 281, 282, 283, and 284 are distributed over an area of eye 290 that covers a wider range of pupil positions (e.g., gaze directions) than a single exit pupil (of the same size as any one of exit pupils 281, 282, 283 and 284) on its own. Thus, eyebox 280 is expanded by exit pupil replication in WHUD 200.
Generally, first light signal 270 shown in
In the operation of WHUD 200 depicted in
Optical replicator 250 of WHUD 200 is positioned in the optical path (e.g., in all the optical paths) between SLP 220 and holographic combiner 230 for the total scan range θ of SLP 220. At least one optical element of optical replicator 250 is arranged to receive all of the light signals (e.g., a single optical element may be arranged to receive all of the light signals or multiple optical elements may arranged to all receive all of the light signals, or multiple optical elements may be arranged so that each light signal is received by at least one of the multiple optical elements) generated by SLP 220 during a sweep of the total scan range θ by SLP 220 and redirect respective ones of N (e.g., N=4 in the illustrated example) instances of all of the light signals towards holographic combiner 230. In other words, in a similar way to how a first light signal 270 is replicated by optical replicator 250 as four instances 271, 272, 273, and 274 of the first light signal 270 in the exemplary operation of WHUD 200 illustrated in
In
In a virtual retina display such as scanning laser-based WHUD 100 and/or scanning laser-based WHUD 200, there may not be an “image” formed outside of the eye of the user. There is typically no microdisplay or projection screen or other place where the projected image is visible to a third party; rather, the image may be formed completely within the eye of the user. For this reason, it may be advantageous for a scanning laser-based WHUD to be designed to accommodate the manner in which the eye forms an image.
For a light signal entering the eye (e.g., a light ray, a wavefront, an incident beam from a SLP, or similar), the eye (or more accurately, the combination of the eye and the human brain) may determine “where” the light signal is positioned in the user's field of view based on the region of the retina that is illuminated by the light signal. Two light signals that illuminate the same region of the retina may appear in the same position in the user's field of view. The particular region of the retina that is illuminated by any given light signal is determined by the angle and not the location at which the light signal enters the eye. Thus, two light signals may appear in the same position in the user's field of view even if they enter different location of the user's pupil provided that the two light signals have the same angle of incidence when they enter the user's eye. The geometry of the eye's lens is such that any two light signals entering the eye at the same angle, regardless of the position/location at which the light signals enter the eye, may generally be directed to the same region of the retina and so may generally appear in the same position in the user's field of view.
In at least some implementations, the scanning laser-based WHUDs described herein project multiple instances of the same image onto the retina of the eye substantially concurrently. Even if the multiple instances are temporally-separated, the temporal separation may be small enough to be undetectable by the user. If any two of the multiple instances of the same image do not align/overlap on the eye's retina then those two instances of the image may not align/overlap in the user's field of view and undesirable effects such as ghosting can occur. In order to ensure that multiple instances of the same image (each corresponding to a respective exit pupil) align/overlap on the retina so that multiple instances of the image align/overlap in the user's field of view, a scanning laser-based WHUD may advantageously be configured to direct multiple instances of any given light signal (each corresponding to a respective exit pupil and each representing a respective instance of the same display content) towards the eye spatially in parallel with one another. More specifically and referring to
In the implementation of
In the implementation of
As previously described, holographic combiner 230 comprises at least one hologram embedded, encoded, recorded, or otherwise carried by at least one layer of holographic film. The holographic film may include, as examples, a photopolymer film such as Bayfol®HX from Bayer MaterialScience AG or a silver halide compound. The nature of the at least one hologram may depend on the specific implementation.
As a first example, holographic combiner 230 may include a single hologram that effectively operates as a fast-converging (e.g., convergence within about 1 cm, convergence with about 2 cm, or convergence within about 3 cm) mirror for light having the wavelength(s) provided by SLP 220. In this first example, the holographic film that carries the first hologram may have a relatively wide bandwidth, meaning the hologram recorded in the holographic film may impart substantially the same optical effect or function on all light signals projected by SLP 220 over a relatively wide range of angles of incidence at holographic combiner 230. For the purpose of the present systems, devices, and methods, the term “wide bandwidth” in relation to holograms and holographic films means an angular bandwidth that is greater than or equal to the total range of angles of incidence of all light signals received by any given point, region, or location of the hologram or holographic film from an optical replicator. As an example, WHUD 200 may implement a wide bandwidth hologram in holographic combiner 230 having an angular bandwidth of greater than or equal to about 8°. In this case, the spatial separation between virtual positions 261, 262, 263, and 264 may be such that any given point, region, or location of holographic combiner 230 receives light signals (i.e., spanning all instances 271, 272, 273, and 274) spanning an 8° (or less) range of angles of incidence at holographic combiner 230.
Consistent with conventional mirror behavior, for a single wide-bandwidth fast-converging hologram carried by holographic combiner 230 the angles of incidence for a range of light signals incident on holographic combiner 230 may influence the angles of reflection for that range of light signals redirected by holographic combiner 230. Since holographic combiner 230 is, generally during normal operation of WHUD 200, fixed in place relative to SLP 220, the angles of incidence for a range of light signals are determined, at least in part, by the particular virtual position 261, 262, 263, or 264 for the SLP 220 from which optical replicator 250 causes the range of light signals to effectively originate. The spatial position of the exit pupil 281, 282, 283, or 284 to which the range of light signals is converged by holographic combiner 230 is then determined, at least in part, by the angles of reflection of that range of light signals from holographic combiner 230. Each one of virtual positions 261, 262, 263, and 264 provides light signals over a respective range of angles of incidence (generally but not necessarily with at least some overlap) at holographic combiner 230 and therefore holographic combiner 230 converges light signals from each one of virtual positions 261, 262, 263, and 264 to a respective one of exit pupils 281, 282, 283, and 284. This is why, referring to
As a second example, rather than a single hologram, holographic combiner 230 may instead include any number of multiplexed holograms. Multiplexed holograms may be advantageous when, for example, multiple wavelengths of light signals are used (e.g., red, green, and blue light signals generated by SLP 220) and/or to provide a further means to separate light signals effectively originating from different virtual positions for SLP 220. The “single hologram” example described above may be suitable for an implementation in which SLP 220 only provides light signals of a single wavelength (e.g., only red light signals, only green light signals, or only blue light signals), but for implementations in which SLP 220 provides light signals of multiple wavelengths it may be advantageous for holographic combiner 230 to include a respective wavelength multiplexed hologram for each respective wavelength of light signals provided by SLP 220 (e.g., each respective nominal wavelength of light signals provided by SLP 220, since a laser diode may generally provide light signals over a narrow waveband). Thus, when SLP 220 includes three different laser diodes each providing light signals of a respective nominal wavelength (e.g., a red laser diode, a green laser diode, and a blue laser diode) it may be advantageous for holographic combiner 230 to include three wavelength-multiplexed holograms (e.g., a red hologram, a green hologram, and a blue hologram) each designed to work (e.g., “playback”) for light signals having a respective one of the three nominal wavelengths. In this example, at least one “red hologram” (i.e., at least one hologram that is designed to playback for light signals having a wavelength that corresponds to red light) may converge a respective red component of each one of the N=4 instances of the total scan range θ of SLP 220 to a respective one of the N=4 exit pupils 281, 282, 283, and 284, at least one “green hologram” (i.e., at least one hologram that is designed to playback for light signals having a wavelength that corresponds to green light) may converge a respective green component of each one of the N=4 instances of the total scan range θ of SLP 220 to a respective one of the N=4 exit pupils 281, 282, 283, and 284, and at least one blue hologram (i.e., at least one hologram that is designed to playback for light signals having a wavelength that corresponds to blue light) may converge a respective blue component of each one of the N=4 instances of the total scan range θ of SLP 220 to a respective one of the N=4 exit pupils 281, 282, 283, and 284.
As a third example, either apart from or in addition to multiple wavelength-multiplexed holograms, holographic combiner 230 may include at least N angle-multiplexed holograms. That is, for an implementation with N=4 virtual positions 261, 262, 263, and 264 for the SLP 220 and N=4 exit pupils 281, 282, 283, and 284, holographic combiner 230 may include at least N=4 angle-multiplexed holograms (or N=4 sets of angle-multiplexed holograms when wavelength multiplexing is also employed, as discussed later on). Each of the N=4 angle-multiplexed holograms may be designed to playback for light signals effectively originating from a respective one of the N=4 virtual positions 261, 262, 263, and 264 for SLP 220 and converge such light signals to a respective one of the N=4 exit pupils 281, 282, 283, and 284. That is, a first angle-multiplexed hologram may be designed to playback for light signals effectively originating from first virtual position 261 for SLP 220 and converge such light signals to first exit pupil 281, a second angle-multiplexed hologram may be designed to playback for light signals effectively originating from second virtual position 262 for SLP 220 and converge such light signals to second exit pupil 282, a third angle-multiplexed hologram may be designed to playback for light signals effectively originating from third virtual position 263 for SLP 220 and converge such light signals to third exit pupil 283, and a fourth angle-multiplexed hologram may be designed to playback for light signals effectively originating from fourth virtual position 264 for SLP 220 and converge such light signals to fourth exit pupil 284.
For implementations that employ angle-multiplexing, it may be advantageous for the holographic film that includes an angle-multiplexed hologram to be of relatively narrow bandwidth. Particularly, it may be advantageous for the holographic film to have an angular bandwidth that is less than or about equal to the minimum difference between the respective angles of incidence of two light signals that are incident on the same point, region, or location of holographic combiner 230 but effectively originate from different virtual positions 261, 262, 263. 264. As an example, WHUD 200 may implement a narrow bandwidth angle-multiplexed hologram in holographic combiner 230 having an angular bandwidth of less than or equal to about 4°. In this case, the difference between the angle of incidence (at holographic combiner 230) of a first instance 271 of a light signal that effectively originates from virtual position 261 and is incident at a first point on holographic combiner 230 and the angle of incidence (at holographic combiner 230) of a second instance 272 of a (different) light signal 272 that effectively originates from virtual position 262 and is incident at the same first point on holographic combiner 230 may be less than or equal to about 4°. In this way, each respective angle-multiplexed hologram in holographic combiner 230 may be designed to substantially exclusively playback for light signals effectively originating from a respective one of virtual positions 261, 262, 263, and 264 for SLP 220 and to substantially not playback (e.g., insubstantially playback) for light signals effectively originating from the other ones of virtual positions 261, 262, 263, and 264 for SLP 220.
Generally, holographic combiner 230 may include at least N multiplexed holograms and each one of the at least N multiplexed holograms may converge a respective one of the N instances of a light signal from optical replicator 250 to a respective one of N exit pupils at or proximate the eye 290 of the user.
Some implementations may employ both wavelength multiplexing and angle multiplexing. For example, an implementation that employs angle multiplexing and light signals of multiple wavelengths (e.g., a multi-color SLP) may advantageously also employ wavelength multiplexing. In this case, holographic combiner 230 may include a wavelength-multiplexed and angle-multiplexed holographic combiner that includes at least N angle-multiplexed red holograms, at least N angle-multiplexed green holograms, and at least N angle-multiplexed blue holograms. Each one of the at least N angle-multiplexed red holograms may converge a respective red component of each one of N instances of any given light signal to a respective one of N exit pupils at or proximate the eye of the user, each one of the at least N angle-multiplexed green holograms may converge a respective green component of each one of N instances of any given light signal to a respective one of N exit pupils at or proximate the eye of the user, and each one of the at least N angle-multiplexed blue holograms may converge a respective blue component of each one of N instances of any given light signal to a respective one of N exit pupils at or proximate the eye of the user.
Implementations of holographic combiner 230 that employ multiple multiplexed holograms may include multiple holograms in or on a single layer (i.e., all in or on the same layer) of holographic film or may include multiple layers of holographic film with each layer of holographic film carrying at least one respective hologram. Holographic combiner 230 may or may not comprise at least one volumetric holographic optical element. Generally, holographic combiner 230 may comprise a single layer of holographic film that carries any number of holograms or holographic combiner 230 may comprise multiple layers of holographic film (e.g., multiple layers laminated together) with each respective layer of holographic film carrying any number of respective holograms.
Holographic combiner 230 may be substantially flat or planar in geometry or, as illustrated in
The various embodiments described herein provide systems, devices, and methods for eyebox expansion by exit pupil replication in scanning laser-based WHUDs. Each replicated exit pupil is aligned to a respective spatially-separated position at or proximate the eye of the user because the optical replicator that replicates the light signals spatially separates the replicated light signals so that each replicated light signal appears to effectively originate from a different spatially-separated virtual position for the SLP. The effect is substantially the same as if multiple SLPs were used instead of the optical replicator, with each SLP positioned in a respective one of the virtual positions and with each SLP projecting a respective instance of a light signal towards the holographic combiner; however, the use of the optical replicator has considerable advantages in terms of power savings and minimizing hardware bulk.
While the use of an optical replicator in lieu of multiple spatially-separated SLPs has many advantages, one potential drawback may arise from the fact that replicated instances of a light signal necessarily all embody substantially the same light signal. This can be problematic when, for example, each replicated instance of an image is made to effectively originate from a different spatially-separated virtual position for the SLP. In that case, each replicated instance of the image may be subject to a unique combination of optical distortions. For example, a first replicated instance of an image effectively originating from a first virtual position may be subject to a first set of optical distortions (e.g., image skewing, keystoning, aberrations, and so on) resulting from the unique path of the first instance of the image through the optical replicator and/or from the range of angles of incidence (at holographic combiner 230 and/or at eye 290) that correspond to the first virtual position for the SLP, while a second replicated instance of the image effectively originating from a second virtual position may be subject to a second set of optical distortions resulting from the unique path of the second instance of the image through the optical replicator and/or from the range of angles of incidence (at holographic combiner 230 and/or at eye 290) that correspond to the second virtual position for the SLP. If the first and second replicated instances of the image both correspond to the same initial version of the image defined by the SLP then there may be no opportunity to optically tune, adjust, correct, or otherwise compensate for distortions that are specific to the individual first and second instances of the image. Even though the replicated instances of the image may be optically the same at definition, the resulting replicated instances of the image seen by the user may not be the same because each instance of the image may be subject to individual image distortions. In accordance with the present systems, devices, and methods, this problem may be overcome (if necessary) by including a controllable shutter mechanism to controllably block all but one instance of a projected light signal at any given time, the one instance of the light signal that is not blocked by the shutter corresponding to an instance of the light signal that converges to a particular exit pupil that aligns with the user's gaze direction at that time.
In the illustrated operation of WHUD 300 depicted in
When a controllable shutter 352 is used to selectively block/transmit individual ones of multiple N instances of a light signal (e.g., based on the eye's pupil position as determined by an eye tracker), the SLP may be calibrated to define each light signal in such a way that the light signal accommodates, compensates for, and/or generally accounts for the particular optical distortion(s) that apply to the particular instance of the light signal that current configuration of the controllable shutter 352 will transmit. At least one of eye tracker 351 and/or controllable shutter 352 may provide feedback about the current “active” exit pupil of WHUD 300 and SLP 320 may selectively operate in a mode that applies compensation and/or accommodation measures to light signals in order to account for the optical distortion(s) that are particular to the optical path(s) that correspond to the current “active” exit pupil.
Controllable shutter 352 may comprise any of a variety of different structures depending on the specific implementation. For example, controllable shutter 352 may comprise one or more MEMS-based or piezo-based elements for physically translating and/or rotating one or more opaque surface(s) in the optical path(s) between optical replicator 350 and holographic combiner 330, one or more controllable (e.g., translatable and/or rotatable) reflector(s) or refractor(s), one or more controllable polarization filter(s) together with one or more controllable polarizer(s) in SLP 320 or between SLP 320 and controllable shutter 352, and so on.
Eye tracker 351 may employ any of a variety of different eye tracking technologies depending on the specific implementation. For example, eye tracker 351 may employ any or all of the systems, devices, and methods described in U.S. Provisional Patent Application Ser. No. 62/167,767; U.S. Provisional Patent Application Ser. No. 62/271,135; U.S. Provisional Patent Application Ser. No. 62/245,792; and/or U.S. Provisional Patent Application Ser. No. 62/281,041.
As previously described, WHUD 300 may include at least one processor and at least one non-transitory processor-readable storage medium or memory communicatively coupled thereto. The at least one memory may store processor-executable data and/or instructions that, when executed by the at least one processor, cause the at least one processor to control the operation of any or all of eye tracker 351, controllable shutter 352, and/or SLP 320.
The illustrative examples of the present systems, devices, and methods depicted in
Exemplary optical replicators 250 and 350 drawn in
The partial reflectors 551, 552, and 553 in optical replicator 550 may be substantially parallel with one another. In this case, the numerous instances 571a, 572a, and 573a of light signals 570 emitted from optical replicator 550 may be essentially parallel with one another and the eyebox of WHUD 500 may be effectively increased in size by about the total width spanned by the set of parallel instances 571a, 572a, and 573a of light signal 570. To further expand the eyebox, the partial reflectors 551, 552, and 553 in optical replicator 550 may be oriented at slight angles with respect to one another so that the instances 571a, 572a, and 573a of light signals 570 are divergent with respect to one another when emitted by optical replicator 550. This way, the instances 571a, 572a, and 573a of light signals 570 may span a greater area and reach wider-spread regions on eye 590.
Optical replicator 550 makes use of three partial reflectors 551, 552, and 553 in order to produce three instances 571a, 572a, and 573a of a light signal 570a directed towards holographic combiner 530 at respectively different angles and/or from respectively different virtual positions for scan mirror 522 and/or laser module 521. However, partial reflectors 551, 552, and 553 are used in optical replicator 550 in the implementation of
In addition to various WHUD systems and devices that provide eyebox expansion by exit pupil replication, the various embodiments described herein also include methods of expanding the eyebox of a WHUD by exit pupil replication.
At 601, an SLP of the WHUD generates a first light signal. The first light signal may represent a complete first image or a portion of a first image. For example, the first light signal may represent one or more pixel(s) of an image.
At 602, the optical replicator redirects respective ones of N instances of the first light signal towards the holographic combiner, where N is an integer greater than 1. Generally, in between acts 601 and 602 the optical replicator may receive the first light signal from the SLP and replicate (e.g., optically split, furcate, branch, divide, multiply, or otherwise replicate) the first light signal into the N instances of the first light signal. When the first light signal represents an image comprising at least two pixels, the optical replicator may redirect N respective instances of the image towards the holographic combiner at 602. When the first light signal represents one or more pixel(s) of an image, the optical replicator may redirect N instances of the one or more pixel(s) of the image towards the holographic combiner at 602. As described previously, the optical replicator may redirect respective ones of N instances of the first light signal towards the holographic combiner effectively from respective ones of N spatially-separated virtual positions for the SLP.
At 603, the holographic combiner redirects each instance of the first light signal received from the optical replicator towards the eye of the user. As described in more detail later on, one or more instances of the first light signal may be selectively blocked by a controllable shutter and therefore may not be received from the optical replicator by the holographic combiner. Depending on the specific implementation, the holographic combiner may redirect each instance of the first light signal received from the optical replicator spatially in parallel with one another towards the eye of the user. The holographic combiner may converge each instance of the first light signal received from the optical replicator towards a respective exit pupil at or proximate the eye of the user.
In some implementations, the holographic combiner may include a hologram that redirects instances of the first light signal towards respective exit pupils at the eye of the user based on the angle of incidence (at the holographic combiner) of each instance of the first light signal resulting from the particular virtual position for the SLP to which the instance of the first light signal corresponds. Even in such implementations, the holographic combiner may comprise at least two wavelength multiplexed holograms to respectively playback for (e.g., perform the redirecting and/or converging of act 603) at least two components of the first light signal having different wavelengths, such as at least two color components of the first light signal. For example, the SLP may comprise a red laser diode, a green laser diode, and a blue laser diode and the first light signal may comprise a red component, a green component, and a blue component. In this case, the holographic combiner may comprise a red hologram, a green hologram, and a blue hologram and: the red hologram may converge a respective red component of each instance of the first light signal that is received from the optical replicator to a respective exit pupil at or proximate the eye of the user, the green hologram may converge a respective green component of each instance of the first light signal that is received from the optical replicator to a respective exit pupil at or proximate the eye of the user, and the blue hologram may converge a respective blue component of each instance of the first light signal that is received from the optical replicator to a respective exit pupil at or proximate the eye of the user.
In some implementations, the holographic combiner may include at least two multiplexed holograms and each hologram may converge a respective instance of the first light signal that is received from the optical replicator to a respective exit pupil at or proximate the eye of the user. Continuing on the example above, the holographic combiner may include at least two angle-multiplexed red holograms, at least two angle-multiplexed green holograms, and at least two angle-multiplexed blue holograms. In this case, a respective angle-multiplexed red hologram may converge a respective red component of each instance of the first light signal that is received from the optical replicator to a respective exit pupil at or proximate the eye of the user, a respective angle-multiplexed green hologram may converge a respective green component of each instance of the first light signal that is received from the optical replicator to a respective exit pupil at or proximate the eye of the user, and a respective angle-multiplexed blue hologram may converge a respective blue component of each instance of the first light signal that is received from the optical replicator to a respective exit pupil at or proximate the eye of the user.
Method 600 may be extended in various ways. For example, the SLP may generate at least a second light signal (e.g., corresponding to at least a second image, or at least a second pixel of the first image, or at least a second set of pixels of the first image), the optical replicator may redirect respective ones of N instances of the at least a second light signal towards the holographic combiner, and the holographic combiner may converge each instance of the at least a second light signal that is received from the optical replicator to a respective exit pupil at or proximate the eye of the user. Similarly, the SLP may generate light signals corresponding to a sweep of the total scan range θ of the SLP, the optical replicator may receive the total scan range θ of the SLP and redirect respective ones of N instances of the total scan range θ of the SLP towards the holographic combiner, and the holographic combiner may converge each instance of the total scan range θ of the SLP that is received from the optical replicator to a respective exit pupil at or proximate the eye of the user.
Furthermore, as previously described, the WHUD may include an eye tracker and at least one controllable shutter communicatively coupled to (either directly or through one or more other devices such as a processor and/or memory) the eye tracker. In this case, the eye tracker may determine the gaze direction (e.g., pupil position) of the eye of the user and the at least one controllable shutter may selectively block all but at least one of the N instances of the light signal redirected towards the holographic combiner from the optical replicator. For the at least one of the N instances of the first light signal that is not blocked by the at least one controllable shutter (i.e., the “unblocked instance”), the holographic combiner may redirect the unblocked instance of the first light signal towards a region of the eye of the user that contains the pupil of the eye of the user based on the gaze direction of the eye of the user determined by the eye tracker.
In accordance with the present systems, devices, and methods, the eyebox of a retina-scanning projector may be expanded by replication of one or more exit pupils. In this approach, a given exit pupil may have a defined size that is about equal to or smaller than the diameter of the eye's pupil, such as about 4 mm or less (e.g., about 2 mm), so that all light from an image enters the eye when the exit pupil impinges on (e.g., e.g., aligns with or overlies) the user's (physical) pupil. However, when the user moves their eye, alignment between the exit pupil and the user's pupil may be lost and the projected image may disappear from the user's field of view. Thus, in the “eyebox expansion through exit pupil replication” approaches described herein, multiple exit pupils may be projected and tiled over the user's eye so that at least one exit pupil aligns with the user's eye for multiple, many, most, or all eye positions.
Throughout this specification and the appended claims, the term “about” is sometimes used in relation to specific values or quantities. For example, fast-convergence within “about 2 cm.” Unless the specific context requires otherwise, the term about generally means±15%.
The “optical replicator” described herein is an optical device. A non-limiting example of an optical replicator comprising an arrangement of partial reflectors is illustrated in (and described with reference to)
A person of skill in the art will appreciate that the present systems, devices, and methods may be applied or otherwise incorporated into WHUD architectures that employ one or more light source(s) other than a SLP. For example, in some implementations the SLP described herein may be replaced by another light source, such as a light source comprising one or more light-emitting diodes (“LEDs”), one or more organic LEDs (“OLEDs”), one or more digital light processors (“DLPs”). Such non-laser implementations may advantageously employ additional optics to collimate, focus, and/or otherwise direct projected light signals. Unless the specific context requires otherwise, a person of skill in the art will appreciate that references to a “SLP” throughout the present systems, devices, and methods are representative and that other light sources (combined with other optics, as necessary) may be applied or adapted for application to serve the same general purpose as the SLPs described herein.
A person of skill in the art will appreciate that the present systems, devices, and methods may be applied or otherwise incorporated into WHUD architectures that employ one or more transparent combiner(s) other than a holographic combiner. For example, in some implementations the holographic combiner described herein may be replaced by a non-holographic device that serves substantially the same general purpose, such as prismatic film, a film that carries a microlens array, and/or a waveguide structure. Such non-holographic implementations may or may not employ additional optics. Unless the specific context requires otherwise, a person of skill in the art will appreciate that references to a “holographic combiner” throughout the present systems, devices, and methods are representative and that other transparent combiners (combined with other optics, as necessary) may be applied or adapted for application to serve the same general purpose as the holographic combiners described herein.
A person of skill in the art will appreciate that the various embodiments for eyebox expansion by exit pupil replication described herein may be applied in non-WHUD applications. For example, the present systems, devices, and methods may be applied in non-wearable heads-up displays and/or in other projection displays, including virtual reality displays, in which the holographic combiner need not necessarily be transparent.
In binocular implementations (i.e., implementations in which display content is projected into both eyes of the user), the total field of view may be increased by deliberately projecting a different field of view to each eye of the user. The two fields of view may overlap, so that both eyes see content at the center of the field of view while the left eye sees more content at the left of the field of view and the right eye sees more content at the right of the field of view.
In some implementations that employ multiple exit pupils, all exit pupils may optionally be active at all times. Alternatively, implementations that also employ eye-tracking, may activate only the exit pupil that corresponds to where the user is looking (based on eye-tracking) while one or more exit pupil(s) that is/are outside of the user's field of view may be deactivated.
In some implementations, the scan range of the projector can be actively changed to increase resolution in the direction the eye is looking or in the occupied exit pupil. Such is an example of heterogeneous image resolution as described in U.S. Provisional Patent Application Ser. No. 62/134,347.
Eyebox expansion may advantageously enable a user to see displayed content while gazing in a wide range of directions. Furthermore, eyebox expansion may also enable a wider variety of users having a wider range of eye arrangements to adequately see displayed content via a given WHUD. Anatomical details such as interpupillary distance, eye shape, relative eye positions, and so on can all vary from user to user. The various eyebox expansion methods described herein may be used to render a WHUD more robust over (and therefore more usable by) a wide variety of users having anatomical differences. In order to even further accommodate physical variations from user to user, the various WHUDs described herein may include one or more mechanical structure(s) that enable the user to controllably adjust the physical position and/or alignment of one or more exit pupil(s) relative to their own eye(s). Such mechanical structures may include one or more hinge(s), dial(s), flexure(s), tongue and groove or other slidably-coupled components, and the like. Alternatively, the approaches taught herein may advantageously avoid the need for inclusion of such additional mechanical structures, allowing a smaller package and less weight than might otherwise be obtainable.
In some implementations, one or more optical fiber(s) may be used to guide light signals along some of the paths illustrated herein.
The various implementations described herein may, optionally, employ the systems, devices, and methods for preventing eyebox degradation described in U.S. Provisional Patent Application Ser. No. 62/288,947.
The WHUDs described herein may include one or more sensor(s) (e.g., microphone, camera, thermometer, compass, altimeter, and/or others) for collecting data from the user's environment. For example, one or more camera(s) may be used to provide feedback to the processor of the WHUD and influence where on the display(s) any given image should be displayed.
The WHUDs described herein may include one or more on-board power sources (e.g., one or more battery(ies)), a wireless transceiver for sending/receiving wireless communications, and/or a tethered connector port for coupling to a computer and/or charging the one or more on-board power source(s).
The WHUDs described herein may receive and respond to commands from the user in one or more of a variety of ways, including without limitation: voice commands through a microphone; touch commands through buttons, switches, or a touch sensitive surface; and/or gesture-based commands through gesture detection systems as described in, for example, U.S. Non-Provisional patent application Ser. No. 14/155,087, U.S. Non-Provisional patent application Ser. No. 14/155,107, PCT Patent Application PCT/US2014/057029, and/or U.S. Provisional Patent Application Ser. No. 62/236,060, all of which are incorporated by reference herein in their entirety.
The various implementations of WHUDs described herein may include any or all of the technologies described in U.S. Provisional Patent Application Ser. No. 62/117,316, U.S. Provisional Patent Application Ser. No. 62/156,736, and/or U.S. Provisional Patent Application Ser. No. 62/242,844.
Throughout this specification and the appended claims the term “communicative” as in “communicative pathway,” “communicative coupling,” and in variants such as “communicatively coupled,” is generally used to refer to any engineered arrangement for transferring and/or exchanging information. Exemplary communicative pathways include, but are not limited to, electrically conductive pathways (e.g., electrically conductive wires, electrically conductive traces), magnetic pathways (e.g., magnetic media), and/or optical pathways (e.g., optical fiber), and exemplary communicative couplings include, but are not limited to, electrical couplings, magnetic couplings, and/or optical couplings.
Throughout this specification and the appended claims, infinitive verb forms are often used. Examples include, without limitation: “to detect,” “to provide,” “to transmit,” “to communicate,” “to process,” “to route,” and the like. Unless the specific context requires otherwise, such infinitive verb forms are used in an open, inclusive sense, that is as “to, at least, detect,” to, at least, provide,” “to, at least, transmit,” and so on.
The above description of illustrated embodiments, including what is described in the Abstract, is not intended to be exhaustive or to limit the embodiments to the precise forms disclosed. Although specific embodiments of and examples are described herein for illustrative purposes, various equivalent modifications can be made without departing from the spirit and scope of the disclosure, as will be recognized by those skilled in the relevant art. The teachings provided herein of the various embodiments can be applied to other portable and/or wearable electronic devices, not necessarily the exemplary wearable electronic devices generally described above.
For instance, the foregoing detailed description has set forth various embodiments of the devices and/or processes via the use of block diagrams, schematics, and examples. Insofar as such block diagrams, schematics, and examples contain one or more functions and/or operations, it will be understood by those skilled in the art that each function and/or operation within such block diagrams, flowcharts, or examples can be implemented, individually and/or collectively, by a wide range of hardware, software, firmware, or virtually any combination thereof. In one embodiment, the present subject matter may be implemented via Application Specific Integrated Circuits (ASICs). However, those skilled in the art will recognize that the embodiments disclosed herein, in whole or in part, can be equivalently implemented in standard integrated circuits, as one or more computer programs executed by one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs executed by on one or more controllers (e.g., microcontrollers) as one or more programs executed by one or more processors (e.g., microprocessors, central processing units, graphical processing units), as firmware, or as virtually any combination thereof, and that designing the circuitry and/or writing the code for the software and or firmware would be well within the skill of one of ordinary skill in the art in light of the teachings of this disclosure.
When logic is implemented as software and stored in memory, logic or information can be stored on any processor-readable medium for use by or in connection with any processor-related system or method. In the context of this disclosure, a memory is a processor-readable medium that is an electronic, magnetic, optical, or other physical device or means that contains or stores a computer and/or processor program. Logic and/or the information can be embodied in any processor-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions associated with logic and/or information.
In the context of this specification, a “non-transitory processor-readable medium” can be any element that can store the program associated with logic and/or information for use by or in connection with the instruction execution system, apparatus, and/or device. The processor-readable medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device. More specific examples (a non-exhaustive list) of the computer readable medium would include the following: a portable computer diskette (magnetic, compact flash card, secure digital, or the like), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM, EEPROM, or Flash memory), a portable compact disc read-only memory (CDROM), digital tape, and other non-transitory media.
The various embodiments described above can be combined to provide further embodiments. To the extent that they are not inconsistent with the specific teachings and definitions herein, all of the U.S. patents, U.S. patent application publications, U.S. patent applications, foreign patents, foreign patent applications and non-patent publications referred to in this specification and/or listed in the Application Data Sheet which are owned by Thalmic Labs Inc., including but not limited to: U.S. Provisional Patent Application Ser. No. 62/214,600, U.S. Provisional Patent Application Ser. No. 62/268,892, U.S. Provisional Patent Application Ser. No. 62/167,767, U.S. Provisional Patent Application Ser. No. 62/271,135, U.S. Provisional Patent Application Ser. No. 62/245,792, U.S. Provisional Patent Application Ser. No. 62/281,041, U.S. Provisional Patent Application Ser. No. 62/134,347, U.S. Provisional Patent Application Ser. No. 62/288,947, U.S. Non-Provisional patent application Ser. No. 14/155,087, U.S. Non-Provisional patent application Ser. No. 14/155,107, PCT Patent Application PCT/US2014/057029, U.S. Provisional Patent Application Ser. No. 62/236,060, U.S. Provisional Patent Application Ser. No. 62/117,316, U.S. Provisional Patent Application Ser. No. 62/156,736, and U.S. Provisional Patent Application Ser. No. 62/242,844, are incorporated herein by reference, in their entirety. Aspects of the embodiments can be modified, if necessary, to employ systems, circuits and concepts of the various patents, applications and publications to provide yet further embodiments.
These and other changes can be made to the embodiments in light of the above-detailed description. In general, in the following claims, the terms used should not be construed to limit the claims to the specific embodiments disclosed in the specification and the claims, but should be construed to include all possible embodiments along with the full scope of equivalents to which such claims are entitled. Accordingly, the claims are not limited by the disclosure.
| Number | Name | Date | Kind |
|---|---|---|---|
| 3408133 | Lee | Oct 1968 | A |
| 3712716 | Cornsweet et al. | Jan 1973 | A |
| 4978213 | El Hage | Dec 1990 | A |
| 5103323 | Magarinos et al. | Apr 1992 | A |
| 5231674 | Cleveland et al. | Jul 1993 | A |
| 5467104 | Furness, III et al. | Nov 1995 | A |
| 5589956 | Morishima et al. | Dec 1996 | A |
| 5596339 | Furness, III et al. | Jan 1997 | A |
| 5742421 | Wells et al. | Apr 1998 | A |
| 6008781 | Furness, III et al. | Dec 1999 | A |
| 6027216 | Guyton et al. | Feb 2000 | A |
| 6184847 | Fateh et al. | Feb 2001 | B1 |
| 6204829 | Tidwell | Mar 2001 | B1 |
| 6236476 | Son et al. | May 2001 | B1 |
| 6317103 | Furness, III et al. | Nov 2001 | B1 |
| 6353503 | Spitzer et al. | Mar 2002 | B1 |
| 6377277 | Yamamoto | Apr 2002 | B1 |
| 6639570 | Furness, III et al. | Oct 2003 | B2 |
| 6972734 | Ohshima et al. | Dec 2005 | B1 |
| 7473888 | Wine et al. | Jan 2009 | B2 |
| 7640007 | Chen et al. | Dec 2009 | B2 |
| 7684105 | Lamontagne et al. | Mar 2010 | B2 |
| 7747113 | Mukawa et al. | Jun 2010 | B2 |
| 7773111 | Cleveland et al. | Aug 2010 | B2 |
| 7850306 | Uusitalo et al. | Dec 2010 | B2 |
| 7925100 | Howell et al. | Apr 2011 | B2 |
| 7927522 | Hsu | Apr 2011 | B2 |
| 8120828 | Schwerdtner | Feb 2012 | B2 |
| 8179604 | Prada Gomez et al. | May 2012 | B1 |
| 8188937 | Amafuji et al. | May 2012 | B1 |
| 8355671 | Kramer et al. | Jan 2013 | B2 |
| 8560976 | Kim | Oct 2013 | B1 |
| 8634119 | Bablumyan et al. | Jan 2014 | B2 |
| 8666212 | Amirparviz | Mar 2014 | B1 |
| 8704882 | Turner | Apr 2014 | B2 |
| 8922481 | Kauf | Dec 2014 | B1 |
| 8922898 | Legerton et al. | Dec 2014 | B2 |
| 8970571 | Wong et al. | Mar 2015 | B1 |
| 8971023 | Olsson et al. | Mar 2015 | B2 |
| 9086687 | Park et al. | Jul 2015 | B2 |
| 9135708 | Ebisawa | Sep 2015 | B2 |
| 20010033402 | Popovich | Oct 2001 | A1 |
| 20020003627 | Rieder | Jan 2002 | A1 |
| 20020007118 | Adachi et al. | Jan 2002 | A1 |
| 20020030636 | Richards | Mar 2002 | A1 |
| 20020093701 | Zhang et al. | Jul 2002 | A1 |
| 20020120916 | Snider, Jr. | Aug 2002 | A1 |
| 20040174287 | Deak | Sep 2004 | A1 |
| 20050012715 | Ford | Jan 2005 | A1 |
| 20060238707 | Elvesjo et al. | Oct 2006 | A1 |
| 20070078308 | Daly | Apr 2007 | A1 |
| 20070132785 | Ebersole, Jr. et al. | Jun 2007 | A1 |
| 20090109241 | Tsujimoto | Apr 2009 | A1 |
| 20090179824 | Tsujimoto et al. | Jul 2009 | A1 |
| 20090207464 | Wiltshire et al. | Aug 2009 | A1 |
| 20090258669 | Nie et al. | Oct 2009 | A1 |
| 20090322653 | Putilin et al. | Dec 2009 | A1 |
| 20100053555 | Enriquez et al. | Mar 2010 | A1 |
| 20100060551 | Sugiyama et al. | Mar 2010 | A1 |
| 20100142015 | Kuwahara et al. | Jun 2010 | A1 |
| 20100149073 | Chaum et al. | Jun 2010 | A1 |
| 20100150415 | Atkinson et al. | Jun 2010 | A1 |
| 20100239776 | Yajima et al. | Sep 2010 | A1 |
| 20120002256 | Lacoste et al. | Jan 2012 | A1 |
| 20120139817 | Freeman | Jun 2012 | A1 |
| 20120169752 | Kurozuka | Jul 2012 | A1 |
| 20120182309 | Griffin et al. | Jul 2012 | A1 |
| 20120188158 | Tan et al. | Jul 2012 | A1 |
| 20120249797 | Haddick et al. | Oct 2012 | A1 |
| 20120290401 | Neven | Nov 2012 | A1 |
| 20120302289 | Kang | Nov 2012 | A1 |
| 20130009853 | Hesselink et al. | Jan 2013 | A1 |
| 20130016292 | Miao et al. | Jan 2013 | A1 |
| 20130016413 | Saeedi et al. | Jan 2013 | A1 |
| 20130088413 | Raffle et al. | Apr 2013 | A1 |
| 20130135722 | Yokoyama | May 2013 | A1 |
| 20130165813 | Chang et al. | Jun 2013 | A1 |
| 20130169560 | Cederlund et al. | Jul 2013 | A1 |
| 20130198694 | Rahman et al. | Aug 2013 | A1 |
| 20130215235 | Russell | Aug 2013 | A1 |
| 20130222384 | Futterer | Aug 2013 | A1 |
| 20130265437 | Thörn et al. | Oct 2013 | A1 |
| 20130285901 | Lee et al. | Oct 2013 | A1 |
| 20130300652 | Raffle et al. | Nov 2013 | A1 |
| 20130332196 | Pinsker | Dec 2013 | A1 |
| 20130335302 | Crane et al. | Dec 2013 | A1 |
| 20140045547 | Singamsetty et al. | Feb 2014 | A1 |
| 20140125760 | Au et al. | May 2014 | A1 |
| 20140198034 | Bailey et al. | Jul 2014 | A1 |
| 20140198035 | Bailey et al. | Jul 2014 | A1 |
| 20140202643 | Hikmet et al. | Jul 2014 | A1 |
| 20140204455 | Popovich et al. | Jul 2014 | A1 |
| 20140204465 | Yamaguchi | Jul 2014 | A1 |
| 20140226193 | Sun | Aug 2014 | A1 |
| 20140232651 | Kress et al. | Aug 2014 | A1 |
| 20140285429 | Simmons | Sep 2014 | A1 |
| 20140368896 | Nakazono et al. | Dec 2014 | A1 |
| 20150036221 | Stephenson | Feb 2015 | A1 |
| 20150156716 | Raffle et al. | Jun 2015 | A1 |
| 20150205126 | Schowengerdt | Jul 2015 | A1 |
| 20150205134 | Bailey et al. | Jul 2015 | A1 |
| 20150268821 | Ramsby et al. | Sep 2015 | A1 |
| 20150325202 | Lake et al. | Nov 2015 | A1 |
| 20150362734 | Moser et al. | Dec 2015 | A1 |
| 20150378161 | Bailey et al. | Dec 2015 | A1 |
| 20150378162 | Bailey et al. | Dec 2015 | A1 |
| 20150378164 | Bailey et al. | Dec 2015 | A1 |
| 20160033771 | Tremblay | Feb 2016 | A1 |
| 20160202081 | Debieuvre et al. | Jul 2016 | A1 |
| 20160238845 | Alexander et al. | Aug 2016 | A1 |
| 20160274365 | Bailey et al. | Sep 2016 | A1 |
| 20160274758 | Bailey | Sep 2016 | A1 |
| 20160313899 | Noel | Oct 2016 | A1 |
| 20160327796 | Bailey et al. | Nov 2016 | A1 |
| 20160327797 | Bailey et al. | Nov 2016 | A1 |
| 20160349514 | Alexander et al. | Dec 2016 | A1 |
| 20160349515 | Alexander et al. | Dec 2016 | A1 |
| 20160349516 | Alexander et al. | Dec 2016 | A1 |
| 20160377866 | Alexander | Dec 2016 | A1 |
| 20160377886 | Quiroga | Dec 2016 | A1 |
| 20170068095 | Holland et al. | Mar 2017 | A1 |
| 20170097753 | Bailey et al. | Apr 2017 | A1 |
| 20170115483 | Aleem et al. | Apr 2017 | A1 |
| 20170153701 | Mahon et al. | Jun 2017 | A1 |
| 20170205876 | Vidal et al. | Jul 2017 | A1 |
| 20170212290 | Alexander et al. | Jul 2017 | A1 |
| 20170212349 | Bailey et al. | Jul 2017 | A1 |
| 20170219829 | Bailey | Aug 2017 | A1 |
| 20170299956 | Holland et al. | Oct 2017 | A1 |
| 20170343796 | Bailey et al. | Nov 2017 | A1 |
| 20170343797 | Bailey et al. | Nov 2017 | A1 |
| 20180007255 | Tang | Jan 2018 | A1 |
| Number | Date | Country |
|---|---|---|
| 61-198892 | Sep 1986 | JP |
| 10-319240 | Dec 1998 | JP |
| 2013-127489 | Jun 2013 | JP |
| 2013-160905 | Aug 2013 | JP |
| 10-2004-0006609 | Jan 2004 | KR |
| 2015123775 | Aug 2012 | WO |
| 2014155288 | Oct 2014 | WO |
| Entry |
|---|
| International Search Report and Written Opinion dated Apr. 25, 2017 for corresponding International Application No. PCT/US2016/067246, 12 pages. |
| Amitai, “P-27: A Two-Dimensional Aperture Expander for Ultra-Compact, High-Performance Head-Worn Displays,” SID Symposium Digest of Technical Papers 36(1):360-363, 2005. |
| Äyräs et al., “Exit pupil expander with a large field of view based on diffractive optics,” Journal of the SID 17(8):659-664, 2009. |
| Chellappan et al., “Laser-based displays: a review,” Applied Optics 49(25):F79-F98, 2010. |
| Cui et al., “Diffraction from angular multiplexing slanted volume hologram gratings,” Optik 116:118-122, 2005. |
| Curatu et al., “Dual Purpose Lens for an Eye-Tracked Projection Head-Mounted Display,” International Optical Design Conference 2006, SPIE-OSA 6342:63420X-1-63420X-7, 2007. |
| Curatu et al., “Projection-based head-mounted display with eye-tracking capabilities,” Proc. of SPIE 5875:58750J-1-58750J-9, 2005. |
| Essex, “Tutorial on Optomechanical Beam Steering Mechanisms,” OPTI 521 Tutorial, College of Optical Sciences, University of Arizona, 8 pages, 2006. |
| Fernández et al., “Optimization of a thick polyvinyl alcohol-acrylamide photopolymer for data storage using a combination of angular and peristrophic holographic multiplexing,” Applied Optics 45(29):7661-7666, 2009. |
| Hainich et al., “Chapter 10: Near-Eye Displays,” Displays: Fundamentals & Applications, AK Peters/CRC Press, 2011, 65 pages. |
| Hornstein et al., “Maradin's Micro-Mirror—System Level Synchronization Notes,” SID 2012 Digest, pp. 981-984. |
| International Search Report, dated Jun. 8, 2016, for PCT/US2016/018293, 17 pages. |
| International Search Report, dated Jun. 8, 2016, for PCT/US2016/018298, 14 pages. |
| International Search Report, dated Jun. 8, 2016, for PCT/US2016/018299, 12 pages. |
| Itoh et al., “Interaction-Free Calibration for Optical See-Through Head-Mounted Displays based on 3D Eye Localization,” 2014 IEEE Symposium on 3D User Interfaces (3DUI), pp. 75-82, 2014. |
| Kessler, “Optics of Near to Eye Displays (NEDs),” Presentation—Oasis 2013, Tel Aviv, Feb. 19, 2013, 37 pages. |
| Kress et al., “A review of head-mounted displays (HMD) technologies and applications for consumer electronics,” Proc. of SPIE 8720:87200A-1-87200A-13, 2013. |
| Kress et al., “Diffractive and Holographic Optics as Optical Combiners in Head Mounted Displays,” Proceedings of the 2013 ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication, pp. 1479-1482, 2013. |
| Kress, “Optical architectures for see-through wearable displays,” Presentation—Bay Area—SID Seminar, Apr. 30, 2014, 156 pages. |
| Levola, “7.1: Invited Paper: Novel Diffractive Optical Components for Near to Eye Displays,” SID Symposium Digest of Technical Papers 37(1):64-67, 2006. |
| Liao et al., “The Evolution of MEMS Displays,” IEEE Transactions on Industrial Electronics 56(4):1057-1065, 2009. |
| Lippert, “Chapter 6: Display Devices: RSD™ (Retinal Scanning Display),” The Avionics Handbook, CRC Press, 2001, 8 pages. |
| Majaranta et al., “Chapter 3—Eye-Tracking and Eye-Based Human-Computer Interaction,” in Advances in Physiological Computing, Springer-Verlag London, 2014, pp. 17-39. |
| Schowengerdt et al., “Stereoscopic retinal scanning laser display with integrated focus cues for ocular accommodation” Proc. of SPIE-IS&T Electronic Imaging 5291:366-376, 2004. |
| Silverman et al., “58.5L: Late News Paper: Engineering a Retinal Scanning Laser Display with Integrated Accommodative Depth Cues,” SID 03 Digest, pp. 1538-1541, 2003. |
| Takatsuka et al., “Retinal projection display using diffractive optical element,” Tenth International Conference on Intelligent Information Hiding and Multimedia Signal Processing, IEEE, 2014, pp. 403-406. |
| Urey et al., “ Optical performance requirements for MEMS-scanner based microdisplays,” Conf. on MOEMS and Miniaturized Systems, SPIE 4178:176-185, 2000. |
| Urey, “Diffractive exit-pupil expander for display applications,” Applied Optics 40(32):5840-5851, 2001. |
| Viirre et al., “The Virtual Retinal Display: A New Technology for Virtual Reality and Augmented Vision in Medicine,” Proc. of Medicine Meets Virtual Reality, IOS Press and Ohmsha, 1998, pp. 252-257. (6 pages). |
| International Search Report and Written Opinion, dated Dec. 8, 2016, for International Application No. PCT/US2016/050225, 15 pages. |
| International Search Report and Written Opinion, dated Jan. 18, 2017, for International Application No. PCT/US2016/054852, 12 pages. |
| International Search Report and Written Opinion, dated Oct. 13, 2017, for International Application No. PCT/US2017/040323, 16 pages. |
| International Search Report and Written Opinion, dated Sep. 28, 2017, for International Application No. PCT/US2017/027479, 13 pages. |
| Janssen, “Radio Frequency (RF)” 2013, retrieved from https://web.archive.org/web/20130726153946/https://www.techopedia.com/definition/5083/radio-frequency-rf, retrieved on Jul. 12, 2017, 2 pages. |
| Merriam-Webster, “Radio Frequencies” retrieved from https://www.merriam-webster.com/table/collegiate/radiofre.htm, retrieved on Jul. 12, 2017, 2 pages. |
| Number | Date | Country | |
|---|---|---|---|
| 20160377865 A1 | Dec 2016 | US |
| Number | Date | Country | |
|---|---|---|---|
| 62117316 | Feb 2015 | US | |
| 62156736 | May 2015 | US | |
| 62242844 | Oct 2015 | US |
