Non-telecentric emission spatial light modulators, emissive micro-pixel displays, ultra-compact image projectors, directional light modulators, multi-view 2D displays, 3D displays, near-eye displays, head-up displays.
Spatial light modulators (SLMs) are a class of optoelectronic devices having a planar array of micro-scale pixels that are typically used as an image source in display systems. Such light modulators typically fall in one of three distinctive categories: reflective, transmissive or emissive. Examples of reflective SLMs include micro mirror array devices and liquid crystal on silicon (LCoS) devices. Examples of transmissive SLMs include high temperature poly-silicon liquid crystal (HTPS) devices. Examples of emissive SLMs include emissive micro-pixel array devices. One example of an emissive micro-pixel array device may be based on the Quantum Photonic Imager or QPI® imager described in U.S. Pat. Nos. 7,623,560, 7,767,479, 7,829,902, 8,049,231, 8,243,770 and 8,567,960 and organic light emitting diode (OLED) micro-pixel array devices. Both reflective and transmissive SLMs typically require an external light source to modulate images while an emissive SLM generates its own light. In general, all current categories of SLMs modulate telecentric light; meaning the modulated light bundles have their chief rays perpendicular to the plane of the light modulator pixel array. In the case of reflective and transmissive SLMs, telecentric light modulation is dictated by the design limitations of their external light source. Telecentric light emission is the only option available for emissive SLMs with a Lambertian emission profile such as OLED based SLMs.
Example micro-emissive solid state light-emitting display elements suitable for use with the embodiments herein include, without limitation, those described in U.S. Pat. Nos. 7,623,560, 7,767,479, 7,829,902, 8,049,231, 8,243,770 and 8,567,960. These SSL imagers feature high brightness, in a multi-color emissive micro-pixel spatial array with all of its needed drive circuitry in a single device. Within the context of this disclosure the term “SSL imager” is henceforth intended to mean an optoelectronics device that comprises an array of emissive micro-scale solid state light (SSL) emitting pixels. The SSL light emitting pixels of such an imager, hereinafter referred to as simply SSL imagers, are typically either a light emitting diode (LED) or laser diode (LD) whose on-off state is controlled by the drive circuitry contained within a CMOS device upon which the emissive micro-scale pixel array is formed or bonded. The pixels within the emissive micro-scale pixel array of an SSL imager are individually addressable through its drive circuitry, such as CMOS or the comparable, enabling an SSL imager to emit light that is modulated spatially, chromatically and temporally. The multiple colors emitted by an SSL imager share the same pixel optical aperture. In an SSL imager best suited for use with the embodiments herein, each SSL imager pixel emits at least partially collimated (or non-Lambertian) light, in the case of a QPI SSL imager, with an angle of divergence ranging, by design, from ±5° to ±45°. The size of the pixels comprising the emissive array of an SSL imager would typically be in the range of approximately 5-20 microns with the typical emissive surface area of the device being in the range of approximately 15-150 square millimeter. An SSL imager preferably can be designed with minimal gap between its emissive pixel array area and the device physical edge, allowing a multiplicity of SSL imagers, including QPI imagers, to be tiled to create any arbitrary size emissive display area.
Although all current categories of SLMs preferably modulate telecentric light, there is much to be gained from a non-telecentric light emission SLM. Since reflective and transmissive SLM's non-telecentric light modulation capability is limited by their external light source, and an emissive OLED-based SLM cannot achieve non-telecentric light emission by virtue of its Lambertian light emission profile, the SSL imager with its emissive multi-color micro-pixels that emits collimated (or non-Lambertian) light is uniquely qualified to achieve non-telecentric light modulation. It is therefore an objective of this invention to extend the design and manufacturing methods of an SSL imager to include the capability of non-telecentric light emission for the numerous possible applications that stand to benefit from such capability, some few of which are described herein by way of non-limiting examples only.
In the design of 3D displays, directional modulation of the emitted light is necessary to create the 3D viewing perception. In a typical 3D display, a backlight with uniform illumination in multiple illumination directions is required to display images of the same scene from different directions by utilizing some combination of spatial multiplexing and temporal multiplexing in the SLM. In these 3D displays, the light that typically comes from the directional backlight is usually processed by a directionally selective filter (such as diffractive plate or a holographic optical plate for example
Currently, prior art directional light modulators are a combination of an illumination unit comprising multiple light sources and a directional modulation unit that directs the light emitted by the light sources to a designated direction (see
In addition to being slow, bulky and optically lossy, the prior art directional backlight units typically need to have narrow spectral bandwidth, high collimation and individual controllability for being combined with a directionally selective filter for 3D display purposes. Achieving narrow spectral bandwidth and high collimation requires device level innovations and optical light conditioning, increasing the cost and the volumetric aspects of the overall display system. Achieving individual controllability requires additional circuitry and multiple light sources, increasing the system complexity, bulk and cost. It is therefore an objective of this invention to introduce directional light modulators that overcome the limitation of the prior art, thus making it feasible to create distortion free 3D and multi-view 2D displays that provide the volumetric advantages plus a viewing experience over a wide viewing angle.
Additional objectives and advantages of this invention will become apparent from the following detailed description of preferred embodiments thereof that proceeds with reference to the accompanying drawings.
The invention is illustrated by way of example, and not by way of limitation, in the Figs. of the accompanying drawings in which like reference numerals refer to similar elements.
References in the following detailed description of the present invention to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. The appearances of the phrase “in one embodiment” in various places in this detailed description are not necessarily all referring to the same embodiment.
In the description to follow, references are made to the word groups, such as in a directional modulation groups of micro pixels and similar references. When a non-telecentric emissive micro-pixel array light modulator in accordance with the present invention is to display a light field, reference to a group of pixels or to the corresponding micro optical elements is a reference to the pixels or micro optical elements associated with a single hogel. When the present invention is to display one or more two dimensional scenes at the same time, reference to a group of pixels or to the corresponding micro optical elements is a reference to all pixels of the non-telecentric emissive micro-pixel array light modulator or to one of the groups of pixels associated with a respective two dimensional scene.
The embodiments herein combine the emissive micro pixel array capabilities of an SSL imager with a monolithically fabricated pixel level micro optical element (referred to in the claims as a directional modulation layer) to create a non-telecentric spatial light modulator that performs the combined functionalities of color and brightness as well as directional modulation of the light emitted from each of its emissive micro pixels. Pixel level micro optical elements are a class of wafer level optics (WLO) which, in accordance with the present invention, are fabricated monolithically on an SSL imager wafer from semiconductor dielectric materials, such silicon oxide or silicon nitride, or alternatively with a UV curable polymer using ultra violet (UV) imprint lithography. As used herein, wafer level or wafer means a device or matrix of devices having a diameter of at least 2 inches, and more preferably 4 inches or more. Among the primary advantages of WLO are the ability to fabricate small feature micro-scale optical elements and the ability to precisely align multiple layers of WLO optical elements with the optoelectronics elements of devices such as the SSL imager or a CMOS sensor, for example. The alignment precision that can be achieved by typical WLO fabrication techniques can be much less than one micron. The combination of the individual pixel addressability of the emissive micro pixel array of the SSL imager and a precisely aligned micro pixel level optical elements, herein referred to collectively as the “non-telecentric SSL imager”, creates the first non-telecentric emissive SLM which enables the numerous applications highlighted in the previous discussion and detailed henceforth.
In the embodiments herein, pixels' color, brightness and directional light modulation is achieved by the combination of the SSL imager emissive pixels and light bending achieved by their associated micro pixel level optics that collectively comprise the non-telecentric emission SSL imager.
The pixel level micro optical elements 250 of the directional modulation layer 245 would have the same planar dimensions in the (x,y) plane as their respective emissive micro pixel 210. The pixel level micro optical elements 250 could be realized as either a refractive optical element (ROE) or a diffractive optical element (DOE). In either of these cases, the optical design parameters of each of the pixel level micro optical elements 250 would be selected to realize the selected directional modulation pattern across the modulation pixel group 270 as explained earlier. The pixel level micro optical elements 250 would be fabricated monolithically on the SSL imager wafer from either semiconductor dielectric materials, such silicon oxide or silicon nitride, or from polymer using ultra violet (UV) imprint lithography. In one embodiment, the fabrication of the pixel level micro optical elements 250 would be accomplished as a sequence of lithography, etch and deposition steps directly on the topside (the light emissive side) of the SSL imager wafer that already incorporates on its topside the photonic layer 205, which incorporates the array of micro pixels 230, and on its backside the CMOS control layer 225. In another embodiment of this invention, the fabrication of the pixel level micro optical elements 250 would be accomplished as a sequence of lithography, etch and deposition steps on one side of a wafer size cover glass layer 235, then the micro optical elements 250 side of the cover glass wafer is aligned and bonded directly on the top side (the light emissive side) of the SSL imager wafer that already incorporates the photonic layer 205 and the CMOS control layer 225. In yet another embodiment of this invention, the fabrication of the pixel level micro optical elements 250 would be accomplished as a sequence of lithography, etch and deposition steps on one side of the cover glass layer 235, then the micro optical elements 250 side of the glass cover wafer is used as a substrate to which the multiple sub-layers constituting the photonic layer 205, upon which the array of micro pixels 230 are formed, and the CMOS control layer 225 would be bonded sequentially to fabricate the non-telecentric emissive micro pixel light modulator (SSL imager) 200 of the embodiments herein.
In another embodiment, the fabrication of the pixel level micro optical elements 250 would be accomplished using WLO techniques in which the micro optical elements 250 would be formed by embossing a UV curable polycarbonate polymer on one side of the cover glass layer 235, then the formed micro optical elements 250 side of the wafer is aligned and bonded directly on the top side (the light emissive side) of the SSL imager wafer that already incorporates the photonic layer 205 and the CMOS control layer 225. In this method, the layer of pixel level micro optical elements 250 would be fabricated using a master mold of the array of micro optical elements 250 that is first fabricated using micro machining or laser ablation techniques, then copied on a UV transparent mold that would be used to emboss the UV curable polycarbonate polymer on one side of a wafer-size cover glass 235. The embossing of the array of micro optical elements 250 would also incorporate the embossing of alignment marks that would be used to align the micro optical elements 250 to their respective micro pixels 230 during the bonding process with the SSL imager wafer that already incorporates the photonic layer 205 and the CMOS control layer 225. In this case, the bonding of the glass cover wafer 235 with the embossed micro optical elements 250 to the of the SSL imager wafer, that already incorporates the photonic layer 205 and the CMOS control layer 225, would be accomplished using UV curable optical glue that would be spread or sprayed on the bonding surface of either or both wafers, then the wafers are brought into alignment using alignment marks previously incorporated onto the surface of each wafer, then the aligned wafer pair is scanned with a UV laser beam or illuminated by a UV flood light to cure the optical glue bonding layer placed on the bonding surfaces of the two wafers.
“De-Centered” Refractive Micro Optical Element (ROE)—
“Tilted” Refractive Micro Optical Element (ROE)—
Spatially Modulated Refractive Micro Optical Element (ROE)—
In other applications, the spatially modulated refractive micro optical element (ROE) of the directional modulation layer described above need not have the micro optical element 250-3 at the center of the directional modulation group or have the single waveguide 260 of its associated emissive micro pixel 205 aligned with the center of its optical aperture. Further, the directional modulation layer 245 of the embodiment of
“Diffractive” Micro Optical Element (DOE)—
As illustrated in
The SSL imager 200 pixels' diffractive optical elements 250-4 can be realized using transmission grating such as blazed grating or rail grating, for example. The illustration of
As explained earlier, the SSL imager 200 pixels can emit light with multiple wavelengths from each pixel aperture. In the case when the SSL imager 200 pixels emit light with multiple wavelengths, the diffractive optical elements 250-2 would be realized using wideband transmission gratings or multi-level gratings designed to achieve the desired diffraction angle across the light emission bandwidth of the SSL imager 200 pixels. In the case when wideband transmission gratings are used, the SSL imager 200 pixels diffractive optical elements 250-2 could be realized using a multiplicity of layers of dielectric materials with alternating high and low indexes for each such layer index. The index of such layers together with the formed grating slant angle and pitch would be selected to realize the desired pixel's diffractive angle within each sub-band of the SSL imager 200 pixels' light emission bandwidth.
Directional Modulation Layer 245 Fabrication Method—
As explained earlier, the pixel level micro optical elements 250 of the previous embodiments would be used to directionally modulate, collect and/or collimate the light emitted from their corresponding emissive micro pixels 205 and as such would have optical apertures which are equal to and precisely aligned, within at least less than 10% accuracy, with their corresponding micro pixels 230 of the non-telecentric emissive micro pixel light modulator (SSL imager) 200 of this invention. The pixel level micro optical elements 250 of the previous embodiments can be fabricated using a variety of nano-imprint lithographic techniques such as patterning using Phase, Grayscale, entire domain or sub domain masks, additive lithographic sculpting that use binary masks as well as various direct write techniques. Photo resist (heat) reflow and molding could also be employed for fabrication of the micro optical elements 250 to increase the smoothness of the formed surfaces. Additive Lithography using a sub domain binary mask set will be used herein as an example to illustrate a typical method for the fabrication of exemplary ROE pixel level micro optical elements 250 of the non-telecentric emissive micro pixel light modulator (SSL imager) 200 of this invention.
Wafer Level Optics Mask Set—
The fabrication process sequence of the pixel level micro optical elements 250 begins with the fabrication of a set of lithography sub domain masks that captures the binary shape specifications of the micro optical elements 250. The fabrication process of the sub domain masks involves identifying the symmetry within the specified features of the pixel level micro optical elements 250 to be fabricated and breaking down the identified symmetry to an orthogonal or non orthogonal set of basis.
As illustrated in
Although the preceding discussion outlined the process of forming the lithography mask set for the case when the pixel level micro optical elements 250 are realized in accordance with the embodiment described earlier as a de-centered ROE, a person skilled in the art will readily know how to apply the described lithography mask set formation process in the cases of other described embodiments when the micro optical elements 250 are realized as tilted ROE or as a DOE.
Wafer Level Optics Fabrication Sequence—
A photoresist appropriately chosen and characterized is then coated on top of dielectric-1 layer and Additive Lithography is performed using the fabricated mask set to create a negative of that surface of the pixel level micro optical elements 250 (
Once the specification compliance of the photoresist negative surface is confirmed, the micro optical elements 250 would be etched using the appropriate chemistry on a reactive ion etching (RIE) tool that can provide 1:1 selectivity between the photoresist and the dielectric-1 layers (
After the dielectric-1 layer surface is fabricated, the wafer is coated with a relatively thick conformal layer of a high index dielectric-2 layer (
After the fabrication of the 1st and 2nd surfaces of the pixel level micro optical elements 250 as shown in
The described WLO fabrication steps would be repeated on the top of dielectric-3 layer wafer surface if a second optical element is to be added in accordance with the optical prescription of the pixel level micro optical elements 250.
After all the optical elements of the pixel level micro optical elements 250 are fabricated on top of the substrate wafer, the wafer surface is coated with a bonding intermediary layer, polished to the appropriate bonding specification then aligned bonded, using the alignment marks added earlier in the process, to the top photonic layer 205 surface.
In one embodiment of this invention, the WLO fabrication process described in the preceding paragraphs is used to fabricate the pixel level micro optical elements 250 on the top of the cover glass wafer 235 as a substrate then the fabricated wafer is used as a substrate upon which the pixelated photonic layers stack 205 would be bonded and formed sequentially. At the end of that fabrication sequence, the formed wafer would be aligned bonded to the topside of the CMOS control layer 225.
In another embodiment of this invention, the WLO fabrication process described in the preceding paragraphs is used to fabricate the pixel level micro optical elements 250 on top of the cover glass wafer 235 as a substrate then the fabricated wafer is aligned bonded to the photonic layer 205 top surface of the wafer formed separately that comprises the photonic layer stack 205 bonded sequentially to the topside of the CMOS control layer 225.
In yet another embodiment of this invention, the WLO fabrication process described in the preceding paragraphs is used to fabricate the pixel level micro optical elements 250 on top of the wafer formed separately that comprises the photonic layer stack 205 bonded sequentially to the topside of the CMOS control layer 225, then the cover glass wafer is bonded on the top of the fabricated pixel level micro optical elements 250. In the WLO fabrication process for this case, the 2nd surface of the pixel level micro optical elements 250 will be fabricated first on top of the pixelated photonic layers stack 205, then the 1st surface of the pixel level micro optical elements 250 would be fabricated on top. In still other embodiments, particularly using other fabrication processes described herein for the directional modulation layers and using this fabrication process, the glass cover layer 235 would be bonded on the top side of the SSL imager 200 with the addition of spacers that would allow a thin air gap layer 620 between the high index layer 610 and the glass cover 235. Each of the aforementioned embodiments would have its advantages depending on the intended application.
Ultra Compact Projector—
Minimum Cross-Talk Light Field Modulator—
In this embodiment, the light emitted from each of the SSL imager 200 micro pixels 210 would be sufficiently collimated and directionally modulated (or directed) by the pixel's micro optical elements 250 to efficiently fill in their associated hogel lens with minimal or substantially no light leakage into adjacent hogel lenses. Thus in addition to minimizing the cross-talk between directionally modulated views, the directional light modulator of
Multi-View Display—
Waveguide Light Modulator—
The directional modulation design method of this embodiment is further illustrated in
Tapered Waveguide Light Modulator—
In an alternative embodiment, illustrated in
It should be noted that the two previous embodiments of the SSL imager 200 based Waveguide Light Modulator can be used to create optical see-through (OST) near-eye or head-mounted display (HMD). In each of these embodiments, as alternate embodiments, the SSL imager may instead have its output coupled into an input aperture at the end or edge of the waveguide 1110 and achieve the same of similar results. Also the waveguides themselves may have other shapes, and are not limited to rectangular shapes as shown.
Thus, using the wafer level fabrication techniques for the wafer level optics, the entire non-telecentric emissive micro-pixel array light modulator may be fabricated at the wafer level and then the wafer level assembly diced to obtain the final products.
Also, while
In the foregoing description and in the claims, reference is made to a directional modulation layer. It is to be understood that the word layer is used in a general sense to identify the layer or multiple layers that determine the spatial modulation of the non-telecentric emissive micro-pixel array light modulator. Also it should be understood that while certain fabrication techniques for the directional modulation layer have been described herein, each with respect to a respective embodiment, such techniques are also applicable to use in other embodiments using the same form of directional modulation layer and/or fabrication sequence. Thus by way of example, the use of UV curable polycarbonate polymer or the embossing to form a specific embodiment is exemplary, and such techniques and the sequence of operations for forming the resulting ROE are exemplary only, and the use of UV curable polycarbonate polymer or embossing may be used to form any ROE based directional modulation layer and to fabricate the non-telecentric emissive micro-pixel array light modulator of the present invention in any sequence of fabrication operations. Further, unless the context dictates otherwise, references to a pixel group or pixel groups includes a reference to a group that encompasses the entire SSL imager emitter area
Thus those skilled in the art will readily appreciate that various modifications and changes can be applied to the embodiments of the invention without departing from its scope defined in and by the appended claims. It should be appreciated that the foregoing examples of the invention are illustrative only, and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The disclosed embodiments, therefore, should not be considered to be restrictive in any sense. The scope of the invention is indicated by the appended claims, rather than the preceding description, and all variations which fall within the meaning and range of equivalents thereof are intended to be embraced therein.
This application is a divisional application and claims the benefit of U.S. Non-Provisional patent application Ser. No. 15/391,583, filed Dec. 27, 2016, now allowed, which in turn claims the benefit of U.S. Provisional Patent Application No. 62/271,637, filed Dec. 28, 2015, the entirety of each of which is incorporated herein by reference.
Number | Name | Date | Kind |
---|---|---|---|
4427912 | Bui et al. | Jan 1984 | A |
4987410 | Berman et al. | Jan 1991 | A |
5162828 | Furness et al. | Nov 1992 | A |
5368042 | O'neal et al. | Nov 1994 | A |
5619373 | Meyerhofer et al. | Apr 1997 | A |
5696521 | Robinson et al. | Dec 1997 | A |
5818359 | Beach | Oct 1998 | A |
5886822 | Spitzer | Mar 1999 | A |
5986811 | Wohlstadter | Nov 1999 | A |
6128003 | Smith et al. | Oct 2000 | A |
6147807 | Droessler et al. | Nov 2000 | A |
6151167 | Melville | Nov 2000 | A |
6353503 | Spitzer et al. | Mar 2002 | B1 |
6433907 | Lippert et al. | Aug 2002 | B1 |
6456438 | Lee et al. | Sep 2002 | B1 |
6522794 | Bischel et al. | Feb 2003 | B1 |
6529331 | Massof et al. | Mar 2003 | B2 |
6666825 | Smith et al. | Dec 2003 | B2 |
6710902 | Takeyama | Mar 2004 | B2 |
6719693 | Richard | Apr 2004 | B2 |
6795221 | Urey | Sep 2004 | B1 |
6803561 | Dunfield | Oct 2004 | B2 |
6804066 | Ha et al. | Oct 2004 | B1 |
6829095 | Amitai | Dec 2004 | B2 |
6924476 | Wine et al. | Aug 2005 | B2 |
6937221 | Lippert et al. | Aug 2005 | B2 |
6984208 | Zheng | Jan 2006 | B2 |
6999238 | Glebov et al. | Feb 2006 | B2 |
7061450 | Bright et al. | Jun 2006 | B2 |
7071594 | Yan et al. | Jul 2006 | B1 |
7106519 | Aizenberg et al. | Sep 2006 | B2 |
7190329 | Lewis et al. | Mar 2007 | B2 |
7193758 | Wiklof et al. | Mar 2007 | B2 |
7209271 | Lewis et al. | Apr 2007 | B2 |
7215475 | Woodgate et al. | May 2007 | B2 |
7232071 | Lewis et al. | Jun 2007 | B2 |
7369321 | Ren et al. | May 2008 | B1 |
7482730 | Davis et al. | Jan 2009 | B2 |
7486255 | Brown et al. | Feb 2009 | B2 |
7545571 | Garoutte et al. | Jun 2009 | B2 |
7580007 | Brown et al. | Aug 2009 | B2 |
7619807 | Baek et al. | Nov 2009 | B2 |
7623560 | El-Ghoroury et al. | Nov 2009 | B2 |
7724210 | Sprague et al. | May 2010 | B2 |
7747301 | Cheng et al. | Jun 2010 | B2 |
7767479 | El-Ghoroury et al. | Aug 2010 | B2 |
7791810 | Powell | Sep 2010 | B2 |
7829902 | El-Ghoroury et al. | Nov 2010 | B2 |
7952809 | Takai | May 2011 | B2 |
8049231 | El-Ghoroury et al. | Nov 2011 | B2 |
8098265 | El-Ghoroury et al. | Jan 2012 | B2 |
8243770 | El-Ghoroury et al. | Aug 2012 | B2 |
8279716 | Gossweiler et al. | Oct 2012 | B1 |
8292833 | Son et al. | Oct 2012 | B2 |
8405618 | Colgate et al. | Mar 2013 | B2 |
8471967 | Miao et al. | Jun 2013 | B2 |
8477425 | Border et al. | Jul 2013 | B2 |
8482859 | Border et al. | Jul 2013 | B2 |
8508830 | Wang | Aug 2013 | B1 |
8508851 | Miao et al. | Aug 2013 | B2 |
8510244 | Carson et al. | Aug 2013 | B2 |
8553910 | Dong et al. | Oct 2013 | B1 |
8567960 | El-Ghoroury et al. | Oct 2013 | B2 |
8619049 | Harrison et al. | Dec 2013 | B2 |
8725842 | Al-Nasser | May 2014 | B1 |
8743145 | Price | Jun 2014 | B1 |
8773599 | Saeedi et al. | Jul 2014 | B2 |
8854724 | El-Ghoroury et al. | Oct 2014 | B2 |
8928969 | Alpaslan et al. | Jan 2015 | B2 |
8975713 | Sako et al. | Mar 2015 | B2 |
9097890 | Miller et al. | Aug 2015 | B2 |
9110504 | Lewis et al. | Aug 2015 | B2 |
9134535 | Dobschal et al. | Sep 2015 | B2 |
9179126 | El-Ghoroury et al. | Nov 2015 | B2 |
9195053 | El-Ghoroury et al. | Nov 2015 | B2 |
9239453 | Cheng et al. | Jan 2016 | B2 |
9244277 | Cheng et al. | Jan 2016 | B2 |
9244539 | Venable et al. | Jan 2016 | B2 |
9274608 | Katz et al. | Mar 2016 | B2 |
9286730 | Bar-Zeev et al. | Mar 2016 | B2 |
9529191 | Sverdrup et al. | Dec 2016 | B2 |
9538182 | Mishourovsky et al. | Jan 2017 | B2 |
9681069 | El-Ghoroury et al. | Jun 2017 | B2 |
9712764 | El-Ghoroury et al. | Jul 2017 | B2 |
9774800 | El-Ghoroury et al. | Sep 2017 | B2 |
9779515 | El-Ghoroury et al. | Oct 2017 | B2 |
9965982 | Lapstun | May 2018 | B2 |
20020008854 | Travis | Jan 2002 | A1 |
20020017567 | Connolly et al. | Feb 2002 | A1 |
20020024495 | Lippert et al. | Feb 2002 | A1 |
20020075232 | Daum et al. | Jun 2002 | A1 |
20020083164 | Katayama et al. | Jun 2002 | A1 |
20020141026 | Wiklof et al. | Oct 2002 | A1 |
20020158814 | Bright et al. | Oct 2002 | A1 |
20020181115 | Massof et al. | Dec 2002 | A1 |
20020194005 | Lahr | Dec 2002 | A1 |
20030032884 | Smith et al. | Feb 2003 | A1 |
20030086135 | Takeyama | May 2003 | A1 |
20030122066 | Dunfield | Jul 2003 | A1 |
20030138130 | Cohen et al. | Jul 2003 | A1 |
20030184575 | Reho et al. | Oct 2003 | A1 |
20030187357 | Richard | Oct 2003 | A1 |
20040004585 | Brown et al. | Jan 2004 | A1 |
20040024312 | Zheng | Feb 2004 | A1 |
20040051392 | Badarneh | Mar 2004 | A1 |
20040080807 | Chen et al. | Apr 2004 | A1 |
20040080938 | Holman et al. | Apr 2004 | A1 |
20040085261 | Lewis et al. | May 2004 | A1 |
20040119004 | Wine et al. | Jun 2004 | A1 |
20040125076 | Green | Jul 2004 | A1 |
20040138935 | Johnson et al. | Jul 2004 | A1 |
20040179254 | Lewis et al. | Sep 2004 | A1 |
20040240064 | Dutta | Dec 2004 | A1 |
20050002074 | Douglas et al. | Jan 2005 | A1 |
20050024730 | Aizenberg et al. | Feb 2005 | A1 |
20050053192 | Sukovic et al. | Mar 2005 | A1 |
20050116038 | Lewis et al. | Jun 2005 | A1 |
20050117195 | Glebov et al. | Jun 2005 | A1 |
20050168700 | Berg et al. | Aug 2005 | A1 |
20050179976 | Davis et al. | Aug 2005 | A1 |
20050264502 | Sprague et al. | Dec 2005 | A1 |
20060017655 | Brown et al. | Jan 2006 | A1 |
20060132383 | Gally et al. | Jun 2006 | A1 |
20060152812 | Woodgate et al. | Jul 2006 | A1 |
20060253007 | Cheng et al. | Nov 2006 | A1 |
20060285192 | Yang | Dec 2006 | A1 |
20060290663 | Mitchell et al. | Dec 2006 | A1 |
20070052694 | Holmes | Mar 2007 | A1 |
20070083120 | Cain et al. | Apr 2007 | A1 |
20070236450 | Colgate et al. | Oct 2007 | A1 |
20070269432 | Nakamura et al. | Nov 2007 | A1 |
20070276658 | Douglass | Nov 2007 | A1 |
20080002262 | Chirieleison et al. | Jan 2008 | A1 |
20080049291 | Baek et al. | Feb 2008 | A1 |
20080130069 | Cernasov | Jun 2008 | A1 |
20080141316 | Igoe et al. | Jun 2008 | A1 |
20080239452 | Xu et al. | Oct 2008 | A1 |
20090073559 | Woodgate et al. | Mar 2009 | A1 |
20090086170 | El-Ghoroury et al. | Apr 2009 | A1 |
20090096746 | Kruse et al. | Apr 2009 | A1 |
20090161191 | Powell | Jun 2009 | A1 |
20090199900 | Bita et al. | Aug 2009 | A1 |
20090222113 | Fuller et al. | Sep 2009 | A1 |
20090256287 | Fu et al. | Oct 2009 | A1 |
20090268303 | Takai | Oct 2009 | A1 |
20090278998 | El-Ghoroury et al. | Nov 2009 | A1 |
20090327171 | Tan et al. | Dec 2009 | A1 |
20100003777 | El-Ghoroury et al. | Jan 2010 | A1 |
20100026960 | Sprague | Feb 2010 | A1 |
20100046070 | Mukawa | Feb 2010 | A1 |
20100053164 | Imai et al. | Mar 2010 | A1 |
20100066921 | El-Ghoroury et al. | Mar 2010 | A1 |
20100091050 | El-Ghoroury et al. | Apr 2010 | A1 |
20100156676 | Mooring et al. | Jun 2010 | A1 |
20100171922 | Sessner et al. | Jul 2010 | A1 |
20100199232 | Mistry et al. | Aug 2010 | A1 |
20100220042 | El-Ghoroury et al. | Sep 2010 | A1 |
20100241601 | Carson et al. | Sep 2010 | A1 |
20100245957 | Hudman et al. | Sep 2010 | A1 |
20100259472 | Radivojevic et al. | Oct 2010 | A1 |
20100267449 | Gagner et al. | Oct 2010 | A1 |
20110054360 | Son et al. | Mar 2011 | A1 |
20110115887 | Yoo et al. | May 2011 | A1 |
20110221659 | King et al. | Sep 2011 | A1 |
20110285666 | Poupyrev et al. | Nov 2011 | A1 |
20110285667 | Poupyrev et al. | Nov 2011 | A1 |
20120033113 | El-Ghoroury et al. | Feb 2012 | A1 |
20120075173 | Ashbrook et al. | Mar 2012 | A1 |
20120075196 | Ashbrook et al. | Mar 2012 | A1 |
20120105310 | Sverdrup et al. | May 2012 | A1 |
20120113097 | Nam et al. | May 2012 | A1 |
20120120498 | Harrison et al. | May 2012 | A1 |
20120143358 | Adams et al. | Jun 2012 | A1 |
20120154441 | Kim | Jun 2012 | A1 |
20120157203 | Latta et al. | Jun 2012 | A1 |
20120195461 | Ashok | Aug 2012 | A1 |
20120212398 | Border et al. | Aug 2012 | A1 |
20120212399 | Border et al. | Aug 2012 | A1 |
20120218301 | Miller | Aug 2012 | A1 |
20120236201 | Larsen et al. | Sep 2012 | A1 |
20120249409 | Toney et al. | Oct 2012 | A1 |
20120249741 | Maciocci et al. | Oct 2012 | A1 |
20120288995 | El-Ghoroury et al. | Nov 2012 | A1 |
20120290943 | Toney et al. | Nov 2012 | A1 |
20120293402 | Harrison et al. | Nov 2012 | A1 |
20120299962 | White et al. | Nov 2012 | A1 |
20120319940 | Bress et al. | Dec 2012 | A1 |
20120320092 | Shin et al. | Dec 2012 | A1 |
20130016292 | Miao et al. | Jan 2013 | A1 |
20130021545 | Song et al. | Jan 2013 | A1 |
20130021658 | Miao et al. | Jan 2013 | A1 |
20130027341 | Mastandrea | Jan 2013 | A1 |
20130041477 | Sikdar et al. | Feb 2013 | A1 |
20130050260 | Reitan | Feb 2013 | A1 |
20130080890 | Krishnamurthi et al. | Mar 2013 | A1 |
20130083303 | Hoover et al. | Apr 2013 | A1 |
20130100362 | Saeedi et al. | Apr 2013 | A1 |
20130141895 | Alpaslan et al. | Jun 2013 | A1 |
20130162505 | Crocco et al. | Jun 2013 | A1 |
20130169536 | Wexler et al. | Jul 2013 | A1 |
20130176622 | Abrahamsson et al. | Jul 2013 | A1 |
20130187836 | Cheng et al. | Jul 2013 | A1 |
20130196757 | Latta et al. | Aug 2013 | A1 |
20130215516 | Dobschal et al. | Aug 2013 | A1 |
20130225999 | Banjanin et al. | Aug 2013 | A1 |
20130258451 | El-Ghoroury | Oct 2013 | A1 |
20130271679 | Sakamoto et al. | Oct 2013 | A1 |
20130285174 | Sako et al. | Oct 2013 | A1 |
20130286053 | Fleck et al. | Oct 2013 | A1 |
20130286178 | Lewis et al. | Oct 2013 | A1 |
20130321581 | El-Ghoroury et al. | Dec 2013 | A1 |
20140009845 | Cheng et al. | Jan 2014 | A1 |
20140024132 | Jia et al. | Jan 2014 | A1 |
20140049417 | Abdurrahman et al. | Feb 2014 | A1 |
20140049983 | Nichol et al. | Feb 2014 | A1 |
20140055352 | Davis et al. | Feb 2014 | A1 |
20140055692 | Kroll et al. | Feb 2014 | A1 |
20140085177 | Lyons et al. | Mar 2014 | A1 |
20140091984 | Ashbrook et al. | Apr 2014 | A1 |
20140098018 | Kim et al. | Apr 2014 | A1 |
20140098067 | Yang et al. | Apr 2014 | A1 |
20140118252 | Kim et al. | May 2014 | A1 |
20140129207 | Bailey et al. | May 2014 | A1 |
20140139454 | Mistry et al. | May 2014 | A1 |
20140139576 | Costa et al. | May 2014 | A1 |
20140147035 | Ding et al. | May 2014 | A1 |
20140168062 | Katz et al. | Jun 2014 | A1 |
20140176417 | Young et al. | Jun 2014 | A1 |
20140185142 | Gupta et al. | Jul 2014 | A1 |
20140200496 | Hyde et al. | Jul 2014 | A1 |
20140232651 | Kress et al. | Aug 2014 | A1 |
20140292620 | Lapstun et al. | Oct 2014 | A1 |
20140300869 | Hirsch et al. | Oct 2014 | A1 |
20140301662 | Justice et al. | Oct 2014 | A1 |
20140304646 | Rossmann | Oct 2014 | A1 |
20140340304 | Dewan et al. | Nov 2014 | A1 |
20150001987 | Masaki et al. | Jan 2015 | A1 |
20150033539 | El-Ghoroury et al. | Feb 2015 | A1 |
20150035832 | Sugden et al. | Feb 2015 | A1 |
20150054729 | Minnen et al. | Feb 2015 | A1 |
20150058102 | Christensen et al. | Feb 2015 | A1 |
20150125109 | Robbins et al. | May 2015 | A1 |
20150138086 | Underkoffler et al. | May 2015 | A1 |
20150148886 | Rao et al. | May 2015 | A1 |
20150193984 | Bar-Zeev et al. | Jul 2015 | A1 |
20150205126 | Schowengerdt et al. | Jul 2015 | A1 |
20150220109 | Von Badinski et al. | Aug 2015 | A1 |
20150235467 | Schowengerdt et al. | Aug 2015 | A1 |
20150277126 | Hirano et al. | Oct 2015 | A1 |
20150301256 | Takiguchi et al. | Oct 2015 | A1 |
20150301383 | Kimura | Oct 2015 | A1 |
20150323990 | Maltz et al. | Nov 2015 | A1 |
20150323998 | Kudekar et al. | Nov 2015 | A1 |
20150326842 | Huai et al. | Nov 2015 | A1 |
20150331297 | Han et al. | Nov 2015 | A1 |
20150381782 | Park et al. | Dec 2015 | A1 |
20160018948 | Parvarandeh et al. | Jan 2016 | A1 |
20160026059 | Chung et al. | Jan 2016 | A1 |
20160028935 | El-Ghoroury et al. | Jan 2016 | A1 |
20160116738 | Osterhout et al. | Apr 2016 | A1 |
20160182782 | El-Ghoroury et al. | Jun 2016 | A1 |
20160191765 | El-Ghoroury et al. | Jun 2016 | A1 |
20160191823 | El-Ghoroury et al. | Jun 2016 | A1 |
20160220232 | Takada et al. | Aug 2016 | A1 |
20160342151 | Dey et al. | Nov 2016 | A1 |
20170065872 | Kelley et al. | Mar 2017 | A1 |
20170069134 | Shapira et al. | Mar 2017 | A1 |
20170116897 | Ahn et al. | Apr 2017 | A1 |
20170184776 | El-Ghoroury et al. | Jun 2017 | A1 |
20170236295 | El-Ghoroury et al. | Aug 2017 | A1 |
20170261388 | Ma et al. | Sep 2017 | A1 |
20170310956 | Perdices-Gonzalez et al. | Oct 2017 | A1 |
Number | Date | Country |
---|---|---|
103298410 | Sep 2013 | CN |
103546181 | Jan 2014 | CN |
103558918 | Feb 2014 | CN |
104460992 | Mar 2015 | CN |
108701437 | Oct 2018 | CN |
0431488 | Jun 1991 | EP |
2014075486 | Apr 2014 | JP |
2015501951 | Jan 2015 | JP |
2019508725 | Mar 2019 | JP |
201001368 | Jan 2010 | TW |
201531738 | Aug 2015 | TW |
201734527 | Oct 2017 | TW |
2014124173 | Aug 2014 | WO |
Entry |
---|
Ahumada, Jr., Albert J. et al., “Spatio-temporal discrimination model predicts temporal masking functions”, Proceedings of SPIE—the International Society for Optical Engineering, Human vision and electronic imaging III, vol. 3299, 1998, 6 pp. total. |
Beulen, Bart W. et al., “Toward Noninvasive Blood Pressure Assessment in Arteries by Using Ultrasound ”, Ultrasound in Medicine & Biology, vol. 37, No. 5, 2011, pp. 788-797. |
Bickel, Bernd et al., “Capture and Modeling of Non-Linear Heterogeneous Soft Tissue ”, ACM Transactions on Graphics (TOG)—Proceedings of ACM SIGGRAPH 2009, vol. 28, Issue 3, Article No. 89, Aug. 2009, 9 pp. total. |
Castellini, Claudio et al., “Using Ultrasound Images of the Forearm to Predict Finger Positions”, IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 20, No. 6, Nov. 2012, pp. 788-797. |
Cobbold, Richard S., “Foundations of Biomedical Ultrasound”, Oxford University Press, 2007, pp. 3-95. |
Fattal, David et al., “A multi-directional backlight for a wide-angle, glasses-free three-dimensional display”, Nature, vol. 495, Mar. 21, 2013, pp. 348-351. |
Final Office Action for U.S. Appl. No. 15/391,583, dated Jun. 11, 2019, 13 pages. |
First Examination Report for India Patent Application No. 201847028193, dated Jan. 27, 2021, 5 pages. |
Grossberg, Stephen et al., “Neural dynamics of saccadic and smooth pursuit eye movement coordination during visual tracking of unpredictably moving targets”, Neural Networks, vol. 27, 2012, pp. 1-20. |
Guo, Jing-Yi et al., “Dynamic monitoring of forearm muscles using one-dimensional sonomyography system”, Journal of Rehabilitation Research & Development, vol. 45, No. 1, 2008, pp. 187-195. |
Harrison, Chris et al., “Skinput: Appropriating the Body as an Input Surface”, CHI '10 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2010, pp. 453-462. |
Hsiao, Tzu-Yu et al., “Noninvasive Assessment of Laryngeal Phonation Function Using Color Doppler Ultrasound Imaging”, Ultrasound in Med & Biol., vol. 27, No. 8, 2001, pp. 1035-1040. |
Hua, Hong et al., “A 3D integral imaging optical see-through head-mounted display”, Optics Express, vol. 22, No. 11, May 28, 2014, pp. 13484-13491. |
International Preliminary Report on Patentability for International Patent Application No. PCT/US2016/069042, dated Jul. 12, 2018, 12 pages. |
International Search Report and Written Opinion for International Patent Application No. PCT/US2016/069042, dated Apr. 19, 2017, 21 pages. |
International Search Report and Written Opinion of the International Searching Authority dated Jun. 29, 2017 International Application No. PCT/US2017/026238, mailed Jun. 29, 2017, 24 pages. |
Keir, Peter J. et al., “Changes in geometry of the finger flexor tendons in the carpal tunnel with wrist posture and tendon load: an MRI study on normal wrists”, Clinical Biomechanics, vol. 14, 1999, pp. 635-645. |
Khuri-Yakub, Butrus T. et al., “Capacitive micromachined ultrasonic transducers for medical imaging and therapy”, J. Micromech. Microeng., vol. 21, No. 5, May 2011, pp. 054004-054014. |
Koutsouridis, G. G. et al., “Towards a Non-lnvasive Ultrasound Pressure Assessment in Large Arteries”, Technische Universiteit Eindhoven, University of Technology, Mate Poster Award 2010 : 15th Annual Poster Contest, 2010, 1 page total. |
Lanman, Douglas et al., “Near-Eye Light Field Displays”, ACM Transactions on Graphics (TOC), vol. 32, Issue 6, Article 220, Nov. 2013, 27 pp. total. |
Legros, M. et al., “Piezocomposite and CMUT Arrays Assessment Through In Vitro Imaging Performances”, 2008 IEEE Ultrasonics Symposium, Nov. 2-5, 2008, pp. 1142-1145. |
Martin, Joel R. et al., “Changes in the flexor digitorum profundus tendon geometry in the carpal tunnel due to force production and posture of metacarpophalangeal joint of the index finger: An MRI study”, Clinical Biomechanics, vol. 28, 2013, pp. 157-163. |
Martin, Joel R. et al., “Effects of the index finger position and force production on the flexor digitorum superficialis moment arms at the metacarpophalangeal joints—a magnetic resonance imaging study”, Clinical Biomechanics, vol. 27, 2012, pp. 453-459. |
Marwah, Kshitij et al., “Compressive Light Field Photography using Overcomplete Dictionaries and Optimized Projections”, Proc. of SIGGRAPH 2013 (ACM Transactions on Graphics, 32, 4), 2013, 12 pp. total. |
Mujibiya, Adiyan et al., “The Sound of Touch: On-body Touch and Gesture Sensing Based on Transdermal Ultrasound Propagation”, ITS '13 Proceedings of the 2013 ACM international conference on Interactive tabletops and surfaces, Oct. 6-9, 2013, pp. 189-198. |
Notice of Allowance for Taiwan Patent Application No. 105143728, dated Dec. 1, 2020, 3 pages. |
Notice of Allowance for U.S. Appl. No. 15/391,583, dated Nov. 12, 2019, 8 pages. |
Notice of Reasons for Rejection for Japan Patent Application 2018-533893, dated Nov. 10, 2020, 6 pages. |
Office Action for Taiwan Patent Application No. 105143728, dated Feb. 17, 2020, 7 pages. |
Office Action for U.S. Appl. No. 15/391,583, dated Aug. 31, 2018, 15 pages. |
Paclet, Florent et al., “Motor control theories improve biomechanical model of the hand for finger pressing tasks”, Journal of Biomechanics, vol. 45, 2012, pp. 1246-1251. |
Pinton, Gianmarco F. et al., “A Heterogeneous Nonlinear Attenuating Full-Wave Model of Ultrasound”, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, vol. 56, No. 3, Mar. 2009, pp. 474-488. |
Restriction Requirement for U.S. Appl. No. 15/391,583, dated Mar. 5, 2018, 9 pages. |
Richard, William D. et al., “A scalable architecture for real-time synthetic-focus imaging”, Ultrasonic Imaging, vol. 25, 2003, pp. 151-161. |
Rolland, Jannick P. et al., “Dynamic focusing in head-mounted displays”, Part of the IS&T/SPIE Conference on the Engineering Reality of Virtual Reality, SPIE vol. 3639, Jan. 1999, pp. 463-470. |
Shi, Jun et al., “Feasibility of controlling prosthetic hand using sonomyography signal in real time: Preliminary study”, Journal of Rehabilitation Research & Development, vol. 47, No. 2, 2010, pp. 87-97. |
Sikdar, Siddhartha et al., “Novel Method for Predicting Dexterous Individual Finger Movements by Imaging Muscle Activity Using a Wearable Ultrasonic System”, IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 22, No. 1, Jan. 2014, pp. 69-76. |
Sueda, Shinjiro et al., “Musculotendon Simulation for Hand Animation”, ACM Transactions on Graphics (TOG)—Proceedings of ACM SIGGRAPH 2008, vol. 27 Issue 3, Article No. 83, vol. 27 Issue 3, Aug. 2008, 8 pp. total. |
Szabo, Thomas L., “Diagnostic Ultrasound Imaging: Inside Out, Second Edition”, Elsevier Inc., 2013, 829 pp. total. |
Van Den Branden Lambrecht, Christian J., “A Working Spatio-Temporal Model of the Human Visual System for Image Restoration and Quality Assessment Applications”, ICASSP-96, Conference Proceedings of the 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing, May 1996, 4 pp. total. |
Watson, Andrew B., “The search for optimal visual stimuli”, Vision Research, vol. 38, 1998, pp. 1619-1621. |
Watson, Andrew B., “The Spatial Standard Observer: A Human Visual Model for Display Inspection”, Society for Information Display, SID 06 Digest, Jun. 2006, pp. 1312-1315. |
Watson, Andrew B., “Visual detection of spatial contrast patterns: Evaluation of five simple models”, Optics Express, vol. 6, No. 1, Jan. 3, 2000, pp. 12-33. |
Watson, Andrew B. et al., “Model of human visual-motion sensing”, Journal of the Optical Society of America A, vol. 2, No. 2, Feb. 1985, pp. 322-342. |
Natson, Andrew B. et al., “Model of visual contrast gain control and pattern masking”, Journal of the Optical Society of America A, vol. 14, No. 9, Sep. 1997, pp. 2379-2391. |
Wikipedia, “List of refractive indices”, https://en.wikipedia.org/wiki/List_of_refractive_indices, Dec. 7, 2003, 5 pp. total. |
Williams III, T. W., “Progress on stabilizing and controlling powered upper-limb prostheses”, Journal of Rehabilitation Research & Development, Guest Editorial, vol. 48, No. 6, 2011, pp. ix-xix. |
Willis, Kari D. et al., “MotionBeam: A Metaphor for Character Interaction with Handheld Projectors”, CHI '11 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, May 7-12, 2011, pp. 1031-1040. |
Yun, Xiaoping et al., “Design, Implementation, and Experimental Results of a Quaternion-Based Kalman Filter for Human Body Motion Tracking”, IEEE Transactions on Robotics, vol. 22, No. 6, Dec. 2006, pp. 1216-1227. |
Zhang, Cha et al., “Maximum Likelihood Sound Source Localization and Beamforming for Directional Microphone Arrays in Distributed Meetings”, IEEE Transactions on Multimedia, vol. 10, No. 3, Apr. 2008, pp. 538-548. |
Final Office Action for Taiwan Patent Application No. 110107404, dated Apr. 8, 2022, 3 pages. |
First Office Action for China Patent Application No. 201680082778.9, dated Mar. 5, 2021, 16 pages. |
Notice of Allowance for Chinese Patent Application No. 201680082778.9, dated Jan. 7, 2022, 2 pages. |
Office Action for Taiwan Patent Application No. 110107404, dated Sep. 1, 2021, 4 pages. |
Number | Date | Country | |
---|---|---|---|
20200209646 A1 | Jul 2020 | US |
Number | Date | Country | |
---|---|---|---|
62271637 | Dec 2015 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 15391583 | Dec 2016 | US |
Child | 16775878 | US |