Patent Grant
10097780
The present application relates generally to the field of optical and electronic systems and methods, and methods of making and using the devices and systems.
Mobile devices are widely employed for communications, computing, and social media. They include camera systems—often at least two, a rear-facing and front-facing—for the capture of images. These camera systems can also be regarded as input devices for the communication between the user of the mobile device, and the mobile device's software and systems. One means of communication these camera systems can enable is recognition of gestures in space and time implemented by the user of the mobile device.
The consumption of electrical power by mobile devices is an object of continued focus, with the goal being to minimize power and, for a given battery, extend the operating lifetime of the device between required charges.
Various embodiments described herein provide a means of utilizing a camera system as an image-capture device, and also as gesture-recognition device, while minimizing power consumption.
In example embodiments, the imaging array region operates at power only when being used in image capture commensurate with its resolution. During periods of time when gesture recognition, or proximity detection, is desired, power is minimized by providing power only to sensing regions.
In embodiments, the sensing regions reside on the same integrated circuit as the imaging array.
In embodiments, the sensing regions are read using circuitry on the same integrated circuit, but providing for much lower power operation than if the image array region circuitry were being employed.
In embodiments, the sensing regions are read using circuitry provided off-chip. In such embodiments, the sensing regions could be termed passive: they consist of, for example, light-absorbing materials, clad by electrical contacts, where electrically conductive materials provide for the conveyance of electrical signals for subsequent readout using separate low-power circuits.
Depicted in
As depicted in
As a result, Regions 2, 3, 4, 5 will be exposed to illumination that is similarly in-focus as the illumination onto Region 1.
In embodiments, Regions 2, 3, 4, 5 may be employed as light sensors. In embodiments, Regions 2, 3, 4, 5 may be connected to on-chip electronic circuitry that provides for the digitization of levels proportional with the illumination over a period of time. In embodiments, these levels may be reported as a digital number DN.
In alternate embodiments, Regions 2, 3, 4, 5 collect photocurrent into on-chip charge stores that may be read using off-chip electronics.
In further alternate embodiments, Regions 2, 3, 4, 5 collect photocurrent that is carried off-chip for reading using off-chip circuitry.
In embodiments, light sensing using at least one of Regions 2, 3, 4, 5 can be provided while little or no electrical power is drawn by Region 1. As a result, a low-power operating mode may be provided wherein the light projected from a scene is registered using the Regions 2, 3, 4, 5.
In an embodiment, an illumination source may be employed to illuminate a scene; and Regions 2, 3, 4, 5 may be read for their DN, charge, or photocurrent levels to register light returning from the illuminated scene.
In various embodiments, IR sub arrays could be the same or larger pixels, and may use true GS Positioned in both examples to have an increased or maximum parallax error from its opposite.
Referring to
In embodiments, multiple, independently-powered light sensing regions are illuminated using a single optic. For example, the image sensor array represents a first independently-powered light sensing region. The at least one peripheral light sensors consume an electrical power that is substantially independent of the electrical power of the image sensor array.
In embodiments, a specialized IR filter could be employed where its efficiency of filtering IR can drop off as incident angles increase, allowing the IR sub arrays to receive more IR photons, while keeping the visible main array 1 shielded from IR. Some of the visible sensor's corners may be exposed to IR, but could be image processed out, especially since corner artifacts are expected anyways (such as lens shading). In the
In various embodiments, IR sub arrays could be the same or larger pixels, and may use true GS Positioned in both examples to have an increased or maximum parallax error from its opposite.
In an embodiment, an infrared illumination source may be employed to illuminate a scene. Imaging array region 1 will principally register visible light; and Regions 2, 3, 4, 5 will receive the sum of visible and infrared light, and may be read for their DN, charge, or photocurrent levels to register light returning from the illuminated scene. In embodiments, the image sensor circuit may include alignment marks that define a spatial relationship to the sensing array 1, facilitating alignment of the infrared-blocking element placement during module assembly.
In embodiments, the infrared filter may be provided that substantially overlies the array region 1, and offers a substantially rectangular shape similar to, and aligned with, region 1. In this embodiment, the regions outside the rectangular array region are provided with infrared light, whereas the rectangular array region receives principally visible light for imaging. In embodiments, a filter is provided that substantially removes infrared light from the image sensor plane over at least the dimensions of the image sensor array rectangle; and that allows infrared light to reach the image sensor surface in at least one of the peripheral image sensing regions.
In embodiments, the optical system used to collect and focus light onto the imaging array 1 may have a first field of view, such as for example 120°; yet it may be desired that the image sensors register light from a smaller field of view. In this embodiment, angularly-selective elements may be added on top of the sensing regions 2, 3, 4, 5. An example includes an aperture that protrudes vertically from the surface of the integrated circuit; and that ensures that light is collected only over a defined field of view. In embodiments, the image sensor circuit may include alignment marks that define a spatial relationship to the sensing regions 2, 3, 4, 5; facilitating alignment of the angularly-select element placement during module assembly.
In embodiments, image array regions and light sensing regions such as those may be accomplished by: Forming an integrated circuit; monolithically integrating an optically sensitive layer atop the integrated circuit to provide light sensing in the image array regions and the light sensing regions.
In embodiments, image array regions and light sensing regions such as those may be accomplished by: Forming an integrated circuit; monolithically integrating an optically sensitive layer atop the integrated circuit, and patterning it such that the originally-formed integrated circuit is no longer covered by the light-sensing layer, and its light sensing is provided by the originally-formed integrated circuit; while leaving the optically-sensitive material in the light sensing regions to provide light sensing.
In embodiments, image array regions and light sensing regions such as those may be accomplished by: Forming an integrated circuit; heterogeneously integrating light sensing materials or devices atop regions of the integrated circuit; where the light sensing materials or devices do not substantially disrupt image formation on the image array region; and where they provide additional light sensing capabilities such as those described herein.
In embodiments, a single class of monolithically-integrated optically sensitive layer may overlie portions of the integrated circuit. In regions of the integrated circuit where the optically sensitive layer overlies a pixelated region of pixel electrodes, the optically sensitive layer may function as the light absorber associated with numerous pixel regions, and the array of pixelated regions taken together with the optically sensitive layer may constitute an image sensor region portion of the integrated circuit. In regions of the integrated circuit where the optically sensitive layer overlies at least one large-area electrode, the large-area electrode combined with the optically sensitive layer may constitute a passive light sensor. In embodiments the passive light sensor may be used in a first low-power mode to sense a state or a gesture. In embodiments the image sensor portion of the integrated circuit may be employed in a second imaging mode in which the image sensor circuit portion is biased and operating, typically consuming at least 100 times more power than when operating in the first mode. An example embodiment is show in
In embodiments, two distinct classes of monolithically-integrated optically sensitive layer may overlie portions of the integrated circuit. In regions of the integrated circuit where an optically sensitive layer overlies a pixelated region of pixel electrodes, the optically sensitive layer may function as the light absorber associated with numerous pixel regions, and the array of pixelated regions taken together with the optically sensitive layer may constitute an image sensor region portion of the integrated circuit. In an example embodiments in which the image sensor array is required to acquire visible-spectrum images, the optically sensitive layer overlying the image sensor region may be designed to sense visible light with high sensitivity. For example, the optically sensitive material in this image sensor array region may be a strong absorber of visible wavelengths of light in the range 480 nm to 630 nm, but it may be substantially transmissive to light (nonabsorbing of light) in for example the near infrared (NIR) region beyond 830 nm. In regions of the integrated circuit where an optically sensitive layer overlies at least one large-area electrode, the large-area electrode combined with the optically sensitive layer may constitute a passive light sensor. In an example embodiment in which the peripheral sensors are required to acquire infrared-spectrum light intensity related information, the optically sensitive layer overlying the peripheral regions may be designed to sense infrared light with high sensitivity. For example, the optically sensitive material in this peripheral sensor region may be a strong absorber of near infrared light in the spectral range 800 nm to 850 nm. In another embodiment, the optically sensitive material in this peripheral sensor region may be a strong absorber of near infrared light in the spectral range 900 nm to 950 nm. In embodiments the passive light sensor may be used in a first low-power mode to sense a state or a gesture. In embodiments the image sensor portion of the integrated circuit may be employed in a second imaging mode in which the image sensor circuit portion is biased and operating, typically consuming at least 100 times more power than when operating in the first mode. An example embodiment is show in
In prior art, an image sensor array region is chiefly responsible for light absorption and light sensing; and an outer peripheral area beyond the typically rectangular image array is not utilized for light sensing, but instead is primarily employed in electronics. In cases the peripheral-to-the-rectangular-array region may be shielded from light impinging upon optically sensitive material using a light-obscuring material such as a metal or other substantially opaque material.
In embodiments of the disclosed subject matter, the region peripheral to the typically-rectangular image array region may be employed in light sensing. Optically sensitive material in the peripheral region may be disposed for highly efficient light absorption, as opposed to being obscured using substantially opaque material. In embodiments, the peripheral region may contain both analog, mixed-signal (such as an analog-to-digital converter), and/or digital electronic circuitry; and may also include optically sensitive material for light sensing. In embodiments, in the peripheral region, an optically sensitive layer may overlie electronic circuitry, and at least a portion of the electronic circuitry in the peripheral region may perform functions that are not directly related to the performance of the light sensing function implemented by the overlying optically sensitive material. For example, a portion of the electronic circuitry may be devoted to managing and processing signals and digital information related to the image sensor array; while at least a portion of the optically sensitive material overlying the electronic circuitry may be devoted to sensing light from the peripheral (as distinct from the imaging array) region.
Referring now to
In embodiments, photoelectrons generated in the optically sensitive layer 305 are collected by the electrodes 301 while photoholes are collected by common top electrode 308. The currents flowing through the multiple electrodes 301 and vias 302 are aggregated in the common contact 304. The current may be monitored using a current-monitoring circuit that is connected to 304 and/or to 308, for these two contacts will collect photocurrents of similar amplitudes. The photocurrent is related to the rate of absorption of photons in the region of optically sensitive layer 305 that overlies the set of electrodes 301 (shown both in the top view of
Referring again to
In embodiments in which electrons are to be collected using 304, electrode 304 can be positioned at, for example, +2 V, and electrode 308 can be positioned at, for example, −1 V. Ranges are possible, such as electrode 304 may be biased in the range of 1 V to 3 V, and electrode 308 may be biased in the range of −3 V to −1 V. Entirely positive bias ranges are possible, such as electrode 308 biased to ground, for example, 0 V, and electrode 304 biased to voltages in the range of, for example, +2 V to +4 V.
In embodiments in which electrons are to be collected using the electrode 308, the electrode 308 can be positioned at +2 V, and the electrode 304 can be positioned at −1 V. Ranges are possible, such as the electrode 308 being biased in the range of, for example, 1 V to 3 V, and the electrode 304 being biased in the range of, for example, −3 V to −1 V. Entirely positive bias ranges are possible, such as the electrode 304 being biased to ground, for example, 0 V, and the electrode 308 being biased to voltages in the range of, for example, +2 V to +4 V.
Referring now to
With continuing reference to
In the embodiment of
In the embodiment of
In an example embodiment, ground (e.g., 0 V) may be provided to the top contact 308, and a positive potential may be provided by an off-chip source (such as the ISP) to the pixel electrodes 304. In an example embodiment, the light sensors function as purely passive devices, that is, they do not consume appreciable power from the image sensor circuit. In an example embodiment, the light sensor consumes power from an external device or module, such as an ISP, but does not consume appreciable power from the image sensor integrated circuit on which the light sensing film 305 of
In example embodiments, the optically sensitive material 305 of
In embodiments, an image sensor integrated circuit comprises at least a first light sensing region comprised of small and independent pixels, and a second light sensing region comprised of larger pixels, or highly binned (current-combined) regions. For example, in the image sensor array region, pixels may have 1.1 μm×1.1 μm dimensions. 0.7 μm×0.7 μm dimensions, or 0.9 μm×0.9 μm dimensions; and peripheral light sensing regions may aggregate current over a much larger area, such as 10 μm×10 μm, or 100 μm×100 μm. In embodiments, the ratio of the area of the peripheral light sensing regions to the size of the independent pixels in the image array region may exceed 100:1. The use of large areas for light sensing in peripheral regions may facilitate the use of simple, low-cost, and/or off-integrated-circuit biasing and current metering. Embodiments of the disclosed subject matter herein include image sensing circuits that comprise, on a single semiconductor die, both an image sensor array, and also at least one independent light sensor powered and read using an independent circuit.
An example embodiment of low power light sensing system comprises a photosensor able to absorb photons and thereby produce a photocurrent that is related to incident light intensity. The photosensor may be made using the photosensitive material as that which comprises the image sensing region. In alternative embodiments, it may employ a photosensitive that is distinct from that used in the image sensing region.
An example implementation of a low power light sensing system comprises a photosensor that absorbs photons and produces a current that varies in relation with incident light intensity. The photosensor can be made using the same photosensitive material that comprises the imager pixel array. Alternatively, dedicated regions of photosensitive material and constitute photosensor regions. These can potentially be outside of the main image sensing region.
In an example embodiment, a low-power mode may be provided in which the circuitry associated with the image sensing array region are powered down. In embodiments, readout circuits may be powered down with the exception of the pixel reset transistor driver circuit, which will provide the signal necessary to keep pixels in reset (e.g., a Global Reset state).
In embodiments, the photosensor regions are constantly connected to a reset supply. An example embodiment of a reset supply may be on-chip internally generated.
An example embodiment of a reset supply may be externally generated. The photosensor regions can reside directly atop regions of readout circuitry that are associated not with reading the photosensor, but instead with performing other chip functions. Other chip functions may include the readout of at least one pixel comprising the image sensing array region.
In embodiments, a common optically sensitive material may overlie both the image sensing regions, and also the photosensing regions. In embodiments, the optically sensitive layer is in turn covered with a biasing electrode. In embodiments, this biasing electrode is driven using a voltage source generated on-chip. In embodiments, this biasing electrode is driven using an externally provided voltage source. The photosensing regions are biased to provide an electric potential between the top and bottom (pixel) electrodes. The current flowing through the top contact voltage source is related to the light intensity incident on the photosensing region. Ambient light levels can therefore be monitored by measuring this current.
In embodiments, the photocurrent can be monitored internally using on-chip circuitry. In an alternative embodiment, the photocurrent can be monitored externally using discrete devices.
In embodiments, the pixels or adjacent areas can be subdivided in smaller regions connected to different current monitors to produce independent measurements of spatially distributed areas.
In embodiments, such data can be used to detect motion, implement gesture control, or refine the quality of the ambient light measurement.
In embodiments, sensing is provided via a dedicated voltage supply for the drain of the pixels reset transistor. In embodiments, the corresponding current, i, can be monitored and the variation of current through this supply measured using on-chip or off chip circuitry. In embodiments, a low-power light-sensing mode is implemented by powering down the circuitry associated with the image sensing region; and retaining power to the reset transistor drivers, and, in the case of four transistor (4T) or multiple transistor pixels, also to the transfer gate drivers. The total current through the reset transistors of the pixel array can be monitored through the global reset supply. The current flowing though this voltage source is related to the light impinging on the photosensing region, enabling ambient light to be monitored by measuring this current.
In embodiments such data can be used to detect motion, implement gesture control or refine the quality of the ambient light measurement.
Referring to
Image sensors incorporate arrays of photodetectors. These photodetectors sense light, converting it from an optical to an electronic signal.
The QDPC 100 detects electromagnetic radiation 1000, which in embodiments may be any frequency of radiation from the electromagnetic spectrum. Although the electromagnetic spectrum is continuous, it is common to refer to ranges of frequencies as bands within the entire electromagnetic spectrum, such as the radio band, microwave band, infrared band (IR), visible band (VIS), ultraviolet band (UV), X-rays, gamma rays, and the like. The QDPC 100 may be capable of sensing any frequency within the entire electromagnetic spectrum; however, embodiments herein may reference certain bands or combinations of bands within the electromagnetic spectrum. It should be understood that the use of these bands in discussion is not meant to limit the range of frequencies that the QDPC 100 may sense, and are only used as examples. Additionally, some bands have common usage sub-bands, such as near infrared (NIR) and far infrared (FIR), and the use of the broader band term, such as IR, is not meant to limit the QDPCs 100 sensitivity to any band or sub-band. Additionally, in the following description, terms such as “electromagnetic radiation,” “radiation,” “electromagnetic spectrum,” “spectrum,” “radiation spectrum,” and the like are used interchangeably, and the term color is used to depict a select band of radiation 1000 that could be within any portion of the radiation 1000 spectrum, and is not meant to be limited to any specific range of radiation 1000 such as in visible ‘color’.
In the example embodiment of
A quantum dot 1200 may be a nanostructure, typically a semiconductor nanostructure that confines a conduction band electrons, valence band holes, or excitons (bound pairs of conduction band electrons and valence band holes) in all three spatial directions. A quantum dot exhibits in its absorption spectrum the effects of the discrete quantized energy spectrum of an idealized zero-dimensional system. The wave functions that correspond to this discrete energy spectrum are typically substantially spatially localized within the quantum dot, but extend over many periods of the crystal lattice of the material.
Some embodiments of the QD optical devices are single image sensor chips that have a plurality of pixels, each of which includes a QD layer that is radiation 1000 sensitive, e.g., optically active, and at least two electrodes in electrical communication with the QD layer. The current and/or voltage between the electrodes is related to the amount of radiation 1000 received by the QD layer. Specifically, photons absorbed by the QD layer generate electron-hole pairs, such that, if an electrical bias is applied, a current flows. By determining the current and/or voltage for each pixel, the image across the chip can be reconstructed. The image sensor chips have a high sensitivity, which can be beneficial in low-radiation-detecting 1000 applications; a wide dynamic range allowing for excellent image detail; and a small pixel size. The responsivity of the sensor chips to different optical wavelengths is also tunable by changing the size of the QDs in the device, by taking advantage of the quantum size effects in QDs. The pixels can be made as small as 1 square micron or less, or as large as 30 microns by 30 microns or more or any range subsumed therein.
The photodetector structure 1400 of
In embodiments, processing may include binning of pixels in order to reduce random noise associated with inherent properties of the quantum dot structure 1100 or with readout processes. Binning may involve the combining of pixels 1800, such as creating 2×2, 3×3, 5×5, or the like superpixels. There may be a reduction of noise associated with combining pixels 1800, or binning, because the random noise increases by the square root as area increases linearly, thus decreasing the noise or increasing the effective sensitivity. With the QDPC's 100 potential for very small pixels, binning may be utilized without the need to sacrifice spatial resolution, that is, the pixels may be so small to begin with that combining pixels doesn't decrease the required spatial resolution of the system. Binning may also be effective in increasing the speed with which the detector can be run, thus improving some feature of the system, such as focus or exposure. In example embodiments, binning may be used to combine subpixel elements for the same color or range of radiation (including UV and/or IR) to provide separate elements for a superpixel while preserving color/UV/IR resolution as further described below.
In embodiments the chip may have functional components that enable high-speed readout capabilities, which may facilitate the readout of large arrays, such as 5 Mpixels, 6 Mpixels, 8 Mpixels, 12 Mpixels, or the like. Faster readout capabilities may require more complex, larger transistor-count circuitry under the pixel 1800 array, increased number of layers, increased number of electrical interconnects, wider interconnection traces, and the like.
In embodiments, it may be desirable to scale down the image sensor size in order to lower total chip cost, which may be proportional to chip area. However, shrinking chip size may mean, for a given number of pixels, smaller pixels. In existing approaches, since radiation 1000 must propagate through the interconnect layer onto the monolithically integrated silicon photodiode lying beneath, there is a fill-factor compromise, whereby part of the underlying silicon area is obscured by interconnect; and, similarly, part of the silicon area is consumed by transistors used in read-out. One workaround is micro-lenses, which add cost and lead to a dependence in photodiode illumination on position within the chip (center vs. edges); another workaround is to go to smaller process geometries, which is costly and particularly challenging within the image sensor process with its custom implants.
In embodiments, the technology discussed herein may provide a way around these compromises. Pixel size, and thus chip size, may be scaled down without decreasing fill factor. Larger process geometries may be used because transistor size, and interconnect line-width, may not obscure pixels since the photodetectors are on the top surface, residing above the interconnect. In the technology proposed herein, large geometries such as 0.13 μm and 0.18 μm may be employed without obscuring pixels. Similarly, small geometries such as 90 nm and below may also be employed, and these may be standard, rather than image-sensor-customized, processes, leading to lower cost. The use of small geometries may be more compatible with high-speed digital signal processing on the same chip. This may lead to faster, cheaper, and/or higher-quality image sensor processing on chip. Also, the use of more advanced geometries for digital signal processing may contribute to lower power consumption for a given degree of image sensor processing functionality.
An example integrated circuit system that can be used in combination with the above photodetectors, pixel regions and pixel circuits will now be described in connection with
In a particular example embodiment, when 0.11 μm CMOS technology node is employed, the periodic repeat distance of pixels along the row-axis and along the column-axis may be 900 nm, 1.1 μm, 1.2 μm, 1.4 μm, 1.75 μm, 2.2 μm, or larger. The implementation of the smallest of these pixels sizes, especially 900 nm, 1.1 μm, and 1.2 μm, may require transistor sharing among pairs or larger group of adjacent pixels in some embodiments.
Very small pixels can be implemented in part because all of the silicon circuit area associated with each pixel can be used for read-out electronics since the optical sensing function is achieved separately, in another vertical level, by the optically-sensitive layer that resides above the interconnect layer.
Because the optically sensitive layer and the read-out circuit that reads a particular region of optically sensitive material exist on separate planes in the integrated circuit, the shape (viewed from the top) of (1) the pixel read-out circuit and (2) the optically sensitive region that is read by (1); can be generally different. For example it may be desired to define an optically sensitive region corresponding to a pixel as a square; whereas the corresponding read-out circuit may be most efficiently configured as a rectangle.
In an imaging array based on a top optically sensitive layer connected through vias to the read-out circuit beneath, there exists no imperative for the various layers of metal, vias, and interconnect dielectric to be substantially or even partially optically transparent, although they may be transparent in some embodiments. This contrasts with the case of front-side-illuminated CMOS image sensors in which a substantially transparent optical path must exist traversing the interconnect stack.
Example embodiments include an image sensor circuit that includes an image sensing region (such as
Embodiments include an integrated circuit comprising a semiconductor substrate; an image sensing region (such as
Pixel circuitry may be defined to include components beginning at the electrodes in contact with the quantum dot material 200 and ending when signals or information is transferred from the pixel to other processing facilities, such as the functional components 2004 of the underlying chip 200 or another quantum dot pixel 1800. Beginning at the electrodes on the quantum dot material 200, the signal is translated or read. In embodiments, the quantum dot material 200 may provide a change in current flow in response to radiation 1000. The quantum dot pixel 1800 may require bias circuitry 1700 in order to produce a readable signal. This signal in turn may then be amplified and selected for readout. One embodiment of a pixel circuit shown in
Embodiments include a mobile device comprising a semiconductor substrate; an image sensing region (
An example of a first camera module 113 is shown to be situated within the peripheral region 101 of the single-plane computing device 100 and is described in further detail, below. Example light sensors 115A, 115B are also shown to be situated within the peripheral region 101 of the single-plane computing device 100 and are described in further detail, below, with reference to
Examples of light sensors 107A, 107B, shown to be situated in the display region 103 of the single-plane computing device 100 and are described in further detail, below, with reference to
In embodiments, the display region 103 may be a touchscreen display. In embodiments, the single-plane computing device 100 may be a tablet computer. In embodiments, the single-plane computing device 100 may be a mobile handset.
The double-plane computing device 200 is also shown to include examples of a first camera module 213A in the first peripheral region 201A and a second camera module 213B in the second peripheral region 201B. The camera modules 213A, 213B are described in more detail, below, with reference to
A number of examples of light sensors 215A, 215B, 215C, 215D, are shown situated within the peripheral regions 201A, 201B of the double-plane computing device 200. Although a total of four light sensors are shown, a person of ordinary skill in the art will recognize that more or fewer light sensors may be employed. Examples of the light sensors 215A, 215B, 215C, 215D, are described, below, in further detail with reference to
The double-plane computing device 200 is also shown to include examples of a first camera module 205A in the first display region 203A and a second camera module 205B in the second display region 203B. The camera modules 205A, 205B are described in more detail, below, with reference to
In embodiments, the double-plane computing device 200 may be a laptop computer. In embodiments, the double-plane computing device 200 may be a mobile handset.
With reference now to
Elements of a either the single-plane computing device 100 of
The light sensor of
In embodiments, the electrodes 403A, 403B, and, in the case of a conductive encapsulant for 407, the top electrode 407, may be constituted using materials that are substantially transparent across the visible wavelength region approximately 450 nm to 650 nm. In this case, the quality of the images displayed using the display region is not substantially compromised by the incorporation of the light-absorbing material along the optical path of the display.
In embodiments, the light sensor of
In an example embodiment, the absorbance of the light-sensing display-incorporated material may lie in the range 2-20% in the visible; and may lie in the range 0.1-5% in the infrared. In an example embodiment, the presence of visible light in the ambient, and/or emitted from the display, may produce a background signal within the light sensor, as a consequence of the material visible-wavelength absorption within the light-absorbing material of the light sensor. In an example embodiment, sensing in the infrared region may also be achieved. The light sources used in aid of gesture recognition may be modulated using spatial, or temporal, codes, allowing them to be distinguished from the visible-wavelength-related component of the signal observed in the light sensor. In an example embodiment, at least one light source used in aid of gesture recognition may be modulated in time using a code having a frequency component greater than 100 Hz, 1000 Hz, 10 kHz, or 100 kHz. In an example embodiment, the light sensor may have a temporal response having a cutoff frequency greater than said frequency components. In embodiments, circuitry may be employed to ensure that the frequency component corresponding to gesture recognition can be extracted and monitored, with the background components related to the room ambient, the display illumination, and other such non-gesture-related background information substantially removed. In this example, the light sensors, even though they absorb both visible and infrared light, can provide a signal that is primarily related to gestural information of interest in gesture recognition.
In an example embodiment, an optical source having a total optical power of approximately 1 mW may be employed. When illuminating an object a distance approximately 10 cm away, where the object has area approximately 1 cm2 and diffuse reflectance approximately 20%, then the amount of power incident on a light sensor having area 1 cm2 may be of order 100 pW. In an example embodiment, a light sensor having absorbance of 1% may be employed, corresponding to a photocurrent related to the light received as a consequence of the illumination via the optical source, and reflected or scattered off of the object, and thus incident onto the light sensor, may therefore be of order pW. In example embodiments, the electrical signal reported by the light sensor may correspond to approximately pA signal component at the modulation frequency of the optical source. In example embodiments, a large additional signal component, such as in the nA or μA range, may arise due to visible and infrared background, display light, etc. In example embodiments, the relatively small signal components, with its distinctive temporal and/or spatial signature as provided by modulation (in time and/or space) of the illumination source, may nevertheless be isolated relative to other background/signal, and may be employed to discern gestural information.
In embodiments, light-absorbing material 405 may consist of a material that principally absorbs infrared light in a certain band; and that is substantially transparent to visible-wavelength light. In an example embodiment, a material such as PBDTT-DPP, the near-infrared light-sensitive polymer poly(2,60-4,8-bis(5-ethylhexylthienyl)benzo-[1,2-b;3,4-b]dithiophene-alt-5-dibutyloctyl-3,6-bis(5-bromothiophen-2-yl)pyrrolo[3,4-c]pyrrole-1,4-dione), may be employed as a component of the light-absorbing layer.
In embodiments, the electronic signal produced by the light sensor may be communicated to a device for electronic amplification. This device may amplify a specific electronic frequency band more than other bands, producing an enhanced signal component that is related to the gestural information. The signal from the light sensor, possibly with the combination of amplification (potentially frequency-dependent), may be input to an analog-to-digital converter that can produce a digital signal related to the gestural information. The digital information related to gestural information can be further conveyed to other integrated circuits and/or signal processing engines in the context of a system. For example, it may be conveyed to an application processor.
In embodiments, optical sources used to illuminate a volume of space, with the goal of enabling gesture recognition, may use illumination at a near infrared wavelength that is substantially unseen by the human eye. In an example embodiment, a light-emitting diode having center wavelength of approximately 950 nm may be employed.
In embodiments, gesture recognition may be accomplished by combining information from at least one camera, embedded into the computing device, and having a lens providing a substantially focused image onto an image sensor that is part of the camera; and may also incorporate sensors in the peripheral region, and/or integrated into the display region. In embodiments, the distributed sensors may provide general information on the spatio-temporal movements of the object being imaged; and the signals from the at least one camera(s) may be combined with the distributed sensors' signals to provide a more spatially-/temporally-accurate picture of the two- or three-dimensional motion of the object whose gesture is to be recognized. In an example embodiment, the camera may employ an image sensor providing a modest spatial resolution, such as QVGA, VGA, SVGA, etc., and thus be implemented using an image sensor having small die size and thus low cost; and also be implemented using a camera module having small x, y, and z form factor, enabling minimal consumption of peripheral region area, and no substantial addition to the z-height of the tablet or other computing device. In embodiments, a moderate frame rate, such as 15 fps, 30 fps, or 60 fps may be employed, which, combined with a modest resolution, enables a low-cost digital communication channel and moderate complexity of signal processing in the recognition of gestures. In embodiments, the at least one camera module may implement wide field of view imaging in order to provide a wide angular range in the assessment of gestures in relation to a display. In embodiments, at least one camera module may be tilted, having its angle of regard nonparallel to the normal direction (perpendicular direction) to the display, enabling the at least one camera to image an angular extent in closer proximity to the display. In embodiments, multiple cameras may be employed in combination, each having an angle of regard distinct from at least one another, thereby enabling gestures in moderate proximity to the display to be imaged and interpreted. In embodiments, the at least one camera may employ an image sensor sensitized using light-detecting materials that provide high quantum efficiency, for example, greater than 30%, at near infrared wavelength used by the illuminating source; this enables reduced requirement for power and/or intensity in the illuminating source. In embodiments, the illuminating source may be modulated in time at a specific frequency and employing a specific temporal pattern (e.g., a series of pulses of known spacing and width in time); and the signal from the at least one camera and/or the at least one distributed sensor may be interpreted with knowledge of the phase and temporal profile of the illuminating source; and in this manner, increased signal-to-noise ratio, akin to lock-in or boxcar-averaging or other filtering and/or analog or digital signal processing methods, may be used to substantially pinpoint the moduled, hence illuminated signal, and substantially remove or minimize the background signal associated with the background scene.
In embodiments, signals received by at least one of (1) the touch-based interface devices; (2) the camera modules; (3) the light sensors, each of these either within the peripheral and/or the display or display/input regions, may be employed and, singly or jointly, used to determine the presence, and the type, of gesture indicated by a user of the device.
Referring again to
In embodiments, at least one camera module has a wide field of view exceeding about 40°. In embodiments, at least one camera module employs a fisheye lens. In embodiments, at least one image sensor achieves higher resolution at its center, and lower resolution in its periphery. In embodiments, at least one image sensor uses smaller pixels near its center and larger pixels near its periphery.
In embodiments, active illumination via at least one light source; combined with partial reflection and/or partial scattering off of a proximate object; combined with light sensing using at least one optical module or light sensor; may be combined to detect proximity to an object. In embodiments, information regarding such proximity may be used to reduce power consumption of the device. In embodiments, power consumption may be reduced by dimming, or turning off, power-consuming components such as a display.
In embodiments, at least one optical source may emit infrared light. In embodiments, at least one optical source may emit infrared light in the near infrared between about 700 nm and about 1100 nm. In embodiments, at least one optical source may emit infrared light in the short-wavelength infrared between about 1100 nm and about 1700 nm wavelength. In embodiments, the light emitted by the optical source is substantially not visible to the user of the device.
In embodiments, at least one optical source may project a structured light image. In embodiments, spatial patterned illumination, combined with imaging, may be employed to estimate the relative distance of objects relative to the imaging system.
In embodiments, at least two lensing systems may be employed to image a scene, or portions of a scene, onto two distinct regions of a monolithically-integrated single image sensor integrated circuit; and the patterns of light thus acquired using the image sensor integrated circuit may be used to aid in estimating the relative or absolute distances of objects relative to the image sensor system.
In embodiments, at least two lensing systems may be employed to image a scene, or portions of a scene, onto two distinct image sensor integrated circuits housed within a single camera system; and the patterns of light thus acquired using the image sensor integrated circuits may be used to aid in estimating the relative or absolute distances of objects relative to the image sensor system.
In embodiments, at least two lensing systems may be employed to image a scene, or portions of a scene, onto two distinct image sensor integrated circuits housed within separate camera systems or subsystems; and the patterns of light thus acquired using the image sensor integrated circuits may be used to aid in estimating the relative or absolute distances of objects relative to the image sensor systems or subsystems.
In embodiments, the different angles of regard, or perspectives, from which the at least two optical systems perceive the scene, may be used to aid in estimating the relative or absolute distances of objects relative to the image sensor system.
In embodiments, light sensors such as the light sensors 115A, 115B situated in the peripheral region 101 of
In embodiments, the different angles of regard, or perspectives, from which the at least two optical systems perceive the scene, may be used to aid in estimating the relative or absolute distances of objects relative to the image sensor system.
In embodiments, the time sequence of light detector from at least two light sensors may be used to estimate the direction and velocity of an object. In embodiments, the time sequence of light detector from at least two light sensors may be used to ascertain that a gesture was made by a user of a computing device. In embodiments, the time sequence of light detector from at least two light sensors may be used to classify the gesture that was made by a user of a computing device. In embodiments, information regarding the classification of a gesture, as well as the estimated occurrence in time of the classified gesture, may be conveyed to other systems or subsystems within a computing device, including to a processing unit.
In embodiments, light sensors may be integrated into the display region of a computing device, for example, the light sensors 107A, 107B of
In embodiments, structured light of a first type may be employed, and of a second type may also be employed, and the information from the at least two structured light illuminations may be usefully combined to ascertain information regarding a scene that exceeds the information contained in either isolated structured light image.
In embodiments, structured light of a first type may be employed to illuminate a scene and may be presented from a first source providing a first angle of illumination; and structured light of a second type may be employed to illuminate a scene and may be presented from a second source providing a second angle of illumination.
In embodiments, structured light of a first type and a first angle of illumination may be sensed using a first image sensor providing a first angle of sensing; and also using a second image sensor providing a second angle of sensing.
In embodiments, structured light having a first pattern may be presented from a first source; and structured light having a second pattern may be presented from a second source.
In embodiments, structured light having a first pattern may be presented from a source during a first time period; and structured light having a second pattern may be presented from a source during a second time period.
In embodiments, structured light of a first wavelength may be used to illuminate a scene from a first source having a first angle of illumination; and structured light of a second wavelength may be used to illuminate a scene from a second source having a second angle of illumination.
In embodiments, structured light of a first wavelength may be used to illuminate a scene using a first pattern; and structured light of a second wavelength may be used to illuminate a scene using a second pattern. In embodiments, a first image sensor may sense the scene with a strong response at the first wavelength and a weak response at the second wavelength; and a second image sensor may sense the scene with a strong response at the second wavelength and a weak response at the first wavelength. In embodiments, an image sensor may consist of a first class of pixels having strong response at the first wavelength and weak response at the second wavelength; and of a second class of pixels having strong response at the second wavelength and weak response at the first wavelength.
Embodiments include image sensor systems that employ a filter having a first bandpass spectral region; a first bandblock spectral region; and a second bandpass spectral region. Embodiments include the first bandpass region corresponding to the visible spectral region; the first bandblock spectral region corresponding to a first portion of the infrared; and the second bandpass spectral region corresponding to a second portion of the infrared. Embodiments include using a first time period to detect primarily the visible-wavelength scene; and using active illumination within the second bandpass region during a second time period to detect the sum of a visible-wavelength scene and an actively-illuminated infrared scene; and using the difference between images acquired during the two time periods to infer a primarily actively-illuminated infrared scene. Embodiments include using structured light during the second time period. Embodiments include using infrared structured light. Embodiments include using the structured light images to infer depth information regarding the scene; and in tagging, or manipulating, the visible images using information regarding depth acquired based on the structured light images.
In embodiments, gestures inferred may include one-thumb-up; two-thumbs-up; a finger swipe; a two-finger swipe; a three-finger swipe; a four-finger-swipe; a thumb plus one finger swipe; a thumb plus two finger swipe; etc. In embodiments, gestures inferred may include movement of a first digit in a first direction; and of a second digit in a substantially opposite direction. Gestures inferred may include a tickle.
Sensing of the intensity of light incident on an object may be employed in a number of applications. One such application includes estimation of ambient light levels incident upon an object so that the object's own light-emission intensity can be suitable selected. In mobile devices such as cell phones, personal digital assistants, smart phones, and the like, the battery life, and thus the reduction of the consumption of power, are of importance. At the same time, the visual display of information, such as through the use of a display such as those based on LCDs or pixellated LEDs, may also be needed. The intensity with which this visual information is displayed depends at least partially on the ambient illumination of the scene. For example, in very bright ambient lighting, more light intensity generally needs to be emitted by the display in order for the display's visual impression or image to be clearly visible above the background light level. When ambient lighting is weaker, it is feasible to consume less battery power by emitting a lower level of light from the display.
As a result, it is of interest to sense the light level near or in the display region. Existing methods of light sensing often include a single, or a very few, light sensors, often of small area. This can lead to undesired anomalies and errors in the estimation of ambient illumination levels, especially when the ambient illumination of the device of interest is spatially inhomogeneous. For example, shadows due to obscuring or partially obscuring objects may—if they obscure one or a few sensing elements—result in a display intensity that is less bright than desirable under the true average lighting conditions.
Embodiments include realization of a sensor, or sensors, that accurately permit the determination of light levels. Embodiments include at least one sensor realized using solution-processed light-absorbing materials. Embodiments include sensors in which colloidal quantum dot films constitute the primary light-absorbing element. Embodiments include systems for the conveyance of signals relating to the light level impinging on the sensor that reduce, or mitigate, the presence of noise in the signal as it travels over a distance between a passive sensor and active electronics that employ the modulation of electrical signals used in transduction. Embodiments include systems that include (1) the light-absorbing sensing element; (2) electrical interconnect for the conveyance of signals relating to the light intensity impinging upon the sensing element; and (3) circuitry that is remote from the light-absorbing sensing element, and is connected to it via the electrical interconnect, that achieves low-noise conveyance of the sensed signal through the electrical interconnect. Embodiments include systems in which the length of interconnect is more than one centimeter in length. Embodiments include systems in which interconnect does not require special shielding yet achieve practically useful signal-to-noise levels.
Embodiments include sensors, or sensor systems, that are employed, singly or in combination, to estimate the average color temperature illuminating the display region of a computing device. Embodiments include sensors, or sensor systems, that accept light from a wide angular range, such as greater than about ±20° to normal incidence, or greater than about ±30° to normal incidence, or greater than about ±40° to normal incidence. Embodiments include sensors, or sensor systems, that include at least two types of optical filters, a first type passing primarily a first spectral band, a second type passing primarily a second spectral band. Embodiments include using information from at least two sensors employing at least two types of optical filters to estimate color temperature illuminating the display region, or a region proximate the display region.
Embodiments include systems employing at least two types of sensors. Embodiments include a first type constituted of a first light-sensing material, and a second type constituted of a second light-sensing material. Embodiments include a first light-sensing material configured to absorb, and transduce, light in a first spectral band, and a second light-sensing material configured to transduce a second spectral band. Embodiments include a first light-sensing material employing a plurality of nanoparticles having a first average diameter, and a second light-sensing material employing a plurality of nanoparticles have a second average diameter. Embodiments include a first diameter in the range of approximately 1 nm to approximately 2 nm, and a second diameter greater than about 2 nm.
Embodiments include methods of incorporating a light-sensing material into, or onto, a computing device involving ink-jet printing. Embodiments include using a nozzle to apply light-sensing material over a defined region. Embodiments include defining a primary light-sensing region using electrodes. Embodiments include methods of fabricating light sensing devices integrated into, or onto, a computing device involving: defining a first electrode; defining a second electrode; defining a light-sensing region in electrical communication with the first and the second electrode. Embodiments include methods of fabricating light sensing devices integrated into, or onto, a computing device involving: defining a first electrode; defining a light-sensing region; and defining a second electrode; where the light sensing region is in electrical communication with the first and the second electrode.
Embodiments include integration at least two types of sensors into, or onto, a computing device, using ink-jet printing. Embodiments include using a first reservoir containing a first light-sensing material configured to absorb, and transduce, light in a first spectral band; and using a second reservoir containing a second light-sensing material configured to absorb, and transduce, light in a second spectral band.
Embodiments include the use of differential or modulated signaling in order to substantially suppress any external interference. Embodiments include subtracting dark background noise.
Embodiments include a differential system depicted in
Embodiments include the use of a three-electrode system as follows. Each electrode consists of a metal wire. Light-absorbing material may be in electrical communication with the metal wires. Embodiments include the encapsulation of the light-absorbing material using a substantially transparent material that protects the light-absorbing material from ambient environmental conditions such as air, water, humidity, dust, and dirt. The middle of the three electrodes may be biased to a voltage V1, where an example of a typical voltage is about 0 V. The two outer electrodes may be biased to a voltage V2, where a typical value is about 3 V. Embodiments include covering a portion of the device using light-obscuring material that substantially prevents, or reduces, the incidence of light on the light-sensing material.
The light-obscuring material ensures that one pair of electrodes sees little or no light. This pair is termed the dark, or reference, electrode pair. The use of a transparent material over the other electrode pair ensures that, if light is incident, it is substantially incident upon the light-sensing material. This pair is termed the light electrode pair.
The difference in the current flowing through the light electrode pair and the dark electrode pair is equal to the photocurrent—that is, this difference does not include any dark current, but instead is proportional to the light intensity, with any dark offset substantially removed.
In embodiments, these electrodes are wired in twisted-pair form. In this manner, common-mode noise from external sources is reduced or mitigated. Referring to
In another embodiment, biasing may be used such that the light-obscuring layer may not be required. The three electrodes may be biased to three voltages V1, V2, and V3. In one example, V1=6 V, V2=3 V, V3=0 V. The light sensor between 6 V and 3 V, and that between 0 V and 3 V, will generate opposite-direction currents when read between the 6 V and 0 V. The resultant differential signal is then transferred out in twisted-pair fashion.
In embodiments, the electrode layout may itself be twisted, further improving the noise-resistance inside the sensor. In this case, an architecture is used in which an electrode may cross over another.
In embodiments, electrical bias modulation may be employed. An alternating bias may be used between a pair of electrodes. The photocurrent that flows will substantially mimic the temporal evolution of the time-varying electrical biasing. Readout strategies include filtering to generate a low-noise electrical signal. The temporal variations in the biasing include sinusoidal, square, or other periodic profiles. For example, referring to
Embodiments include combining the differential layout strategy with the modulation strategy to achieve further improvements in signal-to-noise levels.
Embodiments include employing a number of sensors having different shapes, sizes, and spectral response (e.g., sensitivities to different colors). Embodiments include generating multi-level output signals. Embodiments include processing signals using suitable circuits and algorithms to reconstruct information about the spectral and/or other properties of the light incident.
Advantages of the disclosed subject matter include transfer of accurate information about light intensity over longer distances than would otherwise be possible. Advantages include detection of lower levels of light as a result. Advantages include sensing a wider range of possible light levels. Advantages include successful light intensity determination over a wider range of temperatures, an advantage especially conferred when the dark reference is subtracted using the differential methods described herein.
Embodiments include a light sensor including a first electrode, a second electrode, and a third electrode. A light-absorbing semiconductor is in electrical communication with each of the first, second, and third electrodes. A light-obscuring material substantially attenuates the incidence of light onto the portion of light-absorbing semiconductor residing between the second and the third electrodes, where an electrical bias is applied between the second electrode and the first and third electrodes and where the current flowing through the second electrode is related to the light incident on the sensor.
Embodiments include a light sensor including a first electrode, a second electrode, and a light-absorbing semiconductor in electrical communication with the electrodes wherein a time-varying electrical bias is applied between the first and second electrodes and wherein the current flowing between the electrodes is filtered according to the time-varying electrical bias profile, wherein the resultant component of current is related to the light incident on the sensor.
Embodiments include the above embodiments where the first, second, and third electrodes consists of a material chosen from the list: gold, platinum, palladium, silver, magnesium, manganese, tungsten, titanium, titanium nitride, titanium dioxide, titanium oxynitride, aluminum, calcium, and lead.
Embodiments include the above embodiments where the light-absorbing semiconductor includes materials taken from the list: PbS, PbSe, PbTe, SnS, SnSe, SnTe, CdS, CdSe, CdTe, Bi2S3, In2S3, In2S3, In2Te3, ZnS, ZnSe, ZnTe, Si, Ge, GaAs, polypyrolle, pentacene, polyphenylenevinylene, polyhexylthiophene, and phenyl-C61-butyric acid methyl ester.
Embodiments include the above embodiments where the bias voltages are greater than about 0.1 V and less than about 10 V. Embodiments include the above embodiments where the electrodes are spaced a distance between about 1 μm and about 20 μm from one another.
Embodiments include the above embodiments where the distance between the light-sensing region and active circuitry used in biasing and reading is greater than about 1 cm and less than about 30 cm.
The capture of visual information regarding a scene, such as via imaging, is desired in a range of areas of application. In cases, the optical properties of the medium residing between the imaging system, and the scene of interest, may exhibit optical absorption, optical scattering, or both. In cases, the optical absorption and/or optical scattering may occur more strongly in a first spectral range compared to a second spectral range. In cases, the strongly-absorbing-or-scattering first spectral range may include some or all of the visible spectral range of approximately 470 nm to approximately 630 nm, and the more-weakly-absorbing-or-scattering second spectral range may include portions of the infrared spanning a range of approximately 650 nm to approximately 24 μm wavelengths.
In embodiments, image quality may be augmented by providing an image sensor array having sensitivity to wavelengths longer than about a 650 nm wavelength.
In embodiments, an imaging system may operate in two modes: a first mode for visible-wavelength imaging; and a second mode for infrared imaging. In embodiments, the first mode may employ a filter that substantially blocks the incidence of light of some infrared wavelengths onto the image sensor.
Referring now to
In embodiments, an imaging system may operate in two modes: a first mode for visible-wavelength imaging; and a second mode for infrared imaging. In embodiments, the system may employ an optical filter, which remains in place in each of the two modes, that substantially blocks incidence of light over a first infrared spectral band; and that substantially passes incidence of light over a second infrared spectral band. In embodiments, the first infrared spectral band that is blocked may span from about 700 nm to about 1450 nm. In embodiments, the second infrared spectral band that is substantially not blocked may begin at about 1450 nm. In embodiments, the second infrared spectral band that is substantially not blocked may end at about 1600 nm. In embodiments, in the second mode for infrared imaging, active illuminating that includes power in the second infrared spectral band that is substantially not blocked may be employed. In embodiments, a substantially visible-wavelength image may be acquired via image capture in the first mode. In embodiments, a substantially actively-infrared-illuminated image may be acquired via image capture in the second mode. In embodiments, a substantially actively-infrared-illuminated image may be acquired via image capture in the second mode aided by the subtraction of an image acquired during the first mode. In embodiments, a periodic-in-time alternation between the first mode and second mode may be employed. In embodiments, a periodic-in-time alternation between no-infrared-illumination, and active-infrared-illumination, may be employed. In embodiments, a periodic-in-time alternation between reporting a substantially visible-wavelength image, and reporting a substantially actively-illuminated-infrared image, may be employed. In embodiments, a composite image may be generated which displays, in overlaid fashion, information relating to the visible-wavelength image and the infrared-wavelength image. In embodiments, a composite image may be generated which uses a first visible-wavelength color, such as blue, to represent the visible-wavelength image; and uses a second visible-wavelength color, such as red, to represent the actively-illuminated infrared-wavelength image, in a manner that is overlaid.
In image sensors, a nonzero, nonuniform, image may be present even in the absence of illumination, (in the dark). If not accounted for, the dark images can lead to distortion and noise in the presentation of illuminated images.
In embodiments, an image may be acquired that represents the signal present in the dark. In embodiments, an image may be presented at the output of an imaging system that represents the difference between an illuminated image and the dark image. In embodiments, the dark image may be acquired by using electrical biasing to reduce the sensitivity of the image sensor to light. In embodiments, an image sensor system may employ a first time interval, with a first biasing scheme, to acquire a substantially dark image; and a second time interval, with a second biasing scheme, to acquire a light image. In embodiments, the image sensor system may store the substantially dark image in memory; and may use the stored substantially dark image in presenting an image that represents the difference between a light image and a substantially dark image. Embodiments include reducing distortion, and reducing noise, using the method.
In embodiments, a first image may be acquired that represents the signal present following reset; and a second image may be acquired that represents the signal present following an integration time; and an image may be presented that represents the difference between the two images. In embodiments, memory may be employed to store at least one of two of the input images. In embodiments, the result difference image may provide temporal noise characteristics that are consistent with correlated double-sampling noise. In embodiments, an image may be presented having equivalent temporal noise considerable less than that imposed by sqrt(kTC) noise.
Embodiments include high-speed readout of a dark image; and of a light image; and high-speed access to memory and high-speed image processing; to present a dark-subtracted image to a user rapidly.
Embodiments include a camera system in which the interval between the user indicating that an image is to be acquired; and in which the integration period associated with the acquisition of the image; is less than about one second. Embodiments include a camera system that includes a memory element in between the image sensor and the processor.
Embodiments include a camera system in which the time in between shots is less than about one second.
Embodiments include a camera system in which a first image is acquired and stored in memory; and a second image is acquired; and a processor is used to generate an image that employs information from the first image and the second image. Embodiments include generating an image with high dynamic range by combining information from the first image and the second image. Embodiments include a first image having a first focus; and a second image having a second focus; and generating an image from the first image and the second image having higher equivalent depth of focus.
Hotter objects generally emit higher spectral power density at shorter wavelengths than do colder objects. Information may thus be extracted regarding the relative temperatures of objects imaged in a scene based on the ratios of power in a first band to the power in a second band.
In embodiments, an image sensor may comprise a first set of pixels configured to sense light primarily within a first spectral band; and a second set of pixels configured to sense light primarily within a second spectral band. In embodiments, an inferred image may be reported that combines information from proximate pixels of the first and second sets. In embodiments, an inferred image may be reported that provides the ratio of signals from proximate pixels of the first and second sets.
In embodiments, an image sensor may include a means of estimating object temperature; and may further include a means of acquiring visible-wavelength images. In embodiments, image processing may be used to false-color an image representing estimated relative object temperature atop a visible-wavelength image.
In embodiments, the image sensor may include at least one pixel having linear dimensions less than approximately 2 μm×2 μm.
In embodiments, the image sensor may include a first layer providing sensing in a first spectral band; and a second layer providing sensing in a second spectral band.
In embodiments, visible images can be used to present a familiar representation to users of a scene; and infrared images can provide added information, such as regarding temperature, or pigment, or enable penetration through scattering and/or visible-absorbing media such as fog, haze, smoke, or fabrics.
In cases, it may be desired to acquire both visible and infrared images using a single image sensor. In cases, registration among visible and infrared images is thus rendered substantially straightforward.
In embodiments, an image sensor may employ a single class of light-absorbing light-sensing material; and may employ a patterned layer above it that is responsible for spectrally-selective transmission of light through it, also known as a filter. In embodiments, the light-absorbing light-sensing material may provide high-quantum-efficiency light sensing over both the visible and at least a portion of the infrared spectral regions. In embodiments, the patterned layer may enable both visible-wavelength pixel regions, and also infrared-wavelength pixel regions, on a single image sensor circuit.
In embodiments, an image sensor may employ two classes of light-absorbing light-sensing materials: a first material configured to absorb and sense a first range of wavelengths; and a second material configured to absorb and sense a second range of wavelengths. The first and second ranges may be at least partially overlapping, or they may not be overlapping.
In embodiments, two classes of light-absorbing light-sensing materials may be placed in different regions of the image sensor. In embodiments, lithography and etching may be employed to define which regions are covered using which light-absorbing light-sensing materials. In embodiments, ink jet printing may be employed to define which regions are covered using which light-absorbing light-sensing materials.
In embodiments, two classes of light-absorbing light-sensing materials may be stacked vertically atop one another. In embodiments, a bottom layer may sense both infrared and visible light; and a top layer may sense visible light principally.
In embodiments, an optically-sensitive device may include: a first electrode; a first light-absorbing light-sensing material; a second light-absorbing light-sensing material; and a second electrode. In embodiments, a first electrical bias may be provided between the first and second electrodes such that photocarriers are efficiently collected primarily from the first light-absorbing light-sensing material. In embodiments, a second electrical bias may be provided between the first and second electrodes such that photocarriers are efficiently collected primarily from the second light-absorbing light-sensing material. In embodiments, the first electrical bias may result in sensitivity primarily to a first wavelength of light. In embodiments, the second electrical bias may result in sensitivity primarily to a second wavelength of light. In embodiments, the first wavelength of light may be infrared; and the second wavelength of light may be visible. In embodiments, a first set of pixels may be provided with the first bias; and a second set of pixels may be provided with the second bias; ensuring that the first set of pixels responds primarily to a first wavelength of light, and the second set of pixels responds primarily to a second wavelength of light.
In embodiments, a first electrical bias may be provided during a first period of time; and a second electrical bias may be provided during a second period of time; such that the image acquired during the first period of time provides information primarily regarding a first wavelength of light; and the image acquired during the second period of time provides information primarily regarding a second wavelength of light. In embodiments, information acquired during the two periods of time may be combined into a single image. In embodiments, false-color may be used to represent, in a single reported image, information acquired during each of the two periods of time.
In embodiments, a focal plane array may consist of a substantially laterally-spatially uniform film having a substantially laterally-uniform spectral response at a given bias; and having a spectral response that depends on the bias. In embodiments, a spatially nonuniform bias may be applied, for example, different pixel regions may bias the film differently. In embodiments, under a given spatially-dependent biasing configuration, different pixels may provide different spectral responses. In embodiments, a first class of pixels may be responsive principally to visible wavelengths of light, while a second class of pixels may be responsive principally to infrared wavelengths of light. In embodiments, a first class of pixels may be responsive principally to one visible-wavelength color, such as blue; and a second class of pixels may be responsive principally to a distinctive visible-wavelength color, such as green; and a third class of pixels may be responsive principally to a distinctive visible-wavelength color, such as red.
In embodiments, an image sensor may comprise a readout integrated circuit, at least one pixel electrode of a first class, at least one pixel electrode of a second class, a first layer of optically sensitive material, and a second layer of optically sensitive material. In embodiments, the image sensor may employ application of a first bias for the first pixel electrode class; and of a second bias to the second pixel electrode class.
In embodiments, those pixel regions corresponding to the first pixel electrode class may exhibit a first spectral response; and of the second pixel electrode class may exhibit a second spectral response; where the first and second spectral responses are significantly different. In embodiments, the first spectral response may be substantially limited to the visible-wavelength region. In embodiments, the second spectral response may be substantially limited to the visible-wavelength region. In embodiments, the second spectral response may include both portions of the visible and portions of the infrared spectral regions.
In embodiments, it may be desired to fabricate an image sensor having high quantum efficiency combined with low dark current.
In embodiments, a device may consist of: a first electrode; a first selective spacer; a light-absorbing material; a second selective spacer; and a second electrode.
In embodiments, the first electrode may be used to extract electrons. In embodiments, the first selective spacer may be used to facilitate the extraction of electrons but block the injection of holes. In embodiments, the first selective spacer may be an electron-transport layer. In embodiments, the light-absorbing material may include semiconductor nanoparticles. In embodiments, the second selective spacer may be used to facilitate the extraction of holes but block the injection of electrons. In embodiments, the second selective spacer may be a hole-transport layer.
In embodiments, only a first selective spacer may be employed. In embodiments, the first selective spacer may be chosen from the list: TiO2, ZnO, ZnS. In embodiments, the second selective spacer may be NiO. In embodiments, the first and second electrode may be made using the same material. In embodiments, the first electrode may be chosen from the list: TiN, W, Al, Cu. In embodiments, the second electrode may be chosen from the list: ZnO, Al:ZnO, ITO, MoO3, Pedot, Pedot:PSS.
In embodiments, it may be desired to implement an image sensor in which the light-sensing element can be configured during a first interval to accumulate photocarriers; and during a second interval to transfer photocarriers to another node in a circuit.
Embodiments include a device comprising: a first electrode; a light sensing material; a blocking layer; and a second electrode.
Embodiments include electrically biasing the device during a first interval, known as the integration period, such that photocarriers are transported towards the first blocking layer; and where photocarriers are stored near the interface with the blocking layer during the integration period.
Embodiments include electrically biasing the device during a second interval, known as the transfer period, such that the stored photocarriers are extracted during the transfer period into another node in a circuit.
Embodiments include a first electrode chosen from the list: TiN, W, Al, Cu. In embodiments, the second electrode may be chosen from the list: ZnO, Al:ZnO, ITO, MoO3, Pedot, Pedot:PSS. In embodiments, the blocking layer be chosen from the list: HfO2, Al2O3, NiO, TiO2, ZnO.
In embodiments, the bias polarity during the integration period may be opposite to that during the transfer period. In embodiments, the bias during the integration period may be of the same polarity as that during the transfer period. In embodiments, the amplitude of the bias during the transfer period may be greater than that during the integration period.
Embodiments include a light sensor in which an optically sensitive material functions as the gate of a silicon transistor. Embodiments include devices comprising: a gate electrode coupled to a transistor; an optically sensitive material; a second electrode. Embodiments include the accumulation of photoelectrons at the interface between the gate electrode and the optically sensitive material. Embodiments include the accumulation of photoelectrons causing the accumulation of holes within the channel of the transistor. Embodiments include a change in the flow of current in the transistor as a result of a change in photoelectrons as a result of illumination. Embodiments include a change in current flow in the transistor greater than 1000 electrons/s for every electron/s of change in the photocurrent flow in the optically sensitive layer. Embodiments include a saturation behavior in which the transistor current versus photons impinged transfer curve has a sublinear dependence on photon fluence, leading to compression and enhanced dynamic range. Embodiments include resetting the charge in the optically sensitive layer by applying a bias to a node on the transistor that results in current flow through the gate during the reset period.
Embodiments include combinations of the above image sensors, camera systems, fabrication methods, algorithms, and computing devices, in which at least one image sensor is capable of operating in global electronic shutter mode.
In embodiments, at least two image sensors, or image sensor regions, may each operate in global shutter mode, and may provide substantially synchronous acquisition of images of distinct wavelengths, or from different angles, or employing different structured light.
Embodiments include implementing correlated double-sampling in the analog domain. Embodiments include so doing using circuitry contained within each pixel.
In embodiments, light sensing may be enabled through the use of a light sensing material that is integrated with, and read using, a readout integrated circuit. Example embodiments of same are included in U.S. Provisional Application No. 61/352,409, entitled, “Stable, Sensitive Photodetectors and Image Sensors Made Therefrom Including Circuits for Enhanced Image Performance,” and U.S. Provisional Application No. 61/352,410, entitled, “Stable, Sensitive Photodetectors and Image Sensors Made Therefrom Including Processes and Materials for Enhanced Image Performance,” both filed Jun. 8, 2010, which are hereby incorporated by reference in their entirety.
In embodiments, a method of gesture recognition is provided where the method includes acquiring a stream, in time, of at least two images from each of at least one camera module; acquiring a stream, in time, of at least two signals from each of at least one light sensor; and conveying the at least two images and the at least two signals to a processor, the processor being configured to generate an estimate of a gesture's meaning, and timing, based on a combination of the at least two images and the at least two signals.
In embodiments, the at least one light sensor includes a light-absorbing material having an absorbance, across the visible wavelength region of about 450 nm to about 650 nm, of less than about 30%.
In embodiments, the light-absorbing material includes PBDTT-DPP, the near-infrared light-sensitive polymer poly(2,60-4,8-bis(5-ethylhexylthienyl)benzo-[1,2-b;3,4-b]dithiophene-alt-5-dibutyloctyl-3,6-bis(5-bromothiophen-2-yl)pyrrolo[3,4-c]pyrrole-1,4-dione).
In embodiments, the at least one light sensor includes a light-sensing material capable of sensing infrared light.
In embodiments, the method includes modulating a light source using at least one code selected from spatial codes and temporal codes.
In embodiments, the light source has an emission wavelength in the range of about 900 nm to about 1000 nm.
In one embodiment, a camera system includes a central imaging array region, at least one light-sensing region outside of the central imaging array region, a first mode, referred to as imaging mode, and a second mode, referred to as sensing mode. The electrical power consumed in the second mode is at least 10 times lower than the electrical power consumed in the first mode.
In embodiments, the at least one light sensor includes a light-sensing material capable of sensing infrared light.
In embodiments, light impinging on the light-sensing material is to be modulated.
In embodiments, a portion of light impinging on the light-sensing material is to be generated using a light emitter device having an emission wavelength in the range of about 800 nm to about 1000 nm.
In embodiments, the central imaging array includes at least six megapixels.
In embodiments, the central imaging array comprises pixels less than approximately 2 μm in width and approximately 2 μm in height.
In one embodiment, an image sensor circuit includes a central imaging array region having a first field of view; and at least one light-sensing region outside of the central imaging array region having a second field of view. The second field of view is less than half, measured in angle, the field of view of the first field of view.
In one embodiment, an integrated circuit includes a substrate, an image sensing array region occupying a first region of said semiconductor substrate and including a plurality of optically sensitive pixel regions, a pixel circuit for each pixel region, each pixel circuit comprising a charge store and a read-out circuit, and a light-sensitive region outside of the image sensing array region. The image sensing array region having a first field of view and the light-sensitive region having a second field of view; the angle of the second field of view is less than half of the angle of the first field of view.
In embodiments, at least one of the image sensing array and the light-sensitive region outside of the image sensing array region includes a light-sensing material capable of sensing infrared light.
In embodiments, light impinging on at least one of the image sensing array and the light-sensitive region outside of the image sensing array region is to be modulated.
In embodiments, a portion of light impinging on at least one of the image sensing array and the light-sensitive region outside of the image sensing array region is to be generated using a light emitter device having an emission wavelength in the range of about 800 nm to about 1000 nm.
In embodiments, the image sensing array includes at least six megapixels.
In embodiments, the image sensing array comprises pixels less than approximately 2 μm in width and approximately 2 μm in height.
In one embodiment, an image sensor includes a central imaging array region to provide pixelated sensing of an image, in communication with a peripheral region that includes circuitry to provide biasing, readout, analog-to-digital conversion, and signal conditioning to the pixelated light sensing region. An optically sensitive material overlies the peripheral region.
In embodiments, the at least one light sensor includes a light-sensing material capable of sensing infrared light.
In embodiments, light impinging on the light-sensing material is to be modulated.
In embodiments, a portion of light impinging on the light-sensing material is to be generated using a light emitter device having an emission wavelength in the range of about 800 nm to about 1000 nm.
In embodiments, the central imaging array includes at least six megapixels.
In embodiments, the central imaging array comprises pixels less than approximately 2 μm in width and approximately 2 μm in height.
In embodiments, the optically sensitive material is chosen to include at least one material from a list, the list including silicon, colloidal quantum dot film, and a semiconducting polymer.
In embodiments, the optically sensitive material is fabricated on a first substrate, and is subsequently incorporated onto the central imaging array region.
In one embodiment, a mobile computing device includes a semiconductor substrate, an image sensor comprising pixel circuitry formed on the semiconductor substrate and an image sensing region, a photosensor comprising read-out circuitry formed on the semiconductor substrate and a light sensitive region, circuitry configured to read an image from the image sensor, a processor configured to process a signal read from the photosensor proportional to an optical signal sensed by the photosensor, and control circuitry configured in at least one mode to provide power to read the photosensor without providing power to read-out the image sensor such that power consumption is reduced compared to a mode where the power is provided to operate the image sensor.
In various ones of the embodiments discussed herein, additional embodiments include the semiconductor substrate comprising a die of silicon. In embodiments, the semiconductor substrate comprises a single die of silicon. In embodiments, the semiconductor substrate comprises an integrated circuit. In embodiments, each pixel circuit comprises circuitry formed on the semiconductor substrate. In embodiments, the read-out circuit for the light-sensitive region comprises circuitry formed on the semiconductor substrate. In embodiments, the image sensing region and the light-sensitive region are formed on the semiconductor substrate. In embodiments, the image sensing region and the light-sensitive region are formed on the single die of silicon. In embodiments, the image sensing region and the light-sensitive region each comprise a layer of photosensitive material on the semiconductor substrate. In embodiments, the control circuitry is formed on the semiconductor substrate. In embodiments, the control circuitry is formed on the semiconductor substrate.
In various ones of the embodiments discussed herein, additional embodiments include the electrical power consumed by the control circuitry and the photosensor during the second mode is at least 100 times lower than electrical power consumed by the control circuitry and the image sensor during the first mode. In embodiments, electrical power consumed to operate the photosensor is at least ten (10) times lower than electrical power consumed to operate the image sensor. In embodiments, electrical power consumed to operate the photosensor is at least fifty (50) times lower than electrical power consumed to operate the image sensor. In embodiments, electrical power consumed to operate the photosensor is at least one hundred (100) times lower than electrical power consumed to operate the image sensor. In embodiments, electrical power consumed to operate the photosensor is less than about 1 milliwatt (mW). In embodiments, electrical power consumed to operate the photosensor is less than about 10 milliwatt (mW).
In various ones of the embodiments discussed herein, additional embodiments include the photosensor is configured to sense light in at least one mode of operation without providing power to the image sensor. In embodiments, a read-out circuit for the photosensor operates independently from the power provided to the semiconductor substrate. In embodiments, a read-out circuit for the photosensor separate from the semiconductor substrate is configured to read-out a signal from the photosensor in at least one mode of operation without power being provided to the semiconductor substrate. In embodiments, a read-out circuit for the photosensor is separate from the semiconductor substrate that is configured to read-out a signal from the photosensor in at least one mode of operation without power being provided to the image sensor. In embodiments, a power management unit is provided for the mobile device, the power management unit is separate from the semiconductor substrate and is configured to monitor current from the, for example, photosensor and/or light sensitive region.
In various ones of the embodiments discussed herein, additional embodiments include that the photosensor is passive. In embodiments, the image sensor is active. In embodiments, the semiconductor substrate includes on-chip power routed to the image sensor. In embodiments, the on-chip power on the semiconductor substrate is not routed to the photosensor. In embodiments, the photosensor is operated independently of the on-chip power on the semiconductor substrate.
In various ones of the embodiments discussed herein, additional embodiments include the image sensing region having an area that is at least 10 times greater than an area of the photosensor. In embodiments, the image sensing region has an area greater than 40 mm2. In embodiments, the photosensor has an area less than 4 mm2. In embodiments, the image sensing region comprises a plurality of pixel regions, with each pixel region is less than about 2 micron (μm) in width and less than about 2 micron (μm) in height. In embodiments, the image sensing region comprises a plurality of pixel regions, with each pixel region has an area less than about 4 micron squared (μm2). In embodiments, the image sensing region comprises more than 2 million pixel regions. In embodiments, the photosensor has an area of photosensitive material coupled to a where the photosensitive region is greater than 1 to 100 micrometers squared (μm2). In embodiments, the photosensor comprises binned pixel regions having a total binned area of at least 100 microns squared (μm2).
In various ones of the embodiments discussed herein, additional embodiments include the image sensing region being in a center region of an image circle. In embodiments, the photosensor is in a peripheral region of an image circle further comprising optics configured to focus light onto the image sensing region on the semiconductor substrate, the image sensing region being positioned substantially near the center of the image circle. In embodiments, the photosensor is positioned in a peripheral region of the image circle. In embodiments, the photosensor is peripheral to the image sensing region. In embodiments, the photosensor is positioned in a peripheral segment of the image circle. In embodiments, the photosensor has a non-rectangular shape.
In various ones of the embodiments discussed herein, additional embodiments include a second photosensor, the second photosensor being positioned in a peripheral region of the image circle. In embodiments, the first and second photosensors are positioned on opposite sides of the image sensing region. In embodiments, the image sensing region is positioned substantially at the center of the image circle such that there are four peripheral regions around the image sensing region. In embodiments, each peripheral region includes at least one photosensor. In embodiments, is the image sensor region is configured to sense visible light, and at least one of the photosensors is configured to sense infrared light
In various ones of the embodiments discussed herein, additional embodiments include the image sensing region and light sensing region comprise the same photosensitive material. In embodiments, the first photosensor and second photosensor comprise the same photosensitive material. In embodiments, a filter is positioned to filter light incident on the image sensor region within a predetermined range. In embodiments, an infrared (IR) filter configured to block infrared light impinging on the image sensing region, and to pass infrared light impinging on the photosensor(s). In embodiments, the light incident on the photosensor is selectively filtered relative to the image sensing region. In embodiments, the image sensing region comprises a first photosensitive material and the light sensing region comprises a second photosensitive material. In embodiments, the first photosensitive material has a different sensitivity to wavelength of light than the second photosensitive material. In embodiments, the first photosensitive material has a higher sensitivity to visible light than the second photosensitive material. In embodiments, the second photosensitive material has a higher sensitivity to infrared (IR) light than the first photosensitive material. In embodiments, the photosensor comprises a first photosensitive material and the second photosensor comprises a second photosensitive material. In embodiments, the first photosensitive material has a different sensitivity to wavelength of light than the second photosensitive material. In embodiments, the first photosensitive material has a higher sensitivity to infrared (IR) light than the second photosensitive material. In embodiments, the second photosensitive material has a higher sensitivity to near infrared (NIR) light than the first photosensitive material. In embodiments, the first photosensitive material comprises a layer of nanocrystal material. In embodiments, the photosensitive material of the image sensing region comprises silicon; and the photosensitive material of the photosensor comprises a layer nanocrystal photosensitive material.
In various ones of the embodiments discussed herein, additional embodiments include optics configured to focus light on an image circle covering both the image sensing region and the photosensor. In embodiments, the image sensing region and the photosensor are positioned in the focal plane of the optics. In embodiments, the image sensing region is in a central region of image circle and the photosensor is in a peripheral region of the image circle.
In various ones of the embodiments discussed herein, additional embodiments include the semiconductor substrate having circuitry in the peripheral region wherein the light sensing region is formed over at least some of the circuitry. In embodiments, the circuitry (e.g., in the peripheral region) comprises circuitry formed on the semiconductor substrate outside the area of the image sensing region. In embodiments, at least some of the circuitry is positioned below the photosensor.
In various ones of the embodiments discussed herein, additional embodiments include the read-out circuit comprising circuitry below the image sensing region. In embodiments, the control circuitry and or the read-out circuit are formed on the substrate (e.g., where the substrate may be a semiconductor substrate).
In various ones of the embodiments discussed herein, additional embodiments include providing a bias across the photosensitive material of the image sensing region of approximately 1 to 4 V. In embodiments, the circuitry provides a bias across the photosensitive material of the photosensor of approximately 1V to 4 V.
In embodiments, the bias to the photosensitive material of the image sensing region can be selectively turned off independently of the bias across the photosensitive material of the photosensor. In embodiments, the photosensor comprises a first electrode and a second electrode, and a bias provided to the first electrode is independent of a bias applied to the second electrode.
In various ones of the embodiments discussed herein, additional embodiments include the read-out circuit for the photosensor comprises circuitry being below the image sensing region. In embodiments, the read-out circuit for the photosensor comprises off-chip circuitry separate from the semiconductor substrate. In embodiments, the read-out circuit for the photosensor comprises a current monitor. In embodiments, the read-out circuit for the photosensor comprises a counter. In embodiments, the read-out circuit for the photosensor comprises a charge pump. In embodiments, the read-out circuit for the photosensor comprises circuitry formed on the semiconductor substrate. In embodiments, the read-out circuit for the photosensor is configured to generate, for example, a signal, a digital number, or a counter signal indicative of light incident on, for example, the photosensor or the light sensing material.
In various ones of the embodiments discussed herein, additional embodiments include the semiconductor substrate including power routed to a photosensor separately from power routed to the image sensor such that power can be provided to the photosensor when the power to the image sensor is turned off.
In various ones of the embodiments discussed herein, additional embodiments include the image sensor comprising a back side illumination (BSI) silicon image sensor. In embodiments, the photosensor comprises pixel regions coupled to a reset gate of a pixel circuit for current collection. In embodiments, the, for example, image sensor or photosensor comprises silicon photodiodes.
In various ones of the embodiments discussed herein, additional embodiments include the image sensing region being configured to have a first field of view and the photosensor being configured to have a second field of view at a different angle than the first field of view. In embodiments, the first photosensor has a first field of view and second photosensor has a second field of view at a different angle than the first field of view. In embodiments, each photosensor has a field of view at a different angle than each other photosensor. In embodiments, the photosensor has a first field of view, and the second photosensor has a second field-of-view.
In various ones of the embodiments discussed herein, additional embodiments include further comprising a light source for illuminating a scene. In embodiments, the light source is modulated. In embodiments, the photosensor is configured to sense modulated light reflected from the scene. In embodiments, a processor is configured to process a signal from the photosensor indicative of the modulated light reflected from the scene for gesture recognition.
In various ones of the embodiments discussed herein, additional embodiments include a processor configured to process a signal from the photosensor indicative of the modulated light reflected from the scene for gesture recognition. In embodiments, a processor is configured to process a signal from the photosensor from a first field of view indicative of the modulated light reflected from the scene for gesture recognition. In embodiments, a processor is configured to process a signal from each of the photosensors for gesture recognition. In embodiments, the signal from each photosensor is indicative of the modulated light reflected from the scene for a different field of than the other photosensors.
The various illustrations of the procedures and apparatuses are intended to provide a general understanding of the structure of various embodiments and are not intended to provide a complete description of all the elements and features of the apparatuses and methods that might make use of the structures, features, and materials described herein. Based upon a reading and understanding of the disclosed subject matter provided herein, a person of ordinary skill in the art can readily envision other combinations and permutations of the various embodiments. The additional combinations and permutations are all within a scope of the present invention.
The Abstract of the disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. The abstract is submitted with the understanding that it will not be used to interpret or limit the claims. In addition, in the foregoing Detailed Description, it may be seen that various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as limiting the claims. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separate embodiment.
The present application claims the benefit of priority of U.S. Provisional Patent Application Ser. No. 62/008,039, filed Jun. 5, 2014, and entitled “SENSORS AND SYSTEMS FOR THE CAPTURE OF SCENES AND EVENTS IN SPACE AND TIME,” which is incorporated herein by reference in its entirety.
| Number | Name | Date | Kind |
|---|---|---|---|
| 7377641 | Piers et al. | May 2008 | B2 |
| 7983455 | Moise et al. | Jul 2011 | B2 |
| 8107706 | Saijo | Jan 2012 | B2 |
| 8194924 | Pryor | Jan 2012 | B2 |
| 8284194 | Zhang et al. | Oct 2012 | B2 |
| 8768006 | Subramanian et al. | Jul 2014 | B2 |
| 8837780 | Jia et al. | Sep 2014 | B2 |
| 8971565 | Anzola | Mar 2015 | B2 |
| 9405376 | Sargent et al. | Aug 2016 | B2 |
| 20030001093 | Wood | Jan 2003 | A1 |
| 20050088567 | Kim | Apr 2005 | A1 |
| 20070176916 | Choe | Aug 2007 | A1 |
| 20070279373 | Tseng et al. | Dec 2007 | A1 |
| 20080062117 | Lu et al. | Mar 2008 | A1 |
| 20080259198 | Chen et al. | Oct 2008 | A1 |
| 20090051796 | Massie et al. | Feb 2009 | A1 |
| 20090303346 | Kanemitsu et al. | Dec 2009 | A1 |
| 20100090996 | Chou | Apr 2010 | A1 |
| 20110080500 | Wang et al. | Apr 2011 | A1 |
| 20110122108 | Kozuma | May 2011 | A1 |
| 20110216212 | Watanabe et al. | Sep 2011 | A1 |
| 20110267510 | Malone et al. | Nov 2011 | A1 |
| 20110310005 | Chen et al. | Dec 2011 | A1 |
| 20110312349 | Forutanpour et al. | Dec 2011 | A1 |
| 20120169241 | Wang et al. | Jul 2012 | A1 |
| 20120182266 | Han | Jul 2012 | A1 |
| 20120274745 | Russell | Nov 2012 | A1 |
| 20120327248 | Tack et al. | Dec 2012 | A1 |
| 20130049641 | Sheynblat et al. | Feb 2013 | A1 |
| 20130057713 | Khawand | Mar 2013 | A1 |
| 20130222603 | Agranov et al. | Aug 2013 | A1 |
| 20130297926 | Eaton et al. | Nov 2013 | A1 |
| 20140063332 | Miyawaki | Mar 2014 | A1 |
| 20140168372 | Chang et al. | Jun 2014 | A1 |
| 20140265040 | Batchelder | Sep 2014 | A1 |
| 20150009375 | Agranov et al. | Jan 2015 | A1 |
| 20150092019 | Asano et al. | Apr 2015 | A1 |
| 20150177865 | Rodzevski et al. | Jun 2015 | A1 |
| 20160037070 | Mandelli | Feb 2016 | A1 |
| 20160342232 | Sargent et al. | Nov 2016 | A1 |
| Number | Date | Country |
|---|---|---|
| 105122271 | Dec 2015 | CN |
| 2905296 | Jun 1999 | JP |
| WO-2014093334 | Jun 2014 | WO |
| WO-2015188146 | Dec 2015 | WO |
| WO-2015188146 | Dec 2015 | WO |
| Entry |
|---|
| “International Application Serial No. PCT/US2015/034559, International Search Report dated Nov. 4, 2015”, 4 pgs. |
| “International Application Serial No. PCT/US2015/034559, Written Opinion dated Nov. 4, 2015”, 10 pgs. |
| “International Application Serial No. PCT/US2015/034559, International Preliminary Report on Patentability dated Dec. 15, 2016”, 7 pgs. |
| “U.S. Appl. No. 14/101,642, Non Final Office Action dated Jun. 22, 2015”, 12 pgs. |
| “European Application Serial No. 13862223.8, Office Action dated Jul. 31, 2015”, 2 pgs. |
| “International Application Serial No. PCT/US2013/074112, International Preliminary Report on Patentability dated Jun. 26, 2015”, 6 pgs. |
| “International Application Serial No. PCT/US2015/034559, Invitation to Pay Add'l Fees and Partial Search Rpt dated Aug. 27, 2015”, 2 pgs. |
| U.S. Appl. No. 14/101,612, filed Dec. 10, 2013, Sensors and Systems for the Capture of Scenes and Events in Space and Time. |
| U.S. Appl. No. 14/811,958, filed Jul. 29, 2015, Multi-Mode Power-Efficient Light and Gesture Sensing in Image Sensors. |
| International Application # PCT/US2017/057766 search report dated Feb. 6, 2018. |
| Number | Date | Country | |
|---|---|---|---|
| 20150358567 A1 | Dec 2015 | US |
| Number | Date | Country | |
|---|---|---|---|
| 62008039 | Jun 2014 | US |
