This relates generally to imaging systems, and more particularly, to multi-array imaging systems.
Modern electronic devices such as cellular telephones, cameras, and computers often use digital image sensors. Imagers (i.e., image sensors) often include a two-dimensional array of image sensing pixels. Each pixel typically includes a photosensor such as a photodiode that receives incident photons (light) and converts the photons into electrical signals.
Some conventional imaging systems include multiple imaging arrays. In particular, some conventional imaging systems include separate red, blue, and green pixel arrays. While such systems may have benefits over monolithic single sensor arrays, conventional multi-array imaging systems leave room for improvement.
It would therefore be desirable to be able to provide improved multi-array image sensor systems and methods.
Digital camera modules are widely used in electronic devices. An electronic device with a digital camera module is shown in
Still and video image data from camera sensor 14 may be provided to image processing and data formatting circuitry 16 via path 26. Image processing and data formatting circuitry 16 may be used to perform image processing functions such as adjusting white balance and exposure and implementing video image stabilization, image cropping, image scaling, etc. Image processing and data formatting circuitry 16 may also be used to compress raw camera image files if desired (e.g., to Joint Photographic Experts Group or JPEG format).
In some arrangements, which is sometimes referred to as a system on chip or SOC arrangement, camera sensor 14 and image processing and data formatting circuitry 16 are implemented as a common unit 15 (e.g., on a common integrated circuit, or stacked together). The use of a single integrated circuit to implement camera sensor 14 and image processing and data formatting circuitry 16 can help to minimize costs. If desired, however, multiple integrated circuits may be used to implement circuitry 15. In arrangements in which device 10 includes multiple camera sensors 14, each camera sensor 14 and associated image processing and data formatting circuitry 16 can be formed on a separate SOC integrated circuit (e.g., there may be multiple camera system on chip modules such as modules 12A and 12B). In other suitable arrangements and when device 10 includes multiple camera sensors 14 (e.g., includes multiple arrays), each camera sensor 14 may be formed on a common integration circuit.
Circuitry 15 conveys data to host subsystem 20 over path 18. Circuitry 15 may provide acquired image data such as captured video and still digital images to host subsystem 20.
Electronic device 10 typically provides a user with numerous high level functions. In a computer or advanced cellular telephone, for example, a user may be provided with the ability to run user applications. To implement these functions, electronic device 10 may have input-output devices 22 such as projectors, keypads, input-output ports, and displays and storage and processing circuitry 24. Storage and processing circuitry 24 may include volatile and nonvolatile memory (e.g., random-access memory, flash memory, hard drives, solid state drives, etc.). Storage and processing circuitry 24 may also include processors such as microprocessors, microcontrollers, digital signal processors, application specific integrated circuits, etc.
Device 10 may include position sensing circuitry 23. Position sensing circuitry 23 may include, as examples, global positioning system (GPS) circuitry, radio-frequency-based positioning circuitry (e.g., cellular-telephone positioning circuitry), gyroscopes, accelerometers, compasses, magnetometers, etc.
As shown in
Device 10 may include additional arrays 14B-141, sometimes referred to herein as secondary array. Secondary arrays 14B-141 may be low-power arrays (e.g., each of the arrays 14B-141 may have a lower power consumption that primary arrays such as array 14A). Secondary arrays 14B-141 may have a relative low resolution compared to array 14A. In general, there may be any desired number of secondary arrays 14B-141. As illustrated in
Secondary arrays 14B-141 may have different functions. In some arrangements, multiple arrays 14B-141 share a similar function and, in other arrangements, each of the arrays 14B-141 has a unique function. One or more of arrays 14B-141 may include focus sensitive imaging pixels such device 10 can obtain focus information from those arrays (e.g., that detects focus depth). The secondary arrays 14B-141 may be configured to continually observe a scene and may trigger other arrays (such as primary array 14A) upon detection of preset conditions in the scene. The pre-set conditions may be based on gesture or interest point tracking or a trigger signal invisible to human vision (e.g., a signal in infrared or ultraviolet wavelengths).
With at least some arrangements, device 10 may include a multi-array imaging system in which at least one of the imaging arrays is a hyperspectral imaging array, a polarization sensing array, or a wavefront sensing array. As illustrated in
If desired, device 10 may include a multi-array imaging system with complementary imaging arrays. For example and as illustrated in
In arrangements in which the primary imaging array 14A includes photosites formed from vertically stacked photodiodes and the imaging array 14B is a monochrome imaging array, the primary imaging array 14A may have excellent low light performance and other features. Additionally, the monochrome imaging channel may provide independent luminescent signals that greatly assist in processing images from imaging array 14A. In particular, without the image data from complementary array 14B, imaging array 14A may value inadequate spatial resolution, color resolution, and robustness. By combining image data from array 14A with image data from complementary array 14B, these deficiencies can be overcome.
With other suitable arrangements, imaging array 14B may be a basic imaging array having an array of green, red, and blue imaging pixels arranged in a Bayer pattern. As another example, imaging array 14B may be a non-RBG (i.e., non-Bayer) imaging sensor such as an imaging sensor that includes an array including only one or two of red, blur, and green pixels.
If desired, the multi-array system may include a third imaging array 14C. As one example, the imaging array 14A may be an imaging array including only pixels sensitive to blue light (e.g., an imaging array with a blue filter that extends over all of the pixels), imaging array 14B may be a monochrome imaging array (e.g., an imaging array that detects only the intensity of incident light summed across visible wavelengths), and imaging array 14C may be an imaging array including only pixels sensitive to red light.
As another example, imaging array 14A may be monochrome imaging array, imaging array 14B may be a stacked photodiode imaging array including photosites formed pixels sensitive to a first color and pixels sensitive to a second color (e.g., an arrangement in which red and blue pixels are vertically stacked), and imaging array 14C may be an imaging array including only pixels sensitive to red light.
As illustrated in
In each of the aforementioned examples of
An illustrative polarization sensing imaging array (which may be incorporated into one or more of the embodiments of
If desired, the polarization sensing imaging array may include a plurality of polarization filters, each filter being located over a different region of the sensor array 14. The region may be as small as a single image sensing pixel in array 14. Each of the polarization filters may be sensitive to a particular type of polarized light (e.g., vertically, horizontally, clockwise, or counter-clockwise polarized light). With this type of arrangement, a single polarization sensing imaging array may be sensitive to more than one type of polarized light and may be able to image differences in types of polarized light across a scene.
Illustrative hyperspectral imaging sensors that may be part of a hyperspectral imaging array (which may be incorporated into one or more of the embodiments of
As shown in
If desired, the boundaries 35 between adjacent combinations of microlens 36 and diffraction gratings 34 and the associated imaging pixels may be transparent or opaque (preventing crosstalk).
In arrangements in which the boundaries 35 are transparent, one or more imaging pixels may be shared by adjacent combinations of microlens 36 and diffraction gratings 34 (e.g., may receive light from multiple combinations of microlens 36 and diffraction gratings 34). In such an example, there may be an average of two imaging pixels 32 per combination of microlens 36 and diffraction gratings 34. A first of the imaging pixels may receive a zero order beam while two other imaging pixels (which are each shared with one other combination of microlens 36 and diffraction gratings 34) may receive a first order beam.
An illustrative photosite in a stacked photodiode imaging array (which may be incorporated into one or more of the embodiments of
An illustrative wavefront sensing imaging array (which may be incorporated into one or more of the embodiments of
CMOS imager 200 is operated by a timing and control circuit 206, which controls decoders 203, 205 for selecting the appropriate row and column lines for pixel readout, and row and column driver circuitry 202, 204, which apply driving voltages to the drive transistors of the selected row and column lines. The pixel signals, which typically include a pixel reset signal Vrst and a pixel image signal Vsig for each pixel are sampled by sample and hold circuitry 207 associated with the column driver 204. A differential signal Vrst-Vsig is produced for each pixel, which is amplified by amplifier 208 and digitized by analog-to-digital converter 209. The analog to digital converter 209 converts the analog pixel signals to digital signals, which are fed to image processor 210 which forms a digital image.
Processor system 300, which may be a digital still or video camera system, may include a lens such as lens 396 for focusing an image onto a pixel array such as pixel array 201 when shutter release button 397 is pressed. Processor system 300 may include a central processing unit such as central processing unit (CPU) 395. CPU 395 may be a microprocessor that controls camera functions and one or more image flow functions and communicates with one or more input/output (I/O) devices 391 over a bus such as bus 393. Imaging device 200 may also communicate with CPU 395 over bus 393. System 300 may include random access memory (RAM) 392 and removable memory 394. Removable memory 394 may include flash memory that communicates with CPU 395 over bus 393. Imaging device 200 may be combined with CPU 395, with or without memory storage, on a single integrated circuit or on a different chip. Although bus 393 is illustrated as a single bus, it may be one or more buses or bridges or other communication paths used to interconnect the system components.
Various embodiments have been described illustrating multi-array imaging devices. An imaging system may include multiple imaging arrays. One or more of the arrays may be a low-power array that detects trigger events in observed scenes and, in response to the detection of a trigger event, activates one or more primary imaging arrays. One or more of the arrays may be a polarization sensing array, a hyperspectral array, a stacked photodiode array, a wavefront sensing array, a monochrome array, a single color array, a dual color array, or a full color array. In at least one embodiment, image data from a stacked photodiode imaging array may be enhanced using image data from a separate monochrome imaging array. In at least another embodiment, image data from a wavefront sensing array may provide focus detection for a full color array.
The foregoing is merely illustrative of the principles of this invention which can be practiced in other embodiments.