This application is based upon and claims the benefit of priority from prior Japanese Patent Application No. 2010-199838 filed on Sep. 7, 2010 in Japan, the entire contents of which are incorporated herein by reference.
Embodiments described herein relate generally to a solid-state imaging device.
As an imaging technique capable of obtaining a distance in the depth direction from two-dimensional array information, a technique using a reference ray, a stereo distance measuring technique using a plurality of cameras, and so on are known. In recent years, needs for such techniques are increasing as new input devices in the use of public welfare.
As an imaging scheme capable of obtaining the distance without using a reference ray in order to hold down the cost, there is a scheme using that a ray angle from a subject via an image formation lens contains distance information as the principle of the distance measurement. For example, a structure in which an image formation microlens array (MLA) is disposed over pixels and an aperture for dividing the ray is disposed on the object side of each image formation microlens has been proposed. Each image formation microlens acquires an image having a peak in intensity distribution according to a distance to the subject by re-imaging an image imaged by an image formation lens from each position. The distance to the subject can be estimated by measuring the number of signal peaks.
a) and 2(b) are diagrams showing a concrete example of aperture elements of a solid-state imaging device according to an embodiment;
a) and 4(b) are diagrams for explaining a suppression effect of a light quantity change in a solid-state imaging device according to an embodiment;
a) and 5(b) are diagrams for explaining a suppression effect of a light quantity change in a solid-state imaging device according to an embodiment;
a) and 10(b) are diagrams for explaining overlapping areas depending on position relations between microlenses and imaging elements in a solid-state imaging device according to an embodiment;
a) to 12(f) are diagrams for explaining images of a subject in aperture elements and imaging elements which depend upon the position of the subject in a solid-state imaging device according to an embodiment;
a) to 13(f) are diagrams showing a signal intensity received by imaging elements as light which depends upon a position of a subject in a solid-state imaging device according to an embodiment;
a) and 14(b) are diagrams showing relations between the number of stripes in imaging elements and a distance to a subject, which depends upon a position of the subject in a solid-state imaging device according to an embodiment; and
a) and 15(b) are diagrams for explaining an acquisition method of a two-dimensional image in a solid-state imaging device according to an embodiment.
A solid-state imaging device according to an embodiment includes: an imaging device including an imaging area including a plurality of pixel blocks each of which includes a plurality of pixels; an image formation lens forming an image on an image formation plane by using light from a subject; an aperture unit including a plurality of aperture elements provided to associate with the plurality of pixel blocks, each of the aperture elements having an aperture portion and a shield portion, light from the image formation lens being filtered by each aperture element; a microlens array including a plurality of microlenses provided to associate with the plurality of aperture elements, each of the microlenses forming an image in the imaging area by using light filtered by an associated aperture element; and a signal processing circuit configured to process a signal of an image taken in the imaging area and estimates a distance to the subject.
Hereafter, a solid-state imaging device according to embodiments will be described more specifically with reference to the drawings. A solid-state imaging device according to the present embodiment is shown in
Light emitted from a subject 100 is incident on a pixel in the imaging element 17 through the image formation lens 11, the 10 aperture 13, and the microlens array 15. Information concerning the subject 100 is converted to an electric signal by the imaging element 17. The electric signal which is output from the imaging element 17 is processed by the signal processing circuit 20 and an image of the subject 100 is obtained. In addition, the distance to 15 the subject 100 is estimated.
In
(Aperture)
The aperture 13 will now be described. The aperture 13 is installed to obtain information concerning the distance to the subject 100. The number of divisions of the aperture 13 (i.e., the number of aperture elements) corresponds to the resolution of the distance information.
Concrete examples of an aperture element (for example, an aperture element 13i) included in the aperture 13 are shown in
Each aperture element 13i (i=a, b, c) is associated with a corresponding microlens 15i. The aperture 13 has a configuration in which the aperture elements 13a, 13b and 13c are arranged in an array form. Each aperture element 13i (i=a, b, c) is disposed to have its center on the same axis as that of the center of the microlens 15i.
A method for obtaining information concerning the distance to the subject 100 based on stripe patterns of an image taken into 25 the imaging element 17 by providing the aperture 13 will be described later.
A change of light quantity incident on the imaging element 17 which depends upon the position relation between the aperture element 13i and the incidence position of the principal ray is incident on the aperture element 13i will now be described with reference to
On the other hand, if the center axis 130 of the aperture is deviated from the center 30 of the optical axis as shown in
In the case of an off-optical axis ray which is incident on the image formation lens at an angle with respect to the optical axis, it is necessary for the aperture to cope with the obliquely incident light in order to form a stripe pattern according to the distance. Specifically, a method for curving the aperture or shifting the aperture from the center of the microlens is necessary. However, the method for measuring the distance based on the number of stripe patterns is the same.
(Microlens Array)
The microlens array 15 needs to be disposed to separate images of aperture elements 13i in the imaging element 17 without overlapping. The microlens array 15 plays an important role in obtaining the distance information in the present embodiment. The diameter of the microlens 15i (i=a, b, c) is equal to the diameter of the corresponding aperture element 13i. The microlens array 15 obtained by arranging a plurality of microlenses 15i in a two-dimensional form is disposed on the side of the imaging element 17 as compared with the aperture 13.
(Position Relations of Aperture, Microlens Array, and Imaging Element)
The aperture 13 is disposed to be located at the focal length “f” of the image formation lens 11. When the subject 100 is located at infinity in this case, the light of the subject 100 forms an image at a minimum point image radius on the plane of the aperture 13. When the subject 100 is located at infinity, the light passed through the aperture 13 forms an image having no stripe pattern. As the subject 100 comes closer to the image formation lens 11, the number of stripe patterns increases.
The distance “L” between the aperture 13 and the microlens array 15 depends upon the focal length “f” of the image formation lens 11 and the radius “r” of the microlens 15i. As shown in
For example, assuming that R=4 mm, f=16 mm, and φ=30 μm, the range which can be assumed by “L” is a range of 60 μm from the focal point of the image formation lens 11 in the direction of the imaging element 17. If the microlens array 15 is installed in the position of L=60 μm, therefore, light rays which are emitted from the subject 100 and which are incident on the imaging element 17 become as shown in
A position relation required of the imaging element 17, i.e., a distance M between the microlens 15i and the imaging element 17 will be now described based on the foregoing description.
Denoting a radius of spread of a real image in the imaging element 17 after being passed through the microlens 15i by x, the value of x is represented as
as appreciated from
If this condition is satisfied, an area where images of the microlens 15i overlap on the imaging element 17 is eliminated. For utilizing effective pixels fully, a value of M which is great as far as possible should be used. The range where images of the microlens overlap in the imaging element 17 will now be described by using the expression with reference to
In
In
Shaded ranges in
(Method for Obtaining Distance Information)
A method for obtaining distance information from the subject 100 based on the stripe pattern in an image taken in the imaging element 17 will now be described.
If a distance “a” between the subject 100 and the camera differs as shown in
Images 102 in the aperture element 13i of the subject 100 at the time when the subject 100 is located in the positions denoted by I, II and III are shown in
Light rays passed through the aperture element 13i and the microlens 15i at the time when the subject 100 is located in the positions denoted by I, II and III are shown in
The distance to the subject 100 is obtained as described hereafter.
First, in the image formation lens 11, the following expression holds true.
Because of the expression and similarity of triangles in the light ray trajectory, a real image radius r′ on the aperture 13 is represented by the following expression.
If a laterally asymmetric aperture 13 is installed between the image formation lens 11 and the microlens 15, therefore, the size of the image formed on the aperture 13 changes depending on the distance. As described with reference to
Both the aperture portion 131 and the shield portion 133 in the aperture 13 have the size of 3 μm. Therefore, the condition of the real image radius r′ on the aperture 13 having N stripe pattern peaks formed thereon can be represented as a range of 3(N−1)<r′<3N because of the size of the aperture portion 131 on which light is incident. Results obtained by substituting r′ in Expression (2) into this condition and conducting a calculation are shown in
Rearranging it, the following Expression (4) is obtained.
Substituting values of f and R into Expression (4), the relation
1.0×104 mm<a<2.1×104 mm
is obtained. If the distance “a” to the subject is in this range, two stripe pattern peaks exist as shown in
(Method for Obtaining Two-Dimensional Image Concurrently with Distance Information)
A method for obtaining a two dimensional image concurrently with distance information will now be described with reference to
Supposing that the imaging element 17 includes M×N pixels in this case, the pixel block 17a includes M×N/(k×k) pixels. In the example shown in
A method for reproducing an original two-dimensional image from the image obtained by the imaging element 17 will now be described. The image of the microlens obtained by each pixel block 17a is formed to be blurred although it depends upon the distance to the subject. Therefore, an image is reconfigured by averaging signals obtained by pixels in each pixel block 17a and converting one pixel block to one pixel. This conversion is also conducted by the signal processing circuit shown in
According to the present embodiment, the resolution of the distance to the subject can be improved by incorporating the aperture as heretofore described.
It is possible to relax the light quantity change depending on the distance to the subject while measuring the distance by making the shape of the aperture asymmetric with respect to the optical axis center as described above.
Furthermore, since distance imaging in an ocellus (for example, a single lens or a single imaging element) is made possible, it is possible to simplify the system, reduce the cost, and suppress yield lowering caused by a component alignment error at the time of assembly as compared with compound eyes (for example, a plurality of lenses or a plurality of imaging elements).
Since the aperture does not have a mechanism part, errors such as metal fatigue caused by time elapse can be decreased at least.
While certain embodiments have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the inventions. Indeed, the novel methods and systems described herein may be embodied in a variety of other forms; furthermore, various omissions, substitutions and changes in the form of the methods and systems described herein may be made without departing from the spirit of the inventions. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the inventions.
Number | Date | Country | Kind |
---|---|---|---|
2010-199838 | Sep 2010 | JP | national |
Number | Name | Date | Kind |
---|---|---|---|
20040027681 | Sunagawa et al. | Feb 2004 | A1 |
20070014019 | Mouli | Jan 2007 | A1 |
20070040828 | Zalevsky et al. | Feb 2007 | A1 |
20080142681 | Takizawa et al. | Jun 2008 | A1 |
20080317454 | Onuki | Dec 2008 | A1 |
20090027542 | Yamamoto et al. | Jan 2009 | A1 |
20090128658 | Hayasaka et al. | May 2009 | A1 |
20090153693 | Onuki et al. | Jun 2009 | A1 |
20100066823 | Westphal et al. | Mar 2010 | A1 |
20100141802 | Knight et al. | Jun 2010 | A1 |
20100283884 | Hayasaka et al. | Nov 2010 | A1 |
20120050589 | Ueno et al. | Mar 2012 | A1 |
20120062771 | Ueno et al. | Mar 2012 | A1 |
Number | Date | Country |
---|---|---|
62-089010 | Apr 1987 | JP |
2004-251984 | Sep 2004 | JP |
2007-155930 | Jun 2007 | JP |
2008-152065 | Jul 2008 | JP |
2009-244662 | Oct 2009 | JP |
Entry |
---|
Japanese Office Action mailed Aug. 17, 2012 for Japanese Application No. 2010-199838. |
Adelson et al, Single Lens Stereo with a Plenoptic Camera, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, No. 2, 1992. |
Number | Date | Country | |
---|---|---|---|
20120057020 A1 | Mar 2012 | US |