This application is based upon and claims the benefit of priority from prior Japanese Patent Application No. 2008-292237, filed Nov. 14, 2008, the entire contents of which are incorporated herein by reference.
1. Field of the Invention
The present invention relates to a solid-state image pickup device having a built-in processing circuit such as an image sensor that processes an image signal, and it is used for, e.g., a mobile phone equipped with an image sensor, a digital camera, or a video camera.
2. Description of the Related Art
A reduction in size of a camera module for a mobile phone has been strongly demanded with a reduction in width of a mobile phone. Additionally, a camera module for a mobile phone, which is hardly destroyed even if it is dropped, has been demanded. Further, in recent years, an increase in the number of pixels, e.g., 5 mega pixels or more or 8 mega pixels or more has advanced to fulfill the demand for an improvement in image quality. In a multi-pixel sensor, a depth of field becomes shallow with a reduction in a pixel size. Therefore, an autofocus (AF) mechanism is required, and hence reducing a size of a module is difficult, and there is also problem that the module is prone to breakage when it is dropped. Each of Alon et al. (U.S. Pat. No. 7,065,256), Mendlovic et al. (U.S. Pat. No. 5,909,312), and Mendlovic et al. (U.S. Pat. No. 6,343,307) discloses a solid-state image pickup device that deepens a depth of field. However, a countermeasure for a problem of the chromatic aberration in a lens is not provided, a problem of occurrence of a false color in an image arises, or a resolution signal is not sufficiently obtained, and hence a color reproduced image having a sufficient resolution or a high quality is not acquired.
According to a first aspect of the present invention, there is provided a solid-state image pickup device comprising:
a sensor unit in which pixels formed of photoelectric transducers which convert a signal of light condensed by an optical lens which condenses light and whose focal position differs depending on a wavelength of the light into electric signals and optical filters which are arranged on front surfaces of the photoelectric transducers and separate the wavelength of the light are two-dimensionally arranged, and which separates the signal of light into at least 2 or more wavelength components to generate electrical wavelength signals; and
a signal processing circuit which processes the at least 2 wavelength signals generated by the sensor unit,
wherein the signal processing circuit includes:
a plurality of contour signal extraction circuits which receive the at least 2 wavelength signals to extract respective contour signals; and
a contour signal selection circuit which receives a control signal and the plurality of contour signals extracted by the plurality of contour signal extraction circuits, and selects and outputs a contour signal having a desired wavelength band in accordance with the control signal.
According to a second aspect of the present invention, there is provided a solid-state image pickup device comprising:
a sensor unit in which pixels formed of photoelectric transducers which convert a signal of light condensed by an optical lens which condenses light and whose focal position differs depending on a wavelength of the light into electric signals and optical filters which are arranged on front surfaces of the photoelectric transducers and separate the wavelength of the light are two-dimensionally arranged, and which separates the signal of light into 3 wavelength components, i.e., a first color wavelength band and a second color wavelength band using 2 out of a blue wavelength band, a green wavelength band, and a red wavelength band, and all wavelength bands to generate electrical wavelength signals; and
a signal processing circuit which processes the 3 wavelength signals generated by the sensor unit,
wherein the signal processing circuit includes:
a wavelength signal generation circuit which receives the 3 wavelength signals having the all wavelength bands, the first color wavelength band, and the second color wavelength band to generate a wavelength signal having a third color wavelength band;
a plurality of contour signal extraction circuits which receive the 3 wavelength signals having the all wavelength bands, the first color wavelength band, and the second wavelength band to extract contour signals, respectively; and
a contour signal selection circuit which receives a control signal and the plurality of contour signals extracted by the plurality of contour signal extraction circuits, and selects and outputs a contour signal having a desired wavelength band in accordance with the control signal.
According to a third aspect of the present invention, there is provided a solid-state image pickup device comprising:
a sensor unit in which pixels formed of photoelectric transducers which convert a signal of light condensed by an optical lens which condenses light and whose focal position differs depending on a wavelength of the light into electric signals and optical filters which are arranged on front surfaces of the photoelectric transducers and separate the wavelength of the light are two-dimensionally arranged, and which separates the signal of light into 3 wavelength components having a blue wavelength band, a green wavelength band, and a red wavelength band to generate electrical wavelength signals; and
a signal processing circuit which processes the 3 wavelength signals generated by the sensor unit,
wherein the signal processing circuit includes:
first to third contour signal extraction circuits which receive the three wavelength signals having the blue wavelength band, the green wavelength band, and the red wavelength band to extract first to third contour signals;
a contour signal synthesis circuit which receives the first to third contour signals and synthesizes a contour signal of a wavelength signal having all wavelength bands to be output as a fourth contour signal; and
a contour signal selection circuit which receives a control signal, the fourth contour signal, and the first to third contour signals, and selects and outputs a contour signal having a desired wavelength band in accordance with the control signal.
Embodiments according to the present invention will now be described hereinafter with reference to the accompanying drawings. It is to be noted that like reference numerals denote like parts in a description of the embodiments, thereby omitting a tautological explanation.
In the sensor unit 21, a pixel array 211 and a column type analog/digital converter (ADC) 212 are arranged. In the pixel array 211, photodiodes (pixels) as photoelectric transducers that convert a signal of light condensed by the optical lens 11 into an electric signal are two-dimensionally arranged on a silicon semiconductor substrate. Respective color filters consisting of four types of wavelength separating means for W (transparent), B (blue), G (green), and R (red) are arranged on front surfaces of the photodiodes. As a color arrangement in the color filters, 8 pixels of W having a checkered pattern, 4 pixels of G, 2 pixels of R, and 2 pixels of B are arranged in a basic 4×4 pixel arrangement. In the sensor unit 21, a wavelength of light that enters the photodiodes is divided into four by the color filters, and they are converted into signal charges by the two-dimensional photodiode array, and further converted into digital signals by the ADC 212 to be output. Further, a non-illustrated microlens is arranged in each pixel.
Output signals from the sensor unit 21 are supplied to the line memory 22, and the output signals corresponding to, e.g., vertical 7 lines are stored in the line memory 22. The signals corresponding to the 7 lines are read out in parallel to be input to the focus adjustment circuit 23.
In the focus adjustment circuit 23, a plurality of pixel interpolation circuits 231a to 231d execute interpolation processing with respect to the respective signals W, B, G, and R. The signals subjected to the interpolation processing are supplied to a plurality of contour extraction circuits 232a to 232d in parallel, thereby extracting respective contour signals Ew, Eb′, Eg′, and Er′. Furthermore, the respective signals B, G, and R subjected to the interpolation processing are supplied to a plurality of low-pass filter (LPF) circuits 233a to 233c in parallel where signals having respective signal bands limited to low frequencies, e.g., signals from which high-frequency signals are removed are generated. The contour signals Ew, Eb′, Eb′, and Er′ extracted by the plurality of contour extraction circuits 232a to 232d are supplied to a contour signal selection circuit 234. A control signal PSEL consisting of, e.g., 4 bits output from the command decoder 26 is supplied to the contour signal selection circuit 234. The contour signal selection circuit 234 outputs a contour signal PEw having a desired wavelength band in accordance with the contour signals Ew, Eb′, Eg′ and Er′ and the control signal PSEL. A level of the contour signal PEw output from the contour signal selection circuit 234 is appropriately adjusted by a level adjustment circuit 235. The level adjustment circuit 235 outputs contour signals PEwa and PEwb. The contour signal PEwa is supplied to a plurality of addition circuits 236a to 236c. In the addition circuits 236a to 236c, the signals B, G, and R limited to low frequencies which are output from the plurality of LPF circuits 233a to 233c are added to the contour signal PEwa subjected to the level adjustment, respectively. The signals Gle, Rle, and Ble subjected to the addition in the plurality of LPF circuits 233a to 233c and the level-adjusted contour signal PEwb are supplied to the downstream signal processing circuit 24. In the downstream signal processing circuit 24, general white balance adjustment, color adjustment (an RGB matrix), γ correction, YUV conversion processing, and others are executed, and these signals are output as digital signals DOUT0 to DOUT7 having a YUV signal format or an RGB signal format.
The coefficients in formula (1) can be varied in accordance with the spectral characteristics, color temperature, color reproduction characteristics of a sensor.
With respect to output signals R, G and B of the γ correction circuit 243, the YUV conversion circuit 244 executes, for example, the operation expressed by the following formula (2):
Normally, the numerical values in formula (2) are fixed values so that the conversion of R, G and B signals and the conversion of YUV signals can be executed in common. A Y signal output from the YUV conversion circuit 244 is added to contour signal PEwb output from the focus adjustment circuit 23, at a downstream position of the YUV conversion circuit 244. The downstream signal processing circuit 24 outputs digital signals DOUT0-DOUT7 of either YUV signal format or RGB signal format.
As can be understood from the above, the addition of contour signals is performed when the addition circuits 236a to 236c add signals B, G and R to contour signal PEwa and/or when the downstream signal processing circuit 24 adds the Y signal obtained after the YUV conversion processing to contour signal PEwb.
A master clock signal MCK is supplied to the system timing generation circuit 25 from the outside. The system timing generation circuit 25 outputs a clock signal that controls operations of the sensor unit 21, the line memory 22, and the focus adjustment circuit 23.
Moreover, the operations of the line memory 22, the focus adjustment circuit 23, and the system timing generation circuit 25 are controlled by a command. For example, data DATA which is input from the outside is input to the command decoder 26 through the serial interface 27, and a decoded control signal is input to each circuit, thereby controlling parameters and others for processing based on the external input data DATA.
On the other hand, the digital signals DOUT0 to DOUT7 output from the downstream signal processing circuit 24 are supplied to a processor such as a non-illustrated digital signal processor (DSP), thereby generating focus information that is used for focusing. As an example of focus information generation method, a facial part of a person is specified, and focus information with which the specified part can be focused is generated. The generated focus information is supplied as a part of the data DATA to the serial interface 27 in the sensor chip 12. Moreover, this focus information can be also generated in the downstream signal processing circuit 24.
According to a method depicted in
According to a method depicted in
According to a method depicted in
For generation of the contour signal, various methods other than the above methods can be used. For example, besides the 3×3 pixel area and the 5×5 pixel area, a 7×7 pixel area may be adopted, and weighting (the gain) of each pixel can be changed. The contour signal in the respective pixels R, G, and B other than the pixels W can be generated by the same methods as those depicted in
The arithmetic processing circuit 331 outputs a logic 1 (YES=1) when Eb>Ke and outputs a logic 0 in any other case based on an intensity relationship between the contour signal Eb and a preset fixed reference signal level Ke. The arithmetic processing circuit 332 outputs the logic 1 (YES=1) when Eg>Ke and outputs the logic 0 in any other case based on an intensity relationship between the contour signal Eg and the reference signal level Ke. The arithmetic processing circuit 333 outputs the logic 1 (YES=1) when Er>Ke and outputs the logic 0 in any other case based on an intensity relationship between the contour signal Er and the reference signal level Ke. The arithmetic processing circuit 334 outputs the logic 1 (YES=1) when (Eb−Eg)>Ke and outputs the logic 0 in any other case based on an intensity relationship between a signal (Eb−Eg) as a difference of the contour signals Eb and Eg and the reference signal level Ke. The arithmetic processing circuit 335 outputs the logic 1 (YES=1) when Eb>Ke and Eg>Ke and outputs the logic 0 in any other case based on an intensity relationship between each of the contour signals Eb and Eg and the reference signal level Ke. The arithmetic processing circuit 336 outputs the logic 1 (YES=1) when (Eg−Eb−Er)>Ke and outputs the logic 0 in any other case based on an intensity relationship between a signal (Eg−Eb−Er) obtained by subtracting the contour signals Eb and Er from the contour signal Eg and the reference signal level Ke. The arithmetic processing circuit 337 outputs the logic 1 (YES=1) when Eg>Ke and Er>Ke and outputs the logic 0 in any other case based on an intensity relationship between each of the contour signals Eg and Er and the reference signal level. The arithmetic processing circuit 338 outputs the logic 1 (YES=1) when (Er−Eg)>Ke and outputs the logic 0 in any other case based on an intensity relationship between a signal (Er−Eg) as a difference of the contour signals Er and Eg and the reference signal level Ke. Logic operation output signals from the arithmetic processing circuits 331 to 338 are supplied to input terminals “1” to “8” of a selector 34 in parallel. A signal indicative of the logic 1 is supplied to an input terminal “0” of the selector 34. The selector 34 selects the signals supplied to the input terminals “0” to “8” based on the 4-bit control signal PSEL, and outputs a selected signal from an output terminal Y.
The switch circuit 31 is controlled to be switched in accordance with the output signal from the selector 34, outputs the control signal Ew as PEw when the output signal from the selector 34 corresponds to the logic 1, and outputs 0 as PEw when the output signal from the selector 34 corresponds to the logic 0. That is, the output signal from the selector 34 controls whether the contour signal Ew is enabled to be output as PEw or disabled.
As shown in
In this manner, the contour signal selection circuit 234 outputs the contour signal having a desired wavelength band in accordance with the control signal PSEL, and the distance range with which the resolution is obtained can be finely set based on the control signal PSEL. Further, changing the judgment reference level Ke for the contour signal enables changing the distance range with which the contour signal PEw is obtained.
When this contour signal selection circuit is used, the control signal PSEL can be utilized to generate an image with perspective in which a person is focused and a background blurs like the conventional AF. Furthermore, selecting the contour signal PEw for the signal W as the contour signal Ew can realize a setting in which the depth of field is increased to avoid a defocus failure or avoid missing a time for the click of a shutter.
As explained above, the solid-state image pickup device according to the first embodiment can electronically execute focus control and arbitrarily select a focus depth.
The sensor chip 12 according to this embodiment is different from that according to the first embodiment in a color arrangement of color filters in a sensor unit 21, namely, 8 W pixels having a checkered pattern, 4 pixels G, and 4 pixels R are arranged in a basic 4×4 pixel arrangement. When such color filters are adopted, an output of a signal R is doubled as compared with the sensor chip according to the first embodiment. As a result, a signal of a high resolution equal to that of the pixels G can be obtained as a contour signal of the pixels R. With such a change in color arrangement of the color filters, a focus adjustment circuit is also partially changed. That is, since a focus adjustment circuit 23a according to this embodiment has no input for a signal B, a pixel interpolation circuit and a contour extraction circuit for the signal B are omitted. Furthermore, an LPF output signal of each signal B is generated as follows. An LPF circuit 237 is used to remove a high-frequency signal from a signal that has interpolated the pixel W, and an LPF output of the signal G and an LPF output of the signal R are subtracted from an LPF output of the signal W by using a subtraction circuit 238, namely, subtraction (BLPF=WLPF−GLPF−RLPF) is carried out. Moreover, contour signals Ew, Eg′, and Er′ extracted by a plurality of contour extraction circuits 232b to 232d and a control signal PSEL are supplied to a contour signal selection circuit 234a, and the contour signal selection circuit 234a outputs a contour signal PEw having a desired wavelength band in accordance with the contour signals Ew, Eg′, and Er′ and the control signal PSEL.
The arithmetic processing circuit 351 outputs a logic 1 (YES=1) when (Ewa−Eg−Er)>Ke and outputs a logic 0 in any other case based on an intensity relationship between a signal obtained by subtracting contour signals Eg and Er from the contour signal Ewa and the preset fixed reference signal level Ke. The arithmetic processing circuit 352 outputs the logic 1 (YES=1) when Er<Kr and (Eg−Er)>Ke and outputs the logic 0 in any other case based on an intensity relationship between the contour signal Er and the reference signal level Kr and an intensity relationship between a signal (Eg−Er) as a difference between the contour signals Eg and Er and the reference signal level Ke. The arithmetic processing circuit 353 outputs the logic 1 (YES=1) when (Eg−Er)>Ke and outputs the logic 0 in any other case based on an intensity relationship between the signal (Eg−Er) as a difference between the contour signals Eg and Er and the reference signal level Ke. The arithmetic processing circuit 354 outputs the logic 1 (YES=1) when Er>Kr and (Eg−Er)>Ke and outputs the logic 0 in any other case based on an intensity relationship between the contour signal Er and the preset fixed reference signal level Kr and an intensity relationship between the signal (Eg−Er) as a difference between the contour signals Eg and Er and the reference signal level Ke. Operations of the other arithmetic processing circuits 332, 333, 337, and 338 are the same as those in
The logic operation output signals from the arithmetic processing circuits 351, 332, 333, 352, 353, 354, 337, and 338 are supplied to input terminals “1” to “8” of a selector 34 in parallel. A signal of the logic 1 is supplied to an input terminal “0” of the selector 34. The selector 34 selects a signal supplied to the input terminals “0” to “8” based on the 4-bit control signal PSEL and outputs the selected signal from an output terminal Y.
A switch circuit 31 is controlled to be switched in accordance with an output signal from the selector 34, outputs the contour signal Ew as PEw when an output signal from the selector 34 is indicative of the logic 1, and outputs 0 as PEw when the output signal from the selector 34 is indicative of the logic 0. That is, the output signal from the selector 34 controls whether the contour signal Ew is enabled to be output as PEw or disabled.
As shown in
It is to be noted that the pixel W has the high sensitivity that is approximately double the sensitivity of the signal G, which is also true in the first embodiment. Therefore, signal charges photoelectrically converted in a silicon semiconductor substrate are diffused and mixed into other pixels, color mixture to the signals R, G, and B occurs. The sensitivity can be lowered by decreasing a size of a microlens arranged in each pixel W to be smaller than that in any other pixel, thereby reducing this color mixture.
This embodiment can obtain the same effects as those in the first embodiment. That is, focus control can be electronically carried out, and a depth of focus can be arbitrarily selected.
It is to be noted that, in this embodiment, the description has been given as to the color filters having the color arrangement that the 8 pixels W having a checkered pattern, the 4 pixels G, and the 4 pixels R are arranged in the basic 4×4 pixel arrangement is used and the LPF output of the signal G and the LPF output of the signal R are subtracted from the LPF output of the signal W to obtain the LPF output signal of the signal B by using the subtraction circuit 238. However, as a modification, the color filters having a color arrangement that the pixels W, the pixels G, and the pixels B are arranged may be used and the LPF output of the signal G and the LPF output of the signal B may be subtracted from the LPF output of the signal W to obtain the LPF output signal of the signal R by using the subtraction circuit, or the color filters having a color arrangement that the pixels W, the pixels R, and the pixels B are arranged may be used and the LPF output of the signal R and the LPF output of the signal B may be subtracted from the LPF output of the signal W to obtain the LPF output signal of the signal G by using the subtraction circuit.
The sensor chip 12 according to this embodiment is different from that according to the first embodiment in that a color arrangement of color filters in a sensor unit 21 is a general Bayer arrangement that two pixels G, one pixel B, and one pixel R are arranged in a basic 2×2 pixel arrangement. With such a change in color arrangement of the color filters, a focus adjustment circuit is also partially changed. That is, since a focus adjustment circuit 23b according to this embodiment does not have an input for signals W, a pixel interpolation circuit and a contour extraction circuit for the signals W are omitted. Furthermore, a contour signal synthesis circuit 239 combines the contour signal Eb′ of the signal B, the contour signal Eg′ of the signal G, and the contour signal Er′ of the signal R to generate a contour signal Ew for the signal W. Moreover, the contour signals Eb′, Eg′, and Er′ extracted by a plurality of contour extraction circuits 232a to 232c, the contour signal Ew synthesized by the contour signal synthesis circuit 239, and the control signal PSEL are supplied to a contour signal selection circuit 234. The contour signal selection circuit 234 outputs a contour signal PEw having a desired wavelength band in accordance with these contour signals Eb′, Eg′, Er′, and Ew and the control signal PSEL.
As shown in
In this manner, the contour signal selection circuit 234 outputs the contour signal having a desired wavelength band in accordance with the control signal PSEL, and the distance range with which the resolution is obtained can be finely set based on the control signal PSEL. Further, changing the judgment reference level Ke for the contour signal enables changing the distance range with which the contour signal PEw is obtained.
This embodiment can obtain the same effects as those of the first embodiment. That is, the focus control can be electronically performed, and a depth of focus can be arbitrarily selected.
As explained above, according to the solid-state image pickup device of the present invention, the focus control can be electronically performed, and a depth of focus can be arbitrarily selected.
Moreover, the solid-state image pickup device according to the present invention can obtain the following various effects. Since the mechanical AF is not used, the device is hard to be destroyed even if it is dropped. A height of a module can be reduced. Decreasing the number of lenses enables further reducing the height of the module. When a value F of the lens is reduced, the sensitivity can be improved. In a fixed-focus operation with a large depth of focus, since the AF is not used, a shutter can be rapidly released, and hence a time for the click of the shutter is hardly missed. Additionally, defocus caused due to an erroneous operation of the AF can be avoided. In a digital automatic focus operation, since the mechanical AF operation is not performed, a point of focus can be instantaneously set. Mechanical degradation does not occur. A reproduced image having the same perspective as that in the mechanical AF can be obtained.
Further, although the description has been given as to the single-plate color camera in which the color arrangement of the color filters is RGB or WRGB in each of the foregoing embodiments, even a three-plate camera that uses a prism to obtain RGB signals of 3 primary colors can generate the chromatic aberration, improve the sensitivity based on the same principle, and reduce false colors at edges due to the chromatic aberration. When the color filter arrangement using the pixels W is adopted, the pixels W are arranged in a checkered pattern in order to obtain a resolution signal from each pixel W. The other color filters of R, G, and B have no restriction in arrangement in particular since the LPFs are used to provide hues.
The present invention is not restricted to each of the foregoing embodiments, and it can be modified in many ways without departing from the scope of the invention on an embodying stage. For example, each of the foregoing embodiments includes inventions on various stages, and appropriately combining a plurality of disclosed constituent requirements enables extracting various inventions. For example, even if several constituent requirements are eliminated from all constituent requirements disclosed in each embodiment, the problems described in the section “Problems to be Solved by the Invention” can be solved, and a configuration from which the constituent elements are eliminated can be extracted as the invention when the effects described in the section “Effects of the Invention” are obtained.
Number | Date | Country | Kind |
---|---|---|---|
2008-292237 | Nov 2008 | JP | national |
Number | Name | Date | Kind |
---|---|---|---|
5748371 | Cathey, Jr. et al. | May 1998 | A |
5909312 | Mendlovic et al. | Jun 1999 | A |
6148116 | Park et al. | Nov 2000 | A |
6343307 | Mendlovic et al. | Jan 2002 | B1 |
6373992 | Nagao | Apr 2002 | B1 |
6388706 | Takizawa et al. | May 2002 | B1 |
6714688 | Gallagher et al. | Mar 2004 | B1 |
6842297 | Dowski, Jr. | Jan 2005 | B2 |
7003177 | Mendlovic et al. | Feb 2006 | B1 |
7065256 | Alon | Jun 2006 | B2 |
7148924 | Hiroshige et al. | Dec 2006 | B2 |
7812971 | Jackson et al. | Oct 2010 | B2 |
7889252 | Tamura et al. | Feb 2011 | B2 |
20080291312 | Egawa | Nov 2008 | A1 |
20080303919 | Egawa | Dec 2008 | A1 |
Number | Date | Country |
---|---|---|
1668067 | Sep 2005 | CN |
101247529 | Aug 2008 | CN |
1 662 803 | May 2006 | EP |
62-115408 | May 1987 | JP |
2003-16425 | Jan 2003 | JP |
2005-143038 | Jun 2005 | JP |
2007-318233 | Dec 2007 | JP |
10 2008 00962 | Oct 2008 | KR |
WO 2005018236 | Feb 2005 | WO |
WO 2006077344 | Jul 2006 | WO |
WO 2006095110 | Sep 2006 | WO |
Entry |
---|
Japanese Office Action issued May 8, 2012, in Japan Patent Application No. 2008-292237 (with English translation). |
Office Action issued Jun. 19, 2012, in Chinese Patent Application No. 200910222055.6. |
Combined Taiwanese Office Action and Search Report issued Dec. 18, 2012 in Taiwanese Patent Application No. 098137146 (with English Translation of Categories of Cited Documents). |
Office Action issued on Jul. 29, 2011 in the corresponding Korean Application No. 200910222055.6 (with English Translation). |
Number | Date | Country | |
---|---|---|---|
20100123809 A1 | May 2010 | US |