1. Technical Field
The present invention relates to a color imaging element having a phase difference detection pixel and an imaging apparatus equipped with the color imaging element.
2. Related Art
An imaging element such as a CCD type or a CMOS type is configured by arraying pixels, which are photoelectric conversion elements, in a 2D array pattern.
The plurality of such pixels 1 is arrayed on a surface of a semiconductor board in the 2D array pattern. However, an imaging element in which a phase difference detection pixel coexists in an array of pixels 1 that captures a subject image has been recently proposed and applied to an actual device.
A pair of the phase difference detection pixels 2 and 3 in which the light blocking layer openings 2b and 3b are displaced horizontally are used to acquire left-right-direction, that is, horizontal incident light phase difference information. In addition, when the phase difference detection pixel pair is arranged horizontally in an imaging element, a horizontal distribution of the phase difference information may be acquired and for example, a focus distance up to a subject may be detected from the distribution information. As a result, the phase difference detection pixels 2 and 3 are used as focus detecting pixels in Patent Literature 1 (JP-A-2011-252955) and Patent Literature 2 (JP-A-2011-242514).
When all the pixels for 1 row become the phase difference detection pixel pairs as described above, it is advantageous in that a horizontal resolution of the detected phase difference information is increased. Although a description has been made above on an example in which the phase difference pixel pair is configured by making the narrow light blocking layer openings 2b and 3b be eccentric to the center of the pixel, one ellipsoidal microlens may be mounted with respect to 2 pixels adjacent to the ordinary pixels 1 to be pupil-divided, which become the phase difference pixel pair.
That is, the phase difference detection pixel is handled in the same manner as a defective pixel in regard to imaging a subject. Like the related art illustrated in
In the imaging element 5 illustrated in
With respect to the constitution of
In particular, in an imaging element that captures a color image, by considering a relationship between resolution of phase difference information and an image quality when acquiring a captured image signal at the phase difference detection pixel position through a pixel interpolation operation as well as a relationship with a color filter array, and it needs to be thought which position is preferable for placing the phase difference detection pixel.
An illustrative aspect of the present invention is to provide a color imaging element and a color imaging apparatus that implement a combination of a color filter array and a placement position of a phase difference detection pixel which is capable of detecting phase difference information having high resolution and further, pixel-interpolating a captured image signal at a phase difference detection pixel position with a high definition by a captured image signal of a neighboring pixel.
According to an aspect of the present invention, it is a color imaging element in which color filters of a predetermined color filter array are placed on a plurality of pixels constituted by photoelectric conversion elements arrayed in horizontal and vertical directions, wherein the color filter array includes an array pattern of a 3×3 pixel group in which first filters corresponding to a first color that contributes most to acquiring a luminance signal and second filters corresponding to two or more second colors other than the first color are arrayed, and the first filters are placed at a center and 4 corners in the 3×3 pixel group, and the array pattern is repeatedly placed in the horizontal and vertical directions, the first filter is placed in a line of each of horizontal, vertical and oblique directions of the color filter array, a proportion of a number of pixels of the first color corresponding to the first filters is larger than a proportion of a number of pixels of each color of the second colors corresponding to the second filters, and in a pixel group within a predetermined area of the color imaging element, phase difference detection pixels for acquiring phase difference information are placed in entire components of one direction among components in the horizontal direction and components in the vertical direction in the pixel group.
According to another aspect of the present invention, it is an imaging apparatus equipped with the color imaging element.
With any configuration discussed above, it is possible to detect phase difference information having high resolution and further, to pixel-interpolate a captured image signal at a phase difference detection pixel position with a high definition by a captured image signal of a neighboring pixel.
Hereinafter, exemplary embodiments of the present invention will be described with reference to the accompanying drawings.
An image signal processing circuit 34 that performs known image processing of the output image signal of the imaging element 31, a compression processing circuit 35 that compresses the image signal after the image processing into JPEG image data, or the like, a video encoder 37 that displays a captured image or a through image (live view image) on an image display apparatus 36 installed on a rear surface of the imaging apparatus 30, or the like, a CPU 40 that integrally controls the imaging apparatus 30, a circuit 41 that detects an automatic exposure (AE), an automatic focus (AF), and an automatic white balance (AWB) by processing a signal output from the imaging element 31 as the through image, a main memory 42, a processing circuit 43 that acquires a captured image signal at a phase difference detection pixel position through pixel interpolation, and a media controller 45 that stores JPEG image data in a recording media 44 are connected to the bus 32. The function of the processing circuit 43 is a part of functions of the image signal processing circuit 34.
The imaging element 31 is driven by a timing signal from a timing generator 47 and the timing generator 47 is operated by an instruction from the CPU 40. A focus lens position of the photographing lens 29 is also driven by the instruction from the CPU 40. A shutter release button 48 is connected to the CPU 40.
Before describing a relationship between a layout of the phase difference detection pixels and the color filter array in the imaging element 31, layout positions of the phase difference detection pixels will be described using
In
In an example illustrated in
Further, ‘in a pixel group within a predetermined area of the color imaging element, the phase difference detection pixels are placed in all horizontal components’ represents a state in which the phase difference pixels are arranged in all horizontal pixel positions in the predetermined area when the color imaging element is viewed in a vertical direction. That is, it refers to a state in which, as a result of moving the phase difference detection pixels on each vertical line in the vertical direction and rearranging the phase difference pixels in one horizontal row (for example, a third row), there is not even a gap of one pixel and phase difference detection pixels are placed entirely in the horizontal direction. For example, in
Further, ‘in a pixel group within a predetermined area of the color imaging element, the phase difference pixels are placed in all vertical components’ represents a state in which all of the phase difference detection pixels are arranged in the vertical direction in the predetermined area when viewed in the horizontal direction. That is, it refers to a state in which, as a result of moving the phase difference detection pixels on each horizontal line in the horizontal direction and rearranging the phase difference detection pixels in one vertical column (for example, a third column), there is not even a gap of one pixel and the phase difference detection pixels are placed entirely in the vertical direction. For example, in a 6×6 pixel group in a lower left part of
A combination of XY that makes a pair includes an X pixel and a Y pixel on the same vertical line (for example, the same pixel column 61). The pair pixels XY may deviate from each other in the vertical direction to this extent or several times this extent. This will not cause any problem in acquiring phase difference information since, for example, in a recent imaging element with 10 megapixels or more, it may be though that an image on the same horizontal line of the same subject is incident on the pair pixels XY.
When comparing the example of
Here, descriptions will be made on distribution data of phase difference information obtained when the phase difference detection pixel pairs XY, which are adjacent to each other in the vertical direction, are arranged in the horizontal direction with a pair as a unit.
As can be seen from
That is, an X detection signal line that connects detection signal values detected by the respective phase difference detection pixels X of the horizontal line and a Y detection signal line that connects detection signal values detected by the phase difference detection pixels Y of the horizontal line deviate from each other by a left-right a parallax amount, that is, a phase difference amount when the same subject is viewed with left and right eyes. When the deviation amount (phase difference amount) is obtained, a distance up to the subject may be calculated.
Upon receiving the data of
Plot points on the X detection signal line illustrated in
When the phase difference information having high resolution is obtained, the X pixel and the Y pixel that constitute a pair are spaced apart from each other by 3 pixels in the example of
Therefore, the array of the phase difference detection pixel pairs XY illustrated in
When a vertical displacement is also disregarded in the example of
Moreover, in the case of the configurational example of
According to the array example of
In an array example of
When the pair pixel XY is configured to be formed in the horizontal direction as well as in the vertical direction, it is possible to obtain the distribution data of the vertical phase difference information as well as to obtain the distribution data of the horizontal phase difference information.
In an array example of
According to the array example of
An X pixel and a Y pixel on the same vertical line may be set as a pair pixel or an X pixel and a Y pixel on the same horizontal line may be set as a pair pixel. The distribution data of the phase difference information according to the present example may be obtained either in the horizontal direction or in the vertical direction and the resolution is high either in the horizontal direction or in the vertical direction becomes high like the phase difference pixel array of
In an array example of
Since a lot of ordinary pixels are also present at the same vertical position and the same horizontal position of the phase difference detection pixel in the layout of the phase difference detection pixels in the oblique direction illustrated in
The color filter array is formed by alternately and repeatedly arraying a first array of
In the first array of
When the first array and the second array illustrated in
As a modified example of the color filter array of
The first array of
As a modified example of the color filter array of
Hereinafter, the arrays of the phase difference detection described in
(1) the color filter array is a color filter array which is arrayed on pixels square-lattice-arrayed in the horizontal direction and the vertical direction of the single-plate type color imaging element,
(2) the color filter array includes a predetermined basic array pattern in which a first filter corresponding to a first color (for example, green) that contributes most to acquiring a luminance signal and second filters corresponding to two or more second colors (for example, blue and red) other than the first color are arrayed,
(3) the basic array pattern is repeatedly placed in the horizontal direction and the vertical direction,
(4) the first filter is placed in each line of horizontal, vertical, and oblique (diagonal) lines of the color filter array,
(5) one or more second filters are placed in each line of the horizontal and vertical lines of the color filter array in the basic array pattern, and
(6) the proportion of the number of pixels for the first color corresponding to the first filter is larger than the proportion of the number of pixels for each color of the second colors corresponding to the second filters.
According to the color imaging element that satisfies the above-described conditions, since the first filter corresponding to the first color that contributes most to acquiring the luminance signal is placed in each line of the horizontal, vertical, and oblique lines of the color filter array, reproduction precision of synchronization (interpolation) processing (also referred to as de-mosaic processing) in a high-frequency area may be improved and further, since one or more second filters corresponding to two or more second colors other than the first color are placed in each line of the horizontal and vertical directions of the color filter array, generation of color Moiré (false color) may be suppressed to achieve high resolution.
In the color filter array, since a predetermined basic array pattern is repeatedly placed in the horizontal direction and the vertical direction, when the synchronization processing (interpolation) processing (also referred to as de-mosaic processing) is performed at a later stage, the processing may be performed according to a repetition pattern, and as a result, the latter stage processing may be simplified as compared to a random array in the related art.
Since the proportion of a pixel number for the first color corresponding to the first filter and the proportion of a pixel number for each color of the second colors corresponding to the second filters are made to be different from each other, in particular, since the proportion of the pixel number for the first color that contributes most to acquiring the luminance signal is larger than the proportion of the pixel number for each color of the second colors corresponding to the second filters, aliasing may be suppressed and high-frequency reproducibility may also be improved.
Preferably, the color filter array may include a part where the first filters are consecutive over two or more pixels in each line of the horizontal, vertical, and oblique lines. As a result, a direction (a direction having a high correlation) which is small in change of luminance in the horizontal, vertical, and oblique directions may be determined with a minimum pixel interval.
The color filter array may include a square array corresponding to 2×2 pixels constituted by the first filters. The direction having the high correlation among the horizontal, vertical, and oblique directions may be determined by using pixel values of the 2×2 pixels.
More preferably, color filter arrays in a predetermined basic array pattern may be point-symmetric to each other around the center of the basic array pattern. As a result, it is possible to reduce a circuit scale of the processing circuit at the latter stage.
More preferably, in the color filter array, the first filters may be placed at the center and four corners in the pixel group of 3×3 pixels and the pixel group of the 3×3 pixels is repeatedly placed in the horizontal direction and the vertical direction. Since the first filters are placed at four corners in the pixel group, when the pixel group is repeatedly placed in the horizontal direction and the vertical direction, the color filter array includes a square array corresponding to 2×2 pixels constituted by the first filters. The direction having the higher correlation among the horizontal, vertical, and oblique directions may be determined by using the pixel values of the 2×2 pixels and further, the first filters may be placed in each line of the horizontal, vertical, and oblique lines of the color filter array.
More preferably, the second filters may be placed in each line of the horizontal, vertical, and oblique lines of the color filter array. As a result, oblique color reproducibility may be further improved.
As a suitable exemplary embodiment of a color filter array that satisfies the above conditions, hereinafter, descriptions will be made using the color filter array of
Hereinafter, after
Conditions regarding the layout of phase difference detection pixels:
(1) In order to increase the resolution (resolving power) of the phase difference information, the phase difference detection pixels should be placed compactly without a gap in the horizontal direction, if possible.
(2) In order to calculate a captured image signal of a phase difference detection pixel with high precision by the pixel interpolation, the phase difference detection pixel should be placed at a place where a large number of concolorous ordinary pixels are present around the phase difference detection pixel.
(3) Since an imaging element may image a moving image by thinning reading, a phase difference detection pixel line should be formed at a position where it does not superimpose on a reading pixel line.
(4) In order to suppress an influence by a color mixture, an R pixel should not be positioned adjacent to a phase difference detection pixel.
As can be seen from the color filter array of
Originally, a proportion of G pixels in the color imaging element is higher than those of R pixels and B pixels. As a result, even though the G pixels, of which the number is larger than the number of the R pixels and the B pixels, become the phase difference detection pixels not to be used for capturing for a subject image, the G pixels do not degrade the quality of a captured color image.
In the exemplary embodiment of
When horizontal lines are set as phase difference detection pixel lines as in the present exemplary embodiment, it would be get better, for example, if detection signals of the phase difference detection pixel lines are read when a high-definition still image is picked and further, phase difference information of high resolution is required. In addition, when photographing a moving image, the photographing is terminated when captured image signals of ordinary R, G, and B pixels other than the phase difference detection pixels are read by thinning reading without reading the signal of the phase difference detection pixel line.
When moving image data is read from the color imaging element by a pixel thinning operation, various pixel thinning reading methods such ½ pixel thinning, ⅓ pixel thinning, even line thinning, odd line thinning may be used. However, when the CPU 40 has known in advance which place of the color imaging element 31 the phase difference detection pixel lines are formed, the CPU 40 may correspondingly instruct the timing generator 47 to read the captured image signal by skipping a phase difference detection pixel line.
In the example of
When a G pixel in a horizontal line where G pixels are sparse is set as a phase difference detection pixel, it is difficult to enhance interpolation precision of a captured image signal of the G pixel. As a result, it is desirable that the horizontal line where the G pixels are sparse is not set as a phase difference detection pixel line.
In the example of
Further, in
In
According to the exemplary embodiments described above, phase difference information having high resolution can be obtained and a captured image signal of the phase difference detection pixel position can be interpolated with high precision. When the phase difference detection pixels are placed in such a manner that two phase difference detection pixels are not adjacent to each other in the horizontal and vertical directions, the resolution of the obtained phase difference information and interpolation precision of the captured image signal of the phase difference detection pixel position have a trade-off relationship in any sense.
However, an array as illustrated in
Further, in the above-described exemplary embodiments, an example, in which the phase difference pixels X are compactly placed even without an interval of one pixel in the horizontal direction, for example, as illustrated in
As discussed above, the exemplary embodiments disclose a color imaging element in which color filters of a predetermined color filter array are placed on a plurality of pixels constituted by photoelectric conversion elements arrayed in horizontal and vertical directions, in which the color filter array includes an array pattern of a 3×3 pixel group in which first filters corresponding to a first color that contributes most to acquiring a luminance signal and second filters corresponding to two or more second colors other than the first color are arrayed, and the first filters are placed at a center and 4 corners in the 3×3 pixel group, and the array pattern is repeatedly placed in the horizontal and vertical directions, the first filter is placed in a line of each of horizontal, vertical and oblique directions of the color filter array, a proportion of a number of pixels of the first color corresponding to the first filters is larger than a proportion of a number of pixels of each color of the second colors corresponding to the second filters, and in a pixel group within a predetermined area of the color imaging element, phase difference detection pixels for acquiring phase difference information are placed in entire components of one direction among components in the horizontal direction and components in the vertical direction in the pixel group.
The exemplary embodiments disclose the color imaging element in which in the pixel group within the predetermined area, a first phase difference detection pixel and a second phase difference detection pixel that constitute a pair are placed on a pixel line for ones of horizontal lines arrayed in the horizontal direction and vertical lines arrayed in the vertical direction, in order to detect a phase difference.
The exemplary embodiments disclose the color imaging element, in which the first and second phase difference detection pixels are formed on the pixels having the first filter.
The exemplary embodiments disclose the color imaging element, in which the phase difference detection pixels are placed on a pixel line for ones of the horizontal pixel lines and the vertical pixel lines in which a number of pixels with the first filters is relatively large.
The exemplary embodiments disclose the color imaging element, in which all pixels on a pixel line for ones of the horizontal pixel lines and the vertical pixel lines are set as the phase difference detection pixels and color filters of the phase difference detection pixels on the line are set as concolorous filters, transparent filters, or white filters.
The exemplary embodiments disclose the color imaging element, in which the color filter array includes an oblique line array in which the first filters are consecutively placed in an oblique direction and the phase difference detection pixel is placed on a pixel corresponding to the oblique line array of the first filter.
The exemplary embodiments disclose the color imaging element, in which first phase difference detection pixel of the first and second phase difference detection pixels that constitutes a pair is placed in one of pixels corresponding to two adjacent oblique line arrays in order to detect the phase difference and the second phase difference detection pixel that constitutes the pair is placed in the other one of the pixels corresponding to the two adjacent oblique line arrays.
The exemplary embodiments disclose the color imaging element, in which the phase difference detection pixel is placed at a position of the first filter and the placed phase difference detection pixel is placed at a position different, in the vertical direction and the horizontal direction, from that of another phase difference detection pixel placed on a pixel line of one of the horizontal direction and the vertical direction adjacent to the phase difference detection pixel.
The exemplary embodiments disclose the color imaging element, in which in the color filter array, one or more second filters corresponding to each color of the second colors are placed in each line of one of the horizontal and vertical directions of the color filter array.
The exemplary embodiments disclose the color imaging element, in which the first color is green color and the second colors are red color and blue color, an array pattern, in which, among 4 pixels other than the pixels at the center and 4 corners in the 3×3 pixel group, 2 pixels on a same vertical line are set as red color and 2 remaining pixels are set as blue color, is set as a first array, and an array pattern, in which, among 4 pixels, 2 pixels on a same horizontal line are set as red color and 2 remaining pixels are set as blue color, is set as a second array, and the first array and the second array are alternately placed in both the horizontal and vertical directions.
The exemplary embodiments disclose the color imaging element, in which the first color is green color and the second colors are red color and blue color, 4 pixels other than the pixels at the center and the 4 corners in the 3×3 pixel group are divided into two parts each including two obliquely adjacent pixels, and an array pattern, in which 2 pixels in one part are set as red color and 2 pixels in the other part are set as the blue color, is set as the first array and an array pattern, in which 2 pixels in the one part are set as blue color and 2 pixels in the other part are set as red color, is set as the second array, and the first array and the second array are alternately placed in both the horizontal and vertical directions.
The exemplary embodiments disclose the color imaging element, in which a square lattice shaped 6×6 pixel group which includes two first arrays and two second arrays is set as a basic array pattern, and at least one phase difference detection pixel is placed on each pixel line according to in one of the vertical and horizontal directions in the basic array pattern.
The exemplary embodiments disclose the color imaging element, in which a first phase difference detection pixel and a second phase difference detection pixel that constitute a pair are placed on the each pixel line in one of the vertical and horizontal directions in order to detect a phase difference.
The exemplary embodiments disclose the color imaging element, in which a square lattice shaped 6×6 pixel group which includes two first arrays and two second arrays is set as a basic array pattern, and a first phase difference detection pixel of the first and second phase difference detection pixels that constitute a pair is placed in each pixel line in the vertical direction in the basic array pattern and the second phase difference detection pixel is placed on each pixel line in the vertical direction in another basic array pattern vertically adjacent to the one basic array pattern in order to detect a phase difference.
The exemplary embodiments disclose the color imaging element, in which the second phase difference detection pixel is placed in another basic array pattern which is horizontally adjacent to the basic array pattern in which the first phase difference detection pixel is placed.
The exemplary embodiments disclose the color imaging element, in which a square lattice shaped 6×6 pixel group which includes two first arrays and two second arrays is set as a basic array pattern, two basic array patterns adjacent to each other in one of the vertical and horizontal directions are formed as one set, the phase difference detection pixels are placed at pixel positions where the green first filters are densely arrayed in one of the vertical and horizontal directions and pixels other than the phase difference detection pixels are placed at pixel positions where the green first filters are sparsely placed in one of the vertical and horizontal directions, and the first phase difference detection pixel and the second phase difference detection pixel that constitute the pair are placed over the one set of two adjacent basic array patterns.
The exemplary embodiments disclose an imaging apparatus equipped with the color imaging element.
The exemplary embodiments disclose the imaging apparatus, comprising: a thinning reading unit which generates a moving image by thinning and reading a captured image signal from a pixel on a line where the phase difference detection pixel is not present, among vertical pixel lines and horizontal pixel lines.
The exemplary embodiments disclose the imaging apparatus, further comprising: an AF processing unit which performs AF processing by using a detection signal of the phase difference detection pixel.
According to the exemplary embodiments, both the high-resolution phase difference information and the high pixel interpolation precision of a captured image signal of a phase difference detection pixel position may be obtained.
The color imaging element according to the present invention may obtain both the high-resolution phase difference information and the high pixel interpolation precision of the captured image signal of the phase difference detection pixel position, and may be usefully applied to a digital still camera, a digital video camera, etc.
This application claims priority to and the benefit of Japanese Patent Application No. 2011-288032 filed in the Japan Patent Office on Dec. 28, 2011, the entire contents of which are incorporated herein by reference.
Number | Date | Country | Kind |
---|---|---|---|
2011-288032 | Dec 2011 | JP | national |
This is a continuation of International Application No. PCT/JP2012/082384 filed on Dec. 13, 2012, and claims priority from Japanese Patent Application No. 2011-288032, filed on Dec. 28, 2011, the entire disclosures of which are incorporated herein by reference.
Number | Name | Date | Kind |
---|---|---|---|
5251019 | Moorman et al. | Oct 1993 | A |
5889554 | Mutze | Mar 1999 | A |
6829008 | Kondo et al. | Dec 2004 | B1 |
20080131028 | Pillman et al. | Jun 2008 | A1 |
20090200451 | Conners | Aug 2009 | A1 |
20110249156 | Goto | Oct 2011 | A1 |
20120176520 | Hara | Jul 2012 | A1 |
Number | Date | Country |
---|---|---|
1793620 | Jun 2007 | EP |
H02210996 | Aug 1990 | JP |
5-505718 | Aug 1993 | JP |
2010-512048 | Apr 2010 | JP |
2011-242514 | Dec 2011 | JP |
2011-252955 | Dec 2011 | JP |
Entry |
---|
International Search Report, issued in PCT/JP2012/082384, dated Jan. 29, 2013. |
Written Opinion of the International Searching Authority, issued in PCT/JP2012/082384, dated Jan. 29, 2013. |
Extended European Search Report, dated Aug. 4, 2014, in related application No. EP11859479.5. |
Extended European Search Report, dated Jan. 27, 2015, in related application No. EP12804973.1. |
Extended European Search Report, dated Oct. 28, 2014, in related application No. EP11859950.5. |
Hirakawa, “Spatio-Spectral Color Filter Array Design for Optimal Image Recovery”, IEEE Transactions on Image Processing 17:10:1876-1890 (Oct. 2008). |
Number | Date | Country | |
---|---|---|---|
20140313380 A1 | Oct 2014 | US |
Number | Date | Country | |
---|---|---|---|
Parent | PCT/JP2012/082384 | Dec 2012 | US |
Child | 14319887 | US |