The present invention relates to an optical low pass filter and an imaging device using the same, and in particular, the invention is suitably used for an optical filter to remove a high-frequency component by utilizing a blur condition of an image. Further, the invention is suitably used for a hybrid camera configured to be able to capture both of a dynamic image and a still image by one set of a camera.
In recent years, a so-called hybrid camera configured to be able to capture both of a dynamic image and a still image by one set of a camera has been provided in the market. Together with this, a device for improving the image quality of the dynamic image and the still image to be captured has been conceived. For example, in almost all the hybrid cameras, an imaging element (image sensor) having a large number of pixels to increase a resolution of the still image is employed. Further, to suppress the generation of a false color (color moiré) by sampling is an important factor to improve an image quality.
Here, a description will be made on a principle of the generation of the false color. The imaging element is an assembly of micro pixels, and to capture an optical image by the imaging element is equivalent to the sampling of signals at intervals of a pixel pitch. For example, in the imaging element whose photo diodes (pixels) are disposed in a square lattice shape, assuming that the pixel pitch is taken as d, a sampling frequency fs becomes fs=1/d.
At this time, as shown in
Hence, usually, as shown in
Patent Document 1: Japanese Patent Laid-Open No. 6-292093
Patent Document 2: Japanese Patent No. 2556831
The optical low pass filter is an optical filter composed of a birefringent plate made of quartz crystal, lithium niobate, and the like as a base material, and removing a high-frequency component by utilizing birefringence of the light. That is, as shown in
However, as shown in
However, when seen from the resolution aspect, it is not possible to strengthen the effect of the optical low pass filter until a resolution feeling of the output image is deteriorated. That is, when the sampling frequency of the output image is taken as Fs and the pixel pitch is taken as D (=1/Fs), to allow the output image to carry sufficient resolution, it is necessary to set the cut-off frequency fc of the optical low pass filter approximately at least to Fs/2 (=1/2D) or to a level slightly exceeding this frequency.
In the digital camera having an imaging element of the ordinary Bayer pattern, the number of pixels of the imaging element and the number of pixels of the output image are at the same level (when set to a mode of the highest image quality of the camera). In this kind of the digital camera, the pixel pitch d of the imaging element and the pixel pitch D of the output image are equal to each other. In such a digital camera, the cut-off frequency fc of the optical low pass filter is set to approximately fs/2 (=1/2d) or to a level slightly exceeding this frequency.
This means that importance is attached to the resolution aspect rather than the color aspect. In this case, since the Nyquist frequency of the G color is fs/2, the false color of the G color is not so conspicuous, whereas the Nyquist frequencies of the R color and the B color are fs/4, respectively, and therefore, the false colors by the R color and the B color are left behind. However, in even such a case, if the object itself does not have a high-frequency component, the false color is not generated. Hence, an ordinary user is more likely to give preference to the resolution over the color. In the ordinary digital camera, the cut-off frequency fc of the optical low pass filter is set as described above in accordance to this preference.
In the ordinary digital camera, though preference is given to the resolution aspect over the color aspect, the present applicant devises a camera such that both of the color aspect and the resolution aspect are satisfied simultaneously (for example, see Patent Document 3). In this camera disclosed in Patent Document 3, from N pieces of the pixel signals generated by the imaging element, one piece of the output image signal is generated. The relationship between the pixel pitch d of the imaging element and the pixel pitch D of the output image is d<D. As a result, the cut-off frequency fc of the optical low pass filter can be set so as to satisfy both of the color aspect and the resolution aspect.
Patent Document 3: Japanese Patent Application No. 2004-156083
In general, when the image quality is to be improved, a still image is strongly required to be high in the number of pixels and have high resolution. In contrast to this, though a dynamic image is strongly required to have high resolution, the number of pixels has only to satisfy standards (approximately 350,000 pixels according to SD standards, and approximately 2,080,000 pixels according to HD standards). The number of pixels of the camera for the still image not required to correspond to the standards (several millions of pixels to over ten millions of pixels) is relatively higher as compared with the number of pixels satisfying the standards of the dynamic image. Further, in a camera for taking dynamic images, since a camera hardly generating a false color by using a 3-CCD type imaging element is in widespread use, it is also strongly required to suppress the generation of the false color similarly to this camera. However, both of the still image and the dynamic image are likely to give preference to the high resolution over the low false color.
When a hybrid camera designed to be able to take both of the still image and the dynamic image by one set of a camera is to be fabricated, it is not possible to set an appropriate frequency characteristic to the optical low pass filter simultaneously for both of the still image and the dynamic image. As described above, the cut-off frequency fc of the optical low pass filter most appropriate for the still image is approximately fs/2 (=1/2d) or slightly exceeds this frequency, and the cut-off frequency fc most appropriate for the dynamic image is lower than this frequency.
The present invention has been carried out in view of such circumstances, and an object of the invention is to enable the generation of the false color to be effectively suppressed without sacrificing the resolution of the output image with respect to the dynamic image, and at the same time, to suppress the generation of the false color as much as possible, while increasing the resolution sufficiently with respect to the still image.
To achieve the above described objects, the optical low pass filter of the present invention variably controls a blur condition of the image to be formed in the imaging element according to a still image taking mode or a dynamic image capturing mode. For example, by variably controlling the light beam separation width according to the still image capturing mode or the dynamic image capturing mode, at the time of a still image capturing mode, the light beam separation width is taken as a first width so that the blur condition of the image is taken as a first amount, and at the time of a dynamic image capturing mode, the light beam separation width is taken as a second width wider than the first width, so that the blur condition of the image is taken as a second amount.
Further, the imaging device of the present invention utilizing such an optical low pass filter is provided with the imaging element of many more pixel numbers than the display pixel number set up by the standards of the dynamic image, and at the time of the dynamic image capturing mode, one piece of the output image signal is generated from N pieces of the pixel signals generated by the imaging element.
According to the present invention thus configured, at the time of the still image capturing mode, the light beam separation width at the optical low pass filter is controlled so as to be relatively narrow. Since the cut-off frequency of the optical low pass filter depends on the light beam separation width, the light beam separation width is made relatively narrow and the cut-off frequency is controlled to become approximately fs/2 or to slightly exceed this frequency, so that the resolution of the imaging element can be used as it is while suppressing the generation of the false color to a certain degree. As a result, the still image with high resolution which satisfies high requirements and moreover with the minimum possible false color suppression can be obtained.
On the other hand, at the time of the dynamic image capturing mode, the light beam separation width at the optical low pass filter is controlled so as to be relatively wider, and from N pieces of the pixel signals generated by the imaging element, one piece of the output image signal is generated. The light beam separation width is made relatively wider and a control is made such that the cut-off frequency becomes a frequency equivalent to the Nyquist frequency of the pixel of the output image or slightly exceeds this frequency, so that a high-frequency component equivalent to a resolution component unnecessary for the output image signal can be cutoff by the optical low pass filter. Further, as compared with the still image capturing mode, the suppression of the false color can be strongly performed with allowance. As a result, the dynamic image with the resolution satisfying the requirements under the standards of the dynamic image and the effectively-suppressed false color can be obtained.
One embodiment of the present invention will be described below based on the drawings.
The optical low pass filter 2 plays a role of suppressing a high spatial frequency component in imaging light. The optical low pass filter 2 according to the present embodiment is, for example, composed of the birefringent plate made of liquid crystal as a base material, and is disposed in front of the imaging element 5 on the optical path of the imaging light. When the dielectric constant of the material is anisotropic, an incident light on that material is separated into two lights different in the polarization direction by a relationship with its vibration direction.
As shown in
Further, in the present embodiment, depending on which of the still image capturing mode or the dynamic image capturing mode is set, the frequency characteristic of the optical low pass filter 2 is switched over. The switching over of the frequency characteristic of the optical low pass filter 2 is realized by variously controlling a distance (referred to as a light beam separation width) between the ordinary ray and the extraordinary ray in the birefringent plate. When the light beam separation width becomes larger, the cut-off frequency of the optical low pass filter 2 becomes lower, and the blur condition of the image formed in the imaging element 5 becomes larger. On the contrary, when the light beam separation width becomes smaller, the cut-off frequency of the optical low pass filter 2 becomes higher, and the blur condition of the image formed in the imaging element 5 becomes smaller.
Here, at the time of the still image capturing mode, the light beam separation width is taken as a first width, thereby to take the blur condition of the image as a first amount, and at the time of the dynamic image capturing mode, the light beam separation width is taken as a second width wider than the first width, so that the blur condition of the image is taken as a second amount larger than the first amount. When the color filter 4 is disposed in the Bayer pattern as shown in
Further, at the time of the dynamic image capturing mode, in order to cut the high frequency component equivalent to the resolution component unnecessary for the output image signal and to strongly and sufficiently perform the suppression of the false color, the second width is set to a frequency in which the cut-off frequency of the optical low pass filter 2 is equivalent to the Nyquist frequency of the pixel of the output image or set to a width slightly exceeding this frequency.
The light beam separation width at the optical low pass filter 2 is made variable by controlling the voltage applied to the liquid crystal. That is, as shown in
As shown in
The imaging optical system 3 plays a role of guiding the imaging light to the imaging element 5. In the present embodiment, the system 3 includes the optical low pass filter 2, and is formed of an imaging lens, an infrared removing filter, and the like. The infrared removing filter blocks an infrared ray incident on a photo diode, and is disposed in front of the optical low pass filter 2, and is formed as a panel of glass block.
The color filter 4 is regularly disposed in the predetermined pattern on the light receiving surface of each pixel forming the imaging element 5, and plays a role of filtering the imaging light into the predetermined color components. In the present embodiment, as three colors of a first color, a second color, and a third color forming the color filter 4, the primary color filters for R, G, and B colors are used. However, this is not limitative, and complementary color filters formed of C (cyan), M (Magenta), and Y (Yellow) and a combination of other colors may be used. Moreover, a filter of an emerald color may be added to the three color filters.
Further, in the present embodiment, as a disposition pattern of the color filter 4, as shown in
The imaging element 5 plays a role of photoelectrically converting the received imaging light into electric image information, and storing it as an electric charge amount to be outputted to the output image signal generation unit 6 as an electrical signal. The imaging element 5 has a plurality of pixels (photo diodes) disposed in the predetermined pattern, and regularly disposes the color filters 4 in the predetermined pattern on the light receiving surface of each pixel. The number of pixels of the imaging element 5 according to the present embodiment is set to the number (for example, N times the number of display pixels of HD standards or more (N is a real number of 2 or more)) which is more than the number (for example, approximately 350,000 pixels according to the SD standards, and 2,080,000 pixels also according to the HD standards) of display pixels set up by the standards of the dynamic image.
The output image signal generation unit 6 plays a role of A/D converting the pixel signal obtained from each pixel of the imaging element 5, and performing various image processings, and generating the output image signal. The output image signal generation unit 6 is formed of the A/D converter 7 and the ISP 8, and is electrically connected to the imaging element 5. The A/D converter 7 converts the pixel signal which is an analogue electric signal into digital data.
The CPU 9 performs a control of the voltage applied to the liquid crystal of the optical low pass filter 2 and a control of switching the image capturing mode for the ISP 8. That is, at the time of the still image capturing mode, the CPU 9 applies a voltage V1 to the liquid crystal, and at the same time, performs a control of setting the still image capturing mode for the ISP 8. Further, at the time of the dynamic image capturing mode, the CPU 9 applies a voltage V2 to the liquid crystal, and at the same time, performs a control of setting the dynamic image capturing mode for the ISP 8. The ISP 8 performs various image processings such as optical black processing, white balance processing, color correction processing, color interpolation processing, noise suppression processing, contour enhancement processing, γ correction processing, and resolution conversion processing for the A/D converted pixel signal, and generates an output image signal. At this time, the ISP 8, in response to the imaging mode set from the CPU 9, performs image processing for the still image or the dynamic image.
The resolution conversion processing is executed, for example, only when the dynamic image capturing mode is set. A conversion ratio of the resolution conversion processing is set such that N pieces of the A/D converted pixel signals are made equivalent to one piece of the output image signal. That is, at the time of still image capturing mode, the ISP 8 generates N pieces of the output image signals from N pieces of the pixel signals generated by the imaging element 5, and at the time of the dynamic image capturing mode, generates one piece of the output image signal from N pieces of the pixel signals generated by the imaging element 5. Even at the time of the still image capturing mode, if required by the user, the resolution conversion processing may be performed.
Here, a specific example of the processing for generating one piece of the output image signal from N pieces of the pixel signals will be described.
As shown in
Further, as shown in
The ISP 8 according to the present embodiment is, for example, formed of a CPU, a DSP (Digital Signal Processor) or a hard wired logic. Alternatively, the A/D converted pixel signal is loaded to a PC (personal computer), and the image processing may be performed by various programs.
Next, the operation of the imaging device according to the present embodiment thus configured will be described. First, when the still image capturing mode is set, as shown in
As a result, the false color generated by the sampling in the imaging element 5 can be suppressed to a certain degree. Further, in this case, a filtering effect of the optical low pass filter 2 does not become too strong, and the shortage of the pixel signal of the imaging element 5 is not caused also. Thus, the lowering of the resolution at the optical low pass filter 2 can be suppressed as much as possible.
The imaging light having passed through the optical low pass filter 2 and the color filter 4 is image-formed at the imaging element 5, and by the photoelectric conversion, the pixel signal is generated. The pixel signal generated by the imaging element 5 is outputted to the output image signal generation unit 6, the output image signal is generated in the unit 6. At this time, from N pieces of the pixel signals generated by the imaging element 5, N pieces of the output image signals are generated. That is, with the resolution of the imaging element 5 used as it is, the output image signal of high resolution is generated.
On the other hand, when the dynamic image capturing mode is set, as shown in
The imaging light having passed through the optical low pass filter 2 and the color filter 4 is image-formed in the imaging element 5, and the pixel signal is generated by the photoelectric conversion. The pixel signal generated by the imaging element 5 is outputted to the output image signal generation unit 6, the output image signal is generated in the unit 6. At this time, from N pieces of the pixel signal generated by the imaging element 5, one piece of the output image signal is generated.
That is, N pieces of the pixel signals, obtained by the imaging element 5 wherein the number of pixels is set larger than the number of display pixels required by the standards of the dynamic image so as that the number of pixels sufficient for the dynamic image exists, generates the output image signals corresponding to the number of the display pixels. As a result, even when the resolution is reduced by the optical low pass filter 2 set low in the cut-off frequency fc, since the high-frequency component equivalent to the resolution originally unnecessary for the output image signal is merely cutoff at the time of the generation of the output image signal, the display resolution required by the standards of the dynamic image can be sufficiently satisfied.
As described above in detail, according to the present embodiment, at the time of the still image capturing mode, the still image with the minimally-suppressed false color and high resolution can be obtained. Further, at the time of the dynamic image capturing mode, the dynamic image with the display resolution satisfying the requirements under the dynamic image standard and the effectively-suppressed false color can be obtained. Further, according to the present embodiment, by controlling the voltage applied to the liquid crystal of the optical low pass filter 2, the light beam separation width is changed, so that the frequency characteristic most appropriate to the still image capturing mode and the frequency characteristic most appropriate to the dynamic image capturing mode can be simply switched over.
In the above described embodiment, though a description has been made that the optical low pass filter 2 is composed of the birefringent plate made of the liquid crystal as a base member, the present invention is not limited to this. That is, if a material has a birefringent effect of the light and can electrically control the light beam separation width, it can be applied as a raw material of the optical low pass filter 2.
Further, in the above described embodiment, as a configuration example of the optical low pass filter 2, though a description has been made on an example in which a birefringent index is dynamically changed by a piece of the liquid crystal plate as shown in
In the example of
In the example of
When a voltage is applied to such polarization liquid crystal layer 31, the disposition of the liquid crystal molecules 21 are changed. That is, the liquid crystal molecules 21 are disposed by having an angle corresponding to a size of the voltage applied to the orientation films 31a and 31b. In the present embodiment, the CPU 9, by controlling the presence or absence of the applied voltage, as shown in
In the case of the optical low pass filter 2 shown in
Here, as shown in
Therefore, the ordinary ray emitted from one liquid crystal layer 32 is emitted by being displaced by ΔB by the other liquid crystal layer 33, and the extraordinary ray emitted from one liquid crystal layer 32 is emitted by being not displaced by the other liquid crystal layer 33. As a result, here, since ΔA>ΔB, the first width W1 which is the light beam separation width as the whole of the optical low pass filter 2 becomes W1=ΔA−ΔB.
On the other hand, as shown in
Hence, the ordinary ray emitted from one liquid crystal layer 32 is emitted without being displaced even by the other liquid crystal layer 33, and the extraordinary ray emitted from one liquid crystal layer 32 is emitted by being further displaced by ΔB by the other liquid crystal layer 33. As a result, the second width W2 which is the light beam separation width as the whole of the optical low pass filter 2 becomes W2=ΔA+ΔB.
When the optical low pass filter 2 is configured as
In the case of
Further, in the case of
In the case of
In the case of the optical low pass filter 2 shown in
Here, as shown in
On the other hand, as shown in
When the optical low pass filter 2 is configured as
In the case of
In the optical low pass filter 2 shown in
The configuration in which one piece of the incident light is two-dimensionally separated into a plurality of lights is not limited to the configuration similarly to
The configuration of
The configuration of
The configuration of
Further, in the above described embodiment, as an example of the varifocal layer variably changing the blur condition of the image formed in the imaging element 5, though a description has been made on the example using the birefringent plate made of the liquid crystal and the like as a base material, the present invention is not limited to this. For example, the varifocal layer may be formed of the liquid crystal lens.
The liquid crystal lens is a kind of lens utilizing the liquid crystal. When the liquid crystal is sealed into a lenticular space and the voltage to be applied is adjusted, an apparent refraction index of the liquid crystal changes. Even if it is a lens of the same shape, when the refraction index of the liquid crystal which forms the lens changes, a focal length of the lens changes. Thus, when the liquid crystal lens is employed, the focal length of the lens can be changed by a control of the electrical signal only.
When the liquid crystal lens is used as the varifocal layer, by controlling the voltage applied to the liquid crystal lens according to the still image capturing mode or the dynamic image capturing mode, the focal length can be variably changed through the change of the refraction index of the liquid crystal lens. As a result, at the time of the still image capturing mode, the blur condition of the image is taken as a first amount, and at the time of the dynamic image capturing mode, the blur condition of the image is taken as a second amount.
Further, in the above described embodiment, while a description has been made on the case as an example, in which the pixel disposition of the imaging element 5 is in a square lattice pattern, this is not limitative. For example, the disposition may be in a square lattice pattern inclined 45°.
In the above described embodiment, the setting of the still image capturing mode and the dynamic image capturing mode can be performed by operating a mode setting operation key (not shown), which is provided in the imaging device 1, by a user. Further, even during the taking the dynamic image by setting the dynamic image capturing mode by the user, when the user pushes down an shutter for the still image, it is possible to automatically switch over the mode to the still image capturing mode for the meantime. Mode information thus set up is stored in a memory (not shown) of the imaging device 1. The CPU 8, by referring to the mode information thus stored in the memory, controls the voltage applied to the optical low pass filter 2.
In addition, the above described embodiment has only shown an example of the embodiments at the time of executing the present invention, and it is to be expressively understood that this is not intended as the limits of the scope of the present invention. That is, the present invention can be executed in various forms without being deviated from the spirit or the main features of the invention.
The optical low pass filter of the present invention is useful for a hybrid camera made capable of taking both the dynamic image and the still image by one set of the camera.
Number | Date | Country | Kind |
---|---|---|---|
2006-012672 | Jan 2006 | JP | national |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/JP2007/050861 | 1/16/2007 | WO | 00 | 7/21/2008 |