1. Field of the Invention
The present invention relates to an image sensing system and a method of driving the image sensing system.
2. Description of the Related Art
Known image sensing systems, such as a digital camera and a camcorder, include an image sensing system that performs auto-focusing (AF) based on phase difference detection using sensors for image sensing.
Japanese Patent Laid-Open No. 2000-156823 discloses a solid-state image sensing apparatus configured such that a plurality of photoelectric conversion cells for obtaining ranging signals are arranged in some portions of a pixel array included in the apparatus. The photoelectric conversion cells include a pair of photoelectric conversion cells each having an opening which is offset relative to the center of the cell, the openings being offset in opposite directions away from each other.
As disclosed in Japanese Patent Laid-Open No. 2000-156823, the solid-state image sensing apparatus has a skipping mode and a ranging mode as operation modes. In the skipping mode, signals are read from the photoelectric conversion cells while the photoelectric conversion cells in the portions are skipped, since signals from the photoelectric conversion cells for ranging in the pixel array are not used for image generation. Movie or view finder image data is generated based on data obtained in the skipping mode. Skipping readout enables high-speed photometry. In the ranging mode, signals are read from lines including the photoelectric conversion cells for AF.
However, exposure control may not be properly performed during operation disclosed in Japanese Patent Laid-Open No. 2000-156823. To perform exposure control and image generation in the skipping mode, signals are read from the required number of photoelectric conversion cells for an image. Accordingly, the time required to obtain signals of one frame becomes longer. Under conditions that the quantity of light from an object considerably varies due to, for example, a strobe in a short time, the exposure control may not be able to follow the change of light quantity.
One aspect of the present invention provides an image sensing system including the following elements. A photoelectric conversion apparatus includes a pixel array including a plurality of pixels arranged in a matrix and a plurality of microlenses arranged in correspondence to the plurality of pixels, the plurality pixels including a plurality of image sensing pixels for image sensing and a plurality of ranging pixels for ranging. A signal processing unit is configured to process signals read from the photoelectric conversion apparatus. The ranging pixels include a first ranging pixel and a second ranging pixel. The first ranging pixel has an opening that is offset in a first direction relative to an optical center of the corresponding microlens. The second ranging pixel has an opening that is offset in a second direction opposite to the first direction. The image sensing system has a first operation mode and a second operation mode. In the first operation mode, signals are read from the ranging pixels and the image sensing pixels in the pixel array of the photoelectric conversion apparatus, ranging is performed based on the signals from the ranging pixels, and an image is generated based on the signals from the image sensing pixels. In the second operation mode, signals are read from the image sensing pixels and exposure is controlled based on the signals from the image sensing pixels.
Another aspect of the present invention provides a method of driving an image sensing system that includes a photoelectric conversion apparatus which includes a pixel array including a plurality of pixels arranged in a matrix and a plurality of microlenses arranged in correspondence to the plurality of pixels, the plurality of pixels including a plurality of image sensing pixels for image sensing and a plurality of ranging pixels for ranging, and further includes a signal processing unit configured to process signals read from the photoelectric conversion apparatus and a display unit configured to display an image based on the signals processed by the signal processing unit. The ranging pixels include a first ranging pixel and a second ranging pixel. The first ranging pixel has an opening that is offset in a first direction relative to an optical center of the corresponding microlens. The second ranging pixel has an opening that is offset in a second direction opposite to the first direction. The method includes performing a first operation to read signals from the pixels including the ranging pixels in the pixel array in order to perform ranging and image display based on the read signals, and performing a second operation to read signals from the pixels excluding the ranging pixels in order to control exposure.
Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.
A first embodiment according to the present invention will be described with reference to the drawings.
Light from an object passes through the optical unit 100 and is focused on the photoelectric conversion apparatus 200. The focused light is photoelectrically converted into electrical signals by the photoelectric conversion apparatus 200. The photoelectric conversion apparatus 200 includes image sensing pixels for image generation and AF pixels (ranging pixels).
The signals read from the photoelectric conversion apparatus 200 are transferred to the signal processing unit 300 in which the signals are stored as data in a memory 301. A phase difference processer 302 performs phase difference calculation based on data obtained from the AF pixels of the data stored in the memory 301 to determine an out-of-focus amount (defocus amount). The signal processing unit 300 controls the optical unit 100 based on the determined defocus amount to perform AF control.
Furthermore, in the signal processing unit 300, a light quantity processor 303 determines an exposure amount based on data obtained from the image sensing pixels for image generation of the data stored in the memory 301. The signal processing unit 300 controls the timing generator 400 and the light source 500 based on the determined exposure amount. Thus, the timing generator 400 supplies a driving signal to the photoelectric conversion apparatus 200 to change charging time and the quantity of light emitted from the light source 500 is changed. The light source 500 is, for example, a strobe.
Although not illustrated herein, the image sensing system 1000 further includes an interface configured to transfer data stored in the memory 301 to an external device and a display (display unit) configured to display an image.
The pixel array PA includes a plurality of pixels arranged in a matrix of rows and columns. Each of the pixels includes a photoelectric conversion element and generates charge in response to incident light passing through the optical unit 100. When the row selection portion VSR selects a plurality of pixels on a row-by-row basis, signals are output from the pixels in the selected row. Each of the pixels includes the photoelectric conversion element, an amplifying transistor configured to output a voltage signal based on the amount of generated charge, a reset transistor configured to reset the amount of charge stored in the photoelectric conversion element, and a selection transistor configured to select the pixel.
The column reading portion CC includes a plurality of column reading circuits arranged in correspondence to the columns of the pixel array PA. Each of the column reading circuits is configured to process signals output from the pixels in the corresponding column and may include a noise reduction circuit, such as a correlated double sampling (CDS) circuit, an amplifier, and a sample and hold circuit. The column reading circuit may include an analog-to-digital (AD) converter.
The column selection portion HSR selectively transfers the signals from the column reading circuits through the horizontal transfer portion HTR to the output portion OP. The horizontal transfer portion HTR includes a line to transfer an analog signal and a digital bus to transfer a digital signal and may include a plurality of buffer circuits connected in series to transfer a signal.
The output portion OP transfers signals supplied through the horizontal transfer portion HTR to a circuit subsequent to the image sensing system 1000. The output portion OP may include an amplifier and a circuit that enables differential output in the low voltage differential signaling (LVDS) format.
The principle of phase difference detection using the AF pixels S1 and S2 will be described below.
The following description will focus on the pixel P4 of the pixels P1 to P5 arranged successively. A light beam L1 incident on the region R1 of the pixel P4 passes through a corresponding pupil of the optical unit 100. Similarly, a light beam L2 incident on the region R2 of the pixel P4 passes through a corresponding pupil of the optical unit 100. If an object is in focus, the light beams L1 and L2 are converged on one point on the surface of the microlens ML. The light beams L1 and L2 then enter the regions R1 and R2 of the photoelectric conversion element, respectively. Since the same images are formed in the regions R1 and R2, signals obtained from these regions have the same level.
A case where an object is out of focus will be described with reference to
Comparison between signals obtained from the pixel S1 and those obtained from the pixel S2 in
An operation sequence of the image sensing system 1000 according to this embodiment will be described with reference to
First, the signal processing unit 300 supplies a control signal to the timing generator 400 so that the timing generator 400 operates in a skipping mode which is a first operation mode (S1).
In step S2, signals of one frame are read out in the skipping mode. Specifically, the photoelectric conversion apparatus 200 reads signals from some pixels of the pixel array. In this operation mode, signals are read from the pixels in the rows including the AF pixels in addition to the rows including the image sensing pixels. The signal processing unit 300 generates an image based on the signals obtained from the rows including the image sensing pixels. For example, by displaying the generated image on the display of the image sensing system 1000, the display may serve as an electronic view finder (EVF). In addition, the image can be recorded as a movie in the memory 301. Since the signals are read from only some of the pixels in the pixel array, the time required to read signals of one frame can be reduced. Accordingly, a movie smoother than a movie based on signals read from all of the pixels can be obtained. Furthermore, since signals from the AF pixels do not contribute to image generation, a process of correcting the difference in characteristics between the image sensing pixels and the AF pixels can be eliminated. In step S2, phase difference ranging is performed based on the signals obtained from the AF pixels.
Subsequently, whether an event has occurred is determined in step S3. Examples of events include a pre-flash switch for the strobe 500, serving as the light source, being turned-on by a user and a shutter switch being turned-on by the user. If it is determined in step S3 that no event has occurred, the operation returns to step S2 and signals of one frame are again read out in the skipping mode.
If it is determined in step S3 that the pre-flash switch for the strobe 500 has been turned on, the operation proceeds to step S4. If it is determined in step S3 that the shutter switch has been turned on, the operation proceeds to step S7.
In step S4, the timing generator 400 is switched from the skipping mode to a light control mode which is a second operation mode. In this operation mode, signals are read from the rows including no AF pixels, and signals are not read from the rows including the AF pixels. Accordingly, the process of correcting the difference in characteristics between the AF pixels and the image sensing pixels can be eliminated. Furthermore, since skipping readout is performed, the number of target pixels from which signals are read can be less than that in the skipping mode. As the number of target pixels is smaller, the time required to obtain signals of one frame can be shorter. In the light control mode in which no image is generated, the proper amount of exposure can be obtained in the use of a light source, such as a strobe, whose light quantity considerably varies in a short time.
In step S5, the strobe light quantity suitable for image sensing conditions is determined based on the signals read in step S4. Upon determination of the strobe light quantity, the operation returns to step S1 and the timing generator 400 is set to the skipping mode.
If it is determined in step S3 that the shutter switch has been turned on, the timing generator 400 is switched from the skipping mode to the light control mode (step S7).
In step S8, the strobe light quantity suitable for image sensing conditions is determined based on signals obtained in the light control mode in a manner similar to step S5.
In step S9, the timing generator 400 is switched from the light control mode to a still image shooting mode which is a third operation mode. The still image shooting mode is an operation mode to read signals from the image sensing pixels and the AF pixels without skipping the pixels.
In step S10, the strobe 500 is allowed to emit light on light emission conditions determined in step S9 and shooting is performed. Upon shooting, the operation again returns to step S1.
In the skipping mode in
In step S2, EVF display is performed based on signals obtained from the image sensing pixels in the skipping mode. Specifically, the EVF display is performed and AF is performed based on signals obtained from the AF pixels.
When the turn-on of the pre-flash switch for the strobe 500 is determined in step S3, the mode is switched to the light control mode. The strobe light quantity is determined based on signals read from the image sensing pixels excluding the skipped pixels.
Upon determination of the strobe light quantity, the mode is again switched to the skipping mode. Accordingly, EVF display is performed. After that, the EVF display is repeated until the occurrence of another event is determined in step S3.
In step S2, EVF display is performed based on signals obtained from the image sensing pixels in the skipping mode. Specifically, the EVF display is performed and AF is performed based on signals obtained from the AF pixels.
When the turn-on of the shutter switch is determined in step S3, the mode is switched to the light control mode. The strobe light quantity is determined based on signals read from the image sensing pixels excluding the skipped pixels.
Upon determination of the strobe light quantity, signals are read from the image sensing pixels and the AF pixels to capture a still image (still image capture). After that, the mode is again switched to the skipping mode, so that EVF display is performed. The EVF display is repeated until the occurrence of another event is determined in step S3.
According to this embodiment, signals are read from the rows including only the image sensing pixels and the rows including the AF pixels in the skipping mode. In the light control mode, signals are read from the rows including no AF pixels. Consequently, exposure can be controlled based on signals obtained for a frame which is not related to image generation. Thus, proper exposure control can be immediately achieved on condition that the quantity of light, such as strobe light, considerably varies in a short time.
In the technique disclosed in Japanese Patent Laid-Open No. 2000-156823, EVF display is performed based on signals obtained in the skipping mode and exposure control is also performed based on these signals. In such a method, the exposure control may not be able to follow the change of light quantity as described above. If successive image frames include a frame out of focus, this frame is hardly visible. If successive image frames include a frame having different brightness from the other frames, the difference is easy to notice. According to this embodiment, the image sensing system 1000 has the skipping mode in which phase difference detection and image generation are performed based on signals read from the AF pixels and those from the image sensing pixels and the light control mode in which the amount of exposure is controlled based on signals read only from the image sensing pixels. Advantageously, a natural image can be generated while an object is being in focus.
A second embodiment according to the present invention will be described with reference to the drawings. The second embodiment will be described mainly with respect to the difference from the first embodiment.
In the first embodiment, the AF pixels S1 and S2 are arranged next to each other diagonally in the matrix. Two rows are arranged between the two rows including the diagonally adjacent AF pixels S1 and S2 and the next two rows including the diagonally adjacent AF pixels S1 and S2. In the present embodiment, the AF pixels S1 and S2 are arranged such that two rows are arranged between each row including the AF pixel S1 and each row including the AF pixel S2.
As compared with the arrangement in the first embodiment, the number of AF pixels in the area of the same size can be reduced. The rows including the AF pixels are skipped in the skipping mode. Accordingly, as the number of AF pixels is smaller, the resolution of an image obtained in the skipping mode is higher.
Since a light beam from an object partly enters the photoelectric conversion element of the AF pixel as described above, signals obtained from the AF pixel are not used for image generation. Interpolation is performed based on signals obtained from pixels surrounding the AF pixel. As the number of AF pixels to be subjected to interpolation is larger, processing time necessary for interpolation is longer. This embodiment offers the same advantages as those in the first embodiment and further offers advantages in that since the number of AF pixels per unit area is smaller than that in the first embodiment, a higher definition image can be generated and higher-speed readout can be achieved.
A third embodiment according to the present invention will be described with reference to the drawings. The third embodiment will be described mainly with respect to the difference from the second embodiment.
In the third embodiment, the AF pixels S1 and S2 are arranged such that eight rows are arranged between a row group and the next row group, each of the row groups including a first row including the AF pixel S1, a second row including the AF pixel S2, and two rows arranged between the first and second rows.
According to this embodiment, a higher definition image than the second embodiment can be generated and higher-speed readout than the second embodiment can be achieved.
It should be understood that the above-described embodiments are merely illustrative examples of implementation of the present invention and the technical scope of the present invention is not to be interpreted as limiting. Specifically, the present invention can be embodied in various forms without departing from the technical spirit or essential features of the invention. For example, although the arrangements in which the AF pixels S1 are arranged on one column and the AF pixels S2 are arranged on another column are illustrated, the present invention is not limited to the above examples. For example, the AF pixels S1 may be arranged in one row and the AF pixels S2 may be arranged in another row.
While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
This application claims the benefit of Japanese Patent Application No. 2012-133732 filed Jun. 13, 2012, which is hereby incorporated by reference herein in its entirety.
Number | Date | Country | Kind |
---|---|---|---|
2012-133732 | Jun 2012 | JP | national |
Number | Name | Date | Kind |
---|---|---|---|
20060127084 | Okada | Jun 2006 | A1 |
20060176387 | Kobayashi | Aug 2006 | A1 |
20070140679 | Yumiki | Jun 2007 | A1 |
20070268396 | Kurane | Nov 2007 | A1 |
20090110383 | Fujii | Apr 2009 | A1 |
20090140122 | Suzuki | Jun 2009 | A1 |
20100110272 | Sugawara | May 2010 | A1 |
20110019028 | Kimijima | Jan 2011 | A1 |
20110205423 | Tsukada | Aug 2011 | A1 |
20110267511 | Imafuji | Nov 2011 | A1 |
20110267533 | Hirose | Nov 2011 | A1 |
Number | Date | Country |
---|---|---|
2000-156823 | Jun 2000 | JP |
Number | Date | Country | |
---|---|---|---|
20130335608 A1 | Dec 2013 | US |