The present invention relates to an imaging apparatus, a solid image sensor, and a method for driving the solid image sensor applicable to, for example, digital cameras.
Various image-stabilizing methods for digital still cameras have been realized to correct blur in captured images caused by camera shakes. One of typical methods is an optical image shifting which is realized by, for example, movable lenses, variable optical systems such as a vari-angle prism, or the like. According to the optical image shifting, movement of the optical system shifts the light axis toward the image sensor (for example, CCD (Charge Coupled Device) image sensor, a MOS (Metal Oxide Semiconductor) image sensor, a CMOS (Complementary MOS) image sensor, or the like) during exposure, thus images formed on the image sensor are shifted to compensate blur. Another typical method is an image sensor shifting which drives the image sensor to move in order to shift images formed thereon.
To realize effective image compensation against blurry images, those methods require sensors for detecting shakes occurred on the camera body. Usually, for example, a pair of angular rate sensors for detecting shakes in horizontal and vertical directions is installed in a camera body, and an optical system or an image sensor is controlled to move in an appropriate direction based on the detected shakes. Such the methods have been known generally according to, for example, Unexamined Japanese patent application KOKAI publication No. H10-301157.
According to the conventional image-stabilizing methods above, mechanical components are required to drive the optical system or the image sensor. Such the mechanical components occupy extra spaces in a camera, therefore, it is not appropriate for compact cameras. Moreover, since mechanical components are generally fragile by shocks, the reliability of the camera becomes lower as it is used for a long time.
In consideration of the aforementioned circumstances, the present invention has been made, and an object of the present invention is to provide an imaging apparatus, a solid image sensor, and a method for driving the solid image sensor which are able to realize more reliable image-stabilization.
In order to achieve the above object, an imaging apparatus according to an aspect of the present invention is an imaging apparatus having a function for preventing blurry images caused by shaken apparatus body, comprises:
a solid image sensor having an electronic charge transferor which comprises 2-dimentionally arrayed multiple electronic charge coupling elements having specific electronic charge coupling elements which accumulate electronic charges of each pixel in an optical image generated by photoelectric conversion, and a horizontal transferor which obtains the accumulated electronic charges in the electronic charge transferor line by line as an image signal;
a driver which drives the solid image sensor with a first drive signal which causes the solid image sensor to execute a plurality of exposures during exposure term for image capturing and a second drive signal which causes the solid image sensor to transfer the electronic charges accumulated in the specific electronic charge coupling elements in vertical and/or horizontal directions;
a shake detector which detects directions and amounts of shakes at the apparatus body, and generates shake information representing the detected directions and amounts of the shakes; and
a controller which controls the driver to generate the second drive signal based on the shake information obtained from the shake detector.
To achieve the above object, a solid image sensor according to another aspect of the present invention is a solid image sensor for converting an optical image into image signals by photoelectric conversion, comprises:
an electronic charge transferor which comprises 2-dimentionally arrayed multiple electronic charge coupling elements having specific electronic charge coupling elements which accumulate electronic charges of each pixel in an optical image generated by photoelectric conversion; and
a horizontal transferor which obtains the accumulated electronic charges in the electronic charge transferor line by line as an image signal, wherein
the solid image sensor is driven during an exposure term by a first drive signal which causes the solid image sensor to execute a plurality of exposures and by a second drive signal which causes the solid image sensor to transfer electronic charges accumulated in the specific charge coupling elements in vertical and/or horizontal directions.
To achieve the above object, a method for driving a solid image sensor according to still another aspect of the present invention is a method for driving a solid image sensor having an electronic charge transferor which comprises 2-dimentionally arrayed multiple electronic charge coupling elements having specific electronic charge coupling elements which accumulate electronic charges of each pixel in an optical image generated by photoelectric conversion, and a horizontal transferor which obtains the accumulated electronic charges in the electronic charge transferor line by line as an image signal, comprises the steps of:
causing the solid image sensor by a first drive signal to execute a plurality of exposures during exposure term for image capturing:
detecting directions and amounts of shakes occurred during the exposure term, and generates shake information representing the detected direction and amounts of the shake;
causing the solid image sensor by a second drive signal based on the shake information to transfer electronic charges accumulated in the specific charge coupling elements every time causing the solid image sensor to execute the plurality of exposures.
According to the present invention, it is able to provide an imaging apparatus having highly reliable image stabilizing function, a solid image sensor to be used for the imaging apparatus, and a method for driving the solid image sensor.
These objects and other objects and advantages of the present invention will become more apparent upon reading of the following detailed description and the accompanying drawings in which:
An embodiment of the present invention will now be described with accompanying drawings. In this embodiment, an example where the imaging apparatus according to the present invention is realized as a digital camera for still image capturing (digital still camera). An imaging apparatus of this embodiment according to the present invention will now be described with reference to
As shown in
The lens 1 may be a lens unit having typical lenses for photographing for gathering lights from a target so that an optical image is formed on the image sensor 2. The lens 1 may have, for example, auto-focus function and/or zooming function arbitrary.
The image sensor 2 is a solid image sensor which converts the formed optical image into electric signals by photoelectric conversion, thus image data are generated. In this embodiment, the image sensor 2 may comprise a CCD (Charge Coupled Device) having a dual layer structure (details will be described later).
These lens 1 and the image sensor 2 are major components of the imaging apparatus 10 (hereinafter, referred to as “imaging system”). Principle of imaging by the imaging system of the imaging apparatus 10 will now be described with reference to
The structure of the image sensor 2 will now be described in detail with reference to
The photosensitive layer 21 is an upper layer above the electronic charge transfer layer 22, and comprises a plurality of 2-dimensionally arrayed photoelectric elements (photo sensors) 21a. The photoelectric elements 21a are arrayed in vertical and horizontal directions (hereinafter, referred to as X-Y directions), that is, matrix arrayed. The photosensitive layer 2a is formed by these photoelectric elements 21a. Additionally, for example, Bayer arrayed RGB color filters (not shown) may be formed on each of the photoelectric elements 21a.
The electronic charge transfer layer 22 beneath the photosensitive layer 21 comprises a plurality of 2-dimensional matrix charge coupling elements 22a (that is, CCD) which are arrayed in vertical and horizontal directions (X-Y directions). The number of the charge coupling elements 22a is larger than that of the photoelectric elements 21a. More precisely, each of the photoelectric elements 21a corresponds to a group of plural charge coupling elements 22a.
Though
According to such the structure, electronic charges for all pixels (that is, a whole image) accumulated at the photosensitive layer 21 are transferred downward to the electronic charge transfer layer 22 via the read-out gates 25. In this embodiment, the direction from the photosensitive layer 21 to the electronic charge transfer layer 22 will be referred to as “Z direction”, and the transfer in the Z direction will be referred to as “Z transfer”. Thus transferred electronic charges in the electronic charge transfer layer 22 are further transferred to the horizontal transfer gate 23 sequentially line by line. The horizontal transfer gate 23 further transfers the line unit electronic charges to the output 24 sequentially. Accordingly, packets of the accumulated electronic charges each for 1 horizontal line are constantly available at the output 24 as image signals.
A method of transferring the electronic charges in the electronic charge transfer layer 22 will now be described in detail. In this embodiment, conventional three-phase drive is employed for the transfer.
As shown in
When the drive pluses as shown in
There are other voltage lines are arranged in the image sensor 2 for applying three-phase drive pulses for vertical transfer and horizontal transfer to the electronic charge transfer layer 22. When the three-phase drive pulses for the vertical transfer are sequentially applied to the electronic charge transfer layer 22, each of the charge coupling elements 22a transfers the electronic charges vertically in the direction toward the horizontal transfer gate 23 or in the inverse direction. Additionally, each of the charge coupling elements 22a is able to transfer the electronic charges horizontally in one direction or in the other direction when the three-phase drive pulses for the horizontal transfer are applied to the electronic charge transfer layer 22.
That is, the electronic charges in the electronic charge transfer layer 22 are flexibly controllable to be transferred in X-Y directions, that is, the vertical direction and the horizontal direction (hereinafter, referred to as “X-Y transfer”) by controlling two kinds of the three-phase drive pulses each for the vertical transfer and the horizontal transfer (hereinafter, referred to as “nine-phase drive signals”).
Examples of available patterns for electronic charge transfer realized by the above structure of the image sensor 2 will now be described with reference to FIGS. 6 to 9.
In this case, the electronic charges EC from the corresponding photoelectric element 21a are accumulated in the charge coupling element 22a at θ1 at beginning, and the electronic charges EC will be transferred to the position θ9 in accordance with the route shown in
In
In a case where the electronic charges EC are transferred in accordance with the transfer route shown in
Another example of the electronic charge transfer is shown in FIGS. 8 to 9. Similar to
The image sensor 2 which enables the above flexible transfers is driven by a plurality of drive pulses including the three-phase drive pulses generated by the image sensor driver 4 in accordance with drive timings generated by the timing generator 3 (see
Thus driven image sensor 2 outputs image signals representing levels of the electronic charges for the pixels of the captured image, to the analog signal processor 5.
The analog signal processor 5 may include an AGC (Auto Gain Control) amplifier, a CDS (Correlated Double Sampling) circuit, an ADC (Analog-Digital Converter), and the like. Such structured analog signal processor 5 adjusts gains of the input image signals from the image sensor 2 with sampling the image signals with using a signal being synchronous with the drive timing given by the timing generator 3, thus the image signals are converted into predetermined bits of digital data. The analog signal processor 5 outputs the digitalized data to the control unit 6 as image data.
The control unit 6 (see
The shake detector 7 (see
The program memory 8 may comprise a ROM (Read Only Memory) or a flash memory for storing programs to be executed by the control unit 6. For example, the program memory 8 stores general programs for usual operations of the digital still camera, such as auto exposure (AE) process. In this embodiment, the program memory 8 stores programs by which the control unit 6, during still image capturing, makes the image sensor driver 4 to generate drive signals corresponding to shakes occurred on the imaging apparatus 10 based on the shake information supplied from the shake detector 7. That is, such the programs make the control unit 6 to function as the drive controller according to the present invention.
The key input 9 may comprise various keys or buttons arranged on the outer surface of the imaging apparatus 10 (digital still camera) including, for example, a power key, a shutter button, and the like. If the any one of the keys is operated by a user, the key input 9 generates input signals (for example, a shutter signal) corresponding to the operation, and inputs the signals to the control unit 6.
According to thus structured imaging apparatus 10 of the present embodiment, the control unit 6 controls the image sensor driver 4 to generate the drive signals, when still image capture is instructed by operation on the shutter key, to control the image sensor 2 so as to execute multi-step exposure (details will be described later).
Note that the above described components are essential ones for realizing the present invention. The imaging apparatus 10 may comprise other components necessary for realizing fundamental or extra functions as well as generally used digital cameras, even if those components are not described or illustrated in this specification or drawings.
Operations of the imaging apparatus 10 (digital still camera) according to the present embodiment will now be described with reference to accompanying drawings. “Image sensor control” process executed by the control unit 6 during still image capturing, will now be described with reference to
Under the still image capturing mode, the control unit 6 executes the process in response to the shutter signal. First of all, the control unit 6 determines appropriate exposure time (in other words, shutter speed) for complete a still image forming in accordance with 1-shot image capturing (hereinafter, referred to as “full-exposure”) based on the AE processing (step S1).
In this embodiment, the control unit 6 controls the image sensor 2 so as to carry out multi-step exposure. That is, the image sensor 2 carries out a plurality of short-time exposures (hereinafter, referred to as “sub-exposures”) sequentially within the exposure term determined at step S1. The exposure time determined at step S1 represents necessary exposure time for complete a still image by 1-shot image capturing. In this embodiment, the image sensor 2 executes a plurality of sub-exposures within the exposure term to complete the full-exposure. In other words, the image sensor 2 carries out a plurality of photoelectric conversions within the full-exposure term.
To execute the multi-step exposure, the control unit 6 carries out calculation to determine the number of sub-exposures (n), exposure time (t1) for each sub-exposure, and interval time (t2) among the sub-exposures (step S2).
Then, the control unit 6 instructs the image sensor driver 4 to generate a first drive signal based on the calculation at step S2 (step S3). The first drive signal is a drive signals for the photosensitive layer 21. More precisely, the first drive signal causes the photosensitive layer 21 to perform a plurality of photoelectric conversions (electronic charge accumulations) and transfer the accumulated electronic charges in the Z direction to the electronic charge transfer layer 22 (Z transfer). The control unit 6 also instructs the image sensor driver 4 to supply the generated first drive signal to the image sensor 2 (step S3).
According to such the operations, the control unit 6 makes the image sensor 2 to start multi-step exposure (step S4, see
The image sensor 2 informs the control unit 6 of each sub-exposure completion. According to the information from the image sensor 2, the control unit 6 counts the completed sub-exposures. If the completed sub-exposure is 1st one (step S5: Yes), the control unit 6 waits for completion of 2nd or later sub-exposures.
If the completed sub-exposure is 2nd or later one (step S5: No), the control unit 6 obtains shake information representing shakes occurred during (N−1)th sub-exposure from the shake detector 7. More precisely, the control unit 6 recognizes the directions and amounts of shakes occurred since beginning of (N−1)th sub-exposure until Nth sub-exposure starts (that is, a term represented by t1+t2).
Based on the shake information for (N−1)th sub-exposure, the control unit 6 calculates transfer of the electronic charges in the electronic charge transfer layer 22 in order to compensate blurry image caused by the shake represented by the shake information (step S6). More precisely, the control unit 6 calculates the directions (X-Y directions) and the number of transfer steps for the compensational electronic charge transfer. In this case, the control unit 6 calculates the directions inverse to the shake direction. The number of transfer steps corresponds to the amount of the shake.
When it comes to a timing where the Z transfer for the former sub-exposure completes, the control unit 6 instructs the image sensor driver 4 to generate a second drive signal and to supply the generated second drive signal to the image sensor 2 (step S7). More precisely, the second drive signal is the aforementioned nine-phase drive signals. The second drive signal is generated based on the results of the calculation at step S6. That is, such the second drive signal causes the electronic charge transfer layer 22 to transfer the electronic charges in the directions (X-Y directions) calculated at step S6 with the amount of transfer according to the number of transfer steps calculated at step S6.
Accordingly, as shown in
That is, whole of the electronic charges in the electronic charge transfer layer 22 are transferred so that the electronic charges corresponding to an arbitrary light spot in the optical image generated by the former sub-exposure ((N−1)th exposure) are previously transferred to a predicted position where newly accumulated electronic charges corresponding to the arbitrary light spot will be transferred by the next sub-exposure (Nth exposure).
Such the processing at steps S4 to S7 will be executed repeatedly until the final sub-exposure (that is, n-th exposure) is completed (step S8: No).
As shown in
After the final sub-exposure is completed (step S8: Yes), the control unit 6 instructs the electronic charge transfer layer 22 to transfer the electronic charges vertically to the horizontal transfer gate 23, and instructs the horizontal transfer gate 23 to transfer the electronic charges horizontally to the output 24 (step S9), then terminates the process.
As described above, the imaging apparatus 10 of the embodiment according to the resent invention realizes effective image stabilization only by transferring the electronic charges in the image sensor 2. That is, any mechanical components for stabilizing image are not required for the image stabilization. Since the mechanical components are unnecessary, any compact digital cameras are able to employ the image stabilizing function without any restrictions. Moreover, such the un-mechanical structure brings not only effective image stabilization but also higher reliability of the apparatus.
Noises in the image data after the image stabilization according to the above embodiment are very few rather than a case where a plurality of images captured by a plurality of very short time exposures are synthesized with adjusting blurs, because it is realized only by transferring the electronic charges in the electronic charge transfer layer 22.
In the above described embodiment, since the image sensor 2 employs the dual layer structure having the photosensitive layer 21 and the electronic charge transfer layer 22, it is able to execute X-Y transfers of the electronic charges in the electronic charge transfer layer 22 and next sub-exposure in parallel. Under such the structure, more efficient image capturing is available by shortening the interval time (t2) among the sub-exposures.
Though the above embodiment exemplified the image sensor 2 which enables transfer of all electronic charges in X-Y directions by driving the electronic charge transfer layer 22 with using two kinds of three-phase drive pulses each for the horizontal transfer and the vertical transfer, arbitrary driving methods may be employed. For example, the electronic charge transfer layer 22 may be driven by four-phase driving pulses. In this case, the electronic charge transfer layer 22 may have groups of the charge coupling elements 22a each having 16 (4×4) charge coupling elements 22a corresponding to each of the photoelectric elements 21a.
Instead of the dual layered image sensor 2 described in the above embodiment, a single layered image sensors may also be applicable. For example, the photosensitive layer 21 may be eliminated from the image sensor 2 with applying photoelectric converter function to the electronic charge transfer layer 22. To realize this structure, the electronic charge transfer layer 22 may utilize generally known technique such as full frame CCD (FF-CCD), thus photoelectric converter function is available. According to such the structure, the electronic charge transfer layer 22 performs both the photoelectric conversion and X-Y transfers of the electronic charges. Though this structure eliminates the Z transfers, it is not able to execute the sub-exposures and the X-Y transfers simultaneously. In this case, the sub-exposures should be separated by using a mechanical shutter or the like which allows separated light introductions.
The present invention may be applicable to any image capturing apparatuses where still image capturing function is available. That is, the present invention may be applicable to any imaging apparatuses embedded in any apparatuses, for example, mobile phones or the like.
This application is based on Japanese Patent Application No. 2005-87394 (filed on Mar. 25, 2005), and including specification, claims, drawings and summary. The disclosures of the above Japanese Patent Application are incorporated herein by reference in its entirety.
Number | Date | Country | Kind |
---|---|---|---|
2005-87394 | Mar 2005 | JP | national |