Patent Application
20060197853
1. Field of the Invention
The present invention relates to a solid-state image pickup apparatus, and more particularly to a solid-state image pickup apparatus of the type capturing an image of a subject to produce a plurality of divided images to combine them into a single picture representative of the subject. The invention also relates to a method of capturing an image of a subject to produce a plurality of divided images to combine them into a single picture representative of the subject.
2. Description of the Background Art
A conventional type of solid-state image pickup apparatus is configured to read out imagewise signal charges from a photosensitive array, or imaging surface, formed by photosensitive cells at high speed through a plurality of horizontal transfer paths and a corresponding plurality of output circuits. Each of the output circuits outputs image data representative of a particular image, each of which forms, together with the other, a complete picture corresponding to the imaging surface.
Another type of conventional solid-state image pickup apparatus has its imaging surface physically divided into a plurality of regions. In this type of apparatus, signal charges are transferred from each region of the imaging surface to a particular horizontal transfer path so that image data are output in the form of corresponding plurality of data streams.
The problem with the solid-state image pickup apparatuses of the types described above is that the reference level of image data is different between the divided regions of the imaging surface and causes a boundary seam between the divided regions to be conspicuously displayed in a frame of image. In order to solve this problem, U.S. patent application publication No. US 2004/0090538 A1, for example, discloses correcting means for correcting the seam in an image between the divided regions of the imaging surface.
The correcting means taught in the above U.S. patent application determine beforehand correction values for correcting a seam. This, however, brings about another problem that various correction values, matching with various image pickup conditions including temperature and of lenses or optics, must be determined beforehand. Further, it is difficult for the correcting means to cope with all possible pickup conditions because the storage capacity of a memory is limited.
It is an object of the present invention to provide a solid-state image pickup apparatus capable of easily obtaining, even when temperature or optical conditions vary, correction values for correcting a seam in an image between the divided regions of an imaging surface without resorting to any particular device.
A solid-state image pickup apparatus of the present invention is selectively operable of producing, at a preliminary pickup stage, an image by picking up a subject field or of obtaining, at an actual pickup stage, an exposure condition and a focus condition. The solid-state image pickup apparatus includes an image sensor including an imaging surface made up of a plurality of divided regions for producing a plurality of analog electric signal streams representative of a plurality of divided images on the basis of the divided regions. An analog signal processor executes analog signal processing on the plurality of analog electric signal streams and converts the resulting processed analog electric signals to a corresponding plurality of digital image signals. A digital signal processor executes digital signal processing on each of the plurality of digital image signals to thereby produce a single frame of image. The digital signal processor includes a detector for detecting out of the plurality of digital image signals a plurality of level decision data for determining level differences between the plurality of divided images. A correction value calculator calculates, at the preliminary pickup stage, correction values for correcting the level differences between the divided images in accordance with the plurality of level decision data. A corrector corrects, at the actual pickup stage, the plurality of divided images with the correction values calculated by the correction value calculator.
Also, an image input apparatus of the present invention inputs a plurality of divided image signals representative of a corresponding plurality of divided images. The image input apparatus includes a signal processor for processing the plurality of divided image signals to thereby produce a combined single image. A detector, included in the signal processor, detects a plurality of level decision data, which are used to determine level differences between the plurality of divided images, out of the plurality of divided image signals. A correction value calculator calculates correction values for correcting the level differences between the divided images in accordance with the plurality of level decision data. A corrector corrects the plurality of divided images.
An image correcting method practicable with the solid-state image pickup apparatus or the image input apparatus is also disclosed.
The objects and features of the present invention will become more apparent from consideration of the following detailed description taken in conjunction with the accompanying drawings in which:
Referring to
In
The image pickup apparatus 10 is selectively operable in an automatic focus (AF) mode for automatically focusing the optical image of a desired subject on the image pickup section 12 or a manual focus (MF) mode for allowing the operator to manually focus the former on the latter.
In the illustrative embodiment, the imaging surface 40 is divided into a plurality of regions, which generate a corresponding plurality of divided image data. In the illustrative embodiment, let boundaries between the plurality of divided regions, e.g. a boundary between two divided regions, be referred to as a seam. As shown in
Further, the image pickup section 12 serves as photo-electrically converting the optical image of a field focused on the imaging surface 40 to a corresponding analog electric image signal under the control of a timing signal 114, see
Referring again to
The system controller 16 controls the operation of the entire image pickup apparatus 10 in response to the operation signal 106 received from the control panel 14. For example, in the illustrative embodiment, the system controller 16 controls the timing generator 18 and signal processor 24 with control signals 108 and 110, respectively. Also, the system controller 16 is responsive to the operation signal 106 for determining whether or not the shutter release button is depressed to its half-stroke or full-stroke position so as to produce the control signals 108 and 110 to indicate preliminary pickup if the shutter release button is in its half-stroke position or actual pickup if it is in its full-stroke position.
Further with the illustrative embodiment, when the shutter release button is in its half-stroke position indicative of preliminary pickup, the system controller 16 determines whether or not the focus control mode is the AF mode and then causes the control signal 108 applied to the timing generator 18 to indicate AF control, automatic exposure (AE) control and so forth in accordance with result of the above decision. Subsequently, the system controller 16 receives from the signal processor 24 an adjustment signal 132 including an estimated luminance value and an estimated contrast value and then determines an exposure and a focus condition for actual pickup in response to the adjustment signal 132.
For example, when the shutter release button is depressed to its half-stroke position, the system controller 16 causes, if the AF mode is selected, the control signal 108 to indicate AF and AE control, or causes, if the MF mode is selected, the control signal 108 to indicate only AE control while indicating MF control at a focus position manually selected by the operator.
Furthermore, when the shutter release button is held in its half-stroke position, the system controller 16 receives level decision data conveyed on an adjustment signal 132 for determining level differences between divided images from the signal processor 24, as data to be used for the calculation of correction values 112 at the seam or boundary. For example, data based on the adjoining pixels, which belong to the divided region in respective divided images, maybe input from the signal processor 24 to the system controller 16 as adjoining pixel data. At this instant, the correction value calculator 28, included in the system controller 16, calculates correction values 112 by using the adjoining pixel data and writes the correction values in the recorder 26. The adjoining pixel data input to the system controller 16 may preferably be adapted to receive adjoining pixel data based on divided images picked up at a defocus position, i.e. a position where a desired subject is inadequately focused.
When the system controller 16 commands AF control, estimated contrast values at a focus position are produced. At this instant, assuming that estimated contrast values 152a and 154a shown in
For example, when the shutter release button is depressed to its half-stroke position while the AF mode is selected, the system controller 16 receives adjoining pixel data based on divided images, which are picked up at the defocus position during automatic focusing control. On the other hand, if the MF mode is selected when the shutter release button is in its half-stroke position, then the system controller 16 outputs the control signal 108 indicative of MF and AE control, and thereafter a control signal 108 indicative of pickup at a defocus position shifted from a focus position manually selected by the operator and then receives adjoining pixel data based on the resulting divided images.
The timing generator 18 functions as generating a timing signal 114, including a vertical and a horizontal synchronous signal, an electronic shutter pulse and so forth, in response to the control signal 108 input from the system controller 16 and delivers the timing control signal 114 to the image pickup section 12. In addition, the timing generator 18 generates a timing control signal 116 including sampling pulses for a correlated double sampling (CDS) circuit, not shown, and feeds the timing control signal 116 to the preprocessor 20. Further, the timing generator 18 feeds a timing control signal 118 including a conversion clock to the AD converter 22.
The preprocessor 20 includes various circuits, e.g. the CDS circuit mentioned above and a gain-controlled amplifier (GCA). The CDS circuit, gain-controlled amplifier and so forth, controlled by the timing control signal 116, execute analog processing on the analog electric signals 102 and 104 for thereby outputting the resulting analog image signals 120 and 122, respectively.
The AD converter 22 is adapted to be responsive to the timing control signal 118 to quantize the signal levels of the analog image signals 120 and 122 by use of a preselected quantizing level to output corresponding digital image signals 124 and 126, respectively.
The signal processor 24 is adapted to be controlled by the control signal 110 fed from the system controller 16 to receive, when the control signal 110 indicates the preliminary image pickup, the digital image signals 124 and 126 provided to produce the previously mentioned adjustment signal 132, including estimated luminance and contrast values, by arithmetic operations and deliver the adjustment signal 132 to the system controller 16. On the other hand, when the control signal 110 is indicative of actual pickup, the signal processor 24 executes white balance control, gamma correction, image combination, synchronization and other digital signal processing on the digital image signals 124 and 126, writing the resulting digital image signal 128 in the recorder 26.
Further, when the control signal 110 from the system controller 16 is indicative of preliminary pickup, the signal processor 24 detects level decision data for determining level differences between the divided image regions out of the digital image signals 124 and 126. For example, the signal processor 24 detects first adjoining pixel data based on the adjoining pixels 156, which belong to the image region 152, out of the digital image signal 124 and detects second adjoining pixel data based on the adjoining pixels 158, which belong to the image region 154, out of the digital image signal 126. The first and second adjoining pixel data thus detected are conveyed to the system controller 16 by the adjustment signal 132. In the illustrative embodiment, the signal processor 24 may preferably be adapted to generate the adjoining pixel data on the basis of the digital image signals 124 and 126 derived from data of an image picked up at the defocus position.
When the control signal 110 from the system controller 16 indicates actual pickup, the signal processor 24 reads out the seam correction values 130 stored in the recorder 26 and causes its seam corrector 30 to use the seam correction values to correct the seam between the divided images represented by the digital image signals 124 and 126.
The recorder 26 includes a data storing medium, not shown, for storing the digital image signal 128 fed from the signal processor 24. The data storing medium may be implemented by, e.g. a memory card loaded with a semiconductor memory device or a package loaded with a magneto-optical disk or similar recordable disk, which may be removably mounted to the image pickup apparatus 10.
Particularly, in the illustrative embodiment, the recorder 26 stores the seam correction values 112 input from the correction value calculator 28 of the system controller 16, and also feeds synchronizing signal values 130 stored therein to the seam corrector 30 included in the signal processor 24.
The correction value calculator 28 is adapted to calculate the seam correction values 112 in accordance with the adjoining pixel data conveyed from the signal processor 24 by the adjustment signal 132 and write them in the recorder 26. Alternatively, the correction value calculator 28 may be configured, if desired, to calculate correction values for correcting the offset of data of preselected adjoining pixels based on data of other adjoining pixels.
In the illustrative embodiment, the correction value calculator 28 uses the first and second adjoining pixel data derived from the adjoining pixels 156 and 158, respectively, included in the imaging surface 40 to calculate the seam correction values 112. For example, as shown in
In the case where R (red), G (green) and B (blue) or similar color filter segments are arranged at the adjoining pixels from which the adjoining pixel data should be produced, the correction value calculator 28 may be adapted to use adjoining pixel data grouped by color to calculate a correction value color by color.
The seam corrector 30 is adapted for using the seam correction values 130 read out from the recorder 26 to correct the seam between divided images respectively represented by the digital image signals 124 and 126. For example, if the seam correction values 130 produced by the correction value calculator 28 are based on the formula (A−B)/B+1, then the seam corrector 30 multiplies the entire digital image signal 126 by the seam correction values 130 for thereby correcting the signal 126. In this manner, the seam corrector 30 matches the digital image signal 126 to the digital image signal 124, so that the resulting combined image is free from differences in level.
Reference will be made to
In the step S204, the system controller 16 determines whether or not the operation signal 106 from the control panel 14 is indicative of the AF mode. The system controller 16 executes a step 206 if the answer of the step 204 is positive, Yes, or executes a step S208 if it is negative, No.
In the step S206, the system controller 16 generates control signals 108 and 110 indicative of AF and AE control, respectively, at the preliminary pickup stage and delivers the signals 108 and 110 to the timing generator 18 and signal processor 24, respectively.
In response to the control signal 108, the timing generator 18 delivers timing signals 114, 116 and 118 to the image pickup section 12, preprocessor 20 and AD converter 22, respectively.
The image pickup section 12 picks up the image of a subject field in response to the timing control signal 114 and produces analog electric signals 102 and 104 representative of the image of the field for the respective divided regions 152 and 154. The preprocessor 20 and AD converter 22 sequentially process the analog electric signals 102 and 104 to thereby output corresponding digital image signals 124 and 126. The digital image signals 124 and 126 are input to the signal processor 24.
The signal processor 24, having received the control signal 110 indicative of preliminary pickup, calculates estimated luminance and contracts values on the basis of the digital image signals 124 and 126 and delivers an adjustment signal 132 representative of such estimated values to the system controller 16. The system controller 16 determines an exposure and a focus condition in accordance with the estimated luminance and contrast values.
Further, the signal processor 24 detects first and second adjoining pixel data based on the adjoining pixels 156 and 158, respectively, out of the digital image signals 124 and 126 picked up at a defocus position. The first and second adjoining pixel data thus detected are conveyed to the system controller 16 by the adjustment signal 132. In this condition, a step S212 is executed.
On the other hand, in the step S208, the system controller 16 generates control signals 108 and 110 indicative of AE control at the preliminary pickup state and feeds the control signals 108 and 110 to the timing generator 18 and signal processor 24, respectively. In this case, because the MF mode is selected by the operator instead of the AF mode, the image pickup section 12 is manually focused at a focus position set by hand.
In the step S208, the timing generator 18 feeds timing control signals 114, 116 and 118 to the image pickup section 12, preprocessor 20 and AD converter 22, respectively, in response to the control signal 108 as in the step S106. Analog electric signals 102 and 104, generated by the image pickup signal 12, are again sequentially processed by the preprocessor 20 and AD converter 22. Subsequently, digital image signals 124 and 126, respectively derived from the analog electric signals 102 and 104, are input to the signal processor 24.
The signal processor 24, having received the control signal 110 indicative of preliminary pickup, calculates estimated luminance values in response to the digital image signals 124 and 126 input from the AD converter 22 and delivers the two kinds of estimated values to the system controller 16 as an adjustment signal 132.
In a step 210 following the step 208, while the system controller 16 determines an exposure condition in accordance with the estimated luminance values fed thereto, as stated earlier, it outputs control signals 108 and 110 indicative of pickup at a defocus position, as distinguished from the set focus position, in order to determine a focus condition. With the control signals 108 and 110, the system controller 16 eventually controls the image pickup section 12, preprocessor 20 and AD converter 22. Consequently, digital image signals 124 and 126 picked up at the defocus position are output from the AD converter 22 to the signal processor 24.
The signal processor 24 detects first and second adjoining pixel data based on the adjoining images 156 and 158 out of the input digital image signals 124 and 126, respectively. The first and second adjoining pixel data are conveyed to the system controller 16 by the adjustment signal 132. This is the end of the defocus pickup step S210.
In the step S212 following the step S206 or 210, the correction value calculator 28, included in the system controller 16, calculates seam correction values 112 on the basis of the first and second adjoining pixel data and writes the seam correction values in the recorder 26.
After the step S212, it is determined whether or not the shutter release button is depressed to its full-stroke position by the operator (step S214). If the shutter release button is depressed to its full-stroke position (Yes, step S214), an operation signal 106 indicative of actual pickup is fed from the control panel 14 to the system controller 16. In response, the system controller 16 causes the image pickup apparatus 10 to perform actual pickup, i.e. actually shoot the desired scene (step S216). If the answer of the step S214 is No, then the procedure returns to the initial stage.
More specifically, in the step S216, the system controller 16 generates control signals 108 and 110 indicative of actual pickup and delivers the control signals 108 and 110 to the timing generator 18 and signal processor 24, respectively. At this instant, the control signals 108 and 110 include the exposure and focus conditions determined at the preliminary pickup stage.
The timing generator 18 feeds timing control signals 114, 116 and 118 based on the control signal 108 to the image pickup section 12, preprocessor 20 and AD converter 22, respectively, as in the preliminary pickup step. Subsequently, analog electric signals 102 and 104, generated by the image pickup section 12 in response to the timing control signal 114, are sequentially processed by the preprocessor 20 and AD converter 22. As a result, digital image signals 124 and 126, respectively corresponding to the analog electric signals 102 and 104, are fed from the AD converter 22 to the signal processor 24.
The signal processor 24, when received the control signal 110 indicative of actual pickup, executes white balance control, gamma correction, image combination, synchronization and other digital signal processing on the digital image signals 124 and 126. At this instant, before the combination of the digital image signals 124 and 126, the seam corrector 30 corrects a seam with the seam correction values 130 read out from the recorder 26. For example, the seam corrector 30 multiples the entire digital image signal 126 by the seam correction values 130 for thereby matching the digital image signal 126 to the digital image signal 124.
A digital image signal 128 thus subjected to digital processing by the signal processor 24 is written to the recorder 26. While the illustrative embodiment is structured to end the operation of the image pickup apparatus 10 after the writing of the digital image signal 128 in the recorder 26 (step S218), the system may be adapted to display the digital image signal 128 on a monitor, not shown, prior to the end of the procedure, if desired.
The image pickup section 12 may be provided with a plurality of horizontal transfer paths which are configured to transfer signal charges read out from the imaging surface separately to output a corresponding plural streams of analog electric signals. For example, in a specific modification shown in
The imaging surface 50 shown in
When the image pickup section 12 is of the divisional readout type, a plurality of horizontal transfer paths are connected to a respective output amplifier, not shown, each. In this case, the correction value calculator 28 of the system controller 16 may be adapted to calculate, based on adjoining pixel data, a particular correction value for correcting the gain of each output amplifier in order to reduce differences between a plurality of adjoining pixel data.
In the modifications to the image pickup apparatus 10 described above, the preprocessor 20 is adapted to receive a plural streams of analog electric signals, and, for that aim, to include a corresponding plurality of gain-controlled amplifiers, not shown, assigned one-to-one to the analog electric signal streams. In this case, the correction value calculator 28, included in the system controller 16, may be adapted to calculate, based on the adjoining pixel data, correction values each for correcting the gain of a particular gain-controlled amplifier in order to reduce differences between adjoining pixel data.
At the preliminary pickup stage, the image pickup apparatus 10 of the illustrative embodiment picks up a preselected subject with different exposure times, so that the signal processor 24 attains estimated luminance values and then calculates linearity representative of a relation between the exposure time and the estimated luminance value. In the illustrative embodiment, the signal processor 24 calculates a plurality of linearities for the plurality of divided regions. At this time, the signal processor 24 may be adapted for obtaining estimated luminance values from adjoining pixel data to calculate linearity.
The correction value calculator 28 may also be configured to calculate correction values to be used for the correction of the digital image signals input to the signal processor 24 and for the improvement on differences between the plurality of linearities mentioned above. For example, the correction value calculator 28 may calculate correction values to be used for the correction of offset of preselected linearity from the other reference linearity, and may use a mean value of the plurality of linearities as reference linearity.
An alternative embodiment of the present invention also practicable with the circuitry of
As shown in
The image pickup section 12 may be configured to output OB image data obtained from the OB regions 186 and 188 at the same time as or independently of image signals derived from the regions 182 and 184, respectively. The OB image data are sequentially processed by the preprocessor 20, AD converter 22 and signal processor 24,
Now, in the following, values produced by processing the OB image data obtained from the OB regions 186 and 188 and then averaging the results of processing will be represented by image data C and D, respectively. The correction values to be used by the seam corrector 30 of the signal processor 24,
The signal processor 24 may be adapted to feed the image data C and D mentioned above to the correction value calculator 28 of the system controller 16,
In the alternative embodiment, too, when color filter segments of, e.g. R, G and B are positioned in the OB regions 186 and 188 that derive the OB image data, the OB image data may be grouped by color so as to allow the correction value calculator 28 to calculate correction values color by color. 67) The image pickup section 12 may be configured such that signal charges are read out from the imaging surface in groups over a plurality of horizontal transfer paths and output as a corresponding plurality of analog electric signal streams. For example,
With the alternative embodiment thus configured, the correction value calculator 28 is adapted to calculate the seams correction values with a function having its gradient implemented as differences between the respective OB image data in the upper and lower and right and left divided regions. For example, in the case of the configuration shown in
In the above condition, assuming that pixel data at the coordinates y are M(y), then the pixel data M(y) may be corrected to pixel data N(y) by M(y)−L(y).
In a further alternative embodiment of the present invention, the image pickup apparatus 10 is configured to determine whether or not the correction of a seam or seams between divided regions is necessary, and execute correction if necessary. For the correction of a seam or seams, there may be used either one of the adjoining data at a seam and OB image data.
In the alternative embodiment stated above, the signal processor 24, for example, may include a decision circuit, not shown, for determining whether or not correction is necessary. The decision circuit may be adapted to make such a decision in response to the control signal 110 output from the system controller 16 and indicative of preliminary pickup.
The decision circuit may be adapted to feed the result of the decision to the system controller 16, in which case the correction value calculator 28 of the system controller 16 calculates correction values for a seam if correction is necessary, and otherwise not. This is also true with the seam corrector 30 included in the signal processor 24. Specifically, the seam corrector 30 calculates correction values for a seam only when correction is necessary. For example, the decision circuit determines, based on the digital image signals 124 and 126 output at the preliminary pickup stage, whether frequency components are low throughout the entire image data, and then determines that correction is necessary if they are substantially low.
More specifically, when the image pickup apparatus 10 shoots a generally flat wall, sky or similar flat subject, luminance varies little over the entire optical image of the subject with the result that the frequency components of image signals output at the actual pickup stage are substantially low in the entire image. In such a condition, a seam or seams between divided images are particularly conspicuous in the resulting image, so that the decision circuit determines that correction is required at the seam or seams.
Alternatively, the decision circuit may produce an estimated AE value during AE control from the digital image signals 124 and 126 at the preliminary exposure stage, and determine whether or not correction is necessary on the basis of the estimated AE value. In this case, the estimated AE value may be implemented by one representative of the light and shade of the entire image picked up, e.g. an estimated AE value represented by the accumulated value of luminance signals based on the digital image signals 124 and 126. Assuming that the estimated AE value lies in, e.g. any one of a light, a dark and a medium range, when classified, the decision circuit may preferably be adapted for determining that correction is necessary if the estimated AE value lies in the medium range.
As shown in
In the instant alternative embodiment, the decision circuit determines that correction is necessary if the estimated contrast values remain the same at any focus position.
Further, the image pickup apparatus 10 shown and described may be modified to operate in the same manner as in the preliminary pickup mode as soon as the apparatus 10 is turned on, shooting a field via the image pickup section 12. The resulting digital image signals 124 and 126 are input to the correction calculator 28 via the signal processor 24, causing the calculator 28 to calculate correction values for a seam. The correction values thus calculated at the time of power-up are written to the recorder 26 as initial information and read out at the time of actual pickup for allowing the seam corrector 30 to correct a seam.
The seam corrector 30 may be adapted to correct a seam with the correction values initially stored in the recorder 26 or correct, if the decision circuit determines that correction is necessary, a seam with correction values newly calculated by the correction value calculator 28.
In accordance with the present invention, correction values are calculated on the basis of image data input to the signal processor to be used for correcting a divided image. Therefore, the present invention is applicable even to an apparatus of the type involving an external sensor in exposure and focus control, or an image input apparatus merely adapted for receiving data of divided images.
In summary, in accordance with the present invention, a solid-state image pickup apparatus uses correction values produced from adjoining image data derived from adjoining pixels at a seam of a divided image, and can therefore correct level differences at the seam with a simple configuration without regard to temperature, optical conditions and other variable image capturing conditions, thereby insuring high image quality. This advantage is also achievable even when the correction values for the seam are produced from image data read out from OB regions.
The entire disclosure of Japanese patent application Nos. 2005-62636 and 2006-27091 filed on Mar. 7, 2005 and Feb. 3, 2006, respectively, including the specification, claims, accompanying drawings and abstract of the disclosure is incorporated herein by reference in its entirety.
While the present invention has been described with reference to the particular illustrative embodiments, it is not to be restricted by the embodiments. It is to be appreciated that those skilled in the art can change or modify the embodiments without departing from the scope and spirit of the present invention.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2005-62636 | Mar 2005 | JP | national |
| 2006-27091 | Feb 2006 | JP | national |
