The present invention, together with other objects and features thereof, will become apparent by understanding the following description of preferred embodiments, and are clearly defined in the appended claims. Those skilled in the art will find many other advantages of the present invention that are not mentioned herein by implementing the present invention.
Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. The embodiments described below are merely given by way of examples, and various modifications may be made including the variations hereinafter described.
The imaging element 101 includes a plurality of pixels, and the plurality of pixels include an effective pixel region used in imaging the subject and an OB pixel region arranged in a light-shielded state at the periphery of the effective pixel region and used in detecting an Optical Black level (hereinafter abbreviated as OB level).
The analog front end 107 includes a Correlated Double Sampler (hereinafter abbreviated as CDS) 102 for performing correlated double sampling to set the signal level of the analog imaged signal outputted from the imaging element 101, an Automatic Gain Controller (hereinafter abbreviated as AGC) 103 for amplifying the signal outputted from the CDS 102 at an adjustable gain, an Analog Digital Converter (hereinafter abbreviated as ADC) 104 for converting the signal amplified by the AGC 103 to the digital imaged signal, a Timing Generator (hereinafter abbreviated as TG) 106 for generating a pulse to be used in imaging, and a vertical driver 105 for outputting the pulse generated by the TG 106 to the imaging element 101.
Furthermore, the pulse DSP 111 includes a variance calculating unit 108 (serving as variation calculating unit for calculating variance of the signal level of each pixel, which is a characteristic constituent element of the present invention), a luminance level detecting unit 109 for detecting luminance by obtaining an average value of the signal levels of the pixels within a predetermined region, and a timing adjusting unit 110 for adjusting the phase (timing) of the pulse to be generated by the TG 106 based on the results of the calculation and detection of the variance calculating unit 108 and the luminance level detecting unit 109. The analog imaged signal outputted from the imaging element 101 is stored in a memory (SDRAM), which is not shown. The variance calculating unit 108 (variance calculating unit) and the luminance level detecting unit 109 read data of each pixel from the SDRAM (not shown), and perform calculation based on the signal thereof.
In the present invention, the data necessary in determining the pulse DS2 is measured while shifting the phase of the pulse DS2 from an initial value after fixing the pulse DS1 and the ADCLK to the initial values defined in advance (S301). An optimum phase for the pulse DS2 is determined by evaluating the data (step S302). After the pulse DS2 is determined, data necessary in determining the pulse DS1 is measured while shifting the phase of the pulse DS1 from the initial value with the pulse DS2 fixed to the determined optimum value and the ADCLK fixed to the initial value (step S303). The optimum phase for the pulse DS1 is determined by evaluating the data (step S304). After the pulse DS1 and the pulse DS2 are determined, data necessary in determining the ADCLK is measured while shifting the phase of the ADCLK from the initial value after fixing the pulses DS1 and DS2 to the respective optimum values (step S305). The data is then evaluated to determine the optimum phase for the ADCLK (step S306, step S307). After the optimum phases for the pulse DS1, the pulse DS2, and the ADCLK are determined, information pertaining to the determined optimum phases is set in a register in the TG 106, so that the pulse is generated at the optimum phase. The details of each step will now be described.
First, the phase adjustment of the pulse DS2 will be described with reference to
In
Description will be made in further detail with reference to
Next, the phase adjustment of the pulse DS1 will be described with reference to
In
Description will be made in further detail with reference to
In step S702, the luminance within the pulse DS1 detection region (second pixel region) of the retrieved analog imaged signal is calculated. That is, the average value of the signal level of each pixel in the pulse DS1 detection region is calculated. Since the pixels whose signal levels are greater than or equal to a predetermined value are assumed as being saturated, sampling is preferably performed excluding such pixels. The process of step S702 is performed in the luminance level detecting unit 109.
In step S703, the phase of the pulse DS1 is shifted one step backward with the pulse DS2 and the ADCLK fixed. In step S704, the analog imaged signal imaged by the imaging element 101 is retrieved, and the luminance within the pulse DS1 detection region of the retrieved analog imaged signal is calculated in step S705.
In step S706, the difference between the luminance calculated from the analog imaged signal retrieved at the phase of the pulse DS1 one step before and the luminance calculated from the analog imaged signal detected at the current phase is taken, and determination is made on whether or not the difference is smaller than or equal to a predetermined threshold value. When the difference is smaller than or equal to the predetermine threshold value, the phase of the current pulse DS1 is determined as being within the stable region and when not smaller than or equal to the predetermined threshold value, the current phase is determined as not being within the stable region in step S707.
In step S708, the phase of the pulse DS1 is shifted one step backward with the pulse DS2 and the ADCLK fixed. After the phase is shifted by one step, the processes of steps S704 to S708 are again performed, and determination is made on whether or not the shifted phase is within the stable region. The process is repeatedly performed for one period, and determination is made from which phase to which phase the stable region includes. Finally, in step S709, the median value of the phase determined as the stable region is determined as the optimum value of the pulse DS1. When determined that a plurality of stable regions exist non-continuously in the final determination of the stable region, the interval (area) of shorter period length may be ignored or the interval (area) of the longest period length may be determined as the stable region.
When the noise component is large, there is a possibility that the stable region may be mistakenly detected or may not be detected with only the difference between two pixels. In such a case, the difference between the average value of the luminance at three or more phases and the average value of the luminance at the current phase may be calculated and compared with a threshold value by using, e.g., a filter calculation. Alternatively, the variance of the luminance at three or more phases may be calculated and compared with the threshold value.
The initial values of the pulse DS1 used in respective adjustment flows of the pulse DS1 and the pulse DS2 may be the same or may be different. For instance, the initial value of the pulse DS1 in the adjustment flow of the pulse DS2 is set so as to be in the vicinity of the reference period anticipated from the design specification, and the initial value of the pulse DS1 for retrieving the first image data in the adjustment flow of the pulse DS1 is set so as to be within the reset period to detect the rapid decrease of the luminance signal.
The adjustment of the ADCLK will now be described with reference to
In
the imaging element 101 is in a light-shielded state; and
basis of the output of a pixel group positioned in a partial region or an entire region (hereinafter referred to as an ADCLK detection region) in the region including at least one of the effective pixel region and the OB pixel region.
That is, the variance is a value indicating to what extent the signal level of each pixel, which is constant in an ideal condition, is varied due to the fact that the imaging element 101 is light-shielded. Therefore, the ADCLK must be set so that the variance becomes small. The pixel region for calculating the luminance and the pixel region for calculating the variance may be the same or may be different. When the state of the pulse is the state shown in 801 in the figure, the variance will have a concave shape indicated by 803 when the ADCLK is shifted from the initial value as in the region indicated by 802 with the pulse DS1 and the pulse DS2 fixed to the optimum value. The phase of the ADCLK must be determined so that the variance becomes the smallest value, but the variance may become minimum at a wrong position for some reason. Therefore, the luminance within the ADCLK detection region is compared with a predetermined expected value at the phase at which the variance is determined as minimum. Since the OB pixel region is light-shielded, the expected value serving as a DC offset in the design specification exists. When the luminance within the ADCLK detection region is far off from the expected value, the ADCLK cannot be concluded as being optimum. Therefore, when the difference between the luminance and the predetermined expected value is less than or equal to a predetermined threshold value at the phase at which the variance is determined as minimum, that phase is determined as the optimum value of the ADCLK. However, when the difference between the luminance and the predetermined expected value is greater than the predetermined threshold value, determination is made on whether or not the difference between the luminance and the predetermined expected value is less than or equal to the predetermined threshold value at the phase at which the variance is the next smallest. The minimum value of the ADCLK is determined by repeating the above processes.
A method of light-shielding the imaging element 101 includes shielding the incident light by closing a mechanical shutter. Since the light-shielded state is obtained from the beginning when having the OB pixel region as the ADCLK detection region, the mechanical shutter does not necessarily need to be closed.
The adjustment of the ADCLK will be described in further detail with reference to
According to the above processes, the distributions for every phase of the luminance and the variance are stored in the memory. The optimum ADCLK is then calculated using the data stored in the memory. First, in step S909, the variance σ(1) of the first phase is set as a minimum value σ(min). In step S910, the variance of the second and subsequent phases is assumed as σ(n) and is compared with σ(min). When σ(n) is smaller, σ(n) is set as a new minimum value σ(min) in step S911. The process of step S910 is repeated until the last phase to calculate the phase at which the variance becomes minimum. In step S912, determination is made whether or not the difference between the expected value defined by the luminance at the phase at which the variance becomes the minimum value and the design specification and the minimum value is smaller than or equal to a predetermined threshold value. When the difference is within the predetermined threshold value, the phase at that point is determined as the optimum phase of the ADCLK in step S913. When the difference is greater than the predetermined threshold value, the process of step S913 is performed for the phase at which the variance becomes the next smallest after the phase at which the variance is σ(min). The processes of steps S912 and S914 are repeated until the optimum phase is determined.
According to the method described above, each phase of the pulse DS1, the pulse DS2, and the ADCLK can be automatically adjusted. Therefore, the phase of the pulse outputted from the TG 106 can be automatically adjusted when the imaging element 101 itself is replaced or when the characteristics of the imaging element are changed due to external factors (temperature, aged deterioration, etc.). Furthermore, automatic adjustment with high accuracy becomes possible since the phase of the pulse is adjusted through individual methods in view of the characteristics of each pulse.
The variance calculating unit 108, the luminance level detecting unit 109, and the timing adjusting unit 110, which are characteristic constituent elements of the present invention, may be configured by circuits as hardware or may be realized by software using a microcomputer. When the variance calculating unit 108 (variation calculating unit) and the luminance level detecting unit 109 are configured by hardware circuits, the present invention can be realized without imposing a load on a CPU.
The embodiment described above is merely an example, and it is needless to say that various modifications may be made other than the main variations to be described below.
The imaging element 101 such as a CCD and a MOS sensor often has defective pixels resulting from manufacturing. At such defective pixels, the signal level is often fixed in the vicinity of the maximum value or the minimum value regardless of the quantity of light of the incident light. Therefore, the value of the defective pixel is desirably not used in phase adjustment even if it is within the detection region of each pulse. In the present variation, the defective pixel detecting unit 113 detects the defective pixel, and stores the address of the detected defective pixel in the storage unit 114 in advance. Accordingly, the defective pixels will not be used in the phase adjustment, and the precision of the phase adjustment can be enhanced.
The defective pixel detecting unit 113 takes various configurations, and may be a configuration in which charges for a certain time are accumulated with the mechanical shutter closed at the time of activation of the digital camera, and the pixels whose signal level is greater than or equal to a predetermined threshold value are determined as defective pixels. The storage unit 114 does not need to be configured to hold all the addresses of the defective pixels, and merely needs to store the addresses of a predetermined number of defective pixels.
In the setting of the pulse DS1, the region in which the difference with respect to the adjacent pixel is less than or equal to the predetermined threshold value is assumed as a stable region, and pulse phase adjustment is performed so that the rising edge of the pulse DS1 comes at the center of the stable region. However, there is a possibility the phase period that becomes the stable region can not be detected when the signal quality is low, as shown in
Furthermore, the method of detecting the stable region does not necessarily include only obtaining the difference with respect to the adjacent pixel as described above. Therefore, the detection of the stable region for the first time and the detection of the stable region for the second time may be different. For instance, the detection of the stable region for the first time may include calculating the difference between the average value of the luminance at three or more phases and the average value of the luminance at the current phase, and comparing the difference with the threshold value set relatively small. The detection of the stable region for the second time may include calculating the difference between two adjacent pixels and comparing the difference with a threshold value set relatively large. Therefore, the subject of the present variation is to alleviate the detecting conditions so that the stable region can be more easily detected in the detection for the second time, so that the pulse DS1 can be set even when the signal quality is low.
In the description of the above embodiments, the phase adjustment is performed while shifting the phase in one period to adjust the phases of the pulse DS1, the pulse DS2, and the ADCLK. However, when the design specification of the imaging element 101 is known in advance, around which phase each pulse needs to be phase-adjusted can be anticipated to a certain extent. Thus, the adjustment range may be narrower than one period as shown in
Since the pulse DS2 is adjusted first in the present variation, around which phase the pulse DS1 and the ADCLK need to be adjusted can be anticipated when the phase of the pulse DS2 is adjusted. In
Obviously, if accuracy does not need to be considered, the phases of all the pulses of the pulse DS1, the pulse DS2, and the ADCLK does not necessarily need to be adjusted, and the phases of the other pulses may be obtained with the fixed phase from the phase of the pulse that is calculated firstly, or the phase of the pulse to be obtained thirdly may be determined from the phase of the pulse obtained secondly.
The optimum phase is assumed to exist in the vicinity of the previously adjusted phase even when the phase adjustment is performed due to occurrence of phase shift by a factor such as temperature change and aged deterioration. The result of phase adjustment is stored in the memory each time, and the phase adjustment is performed with only the vicinity of the previously adjusted phase as the adjustment range when newly performing the phase adjustment.
In the phase adjustment of the pulse DS1 and the pulse DS2, the phase adjustment are difficult to be performed unless the luminance is greater than or equal to a certain amount since the optimum phase is determined from the magnitude of the luminance. Since medical digital cameras etc. often have auxiliary light such as a LED, such auxiliary light may be used when the luminance at the peak was less than or equal to a predetermined value after the phase adjustment was performed once.
A histogram used in the present invention will now be described.
The histogram calculating unit 115 counts the number of appearance for every section with respect to each signal, and outputs the number of appearance for each section after the calculation of all the signals of a specified pixel region is finished. This corresponds to 116 of
Existing digital still cameras have a function of displaying the histogram of the image after image processing, and thus the histogram calculating unit 115 can be realized by utilizing such a block without being newly arranged. The input signal when utilizing such a block is not the signal (analog imaged signal or digital imaged signal) outputted from the imaging element 101 but is the signal after image processing, and thus each parameter of image processing must be changed to a value suited for the automatic adjustment. The configuration of the histogram calculating unit 115 or the configuration using the histogram calculating unit 115 is not limited to the above. Configuration is possible without using the SDRAM by performing the automatic adjustment using the histogram calculating unit 115.
The block memory circuit 117 is adapted to,
integrate data for every pixel color within one block,
output the result of integration for i (horizontal direction)×j (vertical direction) blocks while retrieving one image (while forming a frame), and
output respective integrated value of the R pixel, the Gr pixel, the B pixel, and the Gb pixel for i block(s) after integration of i block(s) is completed.
The size of one block and the number of blocks are preferably adjusted to appropriate values in the phase adjustment. The data with high precision can be obtained by making the size of one block small.
In the variance calculating unit 108 (variation calculating unit) and the luminance level detecting unit 109, the variation value and the luminance level can be calculated without using the SDRAM by using the output result of the block memory circuit 117 instead of acquiring the image data from the SDRAM. The pulse phase may be automatically adjusted while changing the region for calculation for every frame.
A variation including a threshold value detecting unit 118 for counting the number of pixel signals satisfying the condition (threshold condition) where its signal level is
greater than or equal to a first threshold value, and
lower than or equal to a second threshold value, in the pixel signals outputted from a specified pixel region, will now be described.
The pixel signals of R pixel, Gr pixel, B pixel, and Gb pixel outputted from the imaging element 101 are inputted to the threshold value detecting unit 118. The threshold value detecting unit 118 counts the number of pixel signals whose signal level satisfies the above threshold condition in the pixel signal group outputted from the specified pixel region for every pixel color. The output of the threshold value detecting unit 118 can be used for the variation value by setting the first and second threshold values constituting the threshold condition to parameters suited for the automatic adjustment.
The pixel region is set as shown in
A variation in which automatic adjustment of the pulse phase is performed using a frequency component detecting circuit for automatic focus adjustment (hereinafter referred to as AF frequency component detecting circuit) 119 serving as one example of a frequency detecting block will now be described.
The AF frequency component detecting circuit 119 includes a high pass filter (hereinafter abbreviated as HPF), where the edge information of the high frequency component of the signal can be extracted by filtering the input signal and the signals of the neighboring pixels by means of the HPF, the peak value of the edge is integrated for every pixel block based on the edge information, and the result of the integration is outputted.
In the present variation, the ADCLK is adjusted so that the variation of the phase is small when the imaging element 101 is in the light-shielded state. When the peak value of the high frequency region is large in the light-shielded state, the variation is determined as large. The present variation uses such a theory and provides the AF frequency component detecting circuit 119 to calculate the phase variation value.
In the frequency detecting block, the AF frequency component detecting circuit 119 is often mounted on the DSP 111 to realize the normal AF (AutoFocus), as described above. Thus, the present variation can be configured without newly adding a processing block by using the AF frequency component detecting circuit 119 as the frequency detecting block. In performing automatic adjustment of the pulse phase, the parameters suited not for AF but for the automatic adjustment of the pulse phase merely need to be set.
A region of low frequency component may be extracted from the pixel region using the frequency detecting block (AF frequency component detecting circuit 119 etc.), and the variance and the luminance level may be calculated in the extracted low frequency component region. The low frequency component region is a region in which the noise component is few, and thus the variation value of the phase can be calculated at high precision according to the present variation.
In the present variation, the supply of clock to the luminance level detecting unit 109, the variance calculating unit 108, and the timing adjusting unit 110 is stopped while the image data is retrieved. At the timing other than retrieving the image data, supply of power to the vertical driver 105 for generating the imaging element control signal is stopped.
The variations for higher speed will now be described.
The concept of transfer of charges from the sensor will now be described with reference to
The operation of an eleventh variation will now be described. In transfer of charges, a normal image read-out process 2102 is performed during one period of the vertical reference signal 2101, as shown in
In the process of transferring the charges described above, the normal image read-out process 2102 is performed only on the portion to be actually detected for the automatic phase adjustment, and a high-speed image read-out process 2103 is performed on the portion that will not be used for the detection in the present variation. The image is not actually outputted during the period the high-speed image read-out process 2103 is being performed, but after rapidly reaching the line necessary for the detection by performing high-speed operation after such a period, the normal image read-out process 2102 is performed on the line necessary for the detection. Thereafter, the transfer process of the remaining lines is performed by the high-speed image read-out process 2103. According to such an operation, the period 2105 necessary for performing all the image read-out processes by using both the high speed image read-out process 2103 and the normal image read-out process 2102 is shorter than the period 2104 necessary for performing all the image read-out processes by the normal image read-out process 2102, thereby enabling the high-speed operation. The operation of the present variation will now be described in detail with reference to
The vertical reference signal period 2101 corresponds to the time length of one screen, and the pixels worth one line are outputted with the horizontal reference signal 2202 as the reference. The specific drive is as follows. That is, the charges worth one line are transferred in the vertical direction by operating the vertical transfer path by the vertical transfer process 2203. The charges worth one line are thereby transferred to the horizontal transfer path. The charges worth one line transferred to the horizontal transfer path are outputted one by one pixel at a time by operating the horizontal transfer path by the horizontal transfer process 2204. Such a horizontal transfer process 2204 is repeated for the pixels worth one line to output the data of one line. After the output of data of one line is finished, the vertical transfer process 2203 is performed to transfer the next line. The horizontal transfer process 2204 is repeated for the pixels worth one line, after the vertical transfer process 2203 is performed, to output the next line. The data worth one screen is outputted by repeating such operations.
The vertical reference signal period 2101 corresponds to the time length of one screen, but actually, the vertical transfer process 2213 performed from the horizontal reference transfer line to the Nth line is performed without waiting for the transfer of charges using the horizontal transfer path in the horizontal transfer process 2215 to be completed for the horizontal pixels. The vertical transfer process 2213 operates independently from the horizontal transfer process 2215, and actually, the vertical transfer process 2213 performs the process at high speed up to the line position necessary for the detection. The vertical transfer process 2213 can be performed before all the charges are outputted from the horizontal transfer path by independently driving the horizontal transfer process 2215 and the vertical transfer process 2213, thereby achieving higher speed of the process.
During the above operation period, the data transferred by the horizontal transfer process 2215 becomes invalid data as it is not synchronized with the vertical transfer process 2213. However, the data being invalid will not cause any problem since effective data are not necessary during this period. During the period in which the data is actually necessary, the effective data can be obtained by performing all the image read-out processes by the normal image read-out process 2102 described above. After the effective data are acquired, the high-speed drive operation of the present variation is again performed to achieve higher speed.
The process of switching between the transfer processes may be controlled by at least one of the resolution of the imaged analog imaged signal, the S/N ratio of the analog imaged signal, and the S/N ratio of the digital imaged signal. For instance, when the resolution of the analog imaged signal is sufficiently high to be used for the phase adjustment, horizontal transfer is not carried out and only vertical transfer is performed in the majority of the region, thereby achieving higher speed. When the resolution is low or when the resolution is high but the S/N ratio of the analog imaged signal itself is not satisfactory and thus the signal itself is not reliable unless a great number of lines are sampled and averaged, the percentage of performing the horizontal transfer may be increased. The reliability of the detection value can be enhanced by using only the center of the screen.
A variation for achieving higher speed through a method of sampling the analog imaged signal outputted from the imaging element 101 will now be described.
The method of sampling the analog imaged signal will now be described with reference to
In the present variation, the output timing of the pulse DS2 (2302) is operated with independently different parameters for every horizontal reference signal period 2202 to check the luminance level corresponding to a plurality of phases in one screen. Specifically, in the next horizontal reference signal period 2202, the pulse DS2 (2302) is outputted at a timing shifted by ΔT1 (2303) with respect to the first horizontal reference signal period 2202. In the further next horizontal reference signal period 2202, the pulse DS2 (2302) is outputted at a timing shifted by ΔT2 (2304) with respect to the first horizontal reference signal period 2202.
The phase of the data can be microscopically moved at high precision in one data retrieval by performing the output adjustment of the pulse DS2 (process of shifting the output timing of the pulse DS2 (2302) respectively to Δl−Δ(N−1) for the imaging element 101 in which one screen is configured by N lines).
Further increase in processing speed can be achieved by setting the phase independent for every line and detecting a plurality of sampling patterns in one screen. Such a configuration allows the present invention to be applied even to the field of in-vehicle device etc. in which real-time property of the process is extremely important.
The unit of change in phase is desirably changed based on at least one of the resolution of the analog imaged signal, the S/N ratio of the analog imaged signal, and the S/N ratio of the digital imaged signal. For instance, when the resolutions of the analog imaged signal and the digital imaged signal are sufficiently high as pixels to be used in the phase adjustment, the phase may be changed for every large number of lines. When the resolution is high but the S/N ratio of the signal itself is not satisfactory and thus the signal itself is not reliable unless a great number of lines are sampled and averaged, the phase must be changed for every large number of lines larger than that for the above-mentioned phase adjustment. When the resolution of the analog imaged signal is low but the S/N ratio of the analog imaged signal and the digital imaged signal is satisfactory and thus the process of averaging and processing the large number of lines is not required, the phase may be changed for every small number of lines.
A thirteenth variation in which the method of storing in the memory and various processes are improved for the digital imaged signal digital converted by the ADC 104 after performing the correlated double sampling of the signal by the CDS 102 will now be described with reference to
A effective area control device 2403 is further provided in the present variation. The effective area control device 2403 selectively retrieves only the data of the effective area (hereinafter referred to as digital image output of the effective area) that is necessary in unit data area of the digital image signal digital converted in the ADC 104 after performing the correlated double sampling by the CDS 102, and stores the data in the memory. Specifically, the effective area 2402 is set in the ADC output 2401 during the horizontal reference signal 2202 period, and the effective area control device 2403 retrieves the ADC output 2502 and proceeds to the next operation at the timing 2404 when the effective area 2402 is finished. The effective area control device 2403 stops the clock to be provided to the horizontal and vertical transfer units during the period other than the effective area 2402.
Through such processes, the processes can be performed with the minimum memory size necessary for the automatic adjustment. The process can be started at the point where the necessary data are ready, thereby further realizing higher speed. In other words, when the image has sufficiently high resolution assuming the application is for the automatic adjustment, the data area to be stored may be reduced (specifically, reduce the period length of the effective area 2402) to realize higher speed. When the resolution is low, or when the resolution is high but the S/N ratio of the signal itself is not satisfactory and thus the signal itself is not reliable unless a plurality of lines is sampled and averaged, the data area to be stored may be enlarged (specifically, increase the period length of the effective area 2402) to enhance the reliability.
Moreover, the reliability of the detection value can be enhanced by setting the effective area 2402 only at the center of the screen. As shown in
The higher speed control by the sensor drive described in the eleventh and twelfth variations is performed by the effective area control device 2403 in
Although the present invention has been described in its preferred form with a certain degree of particularity, it should be understood that the combination and arrangement of the parts may be modified in various ways without departing from the spirit and the scope of the invention as hereinafter claimed.
Number | Date | Country | Kind |
---|---|---|---|
2006-205905 | Jul 2006 | JP | national |
2007-176036 | Jul 2007 | JP | national |