1. Field of the Invention
The present invention relates generally to image processing for electronic imaging sensors, and more particularly, to an apparatus and method for stabilizing the black level of electronic image signal produced by an image sensor.
2. Description of the Related Art
Modern color cameras typically use solid state image sensors such as a charge-coupled-device (CCD) or a CMOS Image Sensor (CIS) to capture still images or generate video. It is important that the signal from these sensors has a stable black level. Black-level clamping enables photographs and video to consistently produce quality images regardless of gain (or equivalent ISO speed), scene content and scene illumination. Adjustment circuits providing such capability have previously been referred to as video black clamp, black restoration or DC coupling reduction circuits. Instability in black translates to degradations in image quality resulting in “washed out” or dim pictures, and flickering video. Rapid black-level convergence can help “capture the moment” at a moment's notice.
The video component of a transmitted television signal is typically characterized by its average brightness, i.e., transmitted DC level, and overall dynamic range. The signal that is to be reproduced in a television receiver, for example, must have proper brightness along with a black level that is constant regardless of scene content and scene-to-scene dynamics. The resulting signal should also fully span the dynamic range of the intended display regardless of the camera gain used to maintain brightness.
In early camera technology, black clamping or restoration was performed ex post facto. Since early video cameras required DC blocking circuits to circumvent technology limitations, television receivers had to fix the errors related to DC blocking in order to prevent major video degradation via buildup of systematic DC shifts. U.S. Pat. No. 4,338,630 discloses a single-chip chroma/luma integrated circuit that facilitates black clamping via dynamic DC restoration in the television receiver, rather than at the source of video origination. A DC restoration circuit clamped to a fixed black level is said to be characterized by 100% DC coupling. However, it is often desirable to decrease DC coupling by varying the clamping level to compensate for broadcast transmission inconsistencies. Although U.S. Pat. No. 4,338,630 facilitated black “clamping” in a single integrated circuit, each receiver behaved differently depending on user setting of black level and contrast.
Black level processing in a color video camera is exemplified by U.S. Pat. No. 4,680,624, which teaches black level clamping and video production wholly in the analog domain (
Nevertheless, the '624 circuit uses bandpass filters and quadrature detectors to strip the luminance signal from the base band signal so that the video is actually decomposed prior to being reassembled for broadcast or broadband transmission. Furthermore, the shading correction is empirically supplied based on general characteristics rather than dynamic customization. Depending on errors in estimating the necessary shading correction and other real-time errors, the qualitative shading compensation can generate pulsation or flashing in the televised image. Consequently, an alternative black clamp correction circuit that is out of the feedback loop is also taught in '624, but with concomitant stability issues.
U.S. Pat. No. 4,974,072, No. 5,189,528, and No. 5,038,225 represent black clamping circuits specifically developed to support the special needs of solid state sensors. Each uses black reference data stored during a calibration period to correct and stabilize the black level. In U.S. Pat. No. 4,974,072, as a case in point, the circuit independently corrects the black level for each line sensor within a plurality of sensors; different black level correction is hence applied to each sensor to compensate for differing levels of electrical feed through crosstalk. Nevertheless, application to video sensors is problematic unless a mechanical shutter is available.
U.S. Pat. No. 5,341,218 teaches an improved black clamping circuit specifically intended for video applications. A major improvement is the use of dedicated black reference pixels to dynamically determine the black video level. The circuit further improves black clamping efficacy by digitally averaging the reference data over a plurality of fields or frames. However, analog corrections to the black clamp level are applied only during the vertical blanking period using dedicated sample-and-hold. Horizontal shading non-uniformity generated by the imaging sensor is hence not corrected. Perhaps more importantly, the circuit cannot adapt to different analog gain so that changing analog gain will disrupt black clamp convergence.
U.S. Pat. No. 5,659,355 improves on the '218 patent by adaptively accommodating variable analog gain, in that the programmable gain is switchable to successively handle black pixels followed by active pixels (
In U.S. Pat. No. 6,940,548, black clamping is moved entirely back to the analog domain by performing signal processing prior to A/D conversion. The standard programmable amplifier (with very wide gain range) is split into two PGAs each having narrower gain range. Consequently, the reverse programmable gain amplifier (RPGA) that subsequently calculates the difference between the actual and desired black values also has narrower range of gain reduction. Since the gains of both the direct programmable amplifier and the reverse programmable amplifier are modified using capacitor ratios, the RPGA, which is located in the black clamp loop, has a more practical range compatible with better device matching and higher production yield. Nevertheless, five analog components (PGA1, PGA2, RPGA, Integrator and Difference amplifier) must accurately complement each other with respect to behavioral characteristics across process and operating temperature variations to maximize black stability.
In one embodiment of the present invention, a black clamp circuit has a programmable gain amplifier and a feed-back loop wherein a gain in the feed-back loop is kept constant for all programmable gain amplifier gain settings by adjusting a programmable capacitive network.
More particularly, a black clamp circuit of the present invention may comprise a differential programmable gain amplifier, a programmable capacitive network attached to the programmable gain amplifier, an analog-to-digital converter connected to an output of the programmable gain amplifier, a digital-to-analog converter connected to an input of the programmable gain amplifier, and a controller connected to the analog-to-digital converter and to the digital-to-analog converter, wherein the controller compares a signal from the analog-to-digital converter with a black reference value, and outputs a control signal to the digital-to-analog converter based on a result of comparing the signal from the analog-to-digital converter with the black reference value.
The capacitive network may comprise a plurality of programmable capacitors. For example, the circuit may include a programmable black clamp capacitor Cbc, a programmable signal capacitor Csig and a programmable feedback capacitor Cf.
The digital-to-analog converter may comprise two digital-to-analog converters connected in tandem. One digital-to-analog converter handles coarse adjustments, and the other handles fine adjustments.
In another embodiment of the invention, a method includes receiving a data stream of optical black reference pixel signals, inputting the data stream to a terminal of a differential programmable gain amplifier, comparing an output signal from the programmable gain amplifier with a black reference value, outputting a control signal based on the comparing, and setting a programmable digital-to-analog converter with the control signal in order to adjust an input to the programmable gain amplifier.
The method may further include processing active pixel signals after setting the black level. Additionally, the method may include setting a gain of the programmable gain amplifier for each color of a color filter array corresponding to each active pixel signal. In a further embodiment, the method includes adjusting programmable capacitors to set the gain of the programmable gain amplifier, such that a loop gain is kept constant for all programmable gain amplifier gain settings.
A black clamp circuit according to the present invention may be incorporated into an image sensor having an array of active pixels, and a plurality of optical black pixels. The optical black pixels are processed by the black clamp circuit in order to set an appropriate black level for the active pixels.
The present invention will be readily understood by the following detailed description in conjunction with the accompanying drawings, wherein like reference numerals designate like structural elements, and in which:
The following description is provided to enable any person skilled in the art to make and use the invention and sets forth the best modes contemplated by the inventor for carrying out the invention. Various modifications, however, will remain readily apparent to those skilled in the art. Any and all such modifications, equivalents and alternatives are intended to fall within the spirit and scope of the present invention.
The present invention includes an exemplary design for an active-pixel CMOS imager. A prototype embodiment of the low-noise APS invention can support, for example, a visible imager comprising an active array of 4096 (columns) by 3072 (rows) of visible light detectors (photodetectors). This nominal record area is bounded by a contingent of black reference pixels in the surrounding periphery of the active array.
The rows and columns of pixels can be spaced 4 microns center-to-center using 0.18 μm design rules to provide as-drawn optical fill factor of ˜50%. Several columns and rows of detectors at the perimeter of the light-sensitive region are normally covered with metal and other opaque materials to form optically black pixels. Constantly reading the OB pixels enables the present invention to determine and update the dark level to facilitate low-noise black clamping operation. In addition, the detectors in each row can be covered with color filters to produce color imagers. For example, the odd rows may begin at the left with red, green, then blue filters, and the even rows may begin with blue, red, then green filters, with these patterns repeating to fill the respective rows. A standard Bayer filter pattern can also be applied.
The video signal from the active pixel sensor hence includes:
It is not required to read the various data at a specific time. For example, all black read pixels can be read in the “front porch” of the data or video stream including data from the black and active pixels of each line of the imaging sensor. The active pixels are normally read during a continuous interval that is preceded first by a “front porch” and followed by a “back porch” as defined by various timing standards such as, e.g. SMPTE 274M for high definition video. In both cases, the term “porch” refers to a timing interval during each video line, field or frame, wherein actual imaging data is not being supplied to the camera so that overhead signal processing functions can be supported. The relevant black pixel data can all be read during the front porch, split between the front porch and back porch, or read entirely during the back porch. Furthermore, while the OB data stream is supplied to the digital controller that supervises all operations in the imaging System-on-Chip (iSoC), it can also be supplied to the camera in the data stream along with the data from the active imaging array area.
The table in
In a preferred embodiment, the pixel array shown in
In
Referring again to
In the present invention the circuit blocks of block 600, including PGA 610 and ADC 620 are assembled in a cross-coupled architecture as shown in
An incrementing signal is applied to the positive leg of the dual input PGA 610 and a decrementing signal is applied to the inverting leg. This cross-coupling enables smooth black reference adjustment by segregating positive and negative corrections. In this specific embodiment of the present invention, the single DAC 66 provides at least 14-bit resolution to accurately provide fine adjustment during normal operation, while also supporting coarse adjustment over a much larger overall range to accommodate either long-term operation over wide temperature range or process variation in production. A programmable capacitive network, formed by adjustable capacitors Csig, Cbc and Cf, allows the gain of the amplifier to be changed, as controlled by the register control 64.
The register control block 64 thus may have pre-programmed gain settings for a given image sensor's color filter array, and/or may be dynamically adjusted based on settings from the camera.
The preferred embodiment using tandem DACs 68, 70 better supports the diverse CMOS processes supported by various CMOS foundries. While the disclosed preferred embodiment uses two 8-bit DACs to obtain ⅛ LSB resolution at the output of the PGA, those skilled in the art will appreciate that other combinations of DAC resolutions are useful. By using a coarse DAC in combination with a fine DAC, the scheme enables separating relatively large offsets as constants that are not changed during normal operation. Instead, the fine DAC is most active after initial start-up of the sensor. The overlapping coverage provided by two DACs also better maintains the monotonicity needed for optimum black clamp operation.
The iSoC data stream including reference and active pixel data are supplied to the input of PGA 610 whose gain is set, on a pixel-by-pixel or channel-by-channel basis, by iSoC register control 64. The final outcome is that the resulting output signal from the digital controller is dynamically governed to generate the output signal:
where Vout is the output signal, Vsig is the active pixel signal, Vbc is the optical black pixel signal, Csig is the capacitor setting for the active video, Cbc is the capacitor setting for the black clamp data stream and Cf is the base feedback capacitance. The present invention hence allows separate gain for the active and black pixels in addition to separate gain for each color. Those skilled in the art will appreciate that additional capacitors can be used to increase the total range of gain control including support for cross-coupled Digital-to-Analog Converter (DAC) 66. Those skilled in the art will also appreciate that four PGAs can be used to best handle the GR, GB, R and B channels. Then, for example, the GR, GB, R and B channels are subsequently recombined in the desired order in I/O port 500. The schematic diagram in
Referring once again to
From
For example, if SDACOUT is 60 μV and the PGA gain is increased from 1 to 2 by decreasing Cf by 50 percent, SDACOUT increases to 120 μV. This larger step can decrease the stability of the black reference and also increases the gain of the DAC noise to the output of the PGA.
One way to handle this is to have the DAC minimum step size chosen to work for the maximum PGA gain. Unfortunately at lower gains, this makes the convergence very slow since the step size is exceedingly small. This also makes the noise requirement on the DAC very difficult to meet, as will be described shortly. The digital algorithm could adjust itself for the different gain settings to take larger steps when the gain is lower. However, this is cumbersome and complicates the digital control circuitry needed to control the DAC and still does not address the issue of the high noise requirements on the DAC. Here we take a look at the noise requirements for a DAC in a system with a 12-bit ADC as an example:
In the present invention, this issue is solved in a novel way by having the gain (and therefore step size and range) of the DAC-to-PGA loop constant for all PGA gain settings. The step size is kept constant by keeping the ratio Cbc/Cf constant by automatically adjusting Cbc whenever Cf is changed. In fact, both Cbc and Cf can share the same digital control signals by design. We now examine the previous example again using this scheme.
At 18 dB gain a 14-bit DAC is needed to cover 1.0 V at the PGA output with a step size of 61 μV. At 0 dB gain, both Cf and Cbc are reduced by 8 times. Since the ratio of Cbc/Cf is the same even though the PGA gain has changed, the step size is still 61 μV and the required DAC noise at the PGA output is now 30 μV instead of 3.8 μV. Therefore, a 14-bit DAC may be used for all gain settings. This method also eliminates the need for a difference stage since the subtraction is performed differentially using the PGA itself.
As mentioned above with respect to
For example a 14-bit current steering DAC would normally comprise 214 or 16384 current elements. However, using this scheme, two 8-bit DAC's may be used instead, only requiring 512 current elements. In theory, only two 7-bit DACs are needed, however, in practice some overlap between the minimum step size of the coarse DAC and the full range of the fine DAC is needed to make sure that the output has no missing output values over process, temperature, etc.
Those skilled in the art will appreciate that various adaptations and modifications of the just-described preferred embodiments can be configured without departing from the scope and spirit of the invention. Therefore, it is to be understood that, within the scope of the appended claims, the invention may be practiced other than as specifically described herein.