Patent Application
20060152530
The invention relates to an apparatus and method for digital filtering of a signal, and, in particular to a signal relating to an image.
Generally, the dominant noise sources that affect a CMOS image sensor are random noise (Gaussian), fixed-pattern noise FPN (1-D and 2-D Gaussian, dominated by 1-D column-FPN), and defectivity (low probability, pixels which are stuck-at-white).
However, as feature sizes shrink, different signal degradation mechanisms have an increased significance for CMOS sensors, such as random noise that is not purely Gaussian, and FPN which is 2-D Gaussian with a white quasi-Poisson distribution tail. Furthermore, electrically leaky pixels are more of a problem than pixels that are stuck-at-white. The white distribution tail increases the incidence of couplets (neighbouring defective pixels) as stress on the sensor increases (examples of stresses include gain, temperature and integration time).
Traditionally, defect correction attacks salient pixel defects while noise reduction attacks Gaussian noise. However, as feature sizes shrink and CMOS sensors become sensitive to the different degradation mechanisms mentioned above, these separate tasks are converging, that is, some defect correction techniques have a utility in correcting for noise, and some noise reduction techniques have utility in correcting for pixel defects. However, neither type of technique deals with couplets.
Defect correction can use heavier filtering if targeted at known defects, as opposed to filtering all pixels on-the-fly. However, this requires a defectivity map to be created and maintained, which involves large costs in terms of silicon area and test time. Larger kernel filters for defect correction also present difficulties in implementation, being big in terms of logic, line RAM; difficult to design because multiple RAM read/write (r/w) access per pixel is required; and having a deleterious effect on image quality.
According to a first aspect, there is provided a method of filtering a signal that may comprise a plurality of pixel values each of which is obtained from at least one pixel of an image sensing array. The method may comprise obtaining a central pixel value and a plurality of neighboring pixel values, and determining the average value of each opposing pair of pixel values. The method may also comprise comparing each average value with the central pixel value, and replacing the central pixel value with the average value that is closest to the central pixel value.
Preferably, neighboring pixel values may be derived from neighboring pixels. Preferably, the outputs obtained from the application of the method to a first central pixel value may be used as the inputs for the application of the method to a second pixel value. This recursive method may provide a greater attenuation of defective pixel couplets.
Preferably, the second pixel value may be a neighboring pixel value of the first pixel value. Preferably, the method may be applied to a set of defective pixel couplets in an image.
Preferably, the method may be selectively applied to pixel values that are above or below a predetermined threshold. Preferably, the predetermined threshold may be varied to select a level of filtering severity.
According to a second aspect, there is provided an apparatus for filtering a signal that may comprise a plurality of pixel values each of which is obtained from at least one pixel of an image sensing array. The apparatus may comprise a reading unit or read means for obtaining a central pixel value and a plurality of neighboring pixel values. The apparatus may also comprise a calculator or calculation means for determining the average value of each opposing pair of pixel values, comparing each average value with the central pixel value, and replacing the central pixel value with the average value that is closest to the central pixel value.
Preferably, neighboring pixel values may be derived from neighboring pixels. Preferably, the apparatus may further comprise a memory or memory means to store the outputs obtained from a first central pixel value, and the outputs forming the inputs for the calculator or calculation means when the apparatus operates on a second pixel value. This recursive method may provide a greater attenuation of defective pixel couplets.
Preferably, the second pixel value may be a neighboring pixel value of the first pixel value. Preferably, the apparatus may be arranged to operate on a set of defective pixel couplets in an image.
Preferably, the apparatus may further comprises a selector or means to select pixel values which are above or below a predetermined threshold for operation on by the apparatus. Preferably, the predetermined threshold may be varied to select a level of filtering severity.
According to a third aspect, there is provided a CMOS image sensor that may comprise the apparatus of the second aspect. According to further aspects, there is provided a digital camera, an optical mouse, or a mobile phone comprising a digital camera, which may comprise the CMOS image sensor of the third aspect.
The invention will now be described, by way of example only, with reference to the accompanying drawings, in which:
a and 2b illustrate non-recursive and recursive operations of the filter shown in
The filter of
a shows a non-recursive filter operation. An input line (in) has a series of pixel values, shown as 0,1,1,0. The pixel value 14 is chosen as the central value such that after a first step (s1), the central pixel value 14 is replaced by the average of its two neighbors 12 and 16. That is, the value “1” is replaced by the average of “0” and “1”, thus becoming “½”.
The filter then moves to the right, such that pixel value 16 becomes the central pixel value. After a second step (s2), the central pixel value 16 of the original input (in) is replaced by the average of its two neighbors 14 and 18. That is, the value “1” is replaced by the average of “1” and “0”, thus becoming “½”.
b shows a recursive filter operation, in which the outputs from a first step form the inputs for the next step. Starting from the same input (in), the pixel value 14 is chosen as the central value, such that after a first step (s1), the central pixel value 14 is replaced by the average of its two neighbors 12, 16. That is, the value “1” is replaced by the average of “0” and “1”, thus becoming “½”.
The filter then moves to the right, such that pixel value 16 becomes the central pixel value. After a second step (s2), the central pixel value 16 is replaced by the average of its two neighbors 14, 18 as defined after the first step (s1). That is, the value “1” is replaced by the average of “½” and “0”, thus becoming “¼”.
It can be seen that a non-recursive operation on a pixel couplet 14, 16 attenuates the couplet, reducing its value by half in the example shown. The recursive operation results in a further attenuation, but has a more significant effect on the image than that resulting from the non-recursive operation.
The operation of the filter can be controlled by a feedback loop that monitors the effects of the filter and adjusts its application accordingly. When used in a digital camera, a camera control system can be provided which evaluates the effect the filter has on the image and adjusts the filter operation accordingly.
The filter could simply be switched on or off, or alternatively could be set to apply filtering to values that fall a predetermined amount above or below specific signal thresholds. The predetermined amount could be varied to provide a “fader” allowing an operating system to quantifiably vary the strength of filtering applied based upon the preferred balance between defect correction and image destruction.
The filter provides a number of advantages over prior art filters. Firstly, the filter can be specifically targeted at pixel couplets by being effective to significantly reduce their amplitude, for example. Noise is structured into linear artifacts for further processing by other noise reduction techniques. Furthermore, the logic cost is small, and there is no history requirement.
It will be appreciated that the digital filter could be used in image sensors for a wide range of applications, including digital cameras, mobile telephones with digital cameras, or optical mice to name but a few examples. The practical issues regarding implementation of the digital filter to any of these devices will be readily apparent to those skilled in the art, and so a detailed description thereof is not necessary herein. Improvements and modifications can be made to the above without departing from the scope of the invention.
| Number | Date | Country | Kind |
|---|---|---|---|
| 05250101.2 | Jan 2005 | EP | regional |
