Patent Application
20070274605
The present invention will be better understood from the following detailed description of some preferred embodiments of the invention, taken in conjunction with the accompanying drawings, in which like numbers correspond to like parts, and in which:
a-4c are variations of the image of
a-5c are variations of the image of
a-6c are variations of the image of
As illustrated in
According to the present invention, denoising can be achieved by use of a simple filter, which satisfies all the above criteria and has the Fourier representation
{tilde over (g)}(k)=exp(−a4k4), (17)
where a is a scale length, which determines the width of the filter. The main advantage of filter 8 is its null covariance matrix, which results from the fact that the second derivatives of {tilde over (g)} all vanish at k=0.
All of the odd and anisotropic moments of the filter are also zero, because the filter is only a function of the magnitude k of the vector wave number but not its direction. The first nonzero moment is therefore the isotropic part of the fourth moment μ(4), and it is the first term to contribute to the denoising bias.
The CPF does have a small Gibbs oscillation in image space as a consequence of the null covariance matrix. (The filter must be both positive and negative, so that the second moments can vanish.) The negative dip of the filter, however, is very modest. The ratio of (negative) minimum to (positive) maximum of the filter is −0.06.
An added benefit of the CPF of Equation 17 is that attenuation at large k with the negative exponent proportional to k4. This strong attenuation overcomes the exponential growth of typical inverse point-response functions, whose positive exponents are typically only proportional to k2. For example, for a Gaussian point-response function whose inverse grows exponentially (Equation 16), the result is that the combined deblurring and denoising filter is well behaved.
CPFs can be implemented using either software or hardware, or a combination thereof. Software processing is most easily performed by application of the Fourier convolution theorem. The Fourier-transformed input signal Ĩ(k) is first obtained using a fast-Fourier-transform (FFT) algorithm. The Fourier-transformed output signal {tilde over (J)}(k) is then obtained by multiplying the Fourier-transformed input signal by the filter of Equation 10. For pure denoising, the filter to use is {tilde over (g)}(k) (Equation 17). To perform both denoising and deblurring, the filter {tilde over (g)}(k)/{tilde over (p)}(k) of Equation 19 is used instead. Finally, the output signal is obtained by an inverse fast Fourier transform of {tilde over (J)}(k).
Fast Fourier transforms perform convolutions very efficiently when used on standard desktop computers, i.e., one or more CPUs, memories and interfaces such as a PC or Apple® Macintosh®, however, they require the full data frame to be collected before the computation can begin. This is a significant disadvantage when processing raster video in pipeline fashion as it comes in, because the time to collect an entire data frame often exceeds the computation time. Pipeline convolution of raster data streams is more efficiently performed by massively parallel direct summation techniques, even when the kernel covers as much as a few percent of the area of the frame. In hardware terms, a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC) can be much more efficient than a digital signal processor (DSP) or a microprocessor unit (MPU), since the latter processors require the full data frame to be collected. Commercially available FPGAs or ASICs can be fabricated to perform small-kernel convolutions faster than the rate at which raster video can directly feed them, e.g., at a rate of up to ˜150 megapixels per second using current technologies.
A curvature-preserving small kernel can be designed in a variety of ways, the only requirement being that it have a null covariance matrix. One way is to obtain the inverse Fourier transforms of the filters in Equations 17 or 19. These kernels do not extend significantly beyond the scale length a (defined in Equation 17). They can therefore safely be truncated at a few scale lengths and can then be used in hardware-based small-kernel convolution.
An exemplary application of the present invention to a video input is illustrated in
The chrominance signal 82 is passed forward through the hardware without processing, but appropriately delayed at 84 to provide signal 82′ which is synchronized with the processed luminance signal 92. The luminance 92 and chrominance 82′ signals are merged and switched at 86 for input to two separate output channels. In the first channel 88, the luminance and chrominance signals are converted into an RGB signal, which is passed to a triple digital-to-analog converter (DAC) 94 and output to a high resolution monitor 96, such as an SVGA monitor. The second channel is a NTSC or PAL standard output channel 98. For the latter, the signals are reinterlaced 93 prior to encoding for NTSC or PAL video output 95.
In an alternate embodiment, the de-interlacing process 76 is omitted and the interlaced signals are processed separately. In this embodiment, some loss of vertical resolution may occur in exchange for elimination of delay introduced by the input deinterlacer. In this case, the re-interlacing step 92 for standard video output can also be eliminated.
In yet another embodiment, the input video signal is progressive and the deinterlacing process 76 may not be applied and is therefore omitted. In this case, the lack of de-interlacing does not cause loss of vertical resolution.
The following examples describe both denoising and deblurring of a photographic image of Piccadilly Circus in London.
In the first simulation, only noise is added to the truth image, and there is no blurring. The range of the RGB intensities of the original image is [0,255], and the noise level added is 255/20=12.75. The maximum signal-to-noise ratio is therefore 20. The noisy data image is shown in
The result of applying a CPF with a scale length a=2 pixels is shown in
The quality of the two reconstructions can be compared by inspecting the difference images between the reconstructions and the truth image, where the residual signal indicates how much of the original image was given up to bias during the reconstruction. In a comparison of the two differences (truth−CPF) and (truth−Gaussian filter) (not shown), the CPF residual is much less than that of the Gaussian filter, which means the CPF has smaller bias than the Gaussian filter.
In the second simulation, the truth image of
The results of the reconstructions are shown in
In the third simulation, the truth image in
The results of the reconstructions are shown in
One can trade resolution versus noise by changing the values of σ and a in the filter, Equation 19, to suit the preference of the user. Decreasing σ or increasing a result in less noise amplification at the price of incomplete deblurring. The choice made in the reconstruction shown in
The inventive filter can be generalized to suit specific needs by using functions of k4 other than the exponential function of Equation 17, or functions of higher, even powers of k. Higher powers of k, however, will tend to increase the amplitude of the Gibbs phenomenon, because such filters have more abrupt transitions from the low-k to the high-k regimes.
The foregoing description of the CPFs was made using 2D images as an illustration. It will be readily apparent to those of skill in the art that analogous arguments may be made for images of any dimensionalities. They can be 3D images, 1D profiles, 1D spectra, or “images” of dimensionality higher than 3, in which the additional dimensions are time, temperature, wavelength, or any number of other variables.
The inventive CPFs can be used by both iterative and noniterative image reconstructions. Their advantage in noniterative methods is their ability to limit the artifacts commonly generated by noniterative techniques. But they can also be employed in iterative techniques in lieu of the filters now in use. CPFs therefore form a complementary new technology. They do not replace existing technologies but simply provide improved filters. The CPFs, however, are expected to tip the tradeoff between image quality and processing speed in favor of the faster noniterative methods by making them more robust and artifact free.
In the preceding detailed description, the invention has been described with reference to specific exemplary embodiments thereof. It will, however, be evident that various modifications and changes may be made thereunto without departing from the broader spirit and scope of the invention as set forth in the appended claims and their full scope of equivalents.
This application claims the priority of U.S. Provisional Application No. 60/808,439, filed May 24, 2006, which is incorporated herein by reference in its entirety.
| Number | Date | Country | |
|---|---|---|---|
| 60808439 | May 2006 | US |
