Patent Application
20140140636
This invention relates to a technique for restoring a video image, and more particularly, for denoising the image.
Image restoration generally constitutes the process of estimating an original image (which is unknown) from a noisy or otherwise flawed image. Ideally, the estimated image should be substantially free of noise so that image restoration constitutes a form of de-noising. During the image restoration, various tools can prove useful, such as gradient image analysis. Although the differences between adjacent pixels in natural images often appears small, the /1 and /2 norm of color values in the image gradients usually increase when a natural image becomes distorted so gradient image analysis can provide a measure of image distortion.
Image gradients also play a part in image restoration, and particularly, image de-noising. Total Variation (TV), which makes use of image gradient, serves as a popular tool for image denoising because of its capability of performing denoising while preserving the image edges. In addition, TV denoising generates high resolution images from lower resolution versions very well while serving to recover images with highly incomplete information.
Typically, calculation of the Total variation depends on the horizontal and vertical gradient images. An image can be defined by its horizontal and vertical gradient images, ∇xI and ∇yI, respectively, as follows
∇xI(x,y)=I(x+1,y)−I(x,y)
∇yI(x,y)=I(x,y+1)−I(x,y). (1)
Then Total Variation (TV) is calculated by
TV(I)=Σi,j√{square root over (∇xI(i,j)2+∇yI(i,j)2)}{square root over (∇xI(i,j)2+∇yI(i,j)2)} (2)
or TVII)=Σi,j|∇xI(i,j)|+|∇yIIi,j)|. (3)
Classical TV denoising seeks to minimize the Rudin-Osher-Fatemi (ROF) denoising
where n is the noisy image, TV(ƒ) represents the total variation of ƒ, and λ is a parameter which controls the denoising intensity.
Traditional TV regularization, as provided in Equation. (2) does not consider the content of images. Rather, tradition TV denoising serves to smooth the image with equivalent intensity from both horizontal and vertical directions. Therefore, the edges undergo smoothing more or less after TV denoising, especially the oblique edges.
An improved version of TV, referred to as called Directional Total Variation, makes use of the /2 norm of a pair of gradient images along the edge direction and its orthogonal direction. Directional TV regularization outperforms traditional TV regularization in both subjective and objective quality, and does particularly well in preserving oblique texture and edges. In contrast, the existing TV regularization technique actually presumes the smoothness along all directions. In other words, the existing TV regularization technique tries to smooth the image along all directions by minimizing the norm of gradients along two orthogonal directions. As a result, the existing TV regularization technique inevitably blurs or even removes the edges and textures. Although a proposal exists to focus on smoothing along the edge by applying different larger weights, minimizing the norm of gradients along the other direction incurs difficulties.
Thus a need exists for a denoising technique that overcomes the aforementioned disadvantages.
Briefly, in accordance with a preferred embodiment of the present principles, a method for de-noising an image using Anisotropic Gradient Regulation commences by first choosing edge directions for the image. Thereafter, an anisotropic gradient norm is established for the image from anisotropic gradient norms along the selected edge directions. The image pixels undergo adjustment to minimize the anisotropic gradient norm for the image, thereby removing image noise.
The processor 12 enjoys a connection to a database 22 which can reside on a hard drive or other non-volatile storage device internal to, or separate from the processor. The database 22 can store raw image information as well as processed image information, in addition to storing software and/or data for processor use.
The system 10 further includes an image acquisition device 24 for supplying the processor 12 with data associated with one or more incoming images. The image acquisition device 24 can take many different forms, depending on the incoming images. For instance, if the incoming images are “live”, the image acquisition device 24 could comprise a television camera. In the event the images were previously recorded, the image acquisition device 24 could comprise a storage device for storing such images. Under circumstances where the images might originate from an another location, the image acquisition device 24 could comprise a network adapter for coupling the processor 12 to a network (not shown) for receiving such images. Although
Execution of the Anisotropic Gradient Regulation denoising technique of the present principles commences by first defining candidate directions for generate image gradients. As depicted in
Next, calculation the /2 norm of gradient along each direction occurs in accordance with the relationship Ek=Σi,j|∇kI(i,j)|2, where (k ε {a, b, c, d, e, f, g, h}). Ek can serve as the mechanism for the direction determination.
The chosen edge directions are {k|Ek<th1}, where th is a predefined threshold.
Direction determination occurs in accordance with the following steps:
Next, calculation of the /2 norm of the gradients occurs along the detected directions for each image region. The Anisotropic Gradient Norm (AGN) of a image region ƒl defined as follows:
AGN(ƒl)=Σi,j√{square root over (α∇pƒl(i,j)2+β∇qƒl(i,j)2+γ∇rƒl(i,j)2)}{square root over (α∇pƒl(i,j)2+β∇qƒl(i,j)2+γ∇rƒl(i,j)2)}{square root over (α∇pƒl(i,j)2+β∇qƒl(i,j)2+γ∇rƒl(i,j)2)} (6)
where p, q and r are the detected edge directions; α, β and γ are the weights for the gradients. Generally, smoothing of the image region (e.g., adjusting the pixels within the image region) along the smaller-norm-directions with higher intensity remains preferable.
However, it is unnecessary to use three directions for all image regiones. If there are only 2 edge directions detected in a image region, the other weight can be set to 0. For the entire image, the Anisotropic Gradient Norm is calculated from the sum of AGNs of all the image regiones as follows:
AGN(ƒ)=ΣlAGN(ƒl) (8)
Note that some gradients of the boundary pixels of a image region require the pixels within other image regiones, so the calculation of AGN of an image may occur across image regiones.
The Anisotropic Gradient Regularization technique discussed above tends to enhance the edges and texture. The technique makes real edges sharper but can also generate false edges. This problem can be addressed by making use of intensity adaptation in the regularization loop. Anisotropic Gradient Regularization for image denoising can be formulated as:
Other methods use a constant multiplier to update λ. For example, consider the relationship:
λn=ηλn−(0<η<1) (11)
where λ turns smaller after each iteration since the noise becomes less.
However, better results occur by calculating λ according to the content of each region of images.
Implementation of Regularization Intensity Adaptation occurs in the following manner. Given λ0 as an initial value, λn is updated after each iteration. At the nth iteration, the ratio of the maximum norm of the gradients to the minimum is calculated.
Given a threshold th, ρ can approximately indicate whether the region is smooth or complicated.
if ρ>th, the region is relatively smooth. Then λn=η1λn−1;
If ρ≦th, the region is relatively complicated. λn=η2λn−.
where 1>η2>η1>0. We set η1=0.85, η2=0.95 in practice.
Advantageously, Anisotropic Gradient Regularization with adaptive intensity does not generate obvious false textures.
For the texture/edge directions of the image regiones within a noisy image, Anisotropic Gradient Regularization denoising occurs performed by minimizing the Anisotropic Gradient Norm (AGN) of the image as follows.
where n is the input noisy image. The edge directions are determined as discussed above. Anisotropic Gradient Regularization denoising significantly outperforms the traditional TV denoising.
Keeping the image edges sharp at the high resolution remains a critical problem in interpolation/super resolution Intuitive bi-linear/bi-cubic interpolation usually introduces blur during interpolation. Total Variation (TV) regularization-based interpolation provides a better solution since TV regularization utilizes the intensity continuity of natural images as prior information during the up-sampling process using the following relationship.
where Φ is a down-sampling matrix, γ is the low resolution image and ƒ is the up-sampled version.
Since Total Variation (TV) regularization does not detect and protect the texture and edges in the image, TV regularization cannot generate high resolution images with sharp (oblique) edges. However, as discussed above, the de-noising technique of the present principles depends on the minimization of the AGN in accordance with the following relationship:
The restoration technique of the present principles detects all the probable edges and generates anisotropic gradients; then the interpolation occurs by minimizing the norm the anisotropic gradients and the difference between the down-sampled version and the input image. In this way, the up-sampled images contain shaper edges and less blur.
The foregoing describes a technique for de-noising an image.
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/CN2011/079093 | 8/30/2011 | WO | 00 | 1/9/2014 |
