The invention relates to a method and a system for enhancing the sharpness of a video signal.
The invention may be used, for example, in the field of video processing.
Improving image quality has become an important issue in video products such as television sets. Image quality is in particular perceptually improved if the sharpness of the input images, which define said video, is increased. There are known solutions for increasing the sharpness of a video signal.
A first solution consists in enhancing the sharpness of the input images by means of a filter applied to all successive images. Since input images may also contain a noise component, for example, analog white noise or digital artefacts resulting from previous block-encoding and decoding operations, the noise component is also enhanced, to the detriment of the overall image quality of the resulting output images.
A second solution consists not only in enhancing the sharpness of the input images by means of a filter applied to all successive images, but also in adapting the level of enhancement in accordance with the level of the noise component detected in said input images. If the noise component has a low level, the sharpness of the resulting output images is highly increased. In contrast, if the noise component has a high level, the filter is switched off (or tuned down) for avoiding enhancement of the noise component, but in that case the sharpness of the resulting output images is unfortunately close to that of the input images. This second method is thus limitative in that it is not efficient when dealing with too noisy input images.
It is an object of the invention to propose a method of enhancing the sharpness of a video signal, which is robust to noise.
To this end, the method according to the invention proposes modification of input images for generating output images. Said method comprises the steps of:
This method is based on the assumption that the input images are defined as the sum of a signal component and a noise component, the signal component having a low frequency spectrum and the noise component having a high frequency spectrum. The part of the frequency spectrum to be enhanced in the input images is thus situated at low frequencies, while the part of the frequency spectrum not to be enhanced in the input images is situated at high frequencies.
The step of splitting allows generation of two types of images: coarse images having a low frequency spectrum and fine images having a high frequency spectrum. This splitting allows selective enhancement of the coarse images which contain mainly a signal component. Performing a sharpness enhancement on images having a low frequency spectrum leads to better results than on images having a high frequency spectrum.
This method is robust to noise because, irrespective of the noise level in the input images, the noise component is not enhanced in the resulting output images. After combining, the resulting output images still have a noise component close to that of the input images, but the overall image quality is improved because the sharpness of the signal component has been enhanced.
Said step of splitting preferably comprises the steps of:
Use of the subtracting step allows a perfect split of the input images without losing any data information. Moreover, this leads to a cost-effective solution.
The step of low-pass filtering is preferably adaptive to a first noise signal derived from said input images.
The limit between the low-frequency spectrum corresponding to the signal component and the high-frequency spectrum corresponding to the noise component is defined adaptively and accurately. This allows applying the sharpness enhancement on the overall spectrum of the signal component independently of the noise level in the input images, resulting in output images with an increased sharpness.
The step of enhancing is preferably adaptive to a second noise signal derived from said input images.
This allows adaptation of the sharpness enhancement to the noise level of the input images, for example, in using a more aggressive sharpness enhancement when the noise level is high, and in using a lighter sharpness enhancement when the noise level is low.
The step of low-pass filtering and the step of enhancing are preferably adaptive to noise signals derived from said input images.
This allows applying the sharpness enhancement on the overall frequency spectrum of the signal component, independently of the noise level in the input images, and adapting the sharpness enhancement to the noise level of the input images, resulting in output images with an increased sharpness.
It is also an object of the invention to propose a system for enhancing the sharpness of a video signal, which system is robust to noise and comprises processing means for implementing the various steps of the method according to the invention.
Detailed explanations and other aspects of the invention will be given below.
The particular aspects of the invention will now be explained with reference to the embodiments described hereinafter and considered in connection with the accompanying drawings, in which identical parts or sub-steps are designated in the same manner.
This method comprises a step 103 of splitting said input images 101 into coarse images 104 and fine images 105. The frequency spectrum of the coarse images 104 corresponds to the low frequency spectrum of the input images 101, while the frequency spectrum of the fine images 105 corresponds to the high frequency spectrum of the input images 101.
This method also comprises a step 106 of enhancing the sharpness of said coarse images 104 for generating intermediate sharpness-enhanced images 107.
This method also comprises a step 108 of combining said intermediate sharpness-enhanced images 107 and said fine images 105 for generating said output images 102. This step 108 allows reconstruction of the output images.
In a first example, the step F of low-pass filtering implements a Gaussian filter intended to be applied on the pixels composing said coarse images 204. This linear filter may be defined by the following kernels k1 or k2:
Alternatively, the step F of low-pass filtering may implement:
The step 103 of splitting also comprises a step SUB of subtracting said coarse images 204 from said input images 201 for generating said fine images 205. The subtraction is done between a pixel of an input image 201 and a pixel of a coarse image 204, which pixels have the same coordinates in the images, while the subtraction is repeated for all pixels of both images.
The step 206 of enhancing the sharpness of said coarse images 204 may consist in a non-linear processing (described in the following section) performed on pixels of the lines and/or the columns of the coarse images 204. The sharpness may be seen as a level transition between two data areas, which are flat or slowly varying. The level transitions to be enhanced are detected, for example, by applying a gradient filter to the coarse images 204, and by detecting areas that have the highest levels in these resultant filtered images.
The step of combining comprises a step 208 of adding said intermediate sharpness-enhanced images 207 to said fine images 205, for generating said output images 202.
A first processing operation consists in convoluting the transition signal T1 with the kernel of a derivative filter DF, for generating a first intermediate signal y1 having overshoots either at the beginning or at the end of the transition, as follows:
y1=T1{circle around (x)}DF (1)
wherein {circle around (x)} represents the convolution operation,
A second processing operation consists in adding the transition signal T1 to a fraction/multiple of said first intermediate signal y1, for generating a second intermediate signal y2, as follows:
y2=T1+α*y1 (2)
wherein α is a coefficient, for example, equal to 1.
A third processing operation consists in suppressing the remaining overshoots in the second intermediate signal y2, for generating the enhanced transition signal T2, as follows:
T2=med(S1,S2,y2) (3)
wherein med(S1, S2, y2) represents the median operation among data samples of signals
This arrangement differs from
The higher the noise level σ in input images 201, the lower the cut-off frequency of the low-pass filter must be. It may be decided that the first kernel k1 defined previously is used for a low noise level σ1, while the second kernel k2 defined previously is used for a higher noise level σ2.
Alternatively, the filter coefficients may be adaptive to the noise level σ detected in input images 201, in defining each coefficient of the kernel k by a function (f1, f2, . . . ) depending on the noise level σ, as follows:
The functions (f1, f2, . . . f9) are derived from, for example, a basic experiment.
This arrangement also differs from
y2=T1+α(σ)*y1 (5)
The higher the noise level σ in input images 201, the larger α must be. For example, a linear relation between α and σ may be established.
It is noted that the step F of filtering and the step 206 of enhancing are not necessarily adaptive to the noise level σ simultaneously, and that the adaptation to the noise level σ may concern only one of these two steps.
The noise level σ in input images 201 is measured by a step DET. This step generates a first and a second signal S1 and S2 (which may be the same) proportional to said noise level σ. For example, the noise level σ may be derived from any algorithm known to a skilled person. For example, such signals may reflect an analog noise measure (e.g. in using a frequency spectrum-based algorithm), and/or a digital noise measure (e.g. in using a blocking-effect detector measuring the activity at the periphery of blocks of 8*8 pixels or 16*16 pixels).
The method according to the invention may be implemented in a system for modifying input images 101 so as to generate output images 102. This system comprises processing means for implementing the various steps of the method according to the invention previously described. In particular, this system comprises:
This system may be embodied as an electronic card and implemented in a video apparatus (e.g. a television set, video broadcast equipment, etc.), said video apparatus being intended to receive said input images 101, and to display or broadcast said output images 102 on a display or on a communication channel, respectively.
The invention also relates to a computer program comprising code instructions for implementing the various steps of the method according to the invention.
It is to be noted that the use of the verb “comprise” and its conjugations does not exclude the presence of elements or steps other than those stated in the claims.
Number | Date | Country | Kind |
---|---|---|---|
04300301.1 | May 2004 | EP | regional |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/IB05/51590 | 5/17/2005 | WO | 00 | 11/16/2006 |