The present invention relates to video encoding and decoding methods, more particularly, to a method for image visual effect improvement of video encoding and decoding.
Based on human vision system, color can be described by brightness, hue and saturation. Usually, hue and saturation are generally referred to as chroma, which is used to represent the category and depth of color. In the video encoding process, for different frames, regions people cared about are dynamically changed, which requires that the algorithm is able to adjust transformation function according to the change of the video sequences, so that brightness distribution of the image can be improved according to demand in various scenes. The visual quality of the image can be improved by a constant transformation function of brightness with the parameters obtained by considerable statistical experiments. However, if the same approach used in the ordinary scenes is carried out in some specific scenes (such as a wholly dark scene), visual quality of the image will be decreased.
For color information of an object, people always hope that, the more colorful the better. Considering the requirement of visual comfort, the bigger the transform intensity is, the more color of the image with insufficient chroma information is improved. Skin color of human beings is between yellow and red. If the same model is used for the whole region, taking relatively large adjusting values, uncomfortable feeling to skin color will be generated, and taking relatively small adjusting values, the requirement of enhancing color information of objects in other color gamut will be restricted. If the algorithm is dependent on the detection of skin color regions, firstly, computational complexity is increased, and secondly there isn't a detection algorithm for skin color regions with 100% accuracy, thirdly many problems such as balance transition brought in by incorrect judgment of discrete point field will occur. Although people are more sensitive to luminance than to chrominance, preprocessing should be employed to enhance the color of the image, since chroma information carried by the image sequence (such as image captured by a camera) processed by the video encoder is insufficient at some time. Most conventional color processing methods are based on RGB or HSV color model, while a separate representation mode of luminance and chrominance, i.e., YUV, is used in video encoding. Although transformation between different models can be realized through color space transformation technology, computational complexity bought in by transformation and invert transformation is also considerable.
Image quality will be decreased in varying degrees after encoding. Problems, such as blocking artifacts brought in by block-based encoding and decoding strategy, attenuation and losing of high frequency information and so on, are present in the image sequence after decoding. In order to eliminate blocking artifacts without losing of boundary high frequency information, and take characteristics of block-based encoding and decoding strategy into account that the blocking artifacts always present at the boundary between blocks, a method for block-based boundary adaptive enhancement is employed.
In order to improve visual effect of video sequences at an encoder, the present invention provides a method for image visual effect improvement of video encoding and decoding, wherein a boundary information enhancement technology is used to increase the amount of high frequency information contained in the image, and adaptive enhancement technologies for luminance and chrominance respectively are provided for improving the brightness information distribution of the image and enhancing chroma information of the image.
The method according to present invention comprises the following steps at the encoder:
S11: extracting image boundary information and enhancing a boundary information operation, the step further comprising:
wherein f (x,y) is a brightness value of the original image at the encoder, γ(x) is a boundary information extracting function, φ(f(x,y),h(x,y)) is a transformation function selected according to characteristics of the original image and the boundary information;
S12: Adaptive luminance transforming to improve luminance distribution:
g′(x,y)=ψ(f(x,y),α(k)|k=1,2, . . . ,K),
wherein g′(x,y) is a transformed brightness value, ψ(x,α(k)|k=1, 2, . . . , K) is a transformation function, wherein α(k) is a set of parameters of the transformation function ψ(x,α(k)|k=1, 2, . . . , K), and K is the number of the parameters;
S13: Adaptively enhancing the chrominance information, which is performed in the UV color space,
(u′(x,y),v′(x,y))=w*φ(u(x,y),v(x,y),αu,αv,βu,βv)
wherein φ(u(x,y),v (x,y),αu,αv,βu,βv) is a transformation function, w is a weight function, and a UV chroma deviation position is determined by αu and αv, a chroma adjusting step is determined by βu and βv.
Image quality will be decreased in varying degrees after encoding. Problems, such as blocking artifacts brought in by block-based encoding and decoding strategy, attenuation and losing of high frequency information and so on, are present in the image sequence after decoding. In consideration of a need for improving visual effect of the image at the decoder, a method for image visual effect improvement of video encoding and decoding is provided in the present invention.
The method according to present invention comprises the following steps at the decoder:
S21: selecting a processing mode
according to block statistical characteristic,
wherein f(x,y) is a original image value at the decoder, t—0 is a statistical variable name of a statistical region ,
(j=0, 1, 2) is a statistical characteristic function,
j is a statistical region corresponding to
, and Thres—1 is a threshold for determining whether the current processing region is a flat region or a complex region;
S22: Adaptively transforming the brightness and improving brightness distribution of the image:
g(x,y)=ψ(f(x,y),α(k)|k=1,2, . . . ,K),
wherein f(x,y) is a brightness value of the original image at the decoder, g(x,y) is a transformed brightness value, ψ(x,α(k)|k=1, 2, . . . , K), is a transformation function, wherein α(k) is a set of parameters of the transformation function ψ(x,α(k)|k=1, 2, . . . , K), and K is the number of the parameters;
S23: Adaptively enhancing the chroma information, wherein the chroma information adaptive enhancement is performed in a UV chroma space,
(u′(x,y),v′(x,y))=w*φ(u(x,y),v(x,y),αu,αv,βu,βv)
wherein φ(u(x,y),v (x,y),αu,αv,βu,βv) is a transformation function, w is a weight function, and a UV chroma deviation position is determined by αu and αv, a chroma adjusting step is determined by βu and βv.
Through the above mentioned method, adaptive adjustment can be applied to eliminate the blocking artifacts, and enhance the image luminance and chrominance information, in such a way the object of improving the objective effect and subjective effect of the coded and decoded images can be achieved. When the adaptive boundary information enhancement technology according to present invention is employed at the decoder, the effect of separation method in enhancing boundary information and eliminating blocking artifacts can be maintained while the processing speed is improved, and the objective effect and subjective effect of the image also can be improved remarkably.
The following drawings illustrate preferred, but not exclusive embodiments of the inventions:
Referring
1. The present step implements boundary information enhancement process, and further comprises the following steps:
wherein: f(x,y) is a brightness value of the original image at the encoder, φ(f(x,y),h(x,y)) is a transformation function selected according to characteristics of the original image and its boundary information, γ(x) is a boundary information extracting function, wherein different methods for extraction can be employed according to different applications requirements. With respect to derivative method, for example, a first order derivative, a second order derivative and so on can be employed, such as gradient module extracting method:
2. In video encoding process, for different frames, regions people cared about are dynamically changed, which requires that the algorithm is able to adjust transformation function according to the change of the video sequences, so that brightness distribution of the image can be improved according to demand in various scenes.
The visual quality of the image can be improved by a constant transformation function of brightness with the parameters obtained by considerable statistical experiments. However, if the same approach used in the ordinary scenes is carried out in some specific scenes (such as a wholly dark scene), visual quality of the image will be decreased.
The present step implements adaptive brightness transformation and improvement of image brightness distribution. The principle of adaptive brightness transformation is that, the set of parameters of the transformation function is adaptively updated according to a statistical characteristic of brightness value of the image before being transformed, so that the transformation function is adjusted dynamically along with different image characteristics, and thus the processing method is optimized:
g(x,y)=ψ(f(x,y),α(k)|k=1, 2, . . . , K),
wherein f(x,y) is a brightness value of the original image at the decoder, g(x,y) is a transformed brightness value, ψ(x,α(k)|k=1, 2, . . . , K) is the transformation function, wherein α(k) is the set of parameters of the transformation function ψ(x,α(k)|k=1, 2, . . . , K), and K is the number of the parameters.
As shown in
a2: Given a characteristic space of the current frame image is
ξi∩ξj=φ,i≠j, and a whole statistical characteristic of the image is obtained by statistic of the brightness information,
for (k=0; k<M; k++) if (f(x,y)εξk) Calculating a statistical characteristic φk(f(x,y)) of ξk.
Finally, the statistical characteristic of the current frame image is obtained:
{φk(f(x,y))|k=1, 2, . . . , M};
wherein ξk and φk(f(x,y)) are image characteristic subspace, the statistical characteristic of ξk respectively;
b2: The threshold is adjusted according to visual characteristic together with the regional statistical characteristic, and the image is divided into different regions
The statistical characteristic threshold PH is adjusted to be PH′ according to a statistical relationship between the global area and the regions,
P′H=ratio*η(PH, Φ1, Φ2, . . . , ΦN)
wherein Φk is a statistical characteristic of Ωk,
Φk={φ1(Ωk), φ2(Ωk), . . . , φM(Ωk)}
PH′, P′H are the threshold obtained through the whole statistical information and the adjusted threshold respectively;
c2: Parameter values of the transformation function is obtained based on the statistical characteristic;
α(k)=(P′H)k=1, 2, . . . K
(x) is a adjusting function of the parameter α(k) of the transformation function ψ(x,α(k)|k=1, 2, . . . , K);
d2: By using brightness transformation function ψ(f(x,y),α(k)|k=1, 2, . . . , K), brightness transformation is implemented and distribution of the image brightness information is improved;
wherein f(x,y) is the brightness value of the original image at the decoder, g(x,y) is the adjusted brightness value, ψ(x,α(k)|k=1, 2, . . . , K) is the transformation function, wherein α(k) is the set of parameters of the transformation function ψ(x,α(k)|k=1, 2, . . . , K), K is the number of the parameters.
3. Chroma information is adaptively enhanced, wherein the chroma information adaptive enhancement is performed in a UV chroma space,
(u′(x,y),v′(x,y))=w*φ(u(x,y),v(x,y),αu,αv,βu,βv)
wherein φ(u(x,y),v (x,y),αu,αv,βu,βv) is a transformation function, w is a weight function, and a UV chroma deviation position is determined by αu and αv, a chroma adjusting step is determined by βu and βv.
As shown in
a3: Saturation information of the UV space κ is obtained through a statistic of the UV characteristic of the current image frame;
b3: Adjusting parameters are calculated with the color saturation information;
αu=γu(κ)βu=γu(κ)
αv=γv(κ)βv=γv(κ)
c3: By statistical experiments in the UV space model, empirical value range of skin color distribution is obtained, and the weight function w=η(θ) is determined, wherein θ is the empirical value range of skin color, θε[θ1,θ2].
w=η(θ), η(θ) is a continuous function having only one minimum value, and wmin=η((θ1+θ2)/2).
d3: Chroma transformation is implemented using the chroma transformation function (u′(x,y),v′(x,y))=w*φ(u(x,y),v(x,y),αu,αv,βu,βv), and the chroma information of the image is enhanced;
wherein φ(u(x,y),v(x,y),αu,αv,βu,βv) is the transformation function, w is the weight function, the UV chroma deviation position is determined by αu and αv, and the chroma adjusting step is determined by βu and βv.
Although transformation between different models can be realized through color space transformation technologies, computational complexity bought in by the transformation and invert transformation is also considerable. Considering the document format processed by the encoder, format conversation time should be reduced. The present invention implements a color information process directly in the UV chroma space.
Referring
10. Selecting a processing mode
according to block statistical characteristic
Then, operations for eliminating blocking artifacts and enhancing boundary information are implemented based on the determined processing mode.
For processing of flat regions:
t—1j=(f(x,y)|f(x,y)ε
)
wherein t—1j is a statistical characteristic variable name of the j-th flat region , Thres—2 is a threshold for the currently processed flat region processed by the selected different processing methods.
For processing of complex regions:
t—2j=(f(x,y)|f(x,y)ε
)
wherein f(x,y) is a original image value at the decoder, t—0 is a statistical variable name of a statistical region ,
(j=0, 1, 2) is a statistical characteristic function,
is a statistical region corresponding to
, and Thres—1 is a threshold for determining whether the current processing region is a flat region or a complex region; t—2j is a statistical characteristic variable name of the j-th complex region
, Thres—3 is a threshold for the currently processed complex region processed by the selected different processing methods.
The following table shows comparative experiment between the adaptive boundary information enhancement process of the present step and the separation method process in the prior art. The experiment uses images with a source size of 320×240, and same decoders are used. Comparison of objective effects is as follows:
It can be seen from the above table that, processing speed can be significantly improved by the adaptive boundary information enhancement method in accordance with the present invention.
20. Brightness of the image is adaptively transformed, and the brightness distribution of the image is improved. The principle of adaptive brightness transformation is that, the set of parameters of the transformation function is adaptively updated according to a statistical characteristic of brightness value of the image before being transformed, so that the transformation function is adjusted dynamically along with different image characteristics, and thus the processing method is optimized:
g(x,y)=ψ(f(x,y),α(k)|k=1, 2, . . . , K),
wherein f(x,y) is a brightness value of the original image at the decoder, g(x,y) is a transformed brightness value, ψ(x,α(k)|k=1, 2, . . . , K) is the transformation function, wherein α(k) is the set of parameters of the transformation function ψ(x,α(k)|k=1, 2, . . . , K), and K is the number of the parameters.
30. Chroma information is adaptively enhanced, wherein the chroma information adaptive enhancement is performed in a UV chroma space,
(u′(x,y),v′(x,y))=w*φ(u(x,y),v(x,y),αu,αv,βu,βv)
wherein φ(u(x,y),v (x,y),αu,αv,βu,βv) is a transformation function, w is a weight function, and a UV chroma deviation position is determined by αu and αv, a chroma adjusting step is determined by βu and βv.
The foregoing description of the exemplary embodiments of the invention has been presented only for the purposes of illustration and description and is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in light of the above teaching without departing from the protection scope of the present invention.
Number | Date | Country | Kind |
---|---|---|---|
2009 1 0106472 | Apr 2009 | CN | national |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/CN2009/073593 | 8/28/2009 | WO | 00 | 6/11/2010 |
Publishing Document | Publishing Date | Country | Kind |
---|---|---|---|
WO2010/111855 | 10/7/2010 | WO | A |
Number | Name | Date | Kind |
---|---|---|---|
20050099545 | Zhu | May 2005 | A1 |
20050163393 | Asari | Jul 2005 | A1 |
20080043854 | Kim et al. | Feb 2008 | A1 |
Number | Date | Country |
---|---|---|
1127562 | Jul 1996 | CN |
1545327 | Nov 2004 | CN |
1744687 | Mar 2006 | CN |
101061506 | Oct 2007 | CN |
0723364 | Jul 1996 | EP |
0043954 | Jul 2000 | WO |
2008023856 | Feb 2008 | WO |
2008047291 | Apr 2008 | WO |
Number | Date | Country | |
---|---|---|---|
20120027076 A1 | Feb 2012 | US |