Patent Grant
8406286
The present invention relates to the field of video coding, and particularly relates to a method and a device for selecting the best mode of intra prediction coding for video coding.
The video coding standard H.264/AVC is the latest video compression standard drafted by the Joint Video Team (JVT) established by the International Standardization Organization (ISO)/the International Electrotechnical Commission (IEC) and the International Telecommunication Union Telecommunication Standardization Sector (ITU-T). Compared with the preceding video compression standards, this standard has made remarkable improvements in the efficiency of video compression. Under the same Peak Signal to Noise Ratio (PSNR) quality, the code rate of H.264/AVC coder is reduced by approximately 50% compared with the code rate of MPEG-4 Advance Simple Profile (ASP).
In the design of H.264/AVC coder, a method of Rate Distortion Optimization (RDO) is adopted by both intra and inter predictions to perform decision. The object of introduction of this method is to select the best predictive mode from the possible combinations of all candidate predictive modes. The best coding mode selected based on rate distortion optimization means that in all possible mode combinations, a coding mode having the best video quality is selected in the condition of a given code rate, instead of simply minimizing the code rate or maximizing the video quality.
For an intra-4×4 prediction block, the best predictive mode corresponding to 16 pixels is determined by selecting the optimized rate distortion cost:
RDCost_init=SSD(s,c,IMODE|Qp)+λMODE·R(s,c,IMODE|Qp)
Wherein IMODE represents one of nine 4×4 block intra prediction modes; Qp represents the quantization parameter of a macro block; λMODE represents the Lagrange multiplier; R(·) represents the bit number of the selecting mode corresponding to the required coding; SSD(·) represents the sum of square of the difference between the original 4×4 luminance block s and the reconstructed block c corresponding to each of the candidate mode. Wherein:
In each computing process of SSD, cij is a reconstructed pixel value after being transformed, quantified, inversely quantified and inversely transformed. This means that the corresponding subblock is required to be coded each time when a to candidate mode is detected. The Discrete Cosine Transform (DCT), quantification and computing of entropy coding are very complex. Before the best predictive mode combination of a macro block is determined, the rate distortion cost for the combination of all chromaticity components and luminance components in the macro block should be repeatedly computed, that is, M8×(M4×16+M16), wherein is M8, M4 and M16 respectively represent the mode number of the chroma block prediction, the luminance block I4MB (intra-frame macro block 4×4 luminance manner) prediction and the I16MB (intra-frame macro block 16×16 luminance manner) prediction. This means that the rate distortion optimization process of a macro block should be computed 4×(9×16+4)=592 times, which results in that the complexity of the algorithm of the H.264 coder sharply increases. Thus, it is hard to apply this compression standard to a real time system with a low complexity processing capability, e.g. a mobile device.
In order to solve this problem, JM6.1d provides a new rate distortion cost function to improve the coding performance:
RDCost—SAD=SAD(s,p)+4P·λMODE
Wherein the Sum of Absolute Differences (SAD) is computed by using the following formula:
Wherein pij is the (i, j)th element in the predictive block. When the predictive mode is the same as the most probable predictive mode, the factor P equals to 0, otherwise, it equals to 1 with respect to other modes.
This cost function greatly reduces the complexity of computing. However, since the SAD-based rate distortion cost function RDCost_SAD neglects the effect of the code rate, the SAD-based rate distortion cost RDCost_SAD is more different from the selected result of the cost value RDCost_init in the original rate distortion optimization model. Particularly, when comparing nine modes, if there is a trivial difference between the minimum SAD value and the second minimal SAD value, and the code rate corresponding to the minimum SAD is comparatively big, after multiplying with the constant coefficient λMODE, the mode selected by the original rate distortion optimization model will be different from the mode selected by the SAD-based rate distortion cost.
Meanwhile, the domestic and foreign scholars and researchers also propose some fast algorithms for the intra prediction mode determination in H.264/AVC. F. Pan et al. have proposed a rapid mode selecting algorithm based on local edge information. The local edge map of the block to be coded is determined by means of the Sobel operator, and corresponding edge histogram map is established, so, the rate distortion optimization cost is computed by selecting a mode based on the proximal edge direction.
C. C. Cheng has proposed a three-step rapid algorithm for the intra-4×4 prediction block. In the algorithm, in the first step, three most probable modes are initially compared, and they are the horizontal mode, the perpendicular mode and the Direct Coefficient (DC) mode respectively. In the second step, based on the probable mode determined in the first step, two modes at the adjacent directions are selected for further refining the direction. Finally, the rate distortion costs of all modes after the refinement are computed to determine the final mode.
C. S. Kim has proposed a solution of synthetically considering the characteristics of the target block spatial domain and the transform domain, and filtering out most of candidate modes by analyzing the relationship between the posterior error probability and average rate-distortion loss.
The main idea of theses methods is to select a mode in which is most relative with the gray direction in the block to be predicted. From the view of essence, it is to look for a mode having the minimum SAD compared with the block to be coded from the nine candidate predictive mode blocks, but the effect of the code rate in the rate distortion optimization cost function is not considered. In order to avoid the error caused by merely considering SAD, these rapid algorithms during designing usually rely on selecting much more candidate modes, e.g. two modes at the adjacent directions of the modes corresponding to the minimum SAD and DC mode, to compute the rate distortion optimization cost.
The present invention provides a method and a device for selecting the best mode of intra prediction coding for video coding, thereby the problems of a difficulty in ensuring the selection of intra prediction mode to be close to the best mode selected by the original rate distortion optimization model, and the high computing complexity of repeatedly calculating the coding rate of the block in each predictive mode in the prior art are solved.
The present invention provides a method for selecting the best mode of intra prediction coding for video coding, including the following steps:
constructing a rate distortion cost function based on an estimated rate to be encoded;
computing the rate distortion cost for all predictive modes by using the constructed function, and
selecting the prodictive mode with the minimum rate distortion cost as the best mode of intra prediction coding based on the computing results.
The step of constructing a rate distortion cost function based on an estimated rate to be encoded specifically includes the following steps:
computing to estimate the residual error after intra prediction of the block to be coded;
computing the estimated rate of the block to be coded;
constructing the rate distortion cost function based on the estimated rate to be encoded by using the estimated residual error of the block to be coded as the distortion measurement, together with the estimated rate of the block to be coded.
The step of computing the estimated rate of the block to be coded specifically includes the following steps:
estimating the numbers of zero coefficients and ±1 coefficients comprised in the coefficients of the block to be coded after DCT and quantification during practical coding, based on the estimated residual error of the block to be coded;
predicting the estimated rate of the block to be coded, according to the characteristic of the Context-based Adaptive Variable Length Coding (CAVLC) method together with the estimated numbers of zero coefficients and ±1 coefficients.
The predictive mode is a valid predictive mode.
The present invention further provides a device for selecting the best mode of intra prediction coding for video coding, including:
a rate distortion cost function constructing unit for constructing a rate distortion cost function based on an estimated rate to be encoded;
a rate distortion cost computing unit for computing the rate distortion costs for all predictive modes based on the rate distortion cost function constructed by the rate distortion cost function constructing unit;
an intra prediction coding best mode selecting unit for selecting the predictive mode with the minimum rate distortion cost as the best mode of intra prediction coding based on the result obtained by the computation of the rate distortion cost computing unit.
The rate distortion cost function constructing unit further includes:
an estimated residual error computing subunit for computing to estimate residual error after intra prediction of the block to be coded;
an estimated rate to be encoded computing subunit for computing the estimated rate of the block to be coded;
a rate distortion cost function generating subunit for constructing a rate distortion cost function based on the estimated rate to be encoded by using the estimated residual error of the block to be coded obtained by the computation of the estimated residual error computing subunit as the distortion measurement, together with the estimated rate of the block to be coded obtained by the computation of the estimated rate to be encoded computing subunit.
The estimated rate to be encoded computing subunit further includes:
a coefficient estimating subunit for estimating the numbers of zero coefficients and ±1 coefficients comprised in the coefficients of the block to be coded after DCT and quantification during practical coding, based on the estimated residual error of the block to be coded obtained by the computation of the estimated residual error computing subunit;
a code rate predicting subunit for predicting the estimated rate of the block to be coded according to the characteristic of the Context-based Adaptive Variable Length Coding (CAVLC) method together with the numbers of zero coefficients and ±1 coefficients estimated by the coefficient estimating subunit.
The predictive mode is a valid predictive mode.
The present invention constructs a new rate distortion cost function which employs simple computing means and synthetically refers to the effects of the quantized residual error value and the estimated rate to be encoded with respect to the mode selection, by directly estimating the residual error value between the predictive block and the pixel corresponding to the original block to be coded to predict the numbers of zero coefficients and ±1 coefficients comprised in the quantified DCT coefficients, and further by predicting the code rate required by the coding according to the predicted numbers together with the characteristic of the Context-based Adaptive Variable Length Coding (CAVLC) method, and by synthetically considering the distortion measurement SAD and the estimated rate to be encoded. Therefore, based on the fact that the mode selection in the present invention is closer to the best mode selected by the original rate distortion optimization model, the need to code every block actually in the process of the rate distortion optimization is effectively avoided, and the coding efficiency is greatly improved in the condition that the code rate and PSNR of coding and all direction search method change little.
The present invention provides a method for selecting the best mode of intra prediction coding for video coding. Based on the fact that the mode selection in the present invention is closer to the best mode selected by the original rate distortion optimization model, the need to code every block actually in the process of the rate distortion optimization is effectively avoided, and the coding efficiency is greatly improved in the condition that the code rate and PSNR of coding and all direction search method change little.
As shown in
S110: constructing a rate distortion cost function based on an estimated rate to be encoded;
S120: computing the rate distortion costs for all predictive modes by using the constructed function, and
S130: selecting a predictive mode with the minimum rate distortion cost as the best mode of intra prediction coding based on the computing result.
As shown in
S1101: computing to estimate the residual error after intra prediction of the block to be coded;
S1102: computing the estimated rate of the block to be coded;
S1103: constructing the rate distortion cost function based on the estimated rate to be encoded by using the estimated residual error of the block to be coded as the distortion measurement, together with the estimated rate of the block to be coded.
As shown in
S11021: estimating numbers of zero coefficients and ±1 coefficients comprised in the coefficients of the block to be coded after DCT and quantification during practical coding, based on the estimated residual error of the block to be coded;
S11022: predicting the estimated rate of the block to be coded, according to the characteristic of the Context-based Adaptive Variable Length Coding (CAVLC) method together with the estimated numbers of zero coefficients and ±1 coefficients.
The following preferred embodiment of the selection of the best mode of intra prediction coding for an H.264/AVC 4×4 block according to the present invention is described by reference of the drawings of the Description:
The selection is implemented on the reference software verification model JM8.6 coder. The detailed working environment is a PC with Intel Pentium D 2.80 GHz processor, 1 GB DDR RAM, and Microsoft Windows 2003 Server. Main Profile of JVT is used as the configuration file, and the main parameters of which are shown as follows:
1) the movement vector MV searching scope is ±16 pixel value;
2) using the rate distortion optimization;
3) the reference frame number is set to 0;
4) the entropy coding mode is CAVLC;
5) the precision of the movement vector MV is ¼ pixel;
6) the GOP structure is a full I frame structure.
The method for selecting the best mode of intra prediction coding for an H.264/AVC 4×4 block in the present invention, comprising the following steps:
S1100: selecting valid predictive modes from nine luminance predictive modes based on valid reference pixel value, and performing intra prediction coding for 16 pixels in the original block s to be predicted respectively to obtain the predictive block p.
S1200: for different valid predictive modes, the estimated residual error is calculated between the predictive block obtained in S1100 and the pixel on the position corresponding to the original block, respectively:
S1300: for different valid predictive modes, performing estimation based on the estimated residual error value, and predicting the numbers of zero coefficients and ±1 coefficients comprised in 4×4 block respectively after DCT and quantification during practical coding; the traditional computing of 4×4 forward DCT transformation is shown as follows:
Wherein A is a transformation matrix, and corresponding items are a=½, b=√{square root over (½)} cos(π/8), c=√{square root over (½)} cos(3π/8).
When the integer DCT coefficient Y(u,v), the quantization parameter Qp and the quantization coefficient Z(u,v) are given, it is represented as follows:
|Z(u,v)|=(|Y(u,v)|·M(u,v)+f)>>qbits, 0≦u, v≦3 (3)
Wherein qbits=15+floor(Qp/6), the value range of Qp is 0 to 51. For the intra prediction block, f=(2qbits/3). M(u, v) is a multiplication factor with respect to Qp% 6, and there are three categories based on locations correspondingly, which is shown in Table 1.
Further expanding the DCT transformation in formula (2), constructing a new transformation matrix Aa(u,v)=[A(x,u)·A(y,v)]4×4 correspondingly, and then the transformation expression can be replaced by the following formula:
Wherein Xa=[X(x,y)]4×4, represents that each element of matrix X, is multiplied by the element in the same position of transformation matrix Aa(u,v).
The formula (3) should be considered to obtain a quantized value T(u,v) when determining whether the value of each element in integer DCT coefficient matrix Ŷ is 0.
Then condition of determining that the DCT coefficient after quantization is 0 is represented as follows:
|{circumflex over (Y)}(u,v)|<T(u,v) (5)
Taking u=0, v=1 as an example:
Since the transformation matrix Aa(u,v) is regularly distributed and is highly symmetrical, it is divided into four 4×4 blocks, for example:
to further obtain:
Ŷ(0,1)=2AV0+AV1−AV2−2AV3
So, the condition that the DCT coefficient at the location (0,1) after quantization is 0 is as follows:
|{circumflex over (Y)}(0,1)|=|2AV0+AV1−AV2−2AV3|<T(0,1)
Similarly, for other locations in the residual error matrix, different block divisions can be performed correspondingly to different transformation matrixes respectively to perform a rapid computing, and the statistics of the number Npre
Certainly, the following formula
|Ŷ(u,v)|>T(u,v)
can serve as a condition that the DCT coefficient after quantization is not 0 to determine whether all DCT coefficients after quantization are nonzero coefficients, and the statistics of the number of the nonzero coefficients is performed.
Additionally, considering the following condition of determining ±1 coefficients:
T(u,v)≦|{circumflex over (Y)}(u,v)|<2T(u,v) (6)
This condition is used to determine whether the DCT coefficients after quantization are ±1, and the statistics of the number Npre
S1400: for all valid predictive modes, predicting the code rate required by the block to be coded according to the coding characteristic of the Context-based Adaptive Variable Length Coding (CAVLC) together with the predicted numbers of zero coefficients and ±1 coefficients; based on the analysis of characteristic of the algorithm of CAVLC entropy coding, it is deemed that there are three important parameters affecting the coding length in CAVLC coding, they are: Ncoeff, None and Nzero, wherein Ncoeff represents the number of nonzero coefficients, None represents the number of ±1 coefficients, and NZero represents the number of zero coefficients after the first nonzero coefficient. From CAVLC code table, it can be seen that the bigger the value of NCoeff is, the more the code words will be generated correspondingly, while the bigger the value of NZero is, the less the corresponding code words will be. Therefore, we could consider corresponding NCoeff and NZero, as a whole unit. In order to estimate the bit number of the corresponding block obtained by CAVLC coding, the following coefficient formula to be fitted is constructed:
actual_rate=a×NCoeff+b×NOne+rate_mode (7)
Wherein rate_mode equals to 1 for most probable predictive mode, and equals to 4 for other modes.
By using n given triad (NCoeff, None, actual_rate), the coefficient fitting to the coefficient formula to be fitted is performed to obtain coefficients a and b to be fitted, and to further obtain an estimated rate of the block to be coded. For instance, several sequences of practical coding are adopted, two coefficients to be determined are obtained first as reference coefficient values of the estimated rate formula: five general test sequences and three video clips collected in practical are used for testing, and after using CAVLC to code, the statistics of the numbers of the required nonzero coefficients and ±1 coefficients, and actual code rate corresponding to each 4×4 luminance block are performed, and by using a method of least square fitting, the correspondingly best fitting coefficients a=6.3, b=−2.3 can be obtained, thereby the following estimated rate formula is obtained:
rate_pre=[6.3×(16−Npre
Wherein rate_pre represents the estimated rate to be encoded, Npre
Certainly, the fitting coefficient value can be determined by using an adaptive manner based on the characteristics of the sequences themselves during coding to obtain the estimated rate of the block to be coded, and it is unnecessary to obtain the coefficient to be fitted first and to make it serve as an experience reference value. The estimated rate fitting coefficients of all valid predictive modes obtained by means of an adaptive way may be different from each other.
S1500: for all valid predictive modes, constructing a rate distortion cost function based on the estimated rate to be encoded by using SAD of the block to be coded to estimate the intra prediction residual error and the estimated rate of the block to be coded:
Additionally, if the estimated rate to be encoded uses an adaptive way to determine the fitting coefficient, the fitting coefficients in a rate distortion cost function based on an estimated rate to be encoded constructed in all valid predictive modes may be different from each other.
S1600: using the rate distortion cost function based on the estimated rate to be encoded obtained in S1500 to respectively compute the rate distortion costs for all valid predictive modes, and thereby selecting the mode due to which the rate distortion cost is minimum as the best mode of intra prediction coding.
Mode_best=Mode(min RDCost_modified)
Wherein if the fitting coefficients in the rate distortion cost function constructed in all valid predictive modes are the same with each other, i.e., an identical rate distortion cost function is constructed for all valid modes, thereby this identical rate distortion cost function is used to respectively compute the rate distortion costs for all valid modes; if the fitting coefficients in the rate distortion cost function constructed in all valid predictive modes are different from each other, corresponding rate distortion cost functions are constructed for all valid modes, thereby the rate distortion costs for all valid modes are respectively computed by respectively corresponding rate distortion cost functions.
As shown in
a rate distortion cost function constructing unit 10 for constructing a rate distortion cost function based on an estimated rate to be encoded;
a rate distortion cost computing unit 20 for computing the rate distortion costs for all predictive modes based on the rate distortion cost function constructed by the rate distortion cost function constructing unit 10;
an intra prediction coding best mode selecting unit 30 for selecting a predictive mode with the minimum rate distortion cost as the best mode of intra prediction coding based on the computing result of the rate distortion cost computing unit 20.
Further, the rate distortion cost function constructing unit 10 specifically includes:
an estimated residual error computing subunit 101 for computing to estimate residual error after the intra prediction of the block to be coded;
an estimated rate to be encoded computing subunit 102 for computing the estimated rate of the block to be coded;
a rate distortion cost function generating subunit 103 for constructing a rate distortion cost function based on the estimated rate to be encoded by using the estimated residual error of the block to be coded obtained by the computation of the estimated residual error computing subunit 101 as the distortion measurement, together with the estimated rate of the block to be coded obtained by the computation of the estimated rate to be encoded computing subunit 102.
Further, the estimated rate to be encoded computing subunit 102 specifically includes:
a coefficient estimating subunit 1021 for estimating numbers of zero coefficients and ±1 coefficients comprised in the coefficients of the block to be coded after DCT and quantification during practical coding, based on the estimated residual error of the block to be coded obtained by the computation of the estimated residual error computing subunit 101;
a code rate predicting subunit 1022 for predicting the estimated rate of the block to be coded according to the characteristic of the Context-based Adaptive Variable Length Coding (CAVLC) method together with the numbers of zero coefficients and ±1 coefficients estimated by the coefficient estimating subunit 1021.
Obviously, a person skilled in the art may modify or change the present invention within the spirit and scope of the present invention. Thus, if the modifications or changes for the present invention are within the scopes of the claims of the present invention and equal techniques, the present invention would like to include these modifications and changes.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2007 1 0119777 | Jul 2007 | CN | national |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/CN2008/001343 | 7/18/2008 | WO | 00 | 6/28/2010 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2009/015553 | 2/5/2009 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 6195388 | Choi et al. | Feb 2001 | B1 |
| 6421467 | Mitra | Jul 2002 | B1 |
| 6639942 | Bayazit | Oct 2003 | B1 |
| 6711209 | Lainema et al. | Mar 2004 | B1 |
| 7006811 | Pukkila | Feb 2006 | B2 |
| 7079581 | Noh et al. | Jul 2006 | B2 |
| 7453937 | Henocq et al. | Nov 2008 | B2 |
| 7764740 | Seok et al. | Jul 2010 | B2 |
| 7778459 | Song | Aug 2010 | B2 |
| 7856053 | Chujoh | Dec 2010 | B2 |
| 7916783 | Gao et al. | Mar 2011 | B2 |
| 7936826 | Chen et al. | May 2011 | B2 |
| 7957600 | Yang | Jun 2011 | B2 |
| 8045587 | Park et al. | Oct 2011 | B2 |
| 8094711 | Tourapis et al. | Jan 2012 | B2 |
| 8111914 | Lee et al. | Feb 2012 | B2 |
| 8130828 | Hsu et al. | Mar 2012 | B2 |
| 8155181 | Han et al. | Apr 2012 | B2 |
| 8155190 | Imamura et al. | Apr 2012 | B2 |
| 8160141 | Auyeung | Apr 2012 | B2 |
| 8160150 | Moore | Apr 2012 | B2 |
| 8170102 | Bhaskaran et al. | May 2012 | B2 |
| 8175150 | Zhang et al. | May 2012 | B1 |
| 8194729 | Kim et al. | Jun 2012 | B2 |
| 8204114 | Wang et al. | Jun 2012 | B2 |
| 8265136 | Hong et al. | Sep 2012 | B2 |
| 8290045 | Ramachandran et al. | Oct 2012 | B2 |
| 20070053426 | Lee et al. | Mar 2007 | A1 |
| 20070121728 | Wang et al. | May 2007 | A1 |
| 20080112481 | Hsaing et al. | May 2008 | A1 |
| 20080232463 | Lu et al. | Sep 2008 | A1 |
| 20090274213 | Zhou et al. | Nov 2009 | A1 |
| 20110096832 | Zhang et al. | Apr 2011 | A1 |
| 20110243229 | Kim et al. | Oct 2011 | A1 |
| 20120039389 | Sjoberg et al. | Feb 2012 | A1 |
| Number | Date | Country |
|---|---|---|
| 1455600 | Nov 2003 | CN |
| 1674680 | Sep 2005 | CN |
| 1753500 | Mar 2006 | CN |
| 101141649 | Mar 2008 | CN |
| 2184925 | May 2010 | EP |
| Entry |
|---|
| Kim et al., “Multistage Mode Decision for Intra Prediction in H.264 Codec”, Visual Communications and Image Processing, 2004, p. 355-363, vol. 5308. |
| Cheng et al., “Fast Three Step Intra Prediction Algorithm for 4×4 blocks in H.264”, Dept. Electronics Engineering, National Chiao-Tung University, Hsinchu, Taiwan, R.O.C., p. 1509-1512. |
| Number | Date | Country | |
|---|---|---|---|
| 20100290521 A1 | Nov 2010 | US |
