Method based on Coding Tree Unit Level Rate-Distortion Optimization for Rate Control in Video Coding

Abstract

A method based on CTU level rate-distortion optimization for rate control in video coding which can effectively improve the perceptual rate-distortion performance and coding efficiency is provided. Firstly, a perceptual rate-distortion model is established using a divisive normalization framework, which characterizes the relationship between local visual quality and coding bits. Subsequently, the established perceptual rate-distortion model is applied to overall distortion optimization which is transformed into a global optimization problem and solved with convex optimization algorithms to obtain optimal CTU level coding bit allocation.

Description

FIELD OF THE INVENTION

The present invention relates generally to rate control in video coding. Particularly, the present invention relates to coding tree unit level rate-distortion optimization for rate control in video coding.

BACKGROUND

Recently, several studies have been conducted to improve rate control (RC) optimization in High Efficiency Video Coding (HEVC). There are three categories of RC algorithms for HEVC: quadratic model [1], ρ-domain model [2] and R-λ model. More specifically, Li et al. [3] first proposed the λ domain RC based on the relationship between coding bits and the Lagrange multiplier. Due to the low complexity and high efficiency, the R-λ model has been adopted in HEVC reference software as the default RC algorithm. Lee et al. investigated the Laplacian probability distribution function (PDF) in [4] to model the residue and proposed independent R-Q models to establish the relationship between the quantization parameters and coding bits, including texture and non-texture bits. Moreover, intra frame RC algorithms have also been studied. Li et al. [5] proposed an adaptive bit allocation algorithm to improve the R-λ model RC algorithm on intra frame. In [6], sum of absolute transformed differences (SATD) was used to measure the complexity for intra-frame, which further improves the performance. Wang et al. proposed an intra R-λ, model in [7], and the gradient was used to characterize the picture complexity.

In FIG. 15, the basic process of RC in HEVC is illustrated. Coding tree unit (CTU) level RC is also playing an important role in regulating the bit rate and improving the coding performance. Due to the importance of CTU-level RC, which can greatly influence the Rate-Distortion (R-D) performances, various RC algorithms at CTU level have been proposed for HEVC. An optimized CTU level RC strategy was proposed by Li et al. [8]. Wang et al. [9] proposed a RC scheme based on Lagrange multiplier, which greatly improved the coding efficiency. In [10], Zhou et al. proposed a novel CTU level RC method based on content complexity correlation for HEVC. However, these methods are optimized based on mean square error (MSE), which may not be optimal in terms of perceptual quality. As the ultimate receiver of the video streams is the human visual system [11], it is desirable to have a RC scheme where the perceptual characteristics are fully considered in CTU level bit allocation.

REFERENCES

• [1] H. Choi, J. Nam, J. Yoo, D. Sim, and I. Bajic, “Rate control based on unified RQ model for HEVC,” in Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IECJTC1/SC29/WG118th Meeting, 2012, pp. 1-10.
• [2] X. Liang, Q. Wang, Y. Zhou, B. Luo, and A. Men, “A novel R-Q model based rate control scheme in HEVC,” in Vis. Commun. Image Process. Kuching, 2013, pp. 1-6.
• [3] B. Li, H. Li, L. Li, and J. Zhang, “Rate control by R-λ model for HEVC,” in ITU-T SG16 Contribution, JCTVC-K0103, Shanghai, 2013, pp. 1-5.
• [4] B. Lee, M. Kim, and T. Q. Nguyen, “A frame-level rate control scheme based on texture and nontexture rate models for high efficiency video coding,” IEEE Trans. Circuits Syst. Video Technol., vol. 24, no. 3, pp. 465-479, March 2014.
• [5] X. Wang, M. Karczewicz, “Intra frame rate control based on SATD,” Document: JCTVC-M0257, Joint Collaborative Team on Video Coding.
• [6] B. Li, H. Li, L. Li, “Adaptive bit allocation for R-λ, model rate control in HM,” Document: JCTVC-M0036, Joint Collaborative Team on Video Coding.
• [7] M. Wang, K. N. Ngan, and H. Li, “An efficient frame-content based intra frame rate control for high efficiency video coding,” IEEE Signal Process. Lett. vol. 22, no. 7, pp. 896-900, July 2015.
• [8] S. Li, M. Xu, Z. Wang, and X. Sun, “Optimal bit allocation for CTU level rate control in HEVC,”IEEE Trans. Circuits Syst. Video Technol., vol. PP, no. 99, p. 1, July 2016.
• [9] M. Wang, K. N. Ngan, and H. Li, “Low-delay rate control for consistent quality using distortion-based Lagrange multiplier,” IEEE Trans. Image Process., vol. 25, no. 7, pp. 2943-2955, April 2016.
• [10] M. Zhou, Y. Zhang, B. Li, and X. Lin, “Complexity Correlation-Based CTU level Rate Control with Direction Selection for HEVC,” ACM Trans-actions on Multimedia Computing, Communications, and Applications, Volume 13, Issue 4, August 2017, pp. 1-23.
• [11] W. Gao, S. Kwong, and Y. Jia, “Joint machine learning and game theory for rate control in high efficiency video coding,” IEEE Transactions on Image Processing, vol. 26, no. 12, pp. 6074-6089, 2017.

The disclosures of above references are incorporated herein by reference in their entirety.

SUMMARY OF THE INVENTION

The present disclosure relates to methods based on coding tree unit (CTU) level rate-distortion (R-D) optimization for rate control (RC) in video coding which can effectively improve the perceptual rate-distortion performance and coding efficiency. Firstly, a perceptual R-D model is established using a divisive normalization framework, which characterizes the relationship between local visual quality and coding bits. Subsequently, the established perceptual R-D model is applied to overall distortion optimization which is transformed into a global optimization problem and solved with convex optimization algorithms to obtain optimal CTU level coding bit allocation.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention are described in more detail hereinafter with reference to the drawings, in which:

FIG. 1 shows a block diagram of a video coding system according to one embodiment of the present invention;

FIG. 2 shows a table of Pearson correlation coefficients between the predicted and actual values for different coding tree units (CTUs) according to one embodiment of the present invention;

FIGS. 3A-3F show the actual and fitted relationships between rate and perceptual distortion for several typical test sequences with different quantization parameters according to one embodiment of the present invention; wherein FIG. 3A shows the actual and fitted relationships for the sequence PeopleOnstreet, FIG. 3B shows the actual and fitted relationships for the sequence PartyScene, FIG. 3C shows the actual and fitted relationships for the sequence Johnny, FIG. 3D shows the actual and fitted relationships for the sequence ParkScene, FIG. 3E shows the actual and fitted relationships for the sequence Keiba and FIG. 3F shows the actual and fitted relationships for the sequence BlowingBubbles;

FIGS. 4A-4C show comparisons between the actual and estimated values of model parameters according to one embodiment of the present invention; wherein FIG. 4A shows the comparison for the 60-th frame in BQMall(832×480) sequence; FIG. 4B shows the comparison for the 96-th frame in Keiba(832×480) sequence; and FIG. 4B shows the comparison for the 200-th frame in Johnny(720p) sequence;

FIGS. 5A-5D show the overall rate-SSIM performance of the method of the present invention comparing with some state-of-the-art methods; wherein FIG. 5A shows the comparison with HM16.8, FIG. 5B shows the comparison with Li et al. [8], FIG. 5C shows the comparison with Wang et al. [9] and FIG. 5D shows the comparison with GAO et al. [11];

FIGS. 6A-6B show two tables of the bit rate savings of the method of the present invention comparing with some state-of-the-art methods under non-hierarchical and hierarchical configurations, respectively; wherein FIG. 6A shows the comparison under non-hierarchical configuration and FIG. 6B shows the son under hierarchical configuration;

FIGS. 7A-7F show comparison results of visual qualities of frames encoded from 120-th frame in BQSquare sequence with target bit rate of 160 kbps by the method of the present invention and some state-of-the-art RC methods under hierarchical configuration; wherein FIG. 7A shows the original frame; FIG. 7B shows a frame encoded with the method of HM16.8 at actual bit rate of 161.11 kbps; FIG. 7C shows a frame encoded with the method of Li et al. [8] at actual bit rate of 160.77 kbps; FIG. 7D shows a frame encoded with the method of Wang et al. [9] at actual bit rate of 163.77 kbps; FIG. 7E shows a frame encoded with the method of Gao et al. [11] at actual bit rate of 161.72 kbps, respectively; and FIG. 7F shows a frame encoded with the method of the present invention at actual bit rate of 160.83 kbps;

FIGS. 8A-8F show other comparison results of visual qualities of frames encoded from the 120-th frame in BQSquare sequence with target bit rate of 780 kbps by the method of the present invention and some state-of-the-art RC methods under hierarchical configuration; wherein FIG. 8A shows the original framed; FIG. 8B shows a frame encoded with the method of HM16.8 at actual bit rate of 780.33 kbps; FIG. 8C shows a frame encoded with the method of Li et al. [8] at actual bit rate of 780.11 kbps; FIG. 8D shows a frame encoded with the method of Wang et al. [9] at actual bit rate of 781.34 kbps ; FIG. 8E shows a frame encoded with the method of Gao et al. [11] at actual bit rate of 780.95 kbps; and FIG. 8F shows a frame encoded with the method of the present invention at actual bit rate of 780.17 kbps;

FIG. 9 shows a table of experimental results comparing the quality smoothness among the method of the present invention and some state-of-the-art RC methods under non-hierarchical (N-Hie) and hierarchical (Hie) configurations, respectively;

FIGS. 10A-10B show typical buffer occupancies for two test video sequences under hierarchical configuration; wherein FIG. 10A shows the buffer occupancy for the video sequence atus (1080p) and FIG. 10B shows the buffer occupancy for the video sequence Keiba (832×480);

FIG. 11 shows a table of experimental results comparing the frame level RC accuracies among the method of the present invention and some state-of-the-art RC methods;

FIG. 12 shows a bar chart comparing the computation time among the method of the present invention and some state-of-the-art RC methods;

FIG. 13 shows a table of R-D performance comparison among the method of the present invention and some state-of-the-art RC methods under hierarchical configuration on video sequences with dynamic scene changes; and

FIG. 14 shows a table of frame level RC accuracy comparison among the method of the present invention and some state-of-the-art RC methods under hierarchical configuration on video sequences with dynamic scene changes; and

FIG. 15 shows a basic process of rate control (RC) in High Efficiency Video Coding (HEVC).

DETAILED DESCRIPTION

In the following description, methods and apparatus using coding tree unit (CTU) level rate-distortion optimization for rate control (RC) in video coding are set forth as preferred examples. It will be apparent to those skilled in the art that modifications, including additions and/or substitutions may be made without departing from the scope and spirit of the invention. Specific details may be omitted so as not to obscure the invention; however, the disclosure is written to enable one skilled in the art to practice the teachings herein without undue experimentation.

In one aspect, the present invention may be implemented in the video coding system 100 of FIG. 1. The system 100 may include modules. The term “module” may include software, hardware, or a combination thereof in which software can be machine code, firmware, embedded code, and application software. The hardware may be circuitry, processors, computers, integrated circuits, integrated circuit cores, passive devices, or a combination thereof. In FIG. 1, a video encoder 20 may receive and encode video from a video source 10. “Video” as used herein, is a digital representation of one or more objects.

Encoding is defined as computationally modifying a video source 10 to a different form. Encoding includes compression, in which data amounts are reduced, enhancement, resolution changes, aspect ratio changes. In one aspect, the encoding may be performed according to the High-Efficiency Video Coding (HEVC)/H.265 standard. Frames are generated in a frame generating module 25. Subsequently, the frame may be divided into one or more CTUs in CTU module 22.

Video encoded by the encoder 20 forms a video bitstream 30 that represents information from the video source 10. The video bitstream 30 is transmitted or transferred to decoder 50 over transmission medium 40.

Transmission medium 40 may be a wired or wireless communication network or a file transfer to decoder 50.

Decoder 50 takes the video bitstream 30 and creates a video stream 60, which is a computationally modified version of the video source 10. The decoder may create a video stream 60 that can have different properties from the source such as a different frame rate, different resolution, different color parameters, different view order, different aspect ratio, a different frame rate, or combinations, etc.

The video stream 60 is transmitted to a display medium 70 including a display processor 75. The display processor 75 can receive the video stream 60 from the video decoder 50 for display by the display medium 70.

The video coding system 100 can employ a variety of video coding syntax structures. For example, the video coding system 100 can encode and decode video information using High Efficiency Video Coding/H.265 (HEVC), scalable extensions for HEVC (SHVC), or other video coding syntax structures.

The video encoder 20 and the video decoder 50 may be implemented by hardware, software, or a combination thereof. For example, the video encoder 50 may be implemented with custom circuitry, a digital signal processor, microprocessor, or a combination thereof. In another example, the video decoder 60 can be implemented with custom circuitry, a digital signal processor, microprocessor, or a combination thereof.

According to one embodiment of the present invention, a method based on CTU level rate-distortion optimization for RC in video coding is provided. The method may be implemented in the system of FIG. 1. The method comprises: receiving an input video; dividing a frame of the input video into one or more CTUs; determining coding bits of the one or more CTU level through optimizing rate-distortion (R-D) performance based on a divisive normalization framework; wherein the divisive normalization framework is configured to characterize the perceptual distortion for each CTU in relation with the mean square error (MSE) distortion and a divisive normalized factor f by the equation:

D′(R)=D(R)/f²   (1)

where D(R) is the MSE distortion, D′(R) is the normalized perceptual distortion, and R is the bit rate of CTU. In general, the MSE distortion, D(R), may be defined by:

$\begin{array}{cc}\mathrm{MSE}=\frac{1}{\mathrm{mn}}\sum _{i=0}^{m-1}\sum _{j=0}^{n-1}{I\left(i,j\right)-K\left(i,j\right)}^2& \left(2\right)\end{array}$

where I and k are original frame and reconstructed frame, respectively. I(i, j) and K(i, j) are the pixel values of original frame and reconstructed frame, respectively. m and n are the numbers of rows and columns in a frame, respectively.

To obtain the divisive normalization factor f, each CTU can be divided into I sub-blocks for Discrete cosine transform (DCT), and the factor f is obtained from the Structural Similarity (SSIM) index in DCT domain:

$\begin{array}{cc}f=\frac{\frac{1}{l}\sum _{i=1}^l\sqrt{\frac{\sum _{j=1}^{N_L-1}\left({U_i\left(j\right)}^2+{V_i\left(j\right)}^2\right)}{N_L-1}+C_1}}{E\left(\sqrt{\frac{\sum _{j=1}^{N_L-1}\left({U\left(j\right)}^2+{V\left(j\right)}^2\right)}{N_L-1}}+C_1\right)}& \left(3\right)\end{array}$

where E( ) is the expectation operation in the whole frame. U(j) and V(j) denote the DCT coefficients of the input and reconstructed signals, U_i(j) and V_i(j) are the corresponding j-th DCT coefficient in the i-th sub-block.

In some embodiments, the DCT coefficients of the reconstructed signals are approximated by the original input signals as the frame has not been encoded when deriving the normalization factors. C_I is the constant in accordance with the definition of SSIM index. N_L is the sub-block size, and may be set to be 16. However, it should be understood by those skilled in the art that the sub-block size can be set to any other values for deriving the divisive normalization factors f.

Given the available bit rate allocated to the frame, the CTU level rate control may be achieved by CTU level bit allocation through optimizing the perceptual distortion by minimizing a perceptual rate distortion cost function J defined by:

$\begin{array}{cc}J=\sum _{i=1}^ND^{\prime }\left(R_i\right)+\lambda R_i& \left(4\right)\end{array}$

where λ is the Lagrange multiplier in HEVC, which is also used when the distortion is normalized with the divisive normalization strategy, D′(R_i) is the perceptual distortion of the i-th CTU with a coding bit rate R_i, and N is the number of CTU in one frame.

In some embodiments, a global optimization approach for optimizing the CTU level coding bit allocation may be used, wherein all CTUs in a frame are configured to compete for the resources under the constraint of the target frame-level coding bits. Therefore, the CTU level coding bits allocation can be performed effectively by solving an optimization problem. The scheme of the present invention not only improves the reconstruction quality and coding efficiency in terms of perceptual rate-distortion, but also benefits the future R-D modelling with high accuracy.

In the global optimization approach, each CTUs of a frame, denoted as CTU₁, CTU₂, . . . , CTU_N, may be allocated with utilities of coding bit rates, R₁, R₂, . . . , R_N respectively. Possible utility combination sets may be expressed with a utility vector denoted as U_m=(R₁^m, R₂^m, . . . , R_N^m), m∈[0, M] where M is the quantity of the possible combinations of utility.

As the utility set U=(U₁, U₂, . . . , U_M) is non-empty and bounded, and the set of feasible utility U is convex, the CTU level rate control can be achieved by an optimal bit rate allocation. The optimal bit allocation may be investigated by minimizing average distortion which depends on the perceptual distortion D′(R) . As such, the CTU level bit allocation may be formulated as:

$\begin{array}{cc}\left\{R_1^{*},R_2^{*},\dots ,R_N^{*}\right\}=\arg \min \sum _{i=1}^ND^{\prime }\left(R_i\right)& \left(5\right)\\ s.t.\sum _{i=1}^NR_i\le R_c& \end{array}$

where N is the number of CTUs of one frame and R_c is the frame-level bit rate.

Therefore, the perceptual distortion optimization problem can be converted from a constrained optimization problem into an unconstrained optimization problem and the cost function J of Equation (4) may be converted to:

$\begin{array}{cc}J=\sum _{i=1}^ND^{\prime }\left(R_i\right)+\lambda \left(R_C-\sum _{i=1}^NR_i\right)& \left(6\right)\end{array}$

Typically, Equation (6) is the minimal value of different function and convex function on convex set. Therefore, Karush-Kuhn-Tucker (KKT) condition ensures that the local optimal solution of Equation (6) is a KKT point and the local optimal solution is also the global optimal solution.

Taking the video content into consideration, the relationship between the normalized perceptual distortion D′(R) and the bit rate R may be depicted with a logarithmic R-D model:

D′(R)=ln(c×R^−k)   (7)

where c and k are model parameters depending on the video content.

The prediction accuracy of the logarithmic R-D model may be validated by calculating the average Pearson correlation coefficient between the predicted and actual values for a series of test sequences with different QPs including: PeopleOnstreet (1600p), ParkScene (1080p), FourPeople(720p), BQMall(832×480) and BQsquare(416×240). It can be seen from a table in FIG. 2 that the average Pearson correlation coefficient between the predicted and actual values using the model of the present invention are around 0.94.

The effectiveness of the logarithmic R-D model is also validated using a Low Delay B (LDB) coding structure with reference image in HM 16.8. The values of c and k are obtained by fitting the actual values with the model. FIGS. 3A-3F show the actual and fitted relationships between rate and perceptual distortion for several typical test sequences with different QPs including: PeopleOnstreet, PartyScene, Johnny, ParkScene, Keiba and BlowingBubbles. Again, it can be seen that the prediction accuracy is high for all sequences under test.

Based on the logarithmic R-D model, the optimal coding bit for each CTU may be obtained by solving the equation:

$\begin{array}{cc}\{\begin{array}{c}\frac{\partial J}{\partial R_j}=k_j\frac{1}{R_j}-\lambda =0,\\ R_c-\sum _{i=1}^NR_i=0\end{array}& \left(8\right)\end{array}$

where R_j and k_j are the coding bit and model parameter for the j-th CTU, respectively.

Given Eq. (8), we have,

$\begin{array}{cc}\sum _{i=1}^N\frac{-k_i}{\lambda }=R_c.& \left(9\right)\end{array}$

Then the following relationship can be derived by

$\begin{array}{cc}\lambda =\frac{{\sum }_{i=1}^N-k_i}{R_c}.& \left(10\right)\end{array}$

Subsequently, by substituting Equation (10) into Equation (8), we have

$\begin{array}{cc}\frac{{\sum }_{i=1}^N-k_i}{R_c}=\frac{-k_j}{R_j}& \left(11\right)\end{array}$

Accordingly, the optimal coding bits of the jth CTU, denoted as R*_j, may be determined by

$\begin{array}{cc}{R}_j^{*}=\frac{k_j}{{\sum }_{i=1}^Nk_i}R_c& \left(12\right)\end{array}$

After obtaining the parameter k_j and R*_j for the j-th CTU of the current to-be-encoded i-th frame, the CTU level target bit budget may be further adjusted by:

$\begin{array}{cc}{R}_j^{*}=R_j^{*}\times {\omega }_a{\omega }_a=\left(1-\frac{{\sum }_{p=1}^{j-1}\left(R_{\mathrm{act},p}-R_p^{*}\right)}{R_c}\right)& \left(13\right)\end{array}$

where ω_a is an adjustment term to regularize the CTU level bit such that the frame-level budget can be met. R_act,p and R*_p are the real bits and the target bits after bit allocation, respectively. The corresponding QP can be obtained for each CTU through the R-Q model disclosed in [1].

In some embodiments, optimal values of the parameter k for each CTU in a current to-be-encoded i-th frame may be estimated with an updating strategy based on the coding statistics of a previously encoded (i-I)th frame. In particular, the optimal value of k for a j-th CTU in the i-th frame may be obtained by minimizing the difference between a true distortion D_real of the j-th CTU in the previously encoded i-I frame and an estimated distortion D_comp for the j-th CTU of the i-th frame.

The true distortion D_real may be estimated with Equation (1): D′(R)=D(R)/f². The distortion between two adjacent frames is of great importance to control the consistent quality and the distortion of the current CTU is similar to the co-located position of previous frame. Therefore, the distortion of a co-located CTU may be used to obtain D_comp.

The difference between D_real and D_comp can be represented by a squared error function denoted as e² which is expressed as:

e ²=(D_real−D_comp)².   (14)

By taking the derivative of e² to k, we have

$\begin{array}{cc}\frac{\partial e^2}{\partial k}=\frac{\partial e^2}{\partial D_{\mathrm{comp}}}\frac{\partial D_{\mathrm{comp}}}{\partial k}=-2\left(D_{\mathrm{real}}-D_{\mathrm{comp}}\right)\ln R.& \left(15\right)\end{array}$

Based on the Taylor's expansion, the optimal value of k for the j-th CTU the i-th frame, k_new, may be obtained by:

$\begin{array}{cc}\begin{array}{c}{k}_{\mathrm{new}}=k_{\mathrm{old}}-\delta \left(-2\left(D_{\mathrm{real}}-D_{\mathrm{comp}}\right)\right)\ln R\\ =k_{\mathrm{old}}+{\delta }_k\left(D_{\mathrm{real}}-D_{\mathrm{comp}}\right)\ln R.\end{array}& \left(16\right)\end{array}$

where λ_k is a constant which is preferably set to be 0.05, k_old is the value of k of the co-located CTU in the previous frame, and R is the bit rate for the to-be-encoded CTU.

It should be noted that λ_k in Equation (16) can be adaptive to the video content and the model parameters between two consecutive frames are of great importance to achieve quality control in video coding. Regarding the rate control that produces videos with consistent quality, the model parameters of a CTU are better to be consistent with the co-located CTU in the previous frame. Therefore, the value of k of the co-located CTU in the previous frame may be used as k_old Equation (16) for computing k_new. As to the initial values of k, it may be set to an arbitrary value such as 2.5 used in the experiment. It is also worth mentioning that the initial values of k are not critical for CTU level rate control in the present invention, as the value will keep updating in the actual coding process.

FIGS. 4A-4C show comparison results of the actual and estimated model parameters for the 60-th frame in BQMa1l(832×480) sequence; the 96-th frame in Keiba(832×480) sequence; and the 200-th frame in Johnny(720p) sequence, respectively. It can be seen the model parameter parameters k can be effectively estimated with the method provided in the present invention.

Experiments have been carried out to compare performance of the RC method provided in the present invention in various aspects with some state-of-the-art RC methods. In the experiments, an LDB coding structure was used and both non-hierarchical (N-Hie) and hierarchical (Hie) encoding were involved.

FIGS. 5A-5D show the overall rate-SSIM performance of the method of the present invention comparing with some state-of-the-art methods disclosed in HM16.8, Li et al. [8], Wang et al. [9] and GAO et al. [11], respectively. It can be seen that the method of the present invention performs better in a wide range of bit rate.

FIGS. 6A-6B shows tables of the bit rate savings of the method of the present invention comparing with HM16.8, Li et al. [8], Wang et al. [9] and GAO et al. [11], under non-hierarchical and hierarchical configurations, respectively. The target bitrates are obtained based on compressing a sequence at fixed QP values set to be 37, 32, 27, and 22, respectively. In comparison with the results of HM16.8, the method of the present invention can achieve 16.3% and 6.5% of average bit rate savings under non-hierarchical and hierarchical configurations, respectively. Moreover, it is observed that the method of the present invention performs better under the non-hierarchical configuration can gain better performance as the hierarchical configuration leaves less room for improvement.

The visual qualities of a 120-th frame in BQSquare sequence with target bit rate of 160 kbps after being encoded with some state-of-the-art RC methods and the method of the present invention under hierarchical configuration are compared to investigate the subjective quality improvement. FIG. 7A shows the original frame; FIGS. 7B-7E shows frames encoded with the methods of HM16.8 at actual bit rate of 161.11 kbps, Li et al. [8] at actual bit rate of 160.77 kbps, Wang et al. [9] at actual bit rate of 163.77 kbps and Gao et al. [11] at actual bit rate of 161.72 kbps, respectively; and FIG. 7F shows a frame encoded with the method of the present invention at actual bit rate of 160.83 kbps.

Similarly, the visual qualities for a 120-th frame in BQSquare sequence with target bit rate of 780 kbps after being encoded with some state-of-the-art RC methods and the method of the present invention under hierarchical configuration are also compared to investigate the subjective quality improvement. FIG. 8A shows the original framed; FIGS. 8B-8E shows frames encoded with the methods of HM16.8 at actual bit rate of 780.33 kbps, Li et al. [8] at actual bit rate of 780.11 kbps, Wang et al. [9] at actual bit rate of 781.34 kbps and Gao et al. [11] at actual bit rate of 780.95 kbps, respectively; and FIG. 8F shows a frame encoded with the method of the present invention at actual bit rate of 780.17 kbps.

It can be seen that the method of the present invention can produce better visual quality at similar bit rate. Experimental results also show that, compared to the method of the present invention, other state-of-the-art RC methods are more likely to suffer from structural deformation, blocking effects as well as color artifacts, leading to lower visual quality. As a result, the visual quality is obviously degraded. Moreover, the method of the present invention has better quality in the texture areas.

Quality smoothness is another factor influencing the visual quality of experience. FIG. 9 shows a table of experimental results comparing the quality smoothness of frames encoded with some state-of-the-art RC methods and the method of the present invention under N-Hie and Hie configurations, respectively. In the experiment, the standard variance of SSIM, denoted as S_SSIM, is used to evaluate the quality smoothness. From the experimental results, it is observed that Li et al. [8] has the maximum fluctuation. Since Wang et al. [9] and Gao et al. [11] take the frame-coherence into account, they have achieved smoother quality when comparing to Li et al. [8] and HM 16.8. As the accuracy of the frame-coherence is also fully considered in the method of the present invention, minimal fluctuations in terms of S_SSIM, which are 0.0042 and 0.0053 under non-hierarchical and hierarchical configurations, respectively, can be achieved. Therefore, the method of the present invention can ensure quality smoothness compared with other state-of the-art methods based on the parameter inheritance scheme.

Occupancy of buffer is another important factor in rate control, as the overflow and underflow should be avoided. Therefore, stable buffer occupancy is of great importance in evaluating RC performance. The buffer occupancy is mainly determined by the target bits and actual bits and may be indicated with the buffer size, B_uf, which is defined as:

B _uf =D _elay ×T _ar   (17)

where D_elay is the delay time and T_ar is the bandwidth.

FIGS. 10A-10B show two typical buffer occupancies under hierarchical configuration for the video sequences Catus (1080p) and Keiba (832×480), respectively. It can be seen that, the RC methods of Li et al. [8], Wang et al. [9], Gao et al. [11] and HM16.8, have higher buffer occupancy than the method of the present invention. Generally speaking, the method of the present invention can maintain lower buffer, such that the stalling effects can be prevented for better quality of experience.

The accuracy of the bit rate at the frame level is also investigated for mismatch error, which is calculated as follows,

$\begin{array}{cc}\mathrm{Er}=\frac{R_{\mathrm{tar}}-R_{\mathrm{act}}}{R_{\mathrm{tar}}}\times 100\%& \left(18\right)\end{array}$

where R_act and R_tar are the actual bit and the target bit at the frame level.

FIG. 11 shows a table of experimental results comparing the frame level RC accuracies among some state-of-the-art RC methods and the method of the present invention. It can be seen the actual bit rates is very close to the target bit rates for the method of the present invention. When being compared with the methods of Wang et al. [9] and Gao et al. [11], the method of the present invention has higher accuracy. Moreover, although the mismatch error of the method of the present invention is slightly higher than the method of Li et al. [9] and HM16.8, the difference is marginal.

The computational complexities of the method of the present invention and some state-of-the-art RC methods are also compared and evaluated with the computation time of the RC methods which are calculated by:

$\begin{array}{cc}\Delta T=\frac{T_{\mathrm{pro}}-T_{\mathrm{org}}}{T_{\mathrm{org}}}\times 100\%& \left(19\right)\end{array}$

where T_pro and T_org are the encoding time of the scheme of the present invention and HM16.8 anchor.

FIG. 12 shows a bar chart comparing the computation time of the methods. It can be seen that the method of the present invention is more complex than HM16.8, and the additional complexity of the method of the present invention is close to Wang et al.'s algorithm [9].

The robustness of RC algorithm of the method of the present invention and some state-of-the-art methods under hierarchical configuration are also evaluated and compared on video sequences with dynamic scene changes, including Mobisode, Kimono and Tennis.

FIG. 13 shows a table of R-D performance comparison among these RC methods. It can be seen that the method of the present invention outperforms the state-of-the-art RC methods in term of rate-perceptual distortion. Compared to HM16.8, the method of the present invention can achieve 4.0% bit rate savings on average.

FIG. 14 shows a table of frame level RC accuracy comparison among these RC methods. It shows that the methods of Li et al. [9] and Gao et al. [11] can achieve performance improvement compared to HM16.8 due to the consideration of scene change in frame-level rate control. However, regarding the frame-level bit rate accuracy, the method of the present invention achieves better results than the method of Gao et al. [11], and it is close to the method of Li et al. [9]. The experimental results also imply that the model parameter k can well adapt to the video content when scene change happens, and the method of the present invention is robust for the dynamic scene changed sequences.

The methods based on CTU level rate-distortion optimization for rate control in video coding may be implemented in the apparatus described above and can be incorporated into systems including high definition televisions, mobile or personal computing devices (e.g. “tablet” computer, laptop computer, and personal computer), kiosks, printers, digital cameras, scanners or photocopiers or user terminals having built-in or peripheral electronic displays. The apparatus, including the encoder, may include machine instructions for performing the algorithms; wherein the machine instructions can be executed using general purpose or specialized computing devices, computer processors, or electronic circuitries including, but not limited to, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA), and other programmable logic devices. The apparatus may also comprise computer storage media having computer instructions or software codes stored therein which can be used to program computers or microprocessors to perform any of the processes of the present invention. The storage media can include, but are not limited to, floppy disks, optical discs, Blu-ray Disc, DVD, CD-ROMs, and magneto-optical disks, ROMs, RAMs, flash memory devices, or any type of media or devices suitable for storing instructions, codes, and/or data.

The foregoing description of the present invention has been provided for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Various of the above-disclosed and other features and functions, or alternatives thereof, may be combined into many other different systems or applications. Various presently unforeseen or unanticipated alternatives, modifications, variations or improvements therein may be subsequently made by those skilled in the art, each of which is also intended to be encompassed by the disclosed embodiments.

Claims

1. An encoding method in an encoding device using coding tree unit (CTU) level rate-distortion (R-D) optimization for rate control (RC) in video coding, comprising: generating, in an encoding device, multiple frames of a received input video in a frame generating module;generating, in the coding device, one or more coding tree units from at least one frame of the received input video;dividing a frame of the input video into one or more CTUs in a CTU module;determining one or more CTU level coding bits through R-D performance optimization, comprising using a divisive normalization framework to characterize a CTU level perceptual distortion (D′(R)), in relation with a CTU level mean square error (MSE) distortion (D(R)), and a divisive normalized factor (f), by: D′(R)=D(R)/f2,

2. An encoding method according to claim 1, further comprising: dividing each CTU into a number, I, of sub-blocks for Direct Cosine Transform (DCT); andobtaining the divisive normalization factor, f, from Structural Similarity (SSIM) index in DCT domain by:

3. An encoding method according to claim 2, wherein the sub-block size, NL, is set to be 16.

4. (canceled)

5. (canceled)

6. An encoding method according to claim 1, wherein the optimization of R-D performance further comprises: converting a perceptual rate distortion cost function J to:

7. An encoding method according to claim 6, wherein the optimization of R-D performance further comprises: adjusting the one or more optimal CTU level coding bit R*j by:

8. (canceled)

9. An encoding method according to claim 1, wherein the difference between Dreal and Dcomp is expressed as squared error function, e2, defined as: e2=(Dreal−Dcomp)2 and the optimal value of k for the j-th CTU the i-th frame, knew, is obtained by:

10. (canceled)

11. An encoding method according to claim 1, wherein the true distortion Dreal is obtained by: D′(R)=D(R)/f2.

12. An encoding method according to claim 1, wherein the estimated distortion Dcomp for the j-th CTU of the current frame is obtained from the distortion of a co-located j-th CTU in the previously encoded frame.