The present principles relate generally to video encoding and decoding and, more particularly, to methods and apparatus for improved entropy encoding and decoding.
Video coding standards employ prediction and block-based transforms to leverage redundancy in intra/inter frame correlation and achieve high compression efficiency. Furthermore, entropy coding makes the coded bit-stream achieve its entropy boundary and further improves the coding efficiency.
An important usage of entropy coding in video coding system is the coding of the quantized transform coefficients of a block, which is the residual data block after intra/inter prediction, block transform, and quantization. For such data, entropy coding tools have been developed, ranging from variable length coding, such as the Huffman coding, to arithmetic coding. The state-of-the-art CABAC (context-adaptive binary arithmetic coding) achieves high coding efficiency, but the non-systematic implementation of the CABAC coding procedure results in two scanning passes being performed to code a data block.
CABAC is the entropy coding method for the quantized transform coefficient block in the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) Moving Picture Experts Group-4 (MPEG-4) Part 10 Advanced Video Coding (AVC) Standard/International Telecommunication Union, Telecommunication Sector (ITU-T) H.264 Recommendation (hereinafter the “MPEG-4 AVC Standard”). CABAC codes a block in two main passes. In the first pass, CABAC codes the significance map of the block according to a forward zigzag scanning order. In the second pass, CABAC codes the non-zero values in an inverse zigzag scanning order.
Turning to
In the inverse zigzag coding of the non-zero values, two sub-coding processes are used. In the first sub-coding process, a syntax called Bin_1 (i.e., the first bin) is used to indicate whether or not a non-zero coefficient has an absolute value of one. If the non-zero coefficient has an absolute value of one, then Bin_1=1 and the sign of the non-zero coefficient is sent out. Otherwise, Bin_1=0and the encoding moves to the second sub-coding process. In the second sub-coding process, CABAC codes the coefficients which have an absolute value greater than one, corresponding to Bin_1=0, and then sends out their respective signs.
The disadvantage of CABAC is that the corresponding coding involves two scanning passes (i.e., a forward zigzag scan to code the significance map, and an inverse zigzag scan to code values). In addition, the design of CABAC is mainly for smaller block sizes (e.g., 4×4 and 8×8). CABAC turns out to be less efficient for larger blocks (e.g., 16×16, 32×32, and 64×64).
One prior art approach proposes adding a flag to signal the last position of a discrete cosine transform (DCT) coefficient greater than one. However, the prior art approach is restricted to a flag greater than one and still uses two scanning passes.
These and other drawbacks and disadvantages of the prior art are addressed by the present principles, which are directed to methods and apparatus for improved entropy encoding and decoding.
According to an aspect of the present principles, there is provided an apparatus. The apparatus includes a video encoder for encoding at least a block in a picture by transforming a residue of the block to obtain transform coefficients, quantizing the transform coefficients to obtain quantized transform coefficients, and entropy coding the quantized transform coefficients. The quantized transform coefficients are encoded using a flag to indicate that a current one of the quantized transform coefficients being processed is a last non-zero coefficient for the block having a value greater than or equal to a specified value.
According to another aspect of the present principles, there is provided a method in a video encoder. The method includes encoding at least a block in a picture by transforming a residue of the block to obtain transform coefficients, quantizing the transform coefficients to obtain quantized transform coefficients, and entropy coding the quantized transform coefficients. The quantized transform coefficients are encoded using a flag to indicate that a current one of the quantized transform coefficients being processed is a last non-zero coefficient for the block having a value greater than or equal to a specified value.
According to yet another aspect of the present principles, there is provided an apparatus. The apparatus includes a video decoder for decoding at least a block in a picture by entropy decoding quantized transform coefficients, de-quantizing the quantized transform coefficients to obtain transform coefficients, and inverse transforming the transform coefficients to obtain a reconstructed residue of the block for use in reconstructing the block. The quantized transform coefficients are decoded using a flag to indicate that a current one of the quantized transform coefficients being processed is a last non-zero coefficient for the block having a value greater than or equal to a specified value.
According to still another aspect of the present principles, there is provided a method in a video decoder. The method includes decoding at least a block in a picture by entropy decoding quantized transform coefficients, de-quantizing the quantized transform coefficients to obtain transform coefficients, and inverse transforming the transform coefficients to obtain a reconstructed residue of the block for use in reconstructing the block. The quantized transform coefficients are decoded using a flag to indicate that a current one of the quantized transform coefficients being processed is a last non-zero coefficient for the block having a value greater than or equal to a specified value.
These and other aspects, features and advantages of the present principles will become apparent from the following detailed description of exemplary embodiments, which is to be read in connection with the accompanying drawings.
The present principles may be better understood in accordance with the following exemplary figures, in which:
The present principles are directed to methods and apparatus for improved entropy encoding and decoding.
The present description illustrates the present principles. It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody the present principles and are included within its spirit and scope.
All examples and conditional language recited herein are intended for pedagogical purposes to aid the reader in understanding the present principles and the concepts contributed by the inventor(s) to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions.
Moreover, all statements herein reciting principles, aspects, and embodiments of the present principles, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.
Thus, for example, it will be appreciated by those skilled in the art that the block diagrams presented herein represent conceptual views of illustrative circuitry embodying the present principles. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudocode, and the like represent various processes which may be substantially represented in computer readable media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.
The functions of the various elements shown in the figures may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared. Moreover, explicit use of the term “processor” or “controller” should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (“DSP”) hardware, read-only memory (“ROM”) for storing software, random access memory (“RAM”), and non-volatile storage.
Other hardware, conventional and/or custom, may also be included. Similarly, any switches shown in the figures are conceptual only. Their function may be carried out through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, the particular technique being selectable by the implementer as more specifically understood from the context.
In the claims hereof, any element expressed as a means for performing a specified function is intended to encompass any way of performing that function including, for example, a) a combination of circuit elements that performs that function or b) software in any form, including, therefore, firmware, microcode or the like, combined with appropriate circuitry for executing that software to perform the function. The present principles as defined by such claims reside in the fact that the functionalities provided by the various recited means are combined and brought together in the manner which the claims call for. It is thus regarded that any means that can provide those functionalities are equivalent to those shown herein.
Reference in the specification to “one embodiment” or “an embodiment” of the present principles, as well as other variations thereof, means that a particular feature, structure, characteristic, and so forth described in connection with the embodiment is included in at least one embodiment of the present principles. Thus, the appearances of the phrase “in one embodiment” or “in an embodiment”, as well any other variations, appearing in various places throughout the specification are not necessarily all referring to the same embodiment.
It is to be appreciated that the use of any of the following “/”, “and/or”, and “at least one of”, for example, in the cases of “A/B”, “A and/or B” and “at least one of A and B”, is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of both options (A and B). As a further example, in the cases of “A, B, and/or C” and “at least one of A, B, and C”, such phrasing is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of the third listed option (C) only, or the selection of the first and the second listed options (A and B) only, or the selection of the first and third listed options (A and C) only, or the selection of the second and third listed options (B and C) only, or the selection of all three options (A and B and C). This may be extended, as readily apparent by one of ordinary skill in this and related arts, for as many items listed.
Also, as used herein, the words “picture” and “image” are used interchangeably and refer to a still image or a picture from a video sequence. As is known, a picture may be a frame or a field.
Turning to
A first output of an encoder controller 205 is connected in signal communication with a second input of the frame ordering buffer 210, a second input of the inverse transformer and inverse quantizer 250, an input of a picture-type decision module 215, a first input of a macroblock-type (MB-type) decision module 220, a second input of an intra prediction module 260, a second input of a deblocking filter 265, a first input of a motion compensator 270, a first input of a motion estimator 275, and a second input of a reference picture buffer 280.
A second output of the encoder controller 205 is connected in signal communication with a first input of a Supplemental Enhancement Information (SEI) inserter 230, a second input of the transformer and quantizer 225, a second input of the entropy coder 245, a second input of the output buffer 235, and an input of the Sequence Parameter Set (SPS) and Picture Parameter Set (PPS) inserter 240.
An output of the SEI inserter 230 is connected in signal communication with a second non-inverting input of the combiner 290.
A first output of the picture-type decision module 215 is connected in signal communication with a third input of the frame ordering buffer 210. A second output of the picture-type decision module 215 is connected in signal communication with a second input of a macroblock-type decision module 220.
An output of the Sequence Parameter Set (SPS) and Picture Parameter Set (PPS) inserter 240 is connected in signal communication with a third non-inverting input of the combiner 290.
An output of the inverse quantizer and inverse transformer 250 is connected in signal communication with a first non-inverting input of a combiner 219. An output of the combiner 219 is connected in signal communication with a first input of the intra prediction module 260 and a first input of the deblocking filter 265. An output of the deblocking filter 265 is connected in signal communication with a first input of a reference picture buffer 280. An output of the reference picture buffer 180 is connected in signal communication with a second input of the motion estimator 275 and a third input of the motion compensator 270. A first output of the motion estimator 275 is connected in signal communication with a second input of the motion compensator 270. A second output of the motion estimator 275 is connected in signal communication with a third input of the entropy coder 245.
An output of the motion compensator 270 is connected in signal communication with a first input of a switch 297. An output of the intra prediction module 260 is connected in signal communication with a second input of the switch 297. An output of the macroblock-type decision module 220 is connected in signal communication with a third input of the switch 297. The third input of the switch 297 determines whether or not the “data” input of the switch (as compared to the control input, i.e., the third input) is to be provided by the motion compensator 270 or the intra prediction module 260. The output of the switch 297 is connected in signal communication with a second non-inverting input of the combiner 219 and an inverting input of the combiner 285.
A first input of the frame ordering buffer 210 and an input of the encoder controller 205 are available as inputs of the encoder 200, for receiving an input picture. Moreover, a second input of the Supplemental Enhancement Information (SEI) inserter 230 is available as an input of the encoder 200, for receiving metadata. An output of the output buffer 235 is available as an output of the encoder 200, for outputting a bitstream.
Turning to
A second output of the entropy decoder 345 is connected in signal communication with a third input of the motion compensator 370, a first input of the deblocking filter 365, and a third input of the intra predictor 360. A third output of the entropy decoder 345 is connected in signal communication with an input of a decoder controller 305. A first output of the decoder controller 305 is connected in signal communication with a second input of the entropy decoder 345. A second output of the decoder controller 305 is connected in signal communication with a second input of the inverse transformer and inverse quantizer 350. A third output of the decoder controller 305 is connected in signal communication with a third input of the deblocking filter 365. A fourth output of the decoder controller 305 is connected in signal communication with a second input of the intra prediction module 360, a first input of the motion compensator 370, and a second input of the reference picture buffer 380.
An output of the motion compensator 370 is connected in signal communication with a first input of a switch 397. An output of the intra prediction module 360 is connected in signal communication with a second input of the switch 397. An output of the switch 397 is connected in signal communication with a first non-inverting input of the combiner 325.
An input of the input buffer 310 is available as an input of the decoder 300, for receiving an input bitstream. A first output of the deblocking filter 365 is available as an output of the decoder 300, for outputting an output picture.
As noted above, the present principles are directed to methods and apparatus for improved video encoding and decoding. Advantageously, the present principles overcome the non-systematic weakness of CABAC. The present principles also use a binary arithmetic coding engine, but with a systematic coding procedure that saves binary bins, which reduces the use of the binary arithmetic coding engine in both the encoder and the decoder. With this systematic coding method and the reduced binary bins, a simpler coding system and higher compression efficiency is achieved over prior art CABAC systems.
Thus, the present principles are directed to the systematic entropy coding of coefficient blocks with less syntax bins. Systematically, we code the coefficient value when the coefficient value is found (or processed) in a given scanning order. Specifically, we use a “significance flag” (sig_flag) to indicate the zero and non-zero coefficients. For non-zero, we use the “last coefficient greater or equal to 2” (last_ge2_flag) and “last flag” (last_flag) to indicate whether or not the left set of positions includes a number of non-zero coefficients “greater or equal to 2” (ge2) and significant coefficients, respectively. Whenever a significant coefficient is found or indicated by the sig_flag (sig_flag=1), its value is coded immediately. There are at least three benefits to the approach of the present principles:
(1) The whole block is coded in one scan, which makes the system simpler.
(2) Binary bins are reduced and thus some binary arithmetic coding operations are saved at the encoder and the decoder.
(3) Level information of neighboring coded coefficients is available to design the context models for the syntax such as sig_flag, last_flag, last_ge2_flag, and the level bins.
Thus, the entropy coding system disclosed and described is a simpler and more efficient entropy coding system than prior art systems.
In a video coding system, the raw data is processed with intra or inter prediction to remove intra or inter frame correlation, and then processed with a block-based transform such as the 4×4, 8×8, 16×16, 32×32, and 64×64 DCT (or some other transform) to further remove the correlation. Then quantization is applied to the coefficients in the transform blocks. In an embodiment, entropy coding is performed last to code the quantized coefficients of each transformed block in order to provide the same for the output bitstream.
Turning to
To express the block information with binary bins, we use the following syntaxes. For purposes of ease of description, some of syntax elements are borrowed from existing methods, such as CABAC. Additionally, new syntax elements are introduced to enable the present principles as follows:
Turning to
From the above coding example, we see that at least one novel aspect of the described embodiment is the use of the last_ge2_flag. There are several advantages of last_ge2_flag, including at least the following:
1. Last_ge2_flag saves some last_flag. If last_ge2_flag=0, there must be non-zero (specifically greater than one) coefficients in the following scanning positions. Then the last_flag must be 0, so we save these last_flag's until last_ge2_flag=1.
Compare coding the same example block with CABAC and the proposed method. In
2. At the end of the scanning path it is very likely to observe the occurrence of successive so-called trailing ones, i.e., transform coefficient levels with an absolute value equal to one. For the example in
3. Last_ge2_flag saves other Bin_1 for non-trailing ones. When the last_ge2_flag turns from 0 to 1 at some coefficient, the coefficient must be a ge2 (i.e., the coefficient has an absolute value greater than one), so there is no need to send out the Bin_1, which must be 0. An example is the coefficient 2 of the coding example in
Such a saving in Bin_1 only exists in the block where last_ge2_flag turns from 0 to 1. That is, there are more than one last_ge2_flags sent out for the block. Presuming only one last_ge2_flag is sent out for a block, it must be 1, and the corresponding coefficient could have an absolute value of one or ge2. Some example cases are provided in
Turning to
Turning to
Regarding the function block 703, we loop for the coefficients in the block in some scanning order. There is no need to loop for the coefficients after the coefficient with last_flag=1. Regarding the decision block 705, if sig_flag=1 (significant), then we further code the coefficient by the blocks 706-710. Otherwise, we loop for the next coefficient. Regarding the function block 706, the same deals with last_ge2_flag, noting that processing of last_ge2_flag is further described with respect to
Turning to
Turning to
Turning to
Turning to
The coding order in method 700 is the same as the coding order of the example in
Note there is special handling when encoding the coefficient in the last scanning position in the block.
Turning to
Regarding the function block 803, we loop for the coefficients in the block in the same scanning order as the encoder. There is no need to loop for the coefficients after the coefficient with last_flag=1. Regarding the function block 806, the same deals with last_ge2_flag, noting that processing of last_ge2_flag is further described with respect to
It is to be appreciated that in one embodiment, the decoding order of method 800 matches the encoding order of method 700. However, the decoding order could be flexible for function blocks 806-810 as far as matching the encoding order with respect thereto.
Turning to
Turning to
Turning to
Turning to
Note there is special handling when decoding the coefficient in the last scanning position in the block.
Another advantage of the proposed method is the coding of the coefficient level in the same scanning pass of the other syntaxes, such as the sig_flag, last_flag, and so forth. In CABAC, the coefficient level information is coded in the inverse zigzag scanning order, which can be seen as a second pass to code a block. In this inverse zigzag order coding, the level information can be coded with the context models designed with the inverse zigzag position of the coded coefficients. To be concrete, the first coded coefficient (in an inverse zigzag order) is coded with context model 0, the second coded coefficient is coded with context model 1, and so forth. This context model design shows some gains compared to the coding of the level bins with equal probable (0.5/0.5) models (i.e., output the bins directly with 1bit per bin).
This small penalty can be easily compensated with the properly designed context models in the one pass coding method. Given the coefficient level information of the coded coefficients, the context models for the sig_flag, last_ge2_flag, last_flag, Bin_1, and the level bins can be designed based on the known level information from their coded neighbors to further improve the performance of the context models and achieve higher coding efficiency.
Turning to
Turning to
Thus, the present principles advantageously provide methods and apparatus for improved entropy encoding and decoding that systematically codes a quantized transform block. There are at least two novelties associated with this approach. First, the introduced last_ge2_flag reduces the binary bins to be binary arithmetic coded in the encoder and decoder and, thus, a simpler system is achieved. Second, coding of the coefficient value information in the same scanning order with the other syntaxes makes entropy coding of a block finished in one scanning pass. The value information can be used to improve the context models of the syntaxes to further achieve higher coding efficiency.
It is to be appreciated that last_ge2_flag is just an embodiment. As is readily apparent to one of ordinary skill in this and related arts, the flag can alternatively be referred to as last_geX_flag, where X is any number.
A description will now be given of some of the many attendant advantages/features of the present invention, some of which have been mentioned above. For example, one advantage/feature is an apparatus having a video encoder for encoding at least a block in a picture by transforming a residue of the block to obtain transform coefficients, quantizing the transform coefficients to obtain quantized transform coefficients, and entropy coding the quantized transform coefficients. The quantized transform coefficients are encoded in a single pass using a flag to indicate that a current one of the quantized transform coefficients being processed is a last non-zero coefficient for the block having a value greater than or equal to a specified value.
Another advantage/feature is the apparatus having the video encoder as described above, wherein the specified value is 2.
Yet another advantage/feature is the apparatus having the video encoder as described above, wherein subsequent non-zero coefficients from among the quantized transform coefficients having a value less than the specified value are encoded by encoding only the respective signs of the subsequent non-zero coefficients having the value less than the specified value.
Still another advantage/feature is the apparatus having the video encoder as described above, wherein the specified value is selected from among a plurality of values.
Moreover, another advantage/feature is the apparatus having the video encoder wherein the specified value is selected from among a plurality of values as described above, wherein the picture is one of a plurality of pictures included in a video sequence, and the specified value is adaptively selected responsive to derived statistics from previously processed blocks in the picture or in one or more other pictures from among the plurality of pictures in the video sequence.
Further, another advantage/feature is the apparatus having the video encoder as described above, wherein the specified value is explicitly signaled.
Also, another advantage/feature is the apparatus having the video encoder as described above, wherein the specified value is explicitly signaled at at least one of a sequence level, a frame level, a slice level, and a block level.
Additionally, another advantage/feature is the apparatus having the video encoder as described above, wherein a level of the current one of the quantized transform coefficients is encoded by subtracting the specified value from an actual value of the current one of the quantized transform coefficients to obtain a difference value and encoding the difference value as the level, in order to reproduce the level at a corresponding decoder by adding the difference value to the specified value.
Moreover, another advantage/feature is the apparatus having the video encoder as described above, wherein at least a sig_flag syntax element, the flag, a last_flag syntax element, a Bin_1 syntax element, a level syntax element, and a sign syntax element are encoded in a same scanning order, the sig_flag syntax element for indicating whether the current one of the quantized transform coefficients has a non-zero value, the last_flag for indicating whether the current one of the quantized transform coefficients having the non-zero value is a last quantized transform coefficient having the non-zero value in the block in a given scanning order, the Bin_1 syntax element for indicating that an absolute value of the current one of the quantized transform coefficients has a currently unknown non-zero value, the level syntax clement for indicating an absolute value of the current one of the quantized transform coefficients when the current one of the quantized transform coefficients has the absolute value greater than the specified value, and the sign syntax element for indicating a corresponding sign of the current one of the quantized transform coefficients.
These and other features and advantages of the present principles may be readily ascertained by one of ordinary skill in the pertinent art based on the teachings herein. It is to be understood that the teachings of the present principles may be implemented in various forms of hardware, software, firmware, special purpose processors, or combinations thereof.
Most preferably, the teachings of the present principles are implemented as a combination of hardware and software. Moreover, the software may be implemented as an application program tangibly embodied on a program storage unit. The application program may be uploaded to, and executed by, a machine comprising any suitable architecture. Preferably, the machine is implemented on a computer platform having hardware such as one or more central processing units (“CPU”), a random access memory (“RAM”), and input/output (“I/O”) interfaces. The computer platform may also include an operating system and microinstruction code. The various processes and functions described herein may be either part of the microinstruction code or part of the application program, or any combination thereof, which may be executed by a CPU. In addition, various other peripheral units may be connected to the computer platform such as an additional data storage unit and a printing unit.
It is to be further understood that, because some of the constituent system components and methods depicted in the accompanying drawings are preferably implemented in software, the actual connections between the system components or the process function blocks may differ depending upon the manner in which the present principles are programmed. Given the teachings herein, one of ordinary skill in the pertinent art will be able to contemplate these and similar implementations or configurations of the present principles.
Although the illustrative embodiments have been described herein with reference to the accompanying drawings, it is to be understood that the present principles is not limited to those precise embodiments, and that various changes and modifications may be effected therein by one of ordinary skill in the pertinent art without departing from the scope or spirit of the present principles. All such changes and modifications are intended to be included within the scope of the present principles as set forth in the appended claims.
This application claims the benefit of U.S. Provisional Application Ser. No. 61/393,195, filed Oct. 14, 2010, which is incorporated by reference herein in its entirety.
Number | Date | Country | |
---|---|---|---|
61393195 | Oct 2010 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 18142635 | May 2023 | US |
Child | 18745113 | US | |
Parent | 17533235 | Nov 2021 | US |
Child | 18142635 | US | |
Parent | 16878689 | May 2020 | US |
Child | 17533235 | US | |
Parent | 16123219 | Sep 2018 | US |
Child | 16878689 | US | |
Parent | 15791470 | Oct 2017 | US |
Child | 16123219 | US | |
Parent | 15602482 | May 2017 | US |
Child | 15791470 | US | |
Parent | 13879378 | Apr 2013 | US |
Child | 15602482 | US |