A transform T transforms a block of n pixels into n transformed coefficients. The process is invertible by applying the inverse transform T−1 to the transformed coefficients to get the pixel values back
In some video coding standards, transformed coefficients are put into a two dimensional (2D) block to which is associated a scanning order; typically, from high frequency to low frequency. After quantization of the coefficients, the first, relative to the high to low frequency scan order, non-zero quantized coefficient is called the last coded coefficient, as shown on
In the 2D block topology associated to the coefficients, the last coded coefficient has two coordinates (x,y) as shown on
Additional information indicating significance of transformed coefficients may be added to signal whether the remaining coefficients (question-marked on
Conventionally, in many video and image codecs developed in the last decades, fixed transforms, such as the Discrete Cosine Transform or Discrete Sine Transform, for example, are applied to the pixels of each block to obtain the transformed coefficients. These coefficients are then quantized by a quantifier Q to obtain quantized coefficients which are encoded by an entropy coder, such as VLC, arithmetic coder, or Context Adaptive Binary Arithmetic Coding (CABAC), for example.
It is desired to encode the last coded coefficient position as efficiently as possible.
The HEVC/H.265 standard introduced the coding of the last coded coefficient position by using two coordinates (x,y). Each coordinate is binarized using a truncated unary code, and then each bit, or “bin” in the HEVC terminology, is encoded using CABAC with channel adaptation based on contexts. In HEVC, the two coordinates x and y are encoded and decoded separately.
These and other drawbacks and disadvantages of the prior art are addressed by the present principles, which are directed to a method and apparatus for advanced CABAC context adaptation for last coefficient coding.
According to one aspect of the present principles, a method for coding a set of transformed coefficients is provided comprising a step of transforming a block of image values to obtain transformed coefficients, and a set of entropy coding a position of a last coded coefficient wherein the position of the last coded coefficient is provided by two coordinates such that the entropy coding of the y coordinate depends on the value of the x coordinate.
According to another aspect of the present principles, an apparatus for coding a set of transformed coefficients is provided comprising a transform circuit operating on a block of image values to obtain transformed coefficients, and an entropy coder wherein the position of a last coded coefficient of the transformed coefficients is provided by two coordinates such that the entropy coding of the y coordinate depends on the value of the x coordinate.
According to another aspect of the present principles, a method is provided for decoding a set of transformed coefficients. The method comprises entropy decoding the transformed coefficients to generate a position of a last coded coefficient, wherein the position of the last coded coefficient is provided by two coordinates (x,y) characterized in that said entropy decoding of a y coordinate depends on the value of an x coordinate, and a step of inverse transforming transformed coefficients to obtain a block of image values.
According to another aspect of the present principles, an apparatus is provided for decoding a set of transformed coefficients. The apparatus comprises an entropy decoder that operates on a code representing a position of a last coded coefficient from the transformed coefficients, wherein the position of the last coded coefficient is provided by two coordinates (x,y) characterized in that the entropy coding of a y coordinate depends on the value of an x coordinate. The apparatus further comprises an inverse transform circuit operating on transformed coefficients to obtain a block of image values.
According to another aspect of the present principles, a non-transitory computer readable storage medium is provided having stored thereon instructions for decoding a set of transformed coefficients such that the position of the last coded coefficient is provided by two coordinates (x,y) characterized in that an entropy coding of a y coordinate depends on the value of an x coordinate.
According to another aspect of the present principles, a non-transitory computer readable storage medium is provided having stored thereon a bitstream for decoding a set of transformed coefficients such that the position of the last coded coefficient is provided by two coordinates (x,y) characterized in that an entropy coding of a y coordinate depends on the value of an x coordinate.
These and other aspects, features and advantages of the present principles will become apparent from the following detailed description of exemplary embodiments, which are to be read in connection with the accompanying drawings.
The technical problem solved by the following embodiments is to reduce the cost of coding the position of the last coded coefficient in a transformed block of pixels to which a 2D transform has been applied.
These embodiments are an improvement of the entropy coding scheme commonly used in video coding standards. One such video coding standard is the HEVC/H.265 standard, but the embodiments are not limited to that standard.
The main idea of the present principles is to choose the contexts used to code the coordinate y depending on the value x. This is obviously decodable as one decodes x first, then y. This leads to better coding performance because prior methods do not take into account the dependence between x and y. For instance, the described idea specifically helps coding blocks with one DC coefficient only (x=y=0) relative to blocks with more coefficients in the case x=0 and y>0. In brief, the y channel statistics are better modeled.
In particular, if one uses partial transforms, it implicitly takes into account the fact that not all coordinate pairs (x,y) are acceptable, such that the coding of (x,y) is more efficient. In this particular embodiment with partial transforms, tests have been conducted on a modified HEVC standard using partial transforms and it has been shown that an improved coding of (x,y) leads to a compression gain of about −0.5%
The ideas presented herein propose to infer the context used to code y bins depending on the value of x.
For instance, as in
Once x is coded as in HEVC, the context of y will depend on x. In the example above, the value y=2 is binarized by 001 using a unary code and each of the three bins is coded using dedicated contexts C0, C1 and C2. In HEVC, the contexts do not depend on x, such that C0 is representative of the probability P(y=0) that y is zero. Under the present principles, this context is replaced by C0, x=2 that depends on x, such that it is representative of the probability P(y=0|x=2) instead.
This requires an increase of the number contexts used by the codec. In a variant, these increases may be limited by inferring the y contexts, not by the precise value of x, but by a range of values for x. For instance Ci,x=0 is chosen depending on whether or not x is zero. Also, this inference can be effective for a limited number of bins, for instance the first few indexes i only.
In HEVC, a coordinate x or y is split into a prefix and a suffix as shown in
For instance, if the coordinate value is 14, the prefix is 7 and the suffix is coded in 2 bits. The suffix is the remainder after the first value is subtracted from the coordinate value, 14−12=2. If the coordinate value is 3, then the prefix is 3, but there is no suffix.
The prefix is binarized using a truncated unary coding. The truncation is performed based on the knowledge of the block size that provides an upper bound for the prefix value. Each bit of the binarized prefix is encoded using CABAC and a dedicated context.
For instance, in a 4×4 block the possible prefix are 0, 1, 2 or 3 which are respectively binarized into 1, 01, 001 and 000 (truncated). In another example, in a 8x8 block, the possible prefix are from 0 to 5 and binarized into 1, 01, 001, 0001, 00001 and 00000 (truncated).
The suffix is binarized using a fixed length coding, and the fixed length suffix is encoded as is, using CABAC in bypass mode without using contexts.
For instance, if the coordinate value is 14 in a 16×16 block, the prefix is 7 (binarized into 0000000) and the suffix is 2 (binarized into 10 and coded on 2 bits).
The binarization process into prefix and suffix is kept as in HEVC, using contexts only for the prefix. The main characteristic of the present principles is that the contexts of y depend on the value of x. Of course, this works only for the prefix of y because the suffix is coded without using any context (a bypassed CABAC is used instead). Nothing is changed for the suffix.
Usually, a transform T transforms a block of n pixels into m=n transformed coefficients. The process is invertible by applying the inverse transform T−1 to the transformed coefficients to get the pixel values back. In case of a partial transform P, the n pixels are transform into less m<n transformed coefficients. Equivalently, it may be assumed the lacking m−n coefficients are set to zero. An “inverse” transform P′ (of course not the mathematical inverse because the partial transform is not invertible) is applied to the transformed coefficient to obtain an approximation of the initial pixel values. Typically, partially transformed coefficients are representative of the low frequency information of the pixel block.
If a partial transform is used, it is known that some m−n transformed coefficients are necessarily zero, thus imposing some constraints on the position of the last coded coefficient and on the coordinates (x,y). In this case, by implicitly using these constraints, it is shown that the embodiment automatically handles the efficient encoding of the two coordinates (x,y) in case a partial transform is used to transform the TU pixels to transformed coefficients.
Instead of the systematic transforms such as DCT or DST, an adaptive set of orthogonal transforms may be used instead that are learned offline on a large training set using different classification and transform optimization schemes. This set of transforms is fed to a codec and the best transform out of a set is chosen in a Rate Distortion Optimization (RDO) loop. A more adaptive approach is to learn a set of orthogonal transforms for a particular intra frame of a sequence. This is referred to as an on-the-fly block based transform learning scheme in the rest of the specification. This scheme is shown in
The block diagram of
Due to the energy compaction property of SVD, it is observed that the overhead cost can be considerably reduced by deducing an incomplete representation of the learned transform where, only first ‘m’ vectors are transmitted to the decoder and the remaining (n−m) transform vectors are either generated using a completion algorithm similar to the Gram-Schmidt method or forced to zero, thus leading to a partial transform. In order to illustrate the effect of dropping the last few vectors of a transform on the Bjontegaard Distortion rate (BD-rate), four non-separable optimized transforms of size 64×64 are learned on the 4K sequences ‘PeopleOnStreet’ and ‘Traffic’. Encoding tests are performed on these sequences where the first ‘m’ vectors are retained and the rest of the basis vectors are then completed using a completion algorithm.
It is observed in
Since, the performance drop in terms of BD-rate when encoding only first ‘m’ transform vectors depends on the video content, a content-adaptive method is necessary to estimate the optimal value of ‘m’. Intuitively, at low bit-rates, most of the coefficients are quantized to zero and at high bit rate, the coefficients' energy is significant even at high frequencies. Therefore, the value of ‘m’ depends on the content as well as the quantization parameter QP.
The residual signal energy on the average is concentrated more in the first few coefficients where, the DC coefficient has on the average maximum energy and it decreases as we go towards the higher frequency coefficients. Therefore, most of the high frequency coefficients are quantized to zero. A simple threshold based method can be applied in order to compute the best value of ‘m’ which is required to be coded along with the frame as an overhead.
Let E be the sum of energy of the DCT coefficients. A threshold t is defined by the multiplication of a parameter p with E to get,
t=p·E
The value of ‘m’ can simply be computed from the number of coefficients with average energy greater than this threshold. The value of ‘p’ can be found experimentally. Table 1 shows the variation of the number of vectors ‘m’ above this threshold for a chosen value of ‘p’ in terms of percentage of total number of vectors that are encoded. It is observed from Table 1 that at high QP, the average number of vectors required are much less compared to that required at low QP. Moreover, the number of vectors ‘m’ also varies depending on the content.
When the present principles are applied using partial transforms, the range of the y prefix depends on x and the contexts of y adapts automatically.
Consider again the example of
In HEVC, this context does not depend on the value of x. Instead, following the principles described herein, the contexts used for the coding of the bin for the prefix y does depend on the value of x. So, in this example, when x=3, the only possible value for the prefix y is 0 because of the partial transform that forces the other coefficients to be zero in the last column where x=3, and the coded first bit for y is 0 all the time.
As a consequence, the x=3-dependent context sees a channel with only zeros and it adapts the statistics of the channel such that after a few codings of the TU, the cost of coding a 0 becomes just negligible in term of bit-rate. This is how the context adapts implicitly to code efficiently this first bit y that, ideally, should not be coded at all because one knows from the implicit transform that x=3 infers y=0.
The embodiment with partial transforms easily extends to a coordinate y that uses a suffix, as in
In the HEVC/H.265 standard, a new tool for coding binary data has been proposed in the arithmetic coder, namely the Context-Adaptive Binary Arithmetic Coding (or CABAC). A binary symbol s, which takes value 0 or 1, is coded following a probability p to be 1 and 1−p to be 0. This probability is deduced form a context and is adapted after each symbol coding.
A context value is an 8 bit value, see
Most Probable Symbol (or MPS) and the next 7 bits represent a probability p′ (or state) from which the probability p is deduced.
The update of the context value is made following the process described in
The evolution is made through two tables, transIdxMPS if the coded symbol is the MPS, and transIdxLPS if the coded symbol is not the MPS, i.e., it is the Least Probable Symbol (LPS). These tables are provided in Table 2 for the entry p′, whose name is pStateIdx.
The probability pMPS of the symbol s to be the MPS is quantized linearly on 8 bits, from 0 to 127. It is deduced from the context value by
p
MPS=(p′+64)/127=(pStateIdx+64)/127
and the probability p of the symbol s to be 1 is deduced obviously from pMPS depending on the value of the MPS.
p=pMPS if MPS=1,
p=1−pMPS if MPS=0.
Context-Adaptive coding is a powerful tool that allows the coding to dynamically follow the statistics of the channel to which the symbol belongs. Also, each channel should have its own context in order to avoid mixing statistics and losing the benefit of the process. This has led to the extensive use of many contexts in HEVC/H.265, which uses several hundreds of contexts, in order to model many channels. For instance, among all channels using contexts, there are
All of these contexts/channels also largely depend on the color channel, that is whether the channel is luma or chroma, the Transform Unit size, the position of the transformed coefficient, the neighboring symbol values and other factors.
As an example, a Coding Group (CG) coding flag is chosen depending on whether or not the CG coding flags below and at the right of the current CG are 1 or not, as shown in
All in all, the context choice depends on many things, thus the huge number of contexts. Of course, the decoder must update the context values to correspond with what was performed on the encoder side to ensure synchronization with the encoder and parsing of the stream.
The embodiments described herein propose to add a dependency on the context choice for the coordinate y of the last coded coefficient, namely, this dependency is the value of the coordinate x. It is to be noted that this context choice also depends on (as standardized in HEVC) the color channel (luma or chroma), the TU size, and the scanning order.
There is no specific syntax needed to implement the described principles, except in the case of partial transforms that may be signaled in the bit-stream.
Regarding the decoding process, the principles described herein impact the number of contexts associated to the last coded coefficient position and the process to decode the last coded coefficient position.
In the embodiment with the partial transforms, the improved binarization and truncated unary coding of the y suffix is also impacted.
In addition to the above features of the present principles, the described embodiments provide that the entropy coding of the coordinate y is performed by CABAC and associated contexts, the dependence is an inference of the choice of the contexts by the value of x, the entropy coding of the coordinate y is performed using a prefix and a suffix, the dependence is an inference of the choice of the contexts used to code the prefix of y by the value of x, and as a variant, the transform is a partial transform and the suffix y binarization is truncated using the knowledge of the size m of the partial transform.
The proposed idea is normative in the sense that it is present in the syntax of the video stream and implies a decoding method to be applied. As such, the present ideas can be implemented in a video standard, such as the successor to HEVC.
One embodiment of a method 1100 for coding a set of transformed coefficients is shown in
One embodiment of an apparatus 1200 for coding a set of transformed coefficients is shown in
An embodiment of a method 1300 for decoding a set of transformed coefficients is shown in
An embodiment of an apparatus 1400 for decoding a set of transformed coefficients is shown in
The aforementioned embodiments can be implemented in Set Top Boxes (STBs), modems, gateways or other devices that perform video encoding or decoding.
The functions of the various elements shown in the figures can be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared. Moreover, explicit use of the term “processor” or “controller” should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (“DSP”) hardware, read-only memory (“ROM”) for storing software, random access memory (“RAM”), and non-volatile storage.
Other hardware, conventional and/or custom, may also be included. Similarly, any switches shown in the figures are conceptual only. Their function may be carried out through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, the particular technique being selectable by the implementer as more specifically understood from the context.
The present description illustrates the present principles. It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody the present principles and are included within its spirit and scope.
All examples and conditional language recited herein are intended for pedagogical purposes to aid the reader in understanding the present principles and the concepts contributed by the inventor(s) to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions.
Moreover, all statements herein reciting principles, aspects, and embodiments of the present principles, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.
Thus, for example, it will be appreciated by those skilled in the art that the block diagrams presented herein represent conceptual views of illustrative circuitry embodying the present principles. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudocode, and the like represent various processes which may be substantially represented in computer readable media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.
In the claims hereof, any element expressed as a means for performing a specified function is intended to encompass any way of performing that function including, for example, a) a combination of circuit elements that performs that function or b) software in any form, including, therefore, firmware, microcode or the like, combined with appropriate circuitry for executing that software to perform the function. The present principles as defined by such claims reside in the fact that the functionalities provided by the various recited means are combined and brought together in the manner which the claims call for. It is thus regarded that any means that can provide those functionalities are equivalent to those shown herein.
Reference in the specification to “one embodiment” or “an embodiment” of the present principles, as well as other variations thereof, means that a particular feature, structure, characteristic, and so forth described in connection with the embodiment is included in at least one embodiment of the present principles. Thus, the appearances of the phrase “in one embodiment” or “in an embodiment”, as well any other variations, appearing in various places throughout the specification are not necessarily all referring to the same embodiment.
Number | Date | Country | Kind |
---|---|---|---|
16305918.1 | Jul 2016 | EP | regional |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/EP2017/067602 | 7/12/2017 | WO | 00 |