Patent Application
20110222608
This invention relates to in-loop filtering methods, and more particularly, to in-loop filtering methods that employ hybrid video coding.
Modern audio-visual systems utilize digital coding to reduce data size. Digital coding methods utilize both intra and inter prediction techniques to compress and decompress data for picture frames. These frames are coded, and reconstructed and may be utilized to predict data for a next frame. Reconstructed data forming a frame can often have blocking artifacts at the corners of macroblocks; therefore, most modern encoders and decoders utilize a deblocking filter in order to remove these blocking artifacts. A common technology is H.264. Unfortunately, while blocking artifacts at block boundaries can be removed, quantization errors will still remain.
Therefore, modern coding techniques employ Wiener filtering, which acts to generate filter coefficients according to Wiener-Hopf equations, such that a reconstructed frame can be generated that has reduced quantization errors. For a typical case, an input signal s will undergo some coding processes, introducing additive quantization noise e, which generates a noisy signal s′. The Wiener filter filters the noisy signal s′ to generate a filtered signal ŝ wherein the mean square error between s and s′ is minimized, such that the filtered signal ŝ will be closer to the original signal s.
Wiener filter information is also required at the decoder side to reconstruct the original video. For example, Wiener filter coefficients used by the encoder side are sent to the decoder side as the Wiener filter information. Even with this advanced filtering, however, there can be discrepancies within a particular frame. A single Wiener filter may not provide the optimized filtering for every macroblock within a frame.
Therefore, it is an objective of the present invention to provide adaptive loop filtering methods that are able to utilize a plurality of Wiener filters within a single frame.
A method for coding a parent block of video data comprises: reconstructing the parent block for prediction; providing at least two filters for filtering the parent block after reconstructing the parent block; dividing the parent block into block partitions, wherein at each block partition determining which of the filters is to be used to filter the block partition; and filtering each of the block partitions with the filter determined accordingly.
A method for coding a parent block of video data comprises: reconstructing the parent block for prediction; dividing the parent block into block partitions according to a partitioning method; determining one of a plurality of filtering methods for each of the block partitions; and filtering each of the block partitions with the corresponding filtering method; wherein the partitioning method is inferred from the parent block.
An encoder for encoding a parent block of video data comprises: a reconstruction block, for reconstructing the parent block for prediction; at least two filters for filtering the parent block after reconstructing the parent block; and a filtering information generation unit, for dividing the parent block into block partitions, wherein at each block partition the filtering information generation unit determines which of the filters is to be used to filter the block partition.
A decoder for decoding a parent block of video data comprises: a decoding entropy unit, for decoding a coded bitstream to retrieve the parent block of video data; a reconstruction block, for reconstructing the parent block for prediction; at least two filters for filtering the parent block after reconstructing the parent block; a filtering information generation unit, for dividing the parent block into block partitions, wherein at each block partition the filtering information generation unit determines which of the filters is to be used to filter the block partition.
The present invention aims to provide coding methods that can utilize different filters to code a single frame. Provided below are different embodiments for carrying out the objectives of the invention. In the following embodiments, Wiener filters are utilized to illustrate the methods and workings of the invention; please note, however, that any other filters that can perform the filtering methods detailed in the following also embody the spirit of the invention. Therefore, the Wiener filter should be taken as merely one example and not a limitation of the invention.
Please refer to
The decoder 300 comprises an entropy coding unit 380, an intra prediction unit 305, an inter (Motion Compensation; MC) prediction unit 310, an inverse transform unit 327, an inverse quantization unit 329, a reconstruction unit 330, a deblocking unit 340, a Wiener filtering unit 360, and a frame buffer 350. The entropy coding unit 380 receives a coded bitstream to decode and extract prediction information such as intra mode information and inter mode information, Wiener filtering information, and residues. The Wiener filtering information is sent to the Wiener filtering unit 360 for selecting a Wiener filter from a plurality of Wiener filters for each block partition of a parent block. The Wiener filtering unit 360 will select the same Wiener filter used at the encoder side according to the Wiener filtering information. The Wiener filtering information may comprise filter coefficients for the Wiener filters corresponding to the parent block, and the Wiener filtering information may also comprise block partition information for filtering processes.
Please refer to
It is possible, however, that even within one picture there exist at least two areas that both require Wiener filtering but have different statistical properties and therefore require different Wiener filters. If more than one Wiener filter can be utilized on a parent block (such as a picture), then even smaller errors between original signals and filtered signals can be achieved. The parent block is divided according to quadtree partitioning, or any other partitioning methods.
Please refer to
At the encoder side, the filter coefficients for both the 1st and the 2nd Wiener filters will be generated, and this information as well as the block partition information may be sent to the decoder side. It is also possible that the decoder side derives the filter coefficients and block partition information according to video data and/or information carried in the bitstream. The encoder side generates a filter index for each block partition, which is utilized alongside the block partition information to inform the decoder which Wiener filter has been used for each partition. It is possible to embed this information in the slice header so it is received first by the decoder 300, or to embed it elsewhere in the bitstream and have a pointer that indicates the location of this Wiener information so that the decoder 300 can access the information at the start of the decoding process.
Please refer to
In this embodiment, the decoder is informed of the Wiener filter coefficients and block partition information and also whether or not Wiener filtering takes place. Therefore, in this embodiment, the filter index also carries the information of whether the block partition is filtered, for example, the filter index incorporates a flag which indicates that no filtering occurs. As in the first embodiment, this information can be transmitted in the slice header, or in another part of the bitstream wherein a pointer will be utilized to indicate the location of the information.
A filter calculation and selection example for the first and second embodiments is described below. Video data (e.g. pixel values) of a picture are divided into a dark part and a bright part, and the encoder trains two filters for the dark part and the bright part respectively. Block partitions belonging to the dark part are filtered by one filter and block partitions belonging to the bright part are filtered by another filter. Another filter calculation and selection example is to try both a first filter which is a standard filter and a second filter which is an optimal filter trained from a previous picture, and select the filter with a smallest error between the original signal and the filtered signal.
A third exemplary embodiment of the present invention determines which Wiener filter to use (or whether to use no filtering) according to statistics. As long as the way of determining the filter (e.g. filter coefficients) is the same at both the encoder and decoder side, it is not necessary to send the filter index to the decoder. This reduces the amount of information that needs to be sent in the bitstream.
At the decoder side, the filtering method for each block partition can be derived from the bitstream. In a first modification of this embodiment, the decoder will perform the same operation as the encoder in order to determine which filtering method is optimal for each block partition of a parent block. In a second modification, it is possible for the encoder to send some information (which will be significantly smaller than the filter index information of the first and second embodiments) to the decoder, which will then perform statistical computations on this information in order to determine the filtering methods employed.
A typical example is given here. A number of block partitions, i, within a current picture is between 1 and K, and a statistical value of each block partition, f(i), is calculated. These statistical values are then sorted in ascending or descending order and fitted into a number of bins, N. For each separate bin, a separate set of filter coefficients is utilized (i.e. a different Wiener filter is utilized for each bin). Data of block partitions within a bin will be trained and filtered by the same filter (i.e. employ the same filter coefficients). The bin sorting method can be predefined and known by both encoders and decoders, or the bin information can be sent to the decoder, which can therefore sort each block partition and filter accordingly. For example, the bin information includes N−1 threshold values, so that the decoder can sort the statistical values properly by comparing the statistical values with the threshold values.
Possible statistical values that can be utilized in the above example include: mean value of the reconstructed signal/residual signal/prediction signal, variance value of the reconstructed signal/residual signal/prediction signal, dynamic range of the reconstructed signal/residual signal/prediction signal (which means the absolute value of the difference between the maximum value and the minimum value within each block partition), etc.
In a fourth exemplary embodiment, block partition information for filtering process can be derived by decoders, and therefore, the encoder no longer needs to insert block partition information in the bitstream. A plurality of filters are designed for corresponding block partitions of a picture, and the corresponding block partitions can be either created simultaneously with the generation of the filters or inferred from the block-based structure in coding the picture. For example, the block partition for filtering processes can be the same as the coding unit partition, transform unit partition, or a predefined partition known by both encoders and decoders. Embodiments of the present invention are not limited to quadtree partitioning. In an example, the encoder transmits coefficients of a plurality of filters to the decoder as side information, and the encoder also transmits corresponding block partition information along with the filter coefficients. The corresponding block partition information may be a flag indicating the way of creating the block partitioning for filtering process. In another example, the encoder inferred from the block-based coding structure of the picture as block partition for filtering process, and the decoder utilizes the block-based coding structure for block partitioning of filtering process.
In the above detailed description, the level at which the block partitioning is performed will be called a parent block. It is to be understood that a parent block may mean an entire frame, or a frame may comprise many parent blocks.
To summarize, the above methods describe means for utilizing at least two filters to filter a parent block, wherein the parent block will be sub-divided into block partitions utilizing quadtree partition or other partition methods, and block partition information, and filter indexes are optionally sent to the decoder side. For example, the block partition information for filtering processes can be derived by the decoder in some embodiments, and a filter index for each block partition is not included in the bitstream as the type of filtering can be derived by the decoder in some other embodiments. These methods may offer greater coding accuracy by further reducing mean quantization errors between original signals and filtered signals than in systems where only one filter is used per parent block. These methods may also offer greater coding efficiency by having the decoders derive certain filtering information to reduce the bitstream data size.
Those skilled in the art will readily observe that numerous modifications and alterations of the device and method may be made while retaining the teachings of the invention.
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/CN10/79002 | 11/23/2010 | WO | 00 | 12/14/2010 |
| Number | Date | Country | |
|---|---|---|---|
| 61313840 | Mar 2010 | US |
