Patent Grant
7027654
U.S. Pat. Nos. 6,178,205; 6,167,164; and 6,188,799 entitled “AN EFFICIENT REAL TIME ALGORITHM TO SIMULTANEOUSLY REDUCE BLOCKING AND RINGING ARTIFACTS OF COMPRESSED VIDEO” by Min-Cheol Hong, Chang Mo Yon and Young Man Park, assigned to Digital Media Research Lab IEEE 1999.
The following computer software files, submitted herewith on one (1) CD-R compact disc, are incorporated by reference herein:
This application specifically relates to a method and apparatus for reducing the decode complexity of two dimensional inverse transforms on a vector process.
A typical digital video decoding system involves the following steps (among others).
For each block in a frame:
The 2-dimensional inverse transform functions typically take a large portion of the time to decode a frame due to their complexity.
The invention described here attempts to reduce the decoder complexity on vector processing machines that are capable of doing the same operation to multiple values stored sequentially in a machine's registers by lowering the complexity of the 2 dimensional transform.
A 2-dimensional separable inverse transform performed on a block typically involves performing the following steps:
Since the 1-dimensional inverse transform usually involves performing exactly the same operations on a number of rows or columns in the block, vector processors are often used to reduce the decoding time. This is typically accomplished by filing vector processing registers with a value from each of N rows in the block (see diagram). The operations of the inverse transform are then performed on the N rows in parallel. And then the vector processing registers are filled with values from each of the N columns in the block and the inverse transform is then performed on the N columns in parallel.
In order to fill the vector processing registers quickly with different values from each row a programmer typically has two options:
Choice (a) requires numerous operations to perform the transpose and choice (b) requires numerous bit-mask AND/OR operations to place each coefficient into the register.
This invention attempts to address these issues. To do so:
This works better than performing the transpose or filling the vector registers one at a time as part of the inverse transform because there are typically many more zero than non-zero coefficients. As such we make only a few positional changes, and avoid doing one full transpose altogether.
Additional savings are achieved by an embodiment of this technique whereby the same coefficient for every block in the image is encoded in the bit-stream before moving on to the next coefficient. In this way, one look up can be performed for each coefficient in the transform to determine where to place the transform coefficient.
Since each processor has vector registers with different cardinality it is necessary to place the coefficients in the order that best suits the processor being used.
The specific embodiment uses an IDCT transform but the technique is equally applicable to any separable 2-dimensional transform, for example, the discrete wavelet transform or the generalized orthogonal transform.
The invention includes enhanced video processing and compression and is further described hereinafter.
The encoder uses a motion estimator, block based 8×8 Discrete Cosine Transform (DCT), a quantizer, a variable length encoder, and a loop filter for smoothing block edges in the reconstruction buffer. The decoder uses a variable length decoder and inverse quantizer, a motion compensator and a loop filter for smoothing block edges.
Two separate image artifacts are produced as a result of the quantization step. A blocking artifact is produced when quantization of the DCT coefficients in adjacent blocks produces pixel values on the shared block edge that differ on either side of the edge by a greater amount than in the original image. A ringing or mosquito artifact results from the quantization of higher frequency components of the transform around strong edges in the image. This means that the transform basis vectors do not reinforce and cancel correctly, producing edges in the reconstruction near to the strong edge that were not present in the original image.
The current invention embodies two separate but dependent filters that attempt to remove these image artifacts in a manner that is low on decoder complexity:
A block diagram of the vectorized de-blocker is shown in
The de-blocking filter of
If the following condition is met:
Side1Sad<3*T1/2 and
Side2Sad<3*T2/2 and
Abs(x4−x5)<T3,
then replace values x1 to x8 with low pass filtered values x1 to x8 as follows:
x1′=(x0+x0+x0+x1*2+x2+x3+x4+4)/8
x2′=(x0+x0+x1+x2*2+x3+x4+x5+4)/8
x3′=(x0+x1+x2+x3*2+x4+x5+x6+4)/8
x4′=(x1+x2+x3+x4*2+x5+x6+x7+4)/8
x5′=(x2+x3+x4+x5*2+x6+x7+x8+4)/8
x6′=(x3+x4+x5+x6*2+x7+x8+x9+4)/8
x7′=(x4+x5+x6+x7*2+x8+x9+x9+4)/8
x8′=(x5+x6+x7+x8*2+x9+x9+x9+4)/8
The invention encompasses the following novelty:
b) The edge enhancement and de-ringing filter works as follows:
The strong de-ringing filter works as follows:
Calculate a maximum blurring modifier (HighModifier) and maximum sharpening modifier (LowModifier) by looking up a value based upon the level of quantization applied to the coefficients.
For the jth pixel in the ith image row, Pi,j, do the following:
Where DeringModifier(X,&) is defined as follows:
Replace Pixel Pi,j with Pi,j′ computed as follows:
Pi,j′=(M0*Pi−1j+M1*Pi+1j+M2*Pij−1+M3*Pi,j+1+(128−(M0+M1+M2+M3)*Pi,j)/128
The deblocker uses simple linear calculations and one dimensional filters to remove deblocking artifacts and to gather information used by the deringer filter to determine how many iterations to apply its filter. The deringing filter encompasses a pixel to pixel spatially adaptive filter that can both blur and sharpen. It does so by collecting the difference between neighboring pixels into a two dimensional array, applying a simple function to this array and then using the result as a convolution kernel. Since the function can produce both positive and negative tapes, the filter can perform both deringing and sharpening.
The Weak de-ringing filter works exactly the same as above except that the High and Low Modifiers are smaller magnitude numbers and the DeringModifier Function works as follows:
Enclosed with this application is a CD-Rom with the preferred embodiment illustrated by providing a listing in Source Code of the method, system and steps of this invention. Following a reading of said CD-Rom, the invention herein is again summarized.
The novelty of this approach in comparison to others found in prior art includes:
In summary, the following table identifies some of the advantages, features and benefits of this invention.
Having described this invention with regard to specific embodiments, it is to be understood that the description is not meant as a limitation since further modifications and variations may be apparent or may suggest themselves to those skilled in the art. It is intended that the present application cover all such modifications and variations as fall within the scope of the appended claims.
This nonprovisional application claims domestic priority to prior provisional application Ser. No. 60/312,941, filed Aug. 16, 2001, and to provisional application Ser. No. 60/316,316, filed Aug. 31, 2001. Both provisional applications are incorporated herein by reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 6041145 | Hayashi et al. | Mar 2000 | A |
| 6167164 | Lee | Dec 2000 | A |
| 6188799 | Tan et al. | Feb 2001 | B1 |
| 6240135 | Kim | May 2001 | B1 |
| 6529638 | Westerman | Mar 2003 | B1 |
| 6707952 | Tan et al. | Mar 2004 | B1 |
| Number | Date | Country | |
|---|---|---|---|
| 60316316 | Aug 2001 | US | |
| 60312941 | Aug 2001 | US |
