1. Field of the Invention
The present invention relates to a method of coding data, and more particularly, to a method of removing blocking artifacts when coding image signals such as in a moving picture at low-bit-rate.
2. Background of the Related Art
Generally, to efficiently compress a time variable video sequence, it is necessary to remove redundancy in the temporal domain as well as in the two-dimensional spatial domain. In moving picture experts group (MPEG), discrete cosine transform (DCT) is used to remove the redundancy in the two-dimensional spatial domain while a motion compensation method is used to remove the redundancy in the temporal domain.
The DCT is a method of removing the correlativity between data through a two-dimensional spatial transformation. Each block in a picture is spatially transformed using the DCT after the picture is divided into blocks. Data that has been spatially transformed tends to be driven to a certain direction. Only a group of the data driven in the certain direction is quantized and transmitted.
Pictures, which are consecutive in the temporal domain, tend to form motions of a human being or an object at the center of the frame. This property is used to reduce the redundancy of the temporal domain in the motion compensation method. A volume of data to be transmitted can be minimized by taking out a similar region from the preceding picture to fill a corresponding region, which has not been changed (or has very little change), in the present picture. The operation of finding the most similar blocks between pictures is called a motion estimation. The displacement representing a degree of motion is called a motion vector. MPEG uses a motion compensation-DCT method so that the two methods combine.
When a compression technique is combined with a DCT algorithm, the DCT transform is usually performed after input data is sampled in a unit size of 8×8, and the transform coefficients are quantized with respect to a visual property using quantization values from a quantization table. Then, the data is compressed through a run length coding (RLC). The data processed with the DCT is converted from a spatial domain to a frequency domain and compressed through the quantization with respect to the visual property of human beings, not to be visually recognized. For example, since eyes of human beings are insensitive to a high frequency, a high frequency coefficient is quantized in a large step size.
For the quantized data, the data having a relatively high frequency is coded with a short code word. The quantized data having a low frequency is coded with a long code word. Thus, the data is finally compressed.
In processing a moving picture as discussed above, blocks are individually processed to maximize the compression ratio and coding efficiency. However, the individual process causes blocking artifacts that disturb the eyes of human beings at boundaries between blocks.
A related art method of removing blocking artifacts will be described with reference to
Various algorithms have been presented for removing blocking artifacts that appear in a coding system, which individually processes blocks. For example, MPEG-4 used a deblocking filter by Telenor, which uses the following algorithm:
If B is replaced with B1 and C is replaced with C1,
B1=B+d1,
C1=C−d1, and
d1=sign(d)*(MAX(0,|d|−MAX(0,2*|d|−QP)))
where d=(3A−8B+8C−3D)/16 and QP denotes the quantization parameter of the macroblock where pixel C belongs.
In processing a MPEG-4 moving picture, blocking artifacts are removed using the above algorithm to improve picture quality. However, it is difficult to effectively remove the blocking artifacts with the above with a small operation capacity in a real time operation. For example, coding and decoding a moving picture is a real time operation. In other words, to completely remove the blocking artifacts, a large calculation amount is needed, which is undesirable in efficiency.
Alternatively, to remove the blocking artifacts, there is provided a method of changing processes of coding and decoding. This method increases the amount of bits to be transmitted.
Still another method for removing blocking artifacts is based on the theory of projection onto convex sets (POCS). However, this method is applied only to a still picture because of an iteration structure and long convergence time.
Thus, the related art methods for removing blocking artifacts in a coding system of a moving picture have several problems. First, in performing an algorithm for removing the blocking artifacts, a calculation is complicated and the calculation amount and time become correspondingly large. Further, the blocking artifacts are not removed in either complex regions or smooth regions in a picture. In addition, the amount of bits to be transmitted increases.
An object of the present invention is to provide a method of removing blocking artifacts in a coding system that substantially obviates one or more of the problems due to limitations and disadvantages of the related art.
Another object of the present invention is to remove blocking artifacts when necessary in a smooth portion of a moving picture.
Yet another object of the present invention is to provide a method of removing blocking artifacts in a coding system of a moving picture where blocking artifacts of the moving picture are removed at real time using frequency features around a block boundary without increasing the amount of bits.
To achieve at least the above objects in a whole or in parts, a method of removing blocking artifacts in a coding system according to the present invention includes determining at least pixel sets S0, S1, S2 around a block boundary, selecting one of a default mode and a DC offset mode as a deblocking mode based on an amount of blocking artifacts, deblocking filtering pixels adjacent the block boundary if a default mode is selected, deblocking filtering of pixels adjacent the block boundary if a default mode is selected, and removing artifacts in the DC offset mode when the DC offset mode is selected and a DC offset mode condition is satisfied, where the artifacts are removed in the DC offset mode according to the following equation:
Pm=(|v1−v0|<QP)?v0:v1,
If m<1;
vm, if 1≦m≦8;
(|v8−v9|<QP) v9:v8, if m>8;
{bk:−4≦k≦4}={1,1,2,2,4,2,2,1,1}//16,
wherein v0−v9 are boundary pixels, QP is the quantitation parameter of a block adjacent the block boundary, and vm is an adjusted pixel value.
To further achieve the above advantages and in accordance with the purpose of the present invention, as embodied and broadly described, a method of removing blocking artifacts in a coding system of a moving picture according to the present invention includes the steps of defining pixel sets S0, S1, S2 around block boundary, selectively determining a deblocking mode as a default mode or a DC offset mode depending on the degree of blocking artifacts after obtaining a mode decision value, obtaining frequency information around the block boundary per pixel using 4-point DCT kernel if the default mode is determined, replacing a magnitude of a discontinuous component belonging to the block boundary with a minimum magnitude of discontinuous components belonging to the surroundings of the block boundary in the frequency domain and applying this replacing step to the spatial domain, judging whether or not it is necessary to perform DC offset mode if the DC offset mode is determined, and removing the blocking artifacts in a smooth region when the judgment is to perform the DC offset mode.
The invention will be described in detail with reference to the following drawings in which like reference numerals refer to like elements wherein:
In a preferred embodiment of the present invention, blocking artifacts at a block boundary are removed in a frequency domain not a spatial domain. Frequency features around the block boundary are preferably obtained using a 4-point DCT kernel, which can be easily calculated. Thus, a complex region at the block boundary can effectively be processed by extending the smoothness of a picture from the frequency domain to the spatial domain.
Using the 4-point DCT kernel has advantages that frequency analysis is possible and deblocking can easily be processed. Therefore, the 4-point DCT Kernel can efficiently remove the blocking artifacts of a real time moving picture.
The blocking artifacts appear at the block boundary between fixed block patterns in the form of a line of discontinuity. Accordingly, removal of the blocking artifacts involves transformation of the discontinuity of the block boundary region to continuity.
In the preferred embodiment according to the present invention, frequency information in S1 and S2 is used to reduce the blocking artifacts from S0. When images change smoothly, image features of S0, S1 and S2 are similar to one another. This means that image features of S0, S1 and S2 are also similar to one another in the frequency domain.
Since the frequency features of S0, S1 and S2 are similar, the frequency component of S0 influenced by the blocking artifacts is adjusted considering the frequency components of S1, S2, which can remove the blocking artifacts. Here, DCT, which is widely applied as an image compression technique, is used as a frequency analysis tool.
The blocking artifacts may appear in both horizontal and vertical block boundaries. In the preferred embodiment according to the present invention, after the blocking artifacts at the horizontal block boundary are removed, the blocking artifacts at the vertical block boundary are removed.
Pixel sets S0, S1 and S2, which overlap, can be defined around the horizontal block boundary. S0 is a 4-point pixel set arranged across the block boundary while S1 and S2 are 4-point pixel sets that adjoin the block boundary.
That is to say, the pixel set S0 contains a discontinuity. The discontinuity in S0 is removed in the preferred embodiment using common information (e.g., between S0 and S2), which are not directly influenced by the discontinuity of the block boundary.
The 4-point DCT basis is used to get information around the block boundary and is shown in
As shown in
Operations for reduction or removal of the block discontinuity will now be described. In the preferred embodiment, the magnitude of a3,0 is replaced with the minimum value of the magnitudes of a3,1 and a3,2. By doing this, a large blocking artifact, which appears when one side of the block boundary to be processed is smooth, can be removed. For a complex region where both S1 and S2 are the objects of motion (i.e., all the values of the magnitudes of a3,0, a3,1 and a3,2 are large), there is little influence on the block boundary.
A method for removing the blocking artifacts in a default mode in the preferred embodiment is as follows:
v4′=v4−d;
v5′=v5+d; and
d=CLIP(c2.(a3,0′−a3,0)//c3,0,(v4−v5)/2)*δ(|a3,0|<QP),
where a3,0′=SIGN(a3,0)*MIN(|a3,0|,|a3,1|,|a3,2|),
a3,0=([c1−c2 c2−c1]*[v3v4v5v6]T)//c3,
a3,1=([c1−c2 c2−c1]*[v1v2v3v4]T)//c3, and
a3,2=([c1−c2 c2−c1]*[v5v6v7v8]T)//c3.
Thus, boundary pixels v4 and v5 that adjoin the boundary are replaced with v4′ and v5′, respectively. QP is the quantization parameter of the macroblock where pixel v5 belongs. Values c1, c2, c3 are kernel constants used in the 4-point DCT. The values of c1 and c2 are approximated to an integer, and the value of c3 is approximated to a multiple of 2. The values of a3,0, a3,1, a3,2 are evaluated from the simple inner product of the DCT kernel and the pixel sets S0, S 1 and S2.
The condition |a3,0|<QP is used to count the influence of the quantization parameter on the blocking artifacts. The condition |a3,0|<QP also prevents over-smoothing when the blocking artifacts are not very serious. The clipping operation on the compensated value is performed to prevent the direction of the gradient at the block boundary from being enlarged or changed in an opposite direction.
This filtering process is performed in both horizontal and vertical block boundaries. In this manner, the blocking artifacts in the whole frame can be removed.
In the default mode, only the boundary pixel values v4 and v5 are compensated. Thus, the default mode is not sufficient to remove the blocking artifacts in a very smooth region, such as a setting in a picture. Therefore, in the preferred embodiment the blocking artifacts in the smooth region are removed by a DC offset mode.
A method for removing the blocking artifacts in the DC offset mode in the preferred embodiment is as follows:
max=MAX(v1, v2, v3, v4, v5, v6, v7, v8),
min=MIN(v1, v2, v3, v4, v5, v6, v7, v8),
if(|max−min|<2QP), /*low pass filtering*/
Pm=(|v1−v0|<QP)?v0:v1,
if m<1;
vm, if 1≦m≦8;
(|v8−v9|<QP) v9:v8, if m>8;
{bk:−4≦k≦4}={1,1,2,2,4,2,2,1,1}//16.
If the absolute value of the maximum data value minus minimum data value in the block boundary pixels is smaller than twice the quantization parameter (i.e., if deblocking is required), the blocking artifacts in the smooth region are removed by the DC offset mode.
The decision to use the default mode or to use the DC offset mode is preferably made based on the following condition:
Mode decision value(eq_cnt)−φ(v0−v1)+φ(v1−v2)+φ(v2−v3)+φ(v3−v4)+φ(v4−v5)+φ(v5−v1)+φ(v7−v8)+φ(v8−v9),
If the mode decision value eq_cnt≧THR2(i.e., a second threshold value), the DC offset mode is applied. In the remaining cases, default mode is applied.
A method for removing the blocking artifacts to code a moving picture at low-rate-bit according to the preferred embodiment of the present invention will be described with reference to
In step 402S, the mode decision value (e.g., eq_cnt) is determined and control continues to step 403S. In step 403S, the mode decision value is compared with a decision value (e.g., a second threshold value THR2 preferably set by a user) to perform deblocking filtering process by selecting the mode depending on the degree of the blocking artifacts in the picture.
If the determination in step 403S is negative, control continues to step 404S where the default mode is set. From step 404S, control continues to step 405S where frequency information around the block boundary on each of the pixel is determined, for example, using the 4-point DCT kernel. From step 405S, control continues to step 406S.
In step 406S, the magnitude of the discontinuous component belonging to the block boundary is replaced with the minimum magnitude of the discontinuous components belonging to the surroundings of the block boundary in the frequency domain. This adjusting operation is applied to the spatial domain. That is, the magnitude of the discontinuous component belonging to the block boundary is replaced with the minimum magnitude of the discontinuous components belonging to the surroundings of the block boundary in the spatial domain.
In the default mode of the preferred embodiment, the blocking artifacts are removed in step 406S using the method as described below:
v4′=v4−d;
v5′=v5+d; and
d=CLIP(c2.(a3,0′−a3,0)//c3,0,(v4−v5)/2)*δ(|a3|<QP),
where a3,0′=SIGN(a3,0)*MIN(|a3,0|,|a3,1|,|a3,2|),
If the determination in step 403S is affirmative, control continues to step 407S where the DC offset mode is set to remove the blocking artifacts. From step 407S, control continues to step 408S where the minimum and maximum data values (min, max) are determined. From step 408S, control continues to step 409S where a determination is made to remove the blocking artifacts in the default mode. If the determination in step 409S is negative, the process ends. If the determination in step 409S is affirmative, control continues to step 410S.
In the DC offset mode according to the preferred embodiment, in step 410S, the blocking artifacts are removed using the following algorithm.
max=MAX(v1, v2, v3, v4, v5, v6, v7, v8),
min=MIN(v1, v2, v3, v4, v5, v6, v7, v8),
if(|max−min|<2QP), /*low pass filtering*/
Pm=(|v1−v0|<QP)?v0:v1,
if m<1;
vm, if 1≦m≦8;
(|v8−v9|<QP) v′9:v8, if m>8;
{bk:−4≦k≦4}={1,1,2,2,4,2,2,1,1}//16.
The maximum data value and the minimum data value in the block boundary pixels are obtained in step 408S. Then, if the absolute value of the maximum data value minus the minimum data value is smaller than 2QP (i.e., if deblocking is required), the blocking artifacts in the smooth region are removed by the DC offset mode in steps 409S and 410S.
From step 406S and 410S, control continues to step 411S. If the deblocking filtering process around the horizontal block boundary is completed, the deblocking filtering process around the vertical block boundary is performed in step 411S. From step 411S, control continues to step 412S.
In step 412S, the deblocking filtering processes around the horizontal and vertical block boundaries repeat over the whole frame. From step 412S, the process ends.
300 frames (only the initial frame was coded in intra.);
Fixed QP;
H.263 quantization;
F_code=1;
Enable DC/AC prediction; and
Rectangular shape VOP.
As shown in
As described above, the method for removing the blocking artifacts according to the preferred embodiments of the present invention has various advantages. The deblocking filtering process is performed using features of the frequency domain so that the blocking artifacts are effectively removed. Further, the blocking artifacts are removed in both the complex and smooth regions. Thus, an excellent image or picture quality is provided. In addition, amount of bits does not increase.
The foregoing embodiments are merely exemplary and are not to be construed as limiting the present invention. The present teaching can be readily applied to other types of apparatuses. The description of the present invention is intended to be illustrative, and not to limit the scope of the claims. Many alternatives, modifications, and variations will be apparent to those skilled in the art.
Number | Date | Country | Kind |
---|---|---|---|
97-46368 | Sep 1997 | KR | national |
Notice: More than one reissue application has been filed for the reissue of U.S. Pat. No. 6,240,135. The reissue applications are application Ser. Nos. 11/834,312; 11/834,347; 11/851,551; 11/851,529; and 11/851,517, all of which are divisional reissues of U.S. Pat. No. 6,240,135. The present application is a continuation of reissue application Ser. No. 11/834,312, filed Aug. 6, 2007 now U.S. Pat. No. Re. 42,851, which is a divisional reissue application of U.S. Pat. No. 6,240,135.
Number | Name | Date | Kind |
---|---|---|---|
5229864 | Moronaga et al. | Jul 1993 | A |
5262854 | Ng | Nov 1993 | A |
5367385 | Yuan | Nov 1994 | A |
5436666 | Astle | Jul 1995 | A |
5526295 | Astle | Jun 1996 | A |
5539664 | Haghighi et al. | Jul 1996 | A |
5565921 | Sasaki et al. | Oct 1996 | A |
5585852 | Agarwal | Dec 1996 | A |
5590064 | Astle | Dec 1996 | A |
5596659 | Normile et al. | Jan 1997 | A |
5608652 | Astle | Mar 1997 | A |
5614952 | Boyce et al. | Mar 1997 | A |
5629778 | Reuman | May 1997 | A |
5635985 | Boyce et al. | Jun 1997 | A |
5646686 | Pearlstein | Jul 1997 | A |
5677736 | Suzuki et al. | Oct 1997 | A |
5680129 | Weinberger et al. | Oct 1997 | A |
5740283 | Meeker | Apr 1998 | A |
5764374 | Seroussi et al. | Jun 1998 | A |
5767907 | Pearlstein | Jun 1998 | A |
5777677 | Linzer et al. | Jul 1998 | A |
5808683 | Tong et al. | Sep 1998 | A |
5818530 | Canfield et al. | Oct 1998 | A |
5825424 | Canfield et al. | Oct 1998 | A |
5903679 | Park | May 1999 | A |
5911008 | Niikura et al. | Jun 1999 | A |
6000000 | Hawkins et al. | Dec 1999 | A |
6028967 | Kim et al. | Feb 2000 | A |
6040879 | Park | Mar 2000 | A |
6104434 | Nakagawa et al. | Aug 2000 | A |
6130911 | Lei | Oct 2000 | A |
6314209 | Kweon et al. | Nov 2001 | B1 |
8265149 | Jung et al. | Sep 2012 | B2 |
8270474 | Kim et al. | Sep 2012 | B2 |
8300689 | Cho et al. | Oct 2012 | B2 |
8306342 | Kim et al. | Nov 2012 | B2 |
8325805 | Cho et al. | Dec 2012 | B2 |
8331433 | Kim et al. | Dec 2012 | B2 |
8345748 | Kim et al. | Jan 2013 | B2 |
8355589 | Kim et al. | Jan 2013 | B2 |
8374248 | Cho et al. | Feb 2013 | B2 |
8406291 | Kim et al. | Mar 2013 | B2 |
8406545 | Choi et al. | Mar 2013 | B2 |
8451893 | Cho et al. | May 2013 | B2 |
8472733 | Cho et al. | Jun 2013 | B2 |
8472734 | Cho et al. | Jun 2013 | B2 |
8509310 | Kim et al. | Aug 2013 | B2 |
20120258699 | Kim et al. | Oct 2012 | A1 |
20130011058 | Kim et al. | Jan 2013 | A1 |
Entry |
---|
Byeungwoo Jeon et al, Blocking Artifacts reduction in image compression with block boundary discontinuity criteron, Circuits and System for Video Technology, IEEE Transactions on, vol.: Jun. 8, 1998, pp. 345-357. |
Barzykina, E et al., Removal of blocking artifacts using random pattern filtering, Image Processing, 1999 International Conference, vol. 3, 1999, pp. 904-908 vol. 2. |
Mei-Yin Shen te al., Fast compression artifact reduction technique based on nonlinear filter, Circuits and Systems 1999. ISCAS '99. Proceedings of the 1999 IEEE International Symposium on, vol. 4, 1999, pp. 179-182. |
Yung-Kai Lai et al., Removal of blocking artifacts of DCT transform by classified space-frequency filtering, Signals, System and Computers, 1995, Conference Record of the Twenty-Nine Asilmar Conference on, vol. 2, 1996, pp. 1457-1461. |
Kasezawa, T., Blocking artifacts reduction using discrete cosine transfomr, Consumer Electronics, IEEE Transactions on, vol. 43 Issue: 1, Feb 1997, pp. 48-55. |
Avideh Zakhor, “Iterative Procedures for Reduction of Blocking Effects In Transform Image Coding”, IEEE Transactions On Circuits and Systems For Video Technology, vol. 2, No. 1, pp. 91-95 (Mar. 1992). |
Yongyi Yang et al., “Regularized Reconstructions to Reduce Blocking Artifacts of Block Discrete Cosine Transform Compressed Images”, IEEE Transactions On Circuits and Systems For Video Technology, vol. 3, No. 6, pp. 421-432 (Dec. 1993). |
Gary J. Sullivan et al., “Motion Compensation for Video Compression Using Control Grid Interpolation”, IEEE International Conference, pp. 2713-2716 (1991). |
G. de Haan et al., “IC For Motion-Compression 100Hz TV With Natural-Motion Move-Mode”, IEEE Transactions On Consumer Electronics, vol. 42, pp. 165-174 (Feb. 1996). |
Özcelik, Taner et al., “Image and Video Compression Algorithms Based on Recovery Techniques Using Mean Field Annealing”, Proceedings of the IEEE, vol. 83, No. 2, pp. 304-316, Feb. 1995. |
Nakajima, Yasuyuki, et al., “A PEL Adaptive Reduction of Coding Artifacts for MPEG Video Signals”, pp. 928-932, IEEE, 1994. |
Number | Date | Country | |
---|---|---|---|
Parent | 11834312 | Aug 2007 | US |
Child | 09065577 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 09065577 | Apr 1998 | US |
Child | 13176814 | US |