1. Field of the Invention
The present invention relates to a method and related device for reducing blocking artifacts in a video stream, and more particularly, to a method and related device for providing detection of the blocking artifacts and removing the blocking artifacts with adaptive de-blocking filtering strengths.
2. Description of the Prior Art
A wide range of new applications in visual communications have been made possible due to the rapidly evolving telecommunication and computer industries. Almost all the related applications, including video conferencing, mobile or PSTN video telephony, video over the Internet Protocol, and digital TV broadcasting, require very efficient data compression methods to fit a large amount of visual information into the narrow bandwidth of communication channels while preserving acceptable quality of the reconstructed data.
To efficiently compress a time variable video sequence, redundancy in the temporal domain as well as in the two dimensional spatial domain must be reduced. The Moving Picture Experts Group (MPEG) standards use a discrete cosine transform (DCT) for processing the steps of reducing the redundancy in the two dimensional spatial domain and a motion estimation method for processing the steps of reducing the redundancy in the temporal domain.
The prior art video compression technique performs DCT operations, if necessary, after input data is sampled into 8×8 units, and DCT coefficients are quantized with respect to a visual property using quantization parameters from a quantization table. Then, the data is compressed through a run length coding (RLC). The data processed with DCT is converted from a spatial domain to a frequency domain and compressed with the quantization parameters corresponding to the visual property of human beings, not to be visually recognized. For example, since human eyes are insensitive to high frequency images, a high frequency coefficient is quantized using a large step size. Thus, a quantization table is made according to external parameters, such as a display characteristic, watching distance, and noise, to perform an appropriate quantization.
Please refer to
In the traditional block-based video compression standards such as MPEG-2, the blocking artifact is the most noticeable artifact. Thus, various methods for reducing blocking artifacts in decoding systems that individually processes blocks have been developed. For example, a de-blocking filter can be integrated into a decoding system as either an in-loop filter or a post-processing filter. Compared with the in-loop filter, the post-processing filter performs a de-blocking process for a decoded bit stream outside a decoding loop of the decoding system. Thus, the post-processing filter can operate in conjunction with the original video compression standards. In other words, the post-processing filter can be easily integrated into any standard decoder using various implementation methods designed independent of the standard.
As shown in
Besides, since the video stream inputted to the decoding system may have various image characteristics or may come from different video sources like digital versatile discs (DVDs) and digital TV, if all of the 8×8 pixel blocks are performed the de-blocking process with fixed filtering strength, the blocks may also blur or still exist apparent boundary lines, which influences the image quality of the decoding system. However, the demand for de-blocking filtering of MPEG-2 encoded video is increasing due to the wide acceptance of MPEG-2 applications, such as DVD and digital TV.
It is therefore a primary objective of the present invention to provide a method and related device for reducing blocking artifacts in video streams.
The present invention discloses a method for reducing blocking artifacts in a video stream. The method comprises receiving a picture of the video stream, the picture comprising a plurality of macroblocks and each of the plurality of macroblock comprising four blocks; determining blocks with quantization parameters greater than a first threshold value in the picture; checking if block boundaries of the blocks are sharp and are real edges of objects in the picture according to pixel value differences between two adjacent pixels respectively located at both sides of the block boundaries; selecting filtering strength of a de-blocking process according to the pixel value differences when the block boundaries are sharp and are not real edges of the objects in the picture; and performing the de-blocking process for two adjacent blocks at both sides of the block boundaries.
The present invention further discloses a de-blocking device for reducing blocking artifacts in a video stream. The de-blocking device comprises a reception terminal, a blocking artifact detection unit and a blocking artifact elimination unit. The reception terminal is utilized for receiving a picture of the video stream, wherein the picture includes a plurality of macroblocks and each of the plurality of macroblock further includes four blocks. The blocking artifact detection unit comprises a block determination unit coupled to the reception terminal for determining blocks with quantization parameters greater than a first threshold value in the picture, and a block boundary determination unit coupled to the block determination unit for checking if block boundaries of the blocks are sharp and are real edges of objects in the picture according to pixel value differences between two adjacent pixels respectively located at both sides of the block boundaries. The blocking artifact elimination unit comprises a filtering strength selection unit coupled to the block boundary determination unit for selecting filtering strengths of a de-blocking operation according to the pixel value differences when the block boundaries are sharp and are not real edges of the objects in the picture, and a de-blocking filtering unit coupled to the filtering strength selection unit for performing the de-blocking operation for two adjacent blocks at both sides of the block boundaries.
These and other objectives of the present invention will no doubt become obvious to those of ordinary skill in the art after reading the following detailed description of the preferred embodiment that is illustrated in the various figures and drawings.
Please refer to
Step 200: start.
Step 210: receive a picture of the video stream.
Step 220: determine blocks with quantization parameters greater than a first threshold value in the picture.
Step 230: check if block boundaries of the blocks are sharp and are not real edges of objects in the picture according to pixel value differences between two adjacent pixels respectively located at both sides of the block boundaries.
Step 240: select filtering strengths of a de-blocking operation.
Step 250: perform the de-blocking operation for two adjacent blocks at both sides of the block boundaries.
Step 260: end.
According to the process 20, after receiving the picture of the video stream decompressed by an MPEG-2 decoder, the present invention first determines blocks with the quantization parameters greater than the first threshold value in the picture, and then check if block boundaries of the blocks are sharp and are not real edges of objects in the picture according to pixel value differences between two adjacent pixels respectively located at both sides of the block boundaries. When the block boundaries are sharp and are not real edges of the objects in the picture, the present invention selects appropriate filtering strengths of the de-blocking operation according to the pixel value differences, and performs the de-blocking operation for two adjacent blocks at both sides of the block boundaries. Note that, the picture decompressed from the MPEG-2 decoder includes a plurality of macroblocks, and each of the plurality of macroblocks further includes four 8×8 pixel blocks.
Therefore, before performing the de-blocking operation, the present invention can detect the blocks that exist the blocking artifacts on their bock boundaries according to decoding information of the MPEG-2 decoder (such as quantization parameters) and the image characteristics (such as the pixel value differences), so that the efficiency of the decoding system can be enhanced significantly. Moreover, the present invention can perform the de-blocking operation for the blocks with the blocking artifacts by adaptive filtering strengths, and thus the blocking artifacts can be eliminated effectively and the desired image quality can be obtained as well. As for the detailed description of the process 20, please further refer to the following illustration.
As those skilled in the art recognized, the decoding information, such as the quantization parameter corresponding to each macroblock in the picture, can be obtained during the MPEG-2 decompression process, and a larger quantization parameter means that the step sizes used to quantize the pixel values are larger, so that the blocking artifacts may occurs on block boundaries of the blocks with large quantization parameters. Thus, by determining the blocks with the quantization parameters greater than the first threshold value in the picture (Step 220), the present invention can detect the blocks that exist the blocking artifacts. In other words, if the quantization parameters of the blocks are greater than the first threshold value, the present invention determines that the blocking artifacts may exist on block boundaries of the blocks, so that the following steps of the process 20 are performed for the corresponding boundaries.
For example, please refer to
When the blocks are determined to have the blocking artifacts on their block boundaries, the present invention checks if the block boundaries of the blocks are sharp and are not real edges of the objects in the picture according to pixel value differences between two adjacent pixels respectively located at both sides of the block boundaries (Step 230). Please refer to
Next, the present invention determines whether the block boundaries are real edges of the objects in the picture according to the pixel value differences pix_diff. If the pixel value differences pix_diff are both smaller than two values which are a third threshold value TH3 multiplying the quantization parameters of the two adjacent blocks at both sides of the block boundaries, the block boundaries are then determined to be not the real edges of the objects in the picture. When the block boundaries are sharp and are not real edges of the objects in the pictures, which indicating that there are indeed the blocking artifacts on the block boundaries, the de-blocking operation has to be performed for the blocks corresponding to the block boundaries. Conversely, if the block boundaries are real edges of the objects in the picture, the block boundaries need not to be performed the following de-blocking operation for avoiding blurring the corresponding images.
In order to determine the third threshold value TH3, the present invention can determine in advance that smoothness or local image characteristics around the block boundaries. For example, when the image around the block boundaries is smooth, the pixel value difference between the two adjacent pixels respectively located at both sides of the block boundary is highly possible to be a blocking artifact but not a real edge of the objects in the picture. Thus, for the smooth image region in the picture, the third threshold value TH3 can be set as a larger value. Conversely, when the image around the block boundaries is complex, the pixel value difference is possible to be a real edge of the objects in the picture, and thus for the complex image region in the picture, the third threshold value TH3 can be set as a smaller value. Therefore, in the present invention, the de-blocking operation can be appropriately performed according to the image characteristics and the quantization parameters, so as to prevent from excessively blurring the image or still remaining apparent boundary lines. Preferably, the third threshold value can be a value selected from 0.5, 1, 1.5, 2 and 2.5, and can be further adjusted according to the desired image quality and the processing efficiency.
When the block boundaries are sharp and are not real edges of the objects in the pictures, the block boundaries need to be performed the de-blocking operation for reducing the blocking artifacts. The present invention then selects filtering strengths of the de-blocking operation corresponding to pixels of the two adjacent blocks at both sides of the block boundaries according to the pixel value differences pix_diff (Step 240). In the present invention, the filtering strengths of the de-blocking operation can be obtained by multiplying various weighting values with the pixel value differences pix_diff between the two pixels respectively located at both side of the block boundaries according to the desired image quality, so as to enhance accuracy and flexibility of the de-blocking filtering operation. Preferably, the weighting values can be a value selected from 1/8, 2/8, 3/8, 4/8, 5/8, 6/8, 7/8, 8/8, 9/8, and 10/8, and are not restricted herein.
After selecting the filtering strengths of the de-blocking operation, the present invention can perform the de-blocking operation for the two adjacent blocks at both sides of the block boundaries (Step 250). When performing the de-blocking operation, the present invention selects four pixels at both sides of the block boundaries, i.e. the pixels P1˜P8 shown in
On the other hand, if the picture is formed by two interlaced image fields, the present invention then separately performs the de-blocking operation for the odd fields and the even fields when processing the horizontal block boundaries. That means, the odd row pixels and the even row pixels are separately de-blocking filtered when processing the horizontal block boundaries of the interlaced picture. For example, the filtering range includes two odd row pixels at both the top and bottom sides of the block boundaries, i.e. the pixels P1, P3, P5 and P7 in
As mentioned above, the filtering strengths of each two corresponding pixels at both sides of the block boundaries can be determined according to the pixel difference pix_diff. Thus, when performing the de-blocking operation, the present invention can respectively increase and decrease the pixel values of each two corresponding pixels at both sides of the block boundaries with the appropriate filtering strengths for lessening the pixel value differences according to their original pixel values, so as to reduce the blocking artifacts. In other words, when performing the de-blocking operation, the pixel value of the pixel with a smaller pixel value in a pixel pair are added with the corresponding filtering strength, and the pixel value of the pixel with a larger pixel value in the pixel pair are subtracted with the same filtering strength, so that the pixel value difference between the pixel pair can be lessened. After performing the filtering operation for all of the pixel pairs at both sides of the block boundaries according to the filtering range, the present invention adjusts all of the altered pixel values of the pixels at both sides of the block boundaries to lie in between the values 0 and 255.
Please refer to
Therefore, in the present invention, before performing the de-blocking operation, the blocks that exist the blocking artifacts on their bock boundaries can be detected in advance according to the quantization parameters obtained from the decoding information of the MPEG-2 decoder and the image characteristics, so that the efficiency of the decoding system can be enhanced significantly. Moreover, the present invention can perform the de-blocking operation for the blocks with the blocking artifacts by the adaptive filtering strengths, and thus the blocking artifacts can be eliminated effectively and the desired image quality can be obtained as well.
Please refer to
The de-blocking device 82 is utilized for realizing the process 20 of the present invention, and includes a blocking artifact detection unit 860 and a blocking artifact elimination unit 870. The blocking artifact detection unit 860 is utilized for detecting the blocking artifacts in a picture of the decompressed video stream according to the quantization parameters stored in the memory 83, and includes a block determination unit 861 and a block boundary determination unit 862. The block determination unit 861 is coupled to the video decoder 81 and the memory 83, and is utilized for determining the blocks with the quantization parameters greater than the first threshold value in the picture. The block boundary determination unit 862 is coupled to the block determination unit 861, and is utilized for checking if the block boundaries of the blocks are sharp and are real edges of the objects in the picture according to the pixel value differences pix_diff between two adjacent pixels respectively located at both sides of the block boundaries.
After the blocking artifacts are detected by the blocking artifact detection unit 860, the blocking artifact elimination unit 870 performs the de-blocking operation for the blocks with the blocking artifacts detected by the blocking artifact detection unit 860. The blocking artifact elimination unit 870 is coupled to the blocking artifact detection unit 860, and includes a filtering strength selection unit 871 and a de-blocking filtering unit 872. The filtering strength selection unit 871 is coupled to the block boundary determination unit 862, and is utilized for selecting the filtering strengths of the de-blocking operation according to the pixel value differences pix_diff. The de-blocking filtering unit 872 is coupled to the filtering strength selection unit, and is utilized for performing the de-blocking operation for two adjacent blocks at both sides of the block boundaries that are sharp and are not real edges of the objects in the picture.
Thus, with the blocking artifact detection unit 860, the present invention can detect the blocks that have the blocking artifacts on their bock boundaries, and with blocking artifact elimination unit 870, the present invention can perform the de-blocking operation for the blocks with the blocking artifacts by adaptive filtering strengths, so that not only the efficiency of the decoding system can be enhanced, but the desired image quality can also be obtained.
Note that, the de-blocking device 82 is utilized for realizing the process 20, and thus the related operation will not be narrated in detail. Certainly, those skilled in the art can further implement the de-blocking device with any hardware and firmware or make appropriate modifications according to practical demands, which are still in the range of the present invention.
In summary, the present invention provides the method and related device for detecting the blocking artifacts and reducing the blocking artifacts with adaptive de-blocking filtering strength. Besides, in the present invention, all of the parameters are adaptive, so as to enhance accuracy and flexibility of the de-blocking filtering operation.
Those skilled in the art will readily observe that numerous modifications and alterations of the device and method may be made while retaining the teachings of the invention.