Patent Application
20080240604
Block-encoded video, such as video encoded using techniques compatible with the Moving Picture Experts Group (MPEG) (e.g., MPEG-2, and/or MPEG-4 video data, also in accordance with International Telecommunication Union (ITU-T) Rec. H.264 (2003)), may suffer from discontinuities at block boundaries, particularly when encoded at low bit rates where large quantization errors may occur. Such discontinuities may appear in the reconstructed video frames as blocking artifacts (e.g., visible block edges, mosaic patterns, tiling effects, etc.), particularly in image regions that are smooth.
Deblocking filters implement methods such as variable length filtering and/or edge protection may reduce the magnitude of blocking artifacts to a visually-acceptable level. Further, this filtering is complex and it is difficult to deblock data of different block sizes.
In various embodiments, a deblocking algorithm can reduce blocking artifacts seen in block-encoded video, such as MPEG-2 data. Embodiments may be used to clean blocking artifacts for different grid sizes (e.g., with a minimum grid size of 4). For example, embodiments can be used to deblock H.264 content independently of an available H.264 loop deblocking filter. As used herein, the terms deblocking algorithm and blocking noise reduction algorithm are used interchangeably.
The deblocking algorithm may operate in the spatial domain, and may be used to provide energy redistribution in which pixels may be re-aligned across the edges of an artifact. Re-alignment may be followed by a selective m×m (e.g., 3×3 window) low pass filtering step which attempts to blend the deblocked pixels with neighboring pixels. Soft thresholds may be applied to reduce flicker. The deblocking algorithm can be either applied to only a luminance (Y) component or to all three components of a color space (e.g., a luminance/chromimine (YUV) color space).
In various embodiments, pixel re-alignment may be based on local characteristics of the content. Furthermore, selective low-pass filtering may be applied, along with soft thresholds to avoid flickering at an output image.
Referring now to
Referring now to
blk=(p0−q0) [1]
ablk=abs(blk) [2]
Still referring to
diff1=abs(p1−p0) [3]
diff2=abs(q1−q0) [4]
adiff=abs(diff1−diff2) [5]
If (diff1<ablk/2) && (diff2<ablk/2) && (adiff<ablk/2) [6]
Do Deblocking
Else
Referring still to
In various embodiments, the deblocking may be implemented using a deblocking filter that applies a soft threshold to the artifact strength. In various embodiments, deblocking may be performed by applying a soft threshold to the artifact strength value as set forth in Equations 7 to 11 below. In this way, a faster roll over of the artifact strength to zero over a threshold th may be realized as shown in
assert (th+16>=32); if (ablk<th) [7]
blk=blk, else if (ablk<th+2) [8]
blk=sign*(ablk−4); else if (ablk<th+8) [9]
blk=sign*(ablk−16); else if (ablk<th+16) [10]
blk=sign*(ablk−32); else blk=0 [11]
After soft thresholding is performed, pixel re-alignment may be applied to n pixels (e.g., 2 pixels) on each side of the artifact, in accordance with Equations 12 to 15.
p
1
=p
1
−blk2 [12]
p0=p0=blk1 [13]
q
0
=q
0
+blk1 [14]
q
1
=q
1
+blk2 [15]
where blk1, blk2 are calculated to be points on a line with a given slope k as shown in
blk1=blk/2 and blk2=blk/4. [16]
After pixel re-alignment, a low-pass filtering process may be applied to the re-aligned pixels. While the scope of the present invention is not limited in this regard, a n×n (e.g., 3×3 window) low-pass filter may be applied to the realigned pixels.
In some embodiments, before applying the low-pass filtering, a threshold process is performed on the surrounding pixels (i.e., the pixels surrounding a given one of the re-aligned pixels that is to be filtered). In this thresholding process, if a surrounding pixel and the pixel to be filtered differ by more than a given threshold (i.e., low-pass_Th), the surrounding pixel may be replaced by the center pixel, i.e., the re-aligned pixel. In various embodiments, low-pass_Th may be a function of the absolute value of the artifact strength. While different manners of performing low-pass filtering may be performed, shown in Table 1 is an exemplary 3×3 low-pass filter kernel that may be used.
Accordingly, embodiments may provide for better output quality with minimal complexity. Furthermore, different types of incoming image data such as MPEG and H.264 data may be deblocked using the same process. That is, embodiments may be used for any block sizes, as the processing remains the same.
Embodiments may be implemented in an algorithm that can be executed in many different system types. For example, such systems can include digital imaging devices such as digital cameras, digital televisions, digital video devices and so forth. Similarly, embodiments may be implemented in computer systems such as server computers, personal computers, mobile devices such as cellular telephones and so forth. In such processor-based systems, an algorithm in accordance with an embodiment may be performed in a general-purpose processor such as a microprocessor, a graphics processing unit (GPU) or other such processing unit.
Embodiments may be implemented in code and may be stored on a storage medium having stored thereon instructions which can be used to program a system to perform the instructions. The storage medium may include, but is not limited to, any type of disk including floppy disks, optical disks, compact disk read-only memories (CD-ROMs), compact disk rewritables (CD-RWs), and magneto-optical disks, semiconductor devices such as read-only memories (ROMs), random access memories (RAMs) such as dynamic random access memories (DRAMs), static random access memories (SRAMs), erasable programmable read-only memories (EPROMs), flash memories, electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, or any other type of media suitable for storing electronic instructions.
While the present invention has been described with respect to a limited number of embodiments, those skilled in the art will appreciate numerous modifications and variations therefrom. It is intended that the appended claims cover all such modifications and variations as fall within the true spirit and scope of this present invention.
