Patent Application
20130177081
1. Field
The inventive concept relates to an interpolation filter which interpolates a reference image for motion prediction and motion compensation. In particular, the inventive concept relates to methods and apparatuses for encoding and decoding a video, which extends a corresponding block of a reference picture, filters the extended corresponding block by applying an infinite impulse response (IIR) filter, and interpolating the IIR filtered extended corresponding block in sub-pixel units.
2. Description of the Related Art
In image compression methods, such as Moving Picture Experts Group (MPEG)-1, MPEG-2, MPEG-4, or H.264/MPEG-4 Advanced Video Coding (AVC), a picture is divided into macroblocks in order to encode an image. Each of the macroblocks is encoded in all encoding modes that can be used in inter-prediction or intra-prediction, and then is encoded in an encoding mode that is selected according to a bit rate used to encode the macroblock and a distortion degree of a decoded macroblock based on the original macroblock.
In inter-prediction, a motion vector is generated by searching for a region in a reference picture similar to a current block to be encoded by using at least one reference picture located in front of or at the rear of a currently encoded picture, and a prediction block of the current block is generated by performing motion compensation using the generated motion vector.
In order to increase prediction accuracy, motion compensation is performed in a unit of sub-pixel resolution that is more precise than an integer pixel resolution. For example, a reference picture is interpolated by generating sub-pixels, such as a ½ pixel (half pel), a ¼ pixel (quarter pel), and a ⅛ pixel (one eighth pel), between integer pixels of the reference picture, and then motion compensation is performed by using the interpolated reference picture.
The inventive concept provides methods and apparatuses for encoding and decoding a video, where prediction efficiency is improved during motion compensation of the video by extending a corresponding block of a reference picture used during the motion compensation and then generating an interpolated reference picture via block-based IIR filtering and finite impulse response (FIR) filtering.
The inventive concept also provides methods and apparatuses for encoding and decoding a video, where prediction efficiency is improved without remarkably increasing operation complexity by applying an IIR filter only to a predetermined region of a reference picture.
According to exemplary embodiments of the inventive concept, interpolation is performed after extending a corresponding block of a reference picture used in motion compensation of a current block and applying a filter having an IIR characteristic to the extended corresponding block.
According to the inventive concept, compression efficiency can be improved during motion prediction and compensation of a video.
According to an aspect of the inventive concept, a method of encoding a video is provided, the method including: generating an extended corresponding block by extending a corresponding block of a reference picture used in motion compensation of a current block to a predetermined size; generating a filtered extended corresponding block by applying a predetermined first filter to the extended corresponding block; performing interpolation in sub-pixel units by applying a predetermined second filter to the filtered extended corresponding block; and performing motion prediction and compensation by using the reference picture that is interpolated in the sub-pixel units.
According to another aspect of the inventive concept, a method of decoding a video is provided, the method including: extracting information about a motion vector of a current block to be decoded from a received bitstream; generating an extended corresponding block by extending a corresponding block of a reference picture indicated by the motion vector to a predetermined size; generating a filtered extended corresponding block by applying a predetermined first filter to the extended corresponding block; performing interpolation in sub-pixel units by applying a predetermined second filter to the filtered extended corresponding block; and performing motion compensation by using the reference picture that is interpolated in the sub-pixel units.
According to another aspect of the inventive concept, an apparatus for encoding a video is provided, the apparatus including: a block extender which generates an extended corresponding block by extending a corresponding block of a reference picture used in motion compensation of a current block to a predetermined size; a first filter which generates a filtered extended corresponding block by applying a predetermined first filter to the extended corresponding block; a second which interpolates in sub-pixel units by applying a predetermined second filter to the filtered extended corresponding block; and a motion predictor and compensator which performs motion prediction and compensation by using the reference picture that is interpolated in the sub-pixel units.
According to another aspect of the inventive concept, an apparatus for decoding a video is provided, the apparatus including: an entropy decoder which extracts information relating to a motion vector of a current block to be decoded from a received bitstream; a block extender which generates an extended corresponding block by extending a corresponding block of a reference picture indicated by the motion vector to a predetermined size; a first filter which generates a filtered extended corresponding block by applying a predetermined first filter to the extended corresponding block; a second filter which interpolates in sub-pixel units by applying a predetermined second filter to the filtered extended corresponding block; and a motion compensator which performs motion compensation by using the reference picture that is interpolated in the sub-pixel units.
According to an exemplary embodiment, an apparatus for encoding a video is provided, where the apparatus includes: a block extender which extends a corresponding block of a reference picture to generate an extended corresponding block; a first filter which filters the extended corresponding block to generate a filtered extended corresponding block; a second filter which interpolates in sub-pixel units by applying a second filtering to the filtered extended corresponding block; and a motion predictor and compensator which uses the reference picture that is interpolated in the sub-pixel units to perform motion prediction and compensation.
An exemplary embodiment also provides an apparatus for decoding a video, the apparatus including: an entropy decoder which extracts information of a current block to be decoded; a block extender which generates an extended corresponding block by extending a corresponding block of a reference picture a first filter which generates a filtered extended corresponding block by filtering the extended corresponding block; a second filter which interpolates in sub-pixel units by applying a second filtering to the filtered extended corresponding block; and a motion compensator which uses the reference picture that is interpolated in the sub-pixel units to perform motion compensation.
The information extracted by the entropy decoder may relate to a block of information received from a compressed bitstream. The entropy decoder generates a quantized coefficient by performing entropy decoding upon receiving a compressed bitstream while extracting information relating to a motion vector of a current block to be decoded. In addition, the block extender and the first and second filters comprise a reference picture interpolator.
Hereinafter, the inventive concept will be described more fully with reference to the accompanying drawings, in which exemplary embodiments are shown.
Referring to
The motion estimator 110 generates a motion vector of a current block by performing motion prediction using data of a reference picture. In particular, the motion estimator 110 searches for a corresponding block in the reference picture which is most similar to the current block of a current picture to be encoded, and generates the motion vector based on a location difference between the current block and the corresponding block. Here, the motion estimator 110 generates a motion vector in an integer pixel unit by using a reference picture in integer pixel units stored in the storage device 160 during initial motion prediction. The motion estimator 110 may not only perform motion prediction by using a reference picture in integer pixel units, but may also perform motion prediction by using a reference picture that is interpolated in a resolution of a sub-pixel, such as ½ pixel or ¼ pixel, by the reference picture interpolator 170.
The reference picture interpolator 170 generates an extended corresponding block by extending the corresponding block of the reference picture indicated by the motion vector in the integer pixel units based on a motion vector in integer pixel units of the current block, and generates a filtered extended corresponding block by applying a first filter having an IIR characteristic to the extended corresponding block. In addition, the reference picture interpolator 170 generates a reference picture interpolated in the resolution of a sub-pixel by applying a second filter having a FIR characteristic to the filtered extended corresponding block. Detailed operations of the reference picture interpolator 170 will be described below.
The motion compensator 120 generates a prediction block of the current block by obtaining a motion compensation value of the current block indicated by the motion vector of the current block, from the interpolated reference picture. If a determined optimum motion vector is in integer pixel units, the motion compensator 120 may perform motion compensation by using the reference picture that is not interpolated, instead of the reference picture that is interpolated.
The subtractor 130 calculates a residual block that is a difference between the prediction block and an original input block. The encoder 140 generates a bitstream by transforming, quantizing, and entropy encoding the residual block. Also, the encoder 140 may insert predetermined index information indicating whether motion compensation is performed by using the reference picture interpolated via filtering of the extended corresponding block by the reference picture interpolator 170 into a predetermined region of the generated bitstream along with information relating to the motion vector of the current block. In other words, a one-bit flag having a value of “0” is inserted into the bitstream when a reference picture is interpolated and a one-bit flag having a value of “1” is inserted into the bitstream when the reference picture which is interpolated via filtering of the extended corresponding block is used according to an exemplary embodiment such that a decoder determines whether the reference picture interpolated via filtering of the extended corresponding block is generated. Such predetermined index information may be set in a slice or sequence device and may be included in a slice or sequence header.
Referring to
The block extender 210 generates an extended corresponding block by extending a corresponding block of a reference picture indicated by a motion vector of a determined current block to a predetermined size.
The first filter 220 generates a filtered extended corresponding block by applying a predetermined first filter to the extended corresponding block. The second filter 230 performs interpolation in sub-pixel units by applying a predetermined second filter to the filtered extended corresponding block.
Referring to
The first filter 220 generates a filtered extended corresponding block by applying a first filter having an IIR characteristic to the extended corresponding block 322. An IIR filter performs filtering as a value of an input signal and a value of an output signal are recursively applied, and is a filter having an infinite length of an impulse response that is a characteristic function. Since it is impossible to actually perform an infinite calculation to calculate an output value of the IIR filter, the IIR filter is generally applied within a frame unit during an image process. However, when the IIR filter is applied within a frame unit, compression efficiency of an image may be improved but operation complexity is increased since the IIR filter is applied to an entire frame. Accordingly, in an exemplary embodiment, the IIR filter is applied based on an extended corresponding block so as to reduce the operation complexity while maintaining efficiency of the IIR filter.
According to an exemplary embodiment, the first filter 220 may be implemented as a filter having a transfer function characteristic as shown in Equation 1 below, in a z-region.
In Equation 1, c1 and z1 are each a predetermined constant. Equation 2 below is obtained when Equation 1 is expressed using a recursive algorithm.
c
+(k)=s(k)+z1·c+(k−1)
c
−(k)=z1·(c−(k+1)−c+(k)) [Equation 2]
In Equation 2, s(k) denotes a (k)th sample value to be filtered in a row or column direction. Referring to Equation 2, c+(k) is a casual filter using a (k−1)th result value before a (k)th sample value that is currently being filtered, and c−(k) is a non-casual filter using a (k+1)th result value after the (k)th sample value that is currently being filtered. The first filter 220 generates the filtered extended corresponding block by performing one-dimensional filtering on the extended corresponding block 322 in the row and column directions by using a recursive operation expression, as shown for example in Equation 2. In the above exemplary embodiment, the first filter 220 uses the IIR filter having a transfer function characteristic, such as Equations 1 and 2, but one of ordinary skill in the art would understand from a reading of the detailed description that the transfer function of the IIR filter used by the first filter 220 may be realized via another recursive algorithm.
Referring to
Referring to
c3(k)=s2(k)+z1*c3(k−1) and
c4(k)=z1*(c4(k+1)−c3(k)).
As a result, the first filter 220 finally outputs an extended corresponding block having filtered integer pixels by performing filtering in a column direction by again applying an IIR filter having the same filter characteristic to result in values filtered by applying the IIR filter in a row direction.
Referring back to
Referring to
The second filter 230 generates a prediction signal having ½ pixel precision by applying an FIR filter having a predetermined tap coefficient to integer pixels filtered by the first filter 220. For example, the second filter 230 interpolates pixels having ½ pixel precision by applying a 6-tap FIR filter having a tap coefficient of {(1, −5, 20, 20, −5, 1)/32} to the integer pixels filtered by the first filter 220, and interpolates pixels having ¼ pixel precision by using a 2-tap average value filter. For example, the second filter 230 generates a value of a ½ pixel b having ½ pixel precision according to the equation, b=(E−5F+20G+20H−5I+J)/32, by applying horizontal direction 6-tap filtering to integer pixels E, F, G, H, I, and J which are filtered by the first filter 220. Also, the second filter 230 generates a ½ pixel located in a vertical direction of two integer pixels by performing 6-tap filtering in a vertical direction. For example, the second filter 230 generates a ½ pixel h by using equation, h=(A−5C+20G+20M−5R+T)/32. The second filter 230 generates a ½ pixel located between four integer pixels by applying 6-tap filtering in both horizontal and vertical directions. According to an example of a ½ pixel j, the second filter 230 first generates ½ pixel signals aa, bb, b, j, s, gg, and hh via horizontal direction 6-tap filtering, and then generates the ½ pixel j by applying vertical direction 6-tap filtering, i.e., using the equation, j=(aa−5bb+20b+20s−5gg+hh)/32. Alternatively, the second filter 230 may first generate ½ pixels cc, dd, h, m, ee, and ff via vertical direction filtering, and then generate the ½ pixel j via horizontal direction filtering, such as by using the equation, j=(cc−5dd+20h+20m−5ee+ff)/32.
The second filter 230 may generate a sub-pixel having ½ pixel precision and then generate a ¼ pixel by using an average value filter. ¼ pixels a, c, l and k are generated by using a horizontal direction average value filter on an adjacent integer pixel or ½ pixel. For example, the second filter 230 generates a ¼ pixel a by using an average value filter according to the equation, a=(G+b). Similarly, ¼ pixels d, f, n, and q are generated by using a vertical direction average value filter on an adjacent integer pixel or ½ pixel. For example, the second filtering unit 230 generates a ¼ pixel f by using an average value filter according to the equation, f=(b+j)/2. A diagonal direction average filter may be used for ¼ pixels e, g, p, and r. For example, the second filter 230 generates a ¼ pixel r by using an average value filter according to the equation, r=(m+s)/2.
As described above, when the second filter 230 generates a reference picture interpolated in sub-pixel units, the motion estimator 110 may determine a motion vector having sub-pixel precision by searching for a reference picture region which is most similar to a current block by using the interpolated reference picture. When a final motion vector is determined by comparing rate-distortion (R-D) costs according to each of the motion vectors, information relating to the final motion vector and predetermined index information indicating whether motion compensation is performed by using the reference picture interpolated via filtering of the extended corresponding block may be inserted into an encoded bitstream, as encoding information.
In operation 710, the block extender 210 generates an extended corresponding block by extending a corresponding block of a reference picture used for motion compensation of a current block to a predetermined size. As described above, the block extender 210 may determine a corresponding block of the reference picture based on a motion vector in integer pixel units determined by the motion estimator 110, and then extend the corresponding block of the reference picture in proportion to a size of the current block.
In operation 720, the first filter 220 generates a filtered extended corresponding block by applying a predetermined first filter to the extended corresponding block. As described above, the first filter 220 outputs an extended corresponding block having changed pixel values by performing filtering using an IIR filter based on each integer pixel included in the extended corresponding block.
In operation 730, the second filter 230 performs interpolation in sub-pixel units by applying a predetermined second filter to the filtered extended corresponding block. As described above, the second filter 230 generates an interpolated reference picture by generating values of sub-pixels having ½ or ¼ pixel precision by applying an FIR filter and an average value filter having a predetermined tap coefficient based on integer pixels of the extended corresponding block filtered by the first filter 220.
In operation 740, the motion estimator 110 and the motion compensator 120 perform motion estimation and compensation in sub-pixel units based on the interpolated reference picture.
Referring to
The entropy decoder 810 generates a quantized coefficient by performing entropy decoding upon receiving a compressed bitstream while extracting information relating to a motion vector of a current block to be decoded and predetermined index information indicating whether motion compensation is performed by using a reference picture interpolated via filtering of an extended corresponding block, according to an exemplary embodiment. The re-arranger 815 re-arranges the quantized coefficient, and the inverse quantizer 820 and the inverse transformer 825 restore a residual block by performing inverse quantization and inverse transformation on the quantized coefficient.
The reference picture interpolator 835 interpolates a reference picture for motion compensation of a current block to be decoded, similar to the reference picture interpolators 170 and 200 of
The motion compensator 840 performs motion compensation by using the reference picture interpolated in the sub-pixel units. The adder 830 restores the current block by adding a value obtained via the motion compensation of the current block and the restored residual block. When the current block is intra-predicted, the current block is restored as the restored residual block and a prediction block predicted by the intra-estimator 845 are added together.
In operation 910, the entropy decoder 810 extracts information relating to a motion vector of a current block to be decoded from a received bitstream. The information relating to the motion vector may include a value of the motion vector of the current block to be decoded and predetermined index information which indicates whether motion compensation is performed by using a reference picture interpolated via filtering of an extended corresponding block.
In operation 920, the reference picture interpolator 835 generates an extended corresponding block by extending a corresponding block of a reference picture indicated by the motion vector to a predetermined size. As described above, the reference picture interpolator 835 may determine the corresponding block of the reference picture by using a motion vector component in integer pixel units.
In operation 930, the reference picture interpolator 835 generates a filtered extended corresponding block by applying a predetermined first filter to the extended corresponding block. The predetermined first filter is an IIR filter and is applied to the extended corresponding block as described above, instead of within a frame.
In operation 940, the reference picture interpolator 835 performs interpolation in sub-pixel units by applying a predetermined second filter to the filtered extended corresponding block. Here, the predetermined second filter may be a 6-tap filter or a 4-tap filter, which is an FIR filter having a predetermined tap coefficient, or a 2-tap filter which is an average value filter.
In operation 950, the motion compensator 840 generates a prediction block of the current block by performing motion compensation on the reference picture interpolated in the sub-pixel units.
While this inventive concept has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. The embodiments should be considered in a descriptive sense only and not for purposes of limitation. Therefore, the scope of the invention is defined not by the detailed description of the invention but by the appended claims, and all differences within the scope will be construed as being included in the inventive concept. The invention may also be embodied as computer readable codes on a non-transitory computer readable recording medium. The non-transitory computer readable recording medium is any data storage device that can store data which can be thereafter read by a computer system. Examples of the computer readable recording medium include read-only memory (ROM), random-access memory (RAM), CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, etc. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code may be stored and executed in a distributed fashion.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2011-0045837 | May 2011 | KR | national |
This application claims priority from U.S. Provisional Application No. 61/334,640 filed on May 14, 2010 and from Korean Patent Application No. 10-2011-0045837 filed on May 16, 2011. This application is a National Stage of PCT/KR2011/003606 filed May 16, 2011, the disclosure of which are incorporated herein by reference, in their entirety.
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/KR2011/003606 | 5/16/2011 | WO | 00 | 3/20/2013 |
| Number | Date | Country | |
|---|---|---|---|
| 61334640 | May 2010 | US |
