1. Field of the Invention
The present invention relates to an apparatus and a method for processing uni- and multi-dimensional discrete signals such as radar, sonar, natural images, photographs, drawings, multi-spectral images, volumetric medical image data sets, video sequences, etc. It further relates to the processing of such signals, images, video, and other multi-dimensional signals which are captured directly in digital format or after they have been converted to or expressed in digital format. This invention further relates to use of fast and efficient reconstruction methods to increase the processing speed for digital signal storage and transmission, especially in fixed-point hardware implementations. More particularly, the present invention relates to the processing of image/video data.
2. Description of the Related Art
The discrete cosine transform (DCT), a well-known invertible, and in fact, orthogonal transformation of sets of real-valued coefficients, has been at the heart of the algorithmic structure of many international standards such as JPEG, H.26x, and the MPEG family of standards, as discussed for example in J. Mitchell, W. Pennebaker, C. Fogg, and D. LeGall, MPEG Video Compression Standard, Chapman and Hall, New York, 1996. It has satisfactory performance in terms of signal energy compaction capability, and many fast algorithms for the DCT and its inverse transform, the so-called IDCT, have been proposed. One type of the fast algorithms is based on various sparse factorizations of the DCT/IDCT matrix such as Chen's factorization in W. Chen, C. Harrison, and S. Fralick, “A fast computational algorithm for the discrete cosine transform,” IEEE Trans. Communications, vol. COM-25, pp. 1004.1011, 1977 and Loeffler's factorization as in C. Loeffler, A. Lightberg, and G. Moschytz, “Practical fast 1-D DCT algorithms with 11 multiplications,” in Proc. IEEE ICASSP, vol. 2, pp. 988-991, 1989. To achieve fast implementations, floating-point coefficients in these algorithms are usually scaled and approximated by integers such that floating-point multiplications in DCT/IDCT can be replaced by integer multiplications, and potentially with only additions and bitshifts. The use of fixed-point DCT/IDCT implementations results in approximation errors when compared with the ideal DCT/IDCT. Until recently, existing standards such as MPEG-1, 2, 4, have required that all approximate IDCTs meet certain accuracy requirements (the IEEE -1180 Standard, see below) to the true floating point IDCT. As different video/image decoders have the freedom to employ different IDCT fixed-point approximations, it leads to the mismatched reconstruction of the decoded frame relative to the encoder's model of the intended decoded picture. Subsequent inter-picture prediction from the aforementioned mismatched P-frame results in error accumulation and generates the so-called drifting effects. Annoying drifting problems can degrade decoded video quality quickly, especially when the encoder has a minimal amount of intra-block refresh.
Recently, MPEG considers to develop a new voluntary standard specifying a particular fixed-point approximation to the ideal IDCT function, and has a call for proposals on fixed-point 8×8 IDCT and DCT Standards in G. Sullivan and A. Luthra, “Call for proposals on fixed-point 8×8 IDCT and DCT standard,” MPEG input contribution N7099, April 2005. This Call for Proposals (CfP) requires an approximation that is within a specified degree of precision relative to the ideal function definition of the IDCT. And the accuracy requirements are compliant to the now-obsolete IEEE-1180 standard described in IEEE CAS Standards Committee, “IEEE standard specification for the implementation of 8×8 inverse discrete cosine transform,” IEEE Standard 1180-1990, December 1990, which has specified a set of well-known evaluation criteria for the accuracy of fixed-point IDCT implementations. The more recent ISO/IEC MPEG 23002-1 Standard, “Accuracy requirements for implementation of integer-output 8×8 inverse discrete cosine transform,” which is effectively identical, now replaces this standard; however, since the IEEE-1180 tests are still better known, we will continue to refer to them. In reality, the new MPEG 23002-1 Standard is implied.
Generally, a fixed-point IDCT algorithm implemented within a wider data bus width (e.g., the number of bits of representation needed for the transformation process) can have higher approximation accuracy than one with a lower bus width. However, the added data bus width is costly in VLSI implementation, and especially in mobile devices, where the CPU capability, bus width, and battery power are severely limited.
In the field of fixed-point IDCT designs (such as two fast algorithms in the public-domain JPEG source code), irrational multiplication parameters are directly scaled up and then approximated as the nearest integers, often suffering from poor approximation accuracy despite very high parameter resolution.
Therefore, it is an object of this invention to provide high-accuracy approximations of the IDCT that can still be implemented with a narrow bus width via simple arithmetic operations such as shifting operations and binary additions.
The current invention is related to the invention disclosed in the Reissue U.S. patent application Ser. No. 10/629,303 of U.S. Pat. No. 6,421,464, to Trac D. Tran and Pankaj Topiwala, both of which are hereby incorporated by reference in the entirety.
The invention provides systematic designs as well as implementation methods for a large class of fixed-point DCT and IDCT algorithms, which can all pass the IEEE-1180 tests (also known as the MPEG 23002-1 tests), maintain satisfactory compatibility with the floating-point IDCT, and along with similar forward transform counterparts form perfect invertible systems (signals passing throughout the claimed FDCT and IDCT pair get reconstructed perfectly without any error). The parameters in the structures of current invention are frequently approximated carefully by dyadic rationals, leading to fast and efficient multiplierless hardware implementation.
In one implementation, a series of dyadic-rational lifting steps in combination with the traditional butterfly of [1 1; −1 1] or [1 −1; 1 1] to construct highly-accurate multiplierless IDCTs are used that are compatible with either the floating-point IDCT or to general integer approximations of the IDCTs that are already deployed in practical image/video compression systems.
In accordance with the present invention, a systematic approach toward the design and implementation of the large family of IDCT algorithms includes the following steps:
The main features of the solutions according to the present invention are a series of approximate IDCT solutions that span the range of possibilities, as further explained below. With these examples, we indicate that the entire range of the complexity/accuracy scale can be efficiently achieved according to the present invention.
In one implementation, an apparatus is provided for block coding of windows of digitally represented signal elements of block size M, wherein M is any integer greater than unity, chosen from one of the dimensionalities of the signal. The apparatus comprises an invertible transformation of the signal elements, wherein the invertible transformation is an approximation of a discrete cosine transform (DCT) or an inverse discrete cosine transform (IDCT), and the approximation is representable by a series of at least one of the following elements: i) butterfly steps; ii) lifting steps, wherein the choice of coefficients is real; and iii) planar rotations. In one implementation, the lifting steps have rational coefficients.
In another implementation, the lifting steps have dyadic rational coefficients in the form of
where k and n are integers.
In yet another implementation, the lifting steps have coefficients selected within a
range from their theoretical values listed in Tables 1, 2, or 3 of this application.
In yet another implementation, the approximate transform is representable using only butterflies and lifting steps.
In yet another implementation, an apparatus for coding, storing, transmitting, or decoding length M blocks of digitally represented signal elements is provided, includes:
In yet another implementation, a method is provided for coding, storing, transmitting, or decoding length M blocks of digitally represented signal elements includes the steps of:
The Forward and Inverse Discrete Cosine Transform
Typically, a block transform for image processing is applied to a block (or window) of, for example, an M×N group of pixels (or in the inverse case, group of transform coefficients) and the process is iterated over the entire image. The block-wise IDCT (also known as a type-III DCT, unitary formulation) for an M×N block of inverse-quantized transform coefficients {circumflex over (F)}m,n (u, v) at position (mM, nN) in a picture or frame can be defined as follows:
for x=0 . . . M and y=0 . . . N.
The forward DCT (also known as a type-II DCT, unitary formulation) for an M×N block of spatial-domain samples f(mM+x, nN+y) at position (mM, nN) in a picture can be defined as follows:
for u=0 . . . M and v=0 . . . N. The constants used in these equations defined as follows:
cu=1/√{square root over (2)} for u=0, otherwise 1.
cv=1/√{square root over (2)} for v=0, otherwise 1.
In the JPEG and MPEG standards relevant to this invention, both M and N are equal to 8. Note that since the values of the cosine function that are used in these formulas are irrational numbers, any implementation of these functions requires some amount of approximation (although double-precision floating-point implementation can be considered effectively exact).
IDCT Sparse Factorizations and Rotation-Based Structure
An elegant factorization for eight-point IDCT was proposed in C. Loeffler, A. Lightberg, and G. Moschytz, “Practical fast 1-D DCT algorithms with 11 multiplications,” in Proc. IEEE ICASSP, vol. 2, pp. 988-991, 1989. The resulting structure is depicted in
A variation of this structure is depicted in
Lifting Step or Ladder Structure or Shear Operation
It is also well known that any plane rotation can be performed by 3 “shears” operations, which appear as upper- or lower-triangular matrices, as follows:
All equivalent relationships above can be easily verified by computation. In signal processing terminology, a “lifting” step is a matrix operation that is either upper- or lower-triangular whereas
are called lifting parameters. The signal processing flow diagram of this operation is shown in
One special case is when the rotation angle is 45-degree. In this case, the plane rotation becomes the special matrix
where the scaling factor 1/√{square root over (2)} is usually delayed and/or combined with others. The remaining orthogonal matrix
is commonly referred to as the butterfly operation. Since the butterfly is already integer-friendly (after all, it simply computes the sum and the difference of the two given inputs), we never have to convert the butterfly to lifting steps. Note that the inverse operation of a butterfly is another butterfly (the location of the −1 entry might vary).
To invert a lifting step, we simply need to subtract out what was added in at the forward lifting step. Hence, the original signal can still be perfectly reconstructed even if the floating-point multiplication results in the lifting steps are rounded to integers, as long as the same procedure is applied to both the forward and inverse routines. Therefore, perfect reconstruction is guaranteed by the lifting structure itself. This is the key to our construction of matching FDCT/IDCT pairs.
Lifting-Based Fixed-Point IDCT Construction
Starting with any existing rotation-based structure, each of the rotation angles is then converted to three lifting steps as shown in Section C. The lifting step at this stage usually has an irrational theoretical value, which in practice requires a very accurate floating-point architecture. In this invention, we propose a systematic approach to approximate this irrational parameter with a fixed-point structure that requires only simple binary shift-and-add operations. This method is demonstrated by a concrete example in
is converted into a cascade of three lifting steps with the parameter set
which is represented as the set of floating-point values {0.19891236737966,0.38268343236509,0.19891236737966} in a 64-bit architecture. We replace these cumbersome floating-point multiplications by approximating them as dyadic-rational parameters of the form
which can be expanded as sum/difference of pure power-of-two, and hence can be implemented by only binary shift and addition operations. In the concrete example shown in
Since 407/2048 and 49/128 can both be represented as sum/difference of pure powers-of-two as
we finally arrive at the resulting shift-and-add only implementation depicted in
Another method of implementing these dyadic lifting steps via shift-and-add operations is to realize the numerator by sums and differences of left-shifts (up-shifts) followed by a single right-shift (down-shift) to realize the denominator. For example, the multiplication of
using this approach can be implemented as ((x<<6)−(x<<4)+x+64)>>7 since
Our systematic approach of dyadic lifting approximation is then applied to every rotation angles in any rotation-based sparse IDCT factorization. If the structure in
Another example is the high-accuracy design whose objective is to approximate the true IDCT as closely as possible within a 32-bit fixed-point architecture. The resulting algorithm is shown in
Since the inverse of the butterfly is another butterfly while the inverse of the lifting step is another lifting step with reverse sign polarity, it is trivial to construct the corresponding forward transform as shown in
Another low-complexity 16-bit design is shown in
Dynamic Range in Implementation
In order to improve approximation accuracy, the input DCT vector coefficients X need to be up-scaled by certain K bits before they feed into IDCT. The value of K depends on the dynamic range of our IDCT scheme. However, we cannot directly assume X can be randomly generated. Instead, we calculate the maximum or minimum output of each subband by generating the worse-case inputs X=Cx where the integer vector x can be randomly assigned and C is the ideal type-II DCT matrix as described in Section A. As all lifting parameters are less than unity and implemented with addition and right-shift operations, they minimize the intermediate dynamic range. It can be verified that the absolute value of the worst intermediate result in each lifting steps is less than that of its final output.
For 8-bit video signal, input sample values to the DCT x after motion estimation and compensation are in the range of [−256, 255]. Hence, the DCT coefficients X can be shown to be within the 12-bit range [−2048, 2047]. The outputs of our 1-D IDCT design in
Therefore, the upper bound for the dynamic range of our proposed structure is 13-bit for the 12-bit inputs. That means, for the popular case that the input DCT coefficients are in the range of [−2048, 2047], K=3 is the limit for 16-bit IDCT implementations while K=11 is the upper limit for 24-bit and K=19 is the upper limit for 32-bit architectures.
IEEE-1180 Test
IEEE-1180 provides a set of specific criteria to measure the compliance of 8×8 IDCT to the ideal IDCT as discussed in IEEE CAS Standards Committee, “IEEE standard specification for the implementation of 8×8 inverse discrete cosine transform,” IEEE Standard 1180-1990, December 1990.
In this IEEE-1180 test, an 8×8 block of integers is randomly generated and fed into double-precision floating-point forward DCT. The output DCT coefficients are then passed through 64-bit floating-point IDCT and the proposed fixed-point IDCT, respectively. The accuracy is measured based on the reconstructed integers from these two IDCTs. Specifically, the peak pixel-wise error (ppe), peak mean-squared error (pmse), overall mean-square error (omse), peak mean error (pme), and overall mean error (ome) need to compute for the pseudo-random input blocks generated at 10000 and 1000000 iterations. The randomly generated block integers should cover the following five ranges, i.e., [−5, 5], [−256, 255], [−300, 300], [−384, 383] and [−512, 511] with positive and negative sign. A fixed-point IDCT could be considered to be compliant with IEEE-1180 standard if only it satisfies the conditions of ppe<=1, pmse<=0.06, omse<=0.02, pme<=0.015 and oms<=0.0015 for all the input ranges.
Table 4 lists the IEEE-1180 results of the proposed lifting-based IDCT for different K up-shift values, which represents the 32-bit, 24-bit and 16-bit implementation, respectively. Due to the limited space, Table 4 only includes the results for the three input ranges of common practical interest, i.e., [−256, 255], [−384, 383] and [−512, 511] with the positive sign at 1000000 iterations. In order to show pixel-wise errors more clearly, the percentage of pixel-wise errors (pep) is also included in the table. From these results, we can see that within the 32-bit word length constraint, the proposed lifting-based fixed-point IDCT solution delivers super high-accuracy approximation (omse=1.78e-04 for K=18); and 24-bit implementation also leads to a very high accuracy approximation (omse=5.95e-004 for K=10). The percentage of pixel-wise error clearly shows the proposed algorithm is very accurate (less than 0.06% of the reconstructed video pixels are different from the original video pixels and in those very rare cases, the mismatch is no more than 1 pixel value apart) for 24-bit and 32-bit implementations. Moreover, K=6 is the minimal up-scaling bits for our implementations in order to pass IEEE-1180 tests fully. Although the 16-bit implementation of K=3 in this particular structure cannot pass IEEE-1180 tests, it discloses an interesting observation that only high accurate lifting parameters would not certainly lead to high accuracy approximation. 10
00000 iterations
Drifting Tests in Practical MPEG-2 and MPEG-4 Coders
Due to the closely-matched nature of the forward-inverse lifting, the mismatch between our lifting-based IDCT and DCT is mitigated. In fact, perfect reconstruction (no mismatch at all) for the integer space in the 8-bit range [−256, 255] is achievable when the DCT coefficient range is extended to [−8192, 8191] (14-bit representation is needed instead of 12-bit). This bit expansion comes from the fact that there are still butterflies left in our structure shown in
Drifting is the effectively-random deviation of decoders from the values that are modelled in the encoder. Drifting tests are carried on the coders of MPEG-2 and MPEG-4. In the encoders, double-precision floating-point DCT and IDCT are used, and all pictures are coded as P-frames except the first one is coded as I-frame. To evaluate the drifting effects better, we consider the extreme worse-case scenario by disabling the intra macroblock refresh in encoder and setting quantization step size Qp=1. At the decoder side, the proposed lifting-based IDCT and double-precision floating-point IDCT are used to reconstruct the sequences, respectively. Peak signal-to-noise ratio (PSNR) values are then computed and their absolute difference values are used to evaluate the drifting effects.
The various embodiments come within the scope of the present invention. The inventors' preferred embodiments, which are described in detail herein, are exemplary of all possible embodiments which practice the spirit of the present invention. The discussion of these specific embodiments should not be construed as limiting the scope of the appended claims. It is understood that the above description is illustrative rather than limiting.
This application relates to and claims priority under 35 U.S.C 109(e) from U.S. Provisional Application Ser. No. 60/756,404, filed Jan. 5, 2006, entitled “Fast Fixed-Point Multiplier-less DCT/IDCT Approximation”, which is hereby incorporated by reference in the entirety.
Number | Date | Country | |
---|---|---|---|
60756404 | Jan 2006 | US |