Patent Grant
8660380
The invention pertains to methods and systems for performing two-dimensional (2D) transforms (e.g., inverse discrete cosine transforms or inverse Hadamard transforms) on 2D arrays of data values. In typical embodiments, the invention pertains to methods and circuitry for performing an inverse of a 2D orthogonal transform on a 2D array of image data values, each having a significant probability of being a zero value, in a manner reducing power consumption by avoiding power-consuming operations not needed to transform zero values.
Throughout the disclosure, the term “block” of video data is used to denote a subset of the data comprising a frame of video data having spatial location within a rectangular region of the frame. A block of video data can but need not consist of compressed (or otherwise encoded) video data. Examples of blocks of video data are the conventionally defined macroblocks of MPEG-encoded video frames.
In many conventional applications, image data (e.g., video data) or other data undergo a two-dimensional (“2D”) transform and the transformed data are later inverse transformed to recover the original data. Examples of such transforms include 2D discrete cosine transforms (two-dimensional “DCTs”), 2D Hadamard transforms, and 2D Fourier transforms.
Throughout the disclosure, the expression “bypassing” an operation (that would otherwise generate an operation output value) denotes generating or asserting a substitute output value (in place of the operation output value) without actually performing the operation. An example of “bypassing” an operation of asserting a zero value “z1” and a non-zero constant “c” to inputs of a multiplication circuit to cause the circuit to assert a current “cz1” at its output, asserting another zero value “z2” and different non-zero constant “d” to inputs of a second multiplication circuit to cause that circuit to assert a current “dz2” at its output, and operating an addition circuit in response to the currents “cz1” and “dz2” to assert an output voltage “cz1+dz2” (equal to zero volts above ground potential) at a node, would be to ground the node (thereby forcing it to ground potential) without actually performing the multiplication and addition steps in the multiplication circuits and addition circuit.
The present invention pertains to improved methods and systems for performing 2D transforms on 2D arrays of data values (i.e., arrays consisting of rows and columns of data values), where each of the values has a significant probability of being a zero value. In typical embodiments, the invention pertains to an improved method and system for performing an inverse transform of a 2D orthogonal transform (e.g., a 2D inverse discrete cosine transform or inverse Hadamard transform) on a 2D array of data values, where each of the values has a significant probability of being a zero value. In a class of preferred embodiments, the invention pertains to an improved method and system for performing a two-dimensional IDCT (2D inverse discrete cosine transform) on DCT coefficients. The DCT coefficients have been generated by performing a 2D discrete cosine transform on an array of video data (or other image data), and each has a significant probability of having the value zero.
Throughout this disclosure, the expression “zero value” (or “zero data value”) denotes data indicative of the value zero. Similarly, the expression “zero input data value” denotes input data indicative of the value zero. For example, a zero input value can be a word of input data (e.g., a DCT coefficient, or a color component or pixel of video data) having the value zero.
Throughout this disclosure, the expression “sparse” data (e.g., a sparse block of data to undergo an inverse transform) denotes data indicative of values that are likely to be zero values. For example, a block of input data (e.g., a block of DCT coefficients) indicative of relatively many zero values and relatively few non-zero values is a sparse block of data.
Inverse transform implementation is typically a major part of the implementation of any system to be compliant any video compression and decompression standard. It is a computationally intensive process and contributes significantly to processing cycle and power consumption requirements. Mobile devices that implement video compression and decompression standards (e.g., portable media players) have especially stringent processing cycle and power consumption requirements: they need to meet the stringent performance requirements set by the application and to consume very low power to maximize battery life; and the transform engine typically must be able to support multiple compression standards and varying requirements that come with these standards.
Typical conventional implementations of 2D transforms (including 2D inverse transforms) on blocks of data use the following techniques in different combinations to improve performance or reduce power:
1. avoiding transformation of blocks that are identified by an external means as being uncoded blocks (where each input block provided to the transform engine is identified by the external means as being a coded or uncoded block). However, this technique has disadvantages, including in that it can result in performance of unnecessary transform operations (e.g., transformation of blocks that are identified as coded blocks but consist only of zero DC coefficients);
2. identifying full rows or columns of each input data block that consist entirely of zero values (“zero-rows” or “zero-columns”) and bypassing normal transform operations that would otherwise be performed on each such row or column (e.g., by outputting predetermined values, typically “zero,” for each zero-row or zero-column). The zero-rows and zero-columns can either be specified by an external device or identified internally by the transform engine. However, this conventional technique does not improve performance or reduce power in many common situations in which a row (or column) is not a zero-row (or zero-column) but is a sparsely populated row (or column) including only a very small number of non-zero values;
3. identifying (from the input data) conditions that indicate that the same coefficients (previously determined for use in multiplying data values in an input data row or column) should be used for multiplying data values in a subsequent input data row or column, and avoiding the updating of such coefficients that would otherwise be performed to determine new coefficients for multiplying the data values in the subsequent input data row or column; and
4. implementing a distributed arithmetic transform (a lookup table-based implementation of a 2D transform). A typical lookup table-based implementation reduces overhead by reducing the number of multiplication operations that must be performed to transform a block. However, designing such an implementation is typically very complicated because very large ROM tables and also multi-ported ROM are typically required, and design constraints typically limit the improvement in power consumption that can be achieved.
In another conventional 2D transform, described in US Patent Application Publication No. 2005/0033788 and related U.S. Pat. No. 6,799,192, the last non-zero entry in each column of a block of data is determined (when performing a column transform phase of an IDCT), and the transform system then branches to an appropriate one of eight different “specialized IDCT” program routines for implementing IDCT operations in software to inverse-transform each column. Apparently, simpler transform operations (requiring fewer multiplication and addition operations) could be employed to process a column having relatively many zeros (as indicated by having the last non-zero value in a higher position) and more complicated transform operations (requiring more multiplication and addition operations) could be employed to process a column having fewer zeros (as indicated by having the last non-zero value in a lower position). The references also teach that when performing a row transform phase of the IDCT (after the column transform phase), the last non-zero entry in each row of a block is determined and the transform system then branches to an appropriate one of eight different “specialized IDCT” program routines for implementing IDCT operations in software to inverse-transform each row.
There are a number of problems and limitations with the technique described in US Patent App. Publication No. 2005/0033788 and U.S. Pat. No. 6,799,192, including that the technique is inefficient in the sense that it does not improve performance or reduce power consumption when processing many columns and rows having typical patterns of zero and non-zero values. For example, when a column or row to be transformed includes zeros (especially, many zeros) but has a last entry that is non-zero, the technique would select a complicated (e.g., the most complicated) “specialized IDCT” routine that consumes much power to transform the column or row. In contrast, preferred embodiments of the present invention improve performance and reduce power consumption by avoiding transform operations on portions of rows and columns that consist of zero values (e.g., on each half-row or half-column, or each quarter-row or quarter-column, that consists of zero values) or performing such transform operations in a reduced-power manner. Some preferred embodiments of the present invention improve performance and reduce power consumption by avoiding transform operations on each individual zero value in a row or column to be transformed (or performing transform operations on each individual zero value in a row or column in a reduced-power manner).
There is no suggestion in US Patent App. Publication No. 2005/0033788 or U.S. Pat. No. 6,799,192 that the performance improvement and power consumption reduction benefits achievable by the technique disclosed therein can be increased by independently processing subsets of each row or column to be transformed, and not suggestion as to how to do so or as to whether it is possible to do so. In contrast, preferred embodiments of the present invention can sequentially perform the same operations on different subsets of each row or column to be transformed (e.g., inverse transformed), where the subsets of each row or column determine a partition of the row or column, and the performance improvement and power consumption reduction benefits achievable by such embodiments can be increased simply by decreasing the size of the subsets that determine each such partition. For example, some preferred embodiments of the present invention sequentially perform sets of operations on 2N-bit subsets of each 8N-bit row or column to be transformed (four sets of operations per row or column) to achieve excellent performance improvement and power consumption reduction benefits, and other preferred embodiments of the invention sequentially perform sets of operations on N-bit subsets of each 8N-bit row or column to be transformed (eight sets of operations per row or column) to achieve even better performance improvement and power consumption reduction benefits.
Another conventional 2D transform is described in the paper by Rohini Krishnan, et al., entitled “Design of a 2D DCT/IDCT Application Specific VLIW Processor Supporting Scaled and Sub-sampled Blocks,” 16th International Conference on VLSI Design, six pages (2003). This paper teaches asserting a downscaled version of full data block (e.g., an 8×4 block that has been generated by discarding even rows of an 8×8 block) to IDCT circuitry, and operating the IDCT circuitry to inverse-transform the downscaled block including by bypassing some of the IDCT circuitry that could otherwise have been used to inverse-transform the full block. This method can avoid calculation of output values that will eventually be discarded, but does not detect and skip operations that will not contribute in any way to the final result.
Another conventional 2D transform is described in U.S. Pat. No. 5,883,823. This transform identifies regions of an input block to be transformed, and processes each region differently (e.g., an IDCT is performed on all elements of some regions and an IDCT is performed only on non-zero elements of other regions). For example, U.S. Pat. No. 5,883,823 apparently teaches (at col. 10, line 53-col. 11, line 26) an IDCT computation in which a “regional” IDCT calculation is performed on all elements (whether zero or non-zero) of one quadrant of an 8×8 block (i.e., the 4×4 quadrant corresponding to the lowest frequency ranges), and another IDCT calculation is performed only on non-zero elements of each of the other three 4×4 quadrants of the 8×8 block (i.e., the three 4×4 quadrants corresponding to higher frequency ranges). However, U.S. Pat. No. 5,883,823 does not teach or suggest how to identify non-zero elements of each region for which an IDCT calculation is to be performed only on non-zero elements (or how efficiently to identify such non-zero coefficients), or how to perform an IDCT calculation only on non-zero elements of a region of a block, or how efficiently (and in a manner consuming reduced power) to perform such an IDCT calculation only on such non-zero elements.
In a class of embodiments, the invention is a system configured to perform a 2D transform (e.g., an inverse discrete cosine transform) on each block of a sequence of input data blocks, where each block comprises rows and columns of input data values, and the 2D transform includes a row transform and a column transform. In these embodiments, the system is configured to perform the 2D transform either by performing the row transform on all rows of each block to generate a block of partially transformed data and then the column transform on each column of the block of partially transformed data, or by performing the column transform on all columns of each block to generate a block of partially transformed data and then the row transform on each row of the block of partially transformed data. To simplify the description, we shall describe embodiments in the class that are configured to perform the 2D transform by performing the row transform on all rows of each block to generate a block of partially transformed data and then the column transform on each column of the block of partially transformed data. It should be understood that all references to “row” and “column” can be replaced by references to “column” and “row,” respectively, to describe other embodiments in the class.
Herein, the term “subset” of a set (e.g., a row or column) of data values is used in a broad sense and can denote a row (or column) of data values, even elements of a row (or column) of data values, odd elements of a row (or column) of data values, every Nth data value in a row (or column) of data values, even elements of a row or column of data values in a bit-reversed order (suitable for FFT butterflies), or another subset of data values.
To implement the row transform, a system in the noted class includes circuitry configured to perform transformation operations (typically including multiplications and additions) on the input data values of each row of an input data block to generate a block of partially transformed data. The system typically includes a buffer in which the partially transformed data are stored. To implement the column transform, the system includes circuitry configured to perform transformation operations (typically including multiplications and additions) on the data values of each column of the block of partially transformed data. Each embodiment in the noted class is configured so that, when performing the row transform on each row of input data, it determines whether each of different subsets of the data values comprising a first partition of the row includes at least one zero value (e.g., consists of zero values), determines whether each of different subsets of a first subset of the first partition of the row includes at least one zero value, and determines whether each of different subsets of at least one other subset of the first partition of the row includes at least one zero value, and when performing the row transform on each said row that includes at least one zero input data value and at least one non-zero input data value, at least one transformation operation on at least one (and preferably on each) said zero input data value is bypassed or performed in a reduced-power manner, where such transformation operation would otherwise be performed in a manner consuming full power if the zero value were a non-zero value (e.g., at least one multiplication or addition is bypassed that would otherwise be performed using multiplication and addition circuitry). When implementing the row transform on each row that includes at least one zero input data value and at least one non-zero input data value, circuitry in some such embodiments for performing the transformation operation on the zero input value is operated without updating at least one of its inputs to avoid consuming power that would otherwise be consumed to toggle each such input.
Preferably, an embodiment in the noted class is also configured so that, when performing the column transform on each column of partially transformed data, it determines whether each of different subsets of the data values comprising a first partition of the column includes at least one zero value (e.g., consists of zero values), determines whether each of different subsets of a first subset of the first partition of the column includes at least one zero value, and determines whether each of different subsets of at least one other subset of the first partition of the column includes at least one zero value, and when performing the column transform on each said column that includes at least one zero value of the partially transformed data and at least one non-zero value of the partially transformed data, it bypasses (or performs in a reduced-power manner) at least one of the transformation operations that it would otherwise perform in a manner consuming full power on at least one (and preferably on each) said zero value of the partially transformed data value if said value were a non-zero value (e.g., at least one multiplication or addition that would otherwise be performed using multiplication and addition circuitry). For example, when implementing the column transform on each column that includes at least one zero data value and at least one non-zero data value, one such embodiment is configured to bypass a transformation operation on a zero value of the column that would otherwise be performed if the zero value were a non-zero value (e.g., the embodiment bypasses circuitry for performing the transformation operation).
Preferred embodiments of the invention determine whether each block of data to be transformed consists entirely of zero values. Upon determining that a block consists entirely of zero values, transformation operations (both row and column transform operations) on the values of the block are bypassed or performed in a reduced power manner. These preferred embodiments also sequentially (e.g., iteratively) determine whether each of a number of different subsets of each row or column of a block of data to be transformed includes at least one zero value. An example of such an embodiment will refer to a row (or column) of data consisting of values xi, where i is an integer in the range 0≦i≦N−1, and N is an even integer, a partition of the row (or column) into a first subset of data values and a second subset of data values distinct from the first subset, a partition of the first subset into a third subset of data values and a fourth subset distinct from the third subset, and a partition of the second subset into a fifth subset of data values and a sixth subset distinct from the fifth subset. The exemplary embodiment determines whether the first subset consists entirely of zero values and whether the second subset consists entirely of zero values. Typically, where the row (or column) consists of cosine transform coefficients (generated by performing a DCT on frames of video data), the first subset consists of low frequency coefficients (values xi, where i is an integer in the range 0≦i≦(N/2)−1), the second subset consists of high frequency coefficients (values xi, where i is an integer in the range N/2≦i≦N−1), and the second subset has a significant probability of consisting only of zero values (and has a much higher probability of consisting only of zero values than does the first subset).
The exemplary embodiment, upon determining that the first subset consists entirely of zero values, bypasses transformation operations on the values in the first subset or performs them in a reduced power manner (e.g., circuitry for performing these operations is bypassed, or the circuitry is operated with at least one of its inputs not being updated to avoid consuming power that would otherwise be consumed to toggle each such input). Upon determining that the second subset consists entirely of zero values, transformation operations on the values in the second subset are bypassed or performed in a reduced power manner.
Upon determining that each of the first subset and the second subset includes at least one non-zero value, the exemplary embodiment determines whether each of the third subset, the fourth subset, the fifth subset, and the sixth subset consists entirely of zero values. In a typical implementation in which the first subset consists of low frequency coefficients (values xi, where i is an integer in the range 0≦i≦(N/2)−1), the second subset consists of high frequency coefficients (values xi, where i is an integer in the range N/2≦i≦N−1), the third subset consists of the even values of the first subset (values xi, where i is an even integer in the range 0≦i≦(N/2)−1), the fourth subset consists of the odd values of the first subset (values xi, where i is an odd integer in the range 0≦i≦(N/2)−1), the fifth subset consists of the even values of the second subset, and the sixth subset consists of the odd values of the second subset. For each of the third subset, the fourth subset, the fifth subset, and the sixth subset that is determined to consist entirely of zero values, transformation operations on the values of such subset are bypassed or performed in a reduced power manner (e.g., circuitry for performing these operations is bypassed, or the circuitry is operated without updating at least one of its inputs to avoid consuming power that would otherwise be consumed to toggle each such input). For each of the third subset, the fourth subset, the fifth subset, and the sixth subset that is determined to include at least one non-zero value, transformation operations are performed in a manner consuming full power on the values of such subset.
In variations on the above-described exemplary embodiment (and in other embodiments of the invention), data values comprising each row or column (or a subset of a row or column) of a block to be transformed are reordered prior to or during the determination as to whether each distinct subset comprising a partition of the row or column (or subset thereof) consists entirely of zero values (or is a zero value). For example, if a row consists of values xi, where i is an integer in the range 0≦i≦N−1, where N is an even integer, the partition consists of distinct first and second subsets of the row, the first subset consists of distinct third and fourth subsets of the row, and the second subset consists of distinct fifth and sixth subsets of the row, the first subset consists of values xi, where i is an integer in the range 0≦i≦(N/2)−1), the second subset consists of high frequency coefficients (values xi, where i is an integer in the range N/2≦i≦N−1),
the third subset can consist of the values xi, where i is in the range 0≦i≦(N/4)−1), which are even values of a reordered version of the first subset,
the fourth subset can consist of the values xi, where i is in the range N/4≦i≦(N/2)−1), which are odd values of the reordered version of the first subset,
the fifth subset can consist of the values xi, where i is in the range N/2≦i≦(3N/4)−1), which are even values of a reordered version of the second subset, and
the sixth subset can consist of the values xi, where i is in the range 3N/4≦i≦N−1), which are odd values of the reordered version of the second subset.
In another class of embodiments, the invention is a method for performing a 2D transform on a sequence of input data blocks, each of the blocks comprising rows and columns of input data values, and the 2D transform including a row transform and a column transform. In typical embodiments in the class, each block is a block of DCT coefficients that have been generated by performing a DCT on video data and the 2D transform is an inverse discrete cosine transform. In these typical embodiments and in other embodiments in the class, many or all of the blocks (e.g., at least substantially all of the blocks) in the sequence are blocks of sparse data. The 2D transform can include the steps of performing the row transform on all rows of each input data block to generate a block of partially transformed data and then performing the column transform on each column of the block of partially transformed data, or the steps of performing the column transform on all columns of each input data block to generate a block of partially transformed data and then performing the row transform on each row of the block of partially transformed data. To simplify the description, we shall describe embodiments in the class in which the 2D transform includes the steps of performing the row transform on all rows of each input data block to generate a block of partially transformed data and performing the column transform on each column of the block of partially transformed data. It should be understood that all references to “row” and “column” can be replaced by references to “column” and “row,” respectively, to describe other embodiments in the class.
In some embodiments in the noted class, the method includes the steps of:
(a) performing the row transform on each row of one of the input data blocks, including by performing transformation operations on input data values of each said row, to generate a partially transformed data block; and
(b) performing the column transform on each column of the partially transformed data block, including by performing additional transformation operations on data values of each said column, wherein step (a) includes the steps of:
determining whether each of different subsets of the data values comprising a first partition of each said row includes at least one zero value (e.g., consists of zero values), determining whether each of different subsets of a first subset of the first partition includes at least one zero value, and determining whether each of different subsets of at least one other subset of the first partition includes at least one zero value; and
when performing the row transform on each said row that includes at least one zero input data value and at least one non-zero input data value, bypassing (or performing in a reduced power manner) at least one of the transformation operations that would otherwise be performed on at least one (and preferably on each) said zero input data value in a manner consuming full power if each said zero input value were a non-zero value (e.g., bypassing at least one multiplication or addition that would otherwise be performed using multiplication and addition circuitry on at least one (and preferably on each) said zero input data value of the row).
For example, when implementing the row transform on each row that includes at least one zero input data value and at least one non-zero input data value, step (a) includes the step of operating a multiplication circuit having a first input and a second input to perform a multiplication operation (in which the zero input value, asserted to the first input, is multiplied by a second value asserted to the second input) without updating the value asserted to the second input to avoid consuming power that would otherwise be consumed to toggle the second input.
Preferably, step (b) includes the steps of:
determining whether each of different subsets of the data values comprising a partition of each said column includes at least one zero value (e.g., consists of zero values), determining whether each of different subsets of a first subset of the partition includes at least one zero value, and determining whether each of different subsets of at least one other subset of the partition includes at least one zero value; and
when performing the column transform on each said column that includes at least one zero input data value and at least one non-zero input data value, bypassing (or performing in a reduced power manner) at least one of the additional transformation operations that would otherwise be performed on at least one (and preferably on each) said zero input data value in a manner consuming full power if each said zero input value were a non-zero value (e.g., bypassing at least one multiplication or addition that would otherwise be performed using multiplication and addition circuitry on at least one (and preferably on each) said zero input data value of the column).
Advantages of transform circuitry implemented in accordance with typical embodiments of the present invention include:
improved (and preferably optimized) computation efficiency (which allows lower frequency of operation) due to avoidance of redundant or otherwise unnecessary computations or computation steps (e.g., the performance of typical embodiments of the inventive transform engine, in transforming blocks of input data values, directly scales with the number of non-zero input data values per block. This is at a finer level of granularity than for conventional transform engines whose performance, in transforming blocks of input data values, scales with the number of rows including at least one non-zero value per block);
reduced switching activity reduces power consumption;
provision of intelligent intermediate buffer memory management (in preferred embodiments); and
faster performance of integer transforms by avoiding redundant or otherwise unnecessary computations or computation steps (e.g., avoiding unnecessary multiplier input toggling).
In some embodiments, the inventive system is a video processing system (e.g., a pipelined video decoding system) including a transform engine implemented in accordance with the invention. In some such embodiments, the video processing system is configured to be operable as a video processing subsystem of a portable media player. In other embodiments, the inventive system is a portable media player including a video processing subsystem that includes a transform engine implemented in accordance with the invention.
Other aspects of the invention are transform engines and transform engine circuitry for use in any embodiment of the inventive system, and methods performed during operation of any embodiment of the inventive system.
Embodiments of the inventive system will be described with reference to
The system of
It is contemplated that some embodiments of the invention are implemented by systems that do not have the structure shown in
A typical conventional transform engine is configured to identify full rows or columns (of each block to be transformed) that consist entirely of zero values (i.e., “zero-rows” or “zero-columns”) and to bypass normal transform operations that would otherwise be performed on each identified zero-row or zero-column. Such a conventional transform engine would identify rows R1 and R3 of block I as zero-rows and bypass transform computations that it would otherwise perform on rows R1 and R3.
In contrast, a typical embodiment of the inventive transform engine (e.g., an implementation of the
Consider the case that such an embodiment of the inventive transform engine has the structure shown in
Circuitry 7 of one such embodiment of the inventive transform engine is configured to determine whether the two lowest frequency coefficients of each row of each block of DCT transform coefficients consist entirely of zero values, to determine whether the two highest frequency coefficients of each such row consist entirely of zero values, to bypass row transform computations on the two lowest frequency coefficients of each row if they consist entirely of zero values (where such computations would otherwise be performed in a manner consuming full power on these coefficients if they did not consist entirely of zero values) or to perform such computations in a reduced-power manner, and to bypass row transform computations on the two highest frequency coefficients of each row if they consist entirely of zero values (where such computations would otherwise be performed in a manner consuming full power on these coefficients if they did not consist entirely of zero values) or to perform such computations in a reduced-power manner. For example, circuitry 7 of such embodiment can be configured to bypass row transform computations on the two highest frequency coefficients of each of rows R1, R2, R2, and R4 of block I.
Circuitry 7 of such embodiment of the inventive transform engine is also configured to determine whether the even coefficient of the two lowest frequency components of each row of each block of DCT transform coefficients asserted thereto (e.g., each coefficient in column “b” of block I of
Circuitry 11 of such embodiment of the inventive transform engine is preferably also configured to determine whether the first half (i.e., the first two data values) of each column of each 4×4 block of partially transformed coefficients read from buffer 9 consists entirely of zero values, to determine whether the second half (the last two data values) of each such column consists entirely of zero values, to bypass column transform computations on the first half of each column if it consists entirely of zero values (where such computations would otherwise be performed in a manner consuming full power on these values if they did not consist entirely of zero values) or to perform such computations in a reduced-power manner, and to bypass column transform computations on the second half of each column if it consists entirely of zero values (where such computations would otherwise be performed in a full power manner on these values if they did not consist entirely of zero values) or to perform such computations in a reduced-power manner.
Circuitry 11 of such embodiment of the inventive transform engine is also configured to determine whether the even data value in the first half of each column of each of 4×4 block of partially transformed coefficients read from buffer 9 (e.g., each value in row R2 of block II of
In variations on the above example in which each row and column comprises 8 data values, each step of determining whether the odd (or even) data value in the first (or second) half of a row (or column) is a zero value, is replaced by a step of determining whether the data values in the first (or second) half of said half of the row (or column) consist entirely of zero values, and the engine is configured to bypass row (or column) transform computations on each half of each half row (or half column) consisting entirely of zero values (where such computations would otherwise be performed in a full power manner on these values if they did not consist entirely of zero values) or to perform such computations in a reduced-power manner. Preferably, the engine is also configured to determine whether the odd data value in each such half row (or half column) is a zero value, to determine whether the even data value of each such half row (or half column) is a zero value, and to perform in a reduced-power manner a multiplication computation that would otherwise be performed (in a manner consuming more power) on each such data value that is determined to be a zero value to transform the relevant row or column.
In variations on either above-described example, data values comprising each row or column (or a subset of a row or column) of a block to be transformed by circuitry 7 or 11 are reordered prior to or during the determination as to whether each distinct subset comprising a partition of the row or column (or subset thereof) consists entirely of zero values (or is a zero value). For example, if a row (or column) consists of values xi, where i is an integer in the range 0≦i≦N−1, where N is an even integer, the partition consists of distinct first and second subsets of the row (column), the first subset consists of distinct third and fourth subsets of the row (column), and the second subset consists of distinct fifth and sixth subsets of the row (column), the first subset consists of values xi, where i is an integer in the range 0≦i≦(N/2)−1), the second subset consists of values xi, where i is an integer in the range N/2≦i≦N−1,
the third subset can consist of the values xi, where i is in the range 0≦i≦(N/4)−1), or it can consist of the even values of the first subset, or it can consist of the even values of a reordered version of the first subset,
the fourth subset can consist of the values xi, where i is in the range N/4≦i≦(N/2)−1), or it can consist of the odd values of the first subset, or it can consist of the odd values of a reordered version of the first subset,
the fifth subset can consist of the values xi, where i is in the range N/2≦i≦(3N/4)−1), or it can consist of the even values of the second subset, or it can consist of the even values of a reordered version of the second subset, and
the sixth subset can consist of the values xi, where i is in the range 3N/4≦i≦N−1), or it can consist of the odd values of the second subset, or it can consist of the odd values of a reordered version of the second subset.
With reference again to row R4 of block I of
With reference again to
Preferred embodiments of the inventive transform engine reduce power consumption by detecting whether individual data values of a block to be transformed (e.g., performing zero-detection on individual coefficients of a block of DCT coefficients to undergo an inverse direct cosine transform, on a coefficient by coefficient basis), and for all data values that are determined to be zero values, ensuring that at least one (and preferably each) input to at least one multiplier to be employed to transform at least one said zero value does not switch from its earlier value. This reduces dynamic power consumption in the engine.
In preferred implementations of an embodiment of the inventive transform engine having the structure shown in
Alternatively, when a symmetric transform being performed in accordance with an embodiment of the inventive transform engine having the structure shown in
In order to perform a conventional H264 based transform (an integer transform that can be performed without real multiplication) in accordance with the invention, the inventive transform engine can be implemented without multipliers. The H264 based transform can be implemented using shifters rather than multipliers. Bypassing transform circuitry in accordance with the invention can speed up the transform computations (and reduce the energy consumed thereby) even in these implementations.
We next describe a class of embodiments of the invention in more detail with reference to the flow chart of
A typical block to be inverse transformed in accordance with the invention has four coefficients per row (e.g., if the inverse transform is an H264 based transform) or eight coefficients per row (e.g., if the inverse transform is an IDCT on 8×8 blocks of frequency coefficients).
Before performing the steps shown in
Upon initiation (step 20) of a row transform to be performed in accordance with the
In step 24, the transform engine determines whether the current row (the row read during the most recent performance of step 22) includes at least one non-zero value. If not, the transform engine outputs zeroes (step 25) indicative of a row transformed version of the row (without performing actual row transform computations on the data values of the row) and reads the next row from the buffer memory (the next performance of step 22). If the current row includes at least one non-zero value, the engine determines (in step 26) whether the first half of the current row (e.g., the first two data values of a row consisting of four data values) includes at least one non-zero value.
If the first half of the current row does not include at least one non-zero value, the engine outputs zeroes (step 28) indicative of a row transformed version of the first half of the row (without performing actual row transform computations on the data values of the first half of the row) and determines (in step 40) whether the second half of the current row (e.g., the second two data values of a row consisting of four data values) includes at least one non-zero value.
If the first half of the current row includes at least one non-zero value, the engine determines (in step 30) whether the first half of the first half of the current row includes at least one non-zero value and determines (in step 32) whether the second half of the first half of the current row includes at least one non-zero value.
If the first half of the current row's first half includes at least one non-zero value (e.g., if it consists of a single non-zero value, or consists of two values including at least one non-zero value), the engine performs (in step 36) row transform computations on the data values of the first half of the first half of the row and stores (in step 39) the resulting transformed data in a buffer memory. The buffer memory employed to perform step 39 can either be a buffer memory coupled to an output of circuitry 11 of
If the first half of the current row's first half does not include a non-zero value, the engine performs step 37 in which it:
(a) outputs one or more zeroes indicative of a row transformed version of the first half of the current row's first half (without performing actual row transform computations on the data value or values of the first half of the current row's first half). Each such zero value is then stored (in step 39) in a buffer memory; or
(b) outputs at least one data value generated by performing row transform computations in a reduced-power manner on the data value(s) of the first half of the current row's first half. Each such output value is then stored (in step 39) in a buffer memory. For example, the engine can do this by operating multiplication circuitry having a first input set (comprising at least one input) to which at least one constant is asserted and a second input set (comprising at least one input) to which each data value of the first half of the current row's first half is asserted, to perform at least multiplication operation on each data value of the first half of the current row's first half without updating the value asserted to at least one of the first input set and the second input set, to avoid consuming power that would otherwise be consumed to toggle the relevant input(s) of the multiplication circuitry. In the case that the engine is performing an IDCT, each constant asserted to the first input set is a cosine constant, and in one implementation, whenever a data value asserted to the second input set is a zero value the engine prevents the cosine constant to be multiplied with said zero value from being updated (changed from its previous value) since the result of the multiplication would be a zero regardless of the cosine constant's value. In another implementation, the engine prevents the updating of a cosine constant asserted to the first input set (to be multiplied with a zero data value) and prevents the updating of the data value being asserted to the corresponding input of the second input set (to prevent toggling of either multiplier input), and asserts a zero value (e.g., multiplexes a zero value into output of the processing pipeline) indicative of the result of multiplying the zero data value with the cosine constant (without actually multiplying together these two operands); or
(c) outputs one or more zeroes indicative of a row transformed version of at least one value in the first half of the current row's first half (without performing actual row transform computations on such data value or values), and outputs at least one data value generated by performing row transform computations in a reduced-power manner on at least one other data value of the first half of the current row's first half. Each such zero value and output value is then stored (in step 39) in a buffer memory.
If the second half of the current row's first half includes at least one non-zero value (e.g., if it consists of a single non-zero value or consists of two values including at least one non-zero value), the engine performs (in step 34) row transform computations on the data values of the second half of the first half of the row and stores (in step 39) the resulting transformed data in a buffer memory.
If the second half of the current row's first half does not include a non-zero value, the engine performs step 35 in which it:
(a) outputs one or more zeroes indicative of a row transformed version of the second half of the current row's first half (without performing actual row transform computations on the data value or values of the second half of the current row's first half). Each such zero value is then stored (in step 39) in a buffer memory; or
(b) outputs at least one data value generated by performing row transform computations in a reduced-power manner on the data value(s) of the second half of the current row's first half. Each such output value is then stored (in step 39) in a buffer memory. For example, the engine can do this by operating multiplication circuitry having a first input set (comprising at least one input) to which at least one constant is asserted and a second input set (comprising at least one input) to which each data value of the second half of the current row's first half is asserted, to perform at least multiplication operation on each data value of the second half of the current row's first half without updating the value asserted to at least one of the first input set and the second input set, to avoid consuming power that would otherwise be consumed to toggle the relevant input(s) of the multiplication circuitry. In the case that the engine is performing an IDCT, each constant asserted to the first input set is a cosine constant, and in one implementation, whenever a data value asserted to the second input set is a zero value the engine prevents the cosine constant to be multiplied with said zero value from being updated (changed from its previous value) since the result of the multiplication would be a zero regardless of the cosine constant's value. In another implementation, the engine prevents the updating of a cosine constant asserted to the first input set (to be multiplied with a zero data value) and prevents the updating of the data value being asserted to the corresponding input of the second input set (to prevent toggling of either multiplier input), and asserts a zero value (e.g., multiplexes a zero value into the output of the processing pipeline) indicative of the result of multiplying the zero data value with the cosine constant (without actually multiplying together these two operands); or
(c) outputs one or more zeroes indicative of a row transformed version of at least one value in the second half of the current row's first half (without performing actual row transform computations on such data value or values), and outputs at least one data value generated by performing row transform computations in a reduced-power manner on at least one other data value of the second half of the current row's first half. Each such zero value and output value is then stored (in step 39) in a buffer memory.
After all data values generated or asserted in steps 34 and 36, steps 34 and 37, steps 35 and 36, or steps 35 and 37 have been stored (step 39) in the buffer memory, the engine determines (in step 40) whether the second half of the current row includes at least one non-zero value. If the second half of the current row does not include at least one non-zero value, the engine outputs zeroes (step 41) indicative of a row transformed version of the second half of the row (without performing actual row transform computations on the data values of said second half of the row) and reads the next row to be transformed from buffer memory (another performance of step 22).
If the second half of the current row includes at least one non-zero value, the engine determines (in step 42) whether the first half of the second half of the current row includes at least one non-zero value and determines (in step 44) whether the second half of the second half of the current row includes at least one non-zero value.
If the first half of the second half of the current row includes at least one non-zero value (e.g., if it consists of a single non-zero value, or consists of two values including at least one non-zero value), the engine performs (in step 48) row transform computations on the data values of the first half of the second half of the row and stores (in step 50) the resulting transformed data in a buffer memory. The buffer memory employed to perform step 50 can either be a buffer memory coupled to an output of circuitry 11 of
If the first half of the current row's second half does not include a non-zero value, the engine performs step 49 in which it:
(a) outputs one or more zeroes indicative of a row transformed version of the first half of the current row's second half (without performing actual row transform computations on the data value or values of the first half of the current row's second half). Each such zero value is then stored (in step 50) in a buffer memory; or
(b) outputs at least one data value generated by performing row transform computations in a reduced-power manner on the data value(s) of the first half of the current row's second half. Each such output value is then stored (in step 50) in a buffer memory. For example, the engine can do this by operating multiplication circuitry having a first input set (comprising at least one input) to which at least one constant is asserted and a second input set (comprising at least one input) to which each data value of the first half of the current row's second half is asserted, to perform at least multiplication operation on each data value of the first half of the current row's second half without updating the value asserted to at least one of the first input set and the second input set, to avoid consuming power that would otherwise be consumed to toggle the relevant input(s) of the multiplication circuitry. In the case that the engine is performing an IDCT, each constant asserted to the first input set is a cosine constant, and in one implementation, whenever a data value asserted to the second input set is a zero value the engine prevents the cosine constant to be multiplied with said zero value from being updated (changed from its previous value) since the result of the multiplication would be a zero regardless of the cosine constant's value. In another implementation, the engine prevents the updating of a cosine constant asserted to the first input set (to be multiplied with a zero data value) and prevents the updating of the data value being asserted to the corresponding input of the second input set (to prevent toggling of either multiplier input), and asserts a zero value (e.g., multiplexes a zero value into the output of the processing pipeline) indicative of the result of multiplying the zero data value with the cosine constant (without actually multiplying together these two operands); or
(c) outputs one or more zeroes indicative of a row transformed version of at least one value in the first half of the current row's second half (without performing actual row transform computations on such data value or values), and outputs at least one data value generated by performing row transform computations in a reduced-power manner on at least one other data value of the first half of the current row's second half. Each such zero value and output value is then stored (in step 50) in a buffer memory.
If the second half of the second half of the current row includes at least one non-zero value (e.g., if it consists of a single non-zero value, or consists of two values including at least one non-zero value), the engine performs (in step 46) row transform computations on the data values of the second half of the second half of the row and stores (in step 50) the resulting transformed data in a buffer memory.
If the second half of the current row's second half does not include a non-zero value, the engine performs step 47 in which it:
(a) outputs one or more zeroes indicative of a row transformed version of the second half of the current row's second half (without performing actual row transform computations on the data value or values of the second half of the current row's second half). Each such zero value is then stored (in step 50) in a buffer memory; or
(b) outputs at least one data value generated by performing row transform computations in a reduced-power manner on the data value(s) of the second half of the current row's second half. Each such output value is then stored (in step 50) in a buffer memory. For example, the engine can do this by operating multiplication circuitry having a first input set (comprising at least one input) to which at least one constant is asserted and a second input set (comprising at least one input) to which each data value of the second half of the current row's second half is asserted, to perform at least multiplication operation on each data value of the second half of the current row's second half without updating the value asserted to at least one of the first input set and the second input set, to avoid consuming power that would otherwise be consumed to toggle the relevant input(s) of the multiplication circuitry. In the case that the engine is performing an IDCT, each constant asserted to the first input set is a cosine constant, and in one implementation, whenever a data value asserted to the second input set is a zero value the engine prevents the cosine constant to be multiplied with said zero value from being updated (changed from its previous value) since the result of the multiplication would be a zero regardless of the cosine constant's value. In another implementation, the engine prevents the updating of a cosine constant asserted to the first input set (to be multiplied with a zero data value) and prevents the updating of the data value being asserted to the corresponding input of the second input set (to prevent toggling of either multiplier input), and asserts a zero value (e.g., multiplexes a zero value into the output of the processing pipeline) indicative of the result of multiplying the zero data value with the cosine constant (without actually multiplying together these two operands); or
(c) outputs one or more zeroes indicative of a row transformed version of at least one value in the second half of the current row's second half (without performing actual row transform computations on such data value or values), and outputs at least one data value generated by performing row transform computations in a reduced-power manner on at least one other data value of the second half of the current row's second half. Each such zero value and output value is then stored (in step 50) in a buffer memory.
After all data values generated or asserted in steps 46 and 48, steps 46 and 49, steps 47 and 48, or steps 47 and 49 have been stored (step 50) in the buffer memory, the engine determines (step 52) whether the row transform has been performed on all rows of the current block. If the row transform has been performed on all rows of the current block, the engine enters a state (step 54) in which it stops row transform operations. It can then perform step 20 again to begin processing of the next block of data values to be transformed. If the row transform has not been performed on all rows of the current block, the engine reads the next row (of the current block to be transformed) from buffer memory (another performance of step 22).
Each of multiplication units M1, M2, M3, and M4 of
When the
In a typical implementation, preprocessing logic 100 resets the output values Y0 and Y1 to zero upon receiving (e.g., from buffer 9) each set of four new input data values to be transformed. In this case, when preprocessing logic 100 determines that a zero value should be output (i.e., a zero value of Y0 or Y1) in response to one of the input values (e.g., in step 25, 28, 41, 35, 37, 47, or 49 of
In embodiments which require reordering of data values to be transformed, preprocessing unit 100 is preferably configured to perform such reordering. For example, to perform transforms similar to that described with reference to
Unit 120 is coupled to receive a sequence of blocks of input data values (e.g., input frequency domain coefficients that have been generated by performing a two-dimensional DCT or inverse DCT or other 2D transform on blocks of video pixels). Unit 120 is also coupled to receive data values (identified in
Multiplexer 129 of unit 120 selects either input data values (that have not undergone processing in
Preprocessing unit 120 is typically configured to separate each set of data values asserted thereto into subsets appropriate for engine 122 to perform the transform to be implemented. The manner in which unit 120 accomplishes the separation into subsets will depend on the transform to be implemented (e.g., 8×8 DCT, 8×8 IDCT, or 8×8 Hadamard), and can be determined by control signals asserted to unit 120 from an external unit so that the subset selection can be customized on the basis of the transform to be implemented.
Transform engine 122 performs a 2D transform on each block of data values that it receives from unit 120 to generate blocks of transformed data values (e.g., blocks of partially decoded video pixels), and asserts the transformed data values to post-processing unit 124 for optional further processing. More specifically, transform engine 122 is configured to perform a pipelined row transform or column transform on each set of four data values asserted thereto from unit 120 to generate an output value in response each of these four data values. Each such set of four data values is typically a row (or column) of a block of data values, or a subset of a row (or column) of a block of data values.
Post-processing unit 124 optionally (i.e., when appropriate for the particular transform being performed by the
Transform engine 122 of
Typically, zero detection logic 130 of preprocessing unit 120 is configured to determine whether an entire block of data to be transformed consists (all four rows or columns of a 4×4 block of data to be transformed consist) entirely of zero values. For example, it may include a shift register providing sufficient latency to perform such a determination on all rows (columns) of a block before passing the first row (column) of the block to downstream circuitry. Upon determining that the block consists entirely of zero values, logic 130 asserts appropriate control bits to other elements of the
Whether or not zero detection logic 130 of preprocessing unit 120 is configured to determine whether an entire block of data consists entirely of zero values as described in the previous paragraph, logic 130 is configured to perform zero detection on the data values of each individual row (or column) asserted to logic 130 from multiplexer 129 to determine whether all data values of such a row (or column) are zero values (i.e., it performs step 24 of
In some implementations, logic 130 determines whether the first data value of each individual row (or column) of data values asserted to it from multiplexer 129 is a non-zero value and all other values of the row (or column) are zero values. In response to identifying a row (column) consisting of zero values except for an initial non-zero value, such an implementation of logic 130 asserts appropriate control bits that cause transformation operations that would otherwise be performed subsequently by engine 122 on the row (column) to be bypassed, and typically also causes predetermined values (e.g., zeroes) to be output from multiplexer 139 of unit 124 in response to the row (column) without performance of actual transformation operations by one or both of engine 122 and unit 124 on the values of the row (column).
Zero detection logic 132 of engine 122 is configured to perform zero detection on a first subset (e.g., the first half, which are the first two data values) of each row (or column) asserted to engine 122 from unit 120 to determine whether all the data values of such first subset are zero values (e.g., it performs step 26 of
If logic 135 determines that a first data value of the first subset of the current row (column) is a zero value, it asserts a control bit to multiplication circuit 141 (to whose first input the transform constant C0 is asserted, and to whose second input the zero data value of the current row or column is asserted) to cause circuit 141 to perform a multiplication operation on the value asserted to its second input without updating the previous value asserted to its first input, to avoid consuming power that would otherwise be consumed to toggle the first input to an updated value of the constant C0.
Similarly, if logic 136 determines that a second data value of the first subset of the current row (column) is a zero value, it asserts a control bit to multiplication circuit 142 (to whose first input the transform constant C1 is asserted, and to whose second input such zero data value of the current row or column is asserted) to cause circuit 142 to perform a multiplication operation on the value asserted to its second input without updating the previous value asserted to its first input, to avoid consuming power that would otherwise be consumed to toggle the first input to an updated value of the constant C1.
Zero detection logic 134 of engine 122 is configured to perform zero detection on a second subset (e.g., the second half, which are the second two data values) of each row (or column) asserted to engine 122 from unit 120 to determine whether all the data values of such second subset are zero values (i.e., it performs step 40 of
If logic 137 determines that a first data value of the second subset of the current row (column) is a zero value, it asserts a control bit to multiplication circuit 143 (to whose first input the transform constant C2 is asserted, and to whose second input the zero data value of the current row or column is asserted) to cause circuit 143 to perform a multiplication operation on the value asserted to its second input without updating the previous value asserted to its first input, to avoid consuming power that would otherwise be consumed to toggle the first input to an updated value of the constant C2.
Similarly, if logic 138 determines that a second data value of the second subset of the current row (column) is a zero value, it asserts a control bit to multiplication circuit 144 (to whose first input the transform constant C3 is asserted, and to whose second input such zero data value of the current row or column is asserted) to cause circuit 144 to perform a multiplication operation on the value asserted to its second input without updating the previous value asserted to its first input, to avoid consuming power that would otherwise be consumed to toggle the first input to an updated value of the constant C3.
Each of multiplexers 129, 140, 141, 150, 151, 152, and 153 of
Some alternative embodiments of the invention consist of or include circuitry identical to
The system of
It should be understood that in order to implement various embodiments of the invention to perform any of many different 2D transforms on blocks of data having any of many different formats, variations on the specific steps shown and described with reference to
It should also be understood that while some embodiments of the present invention are illustrated and described herein, the invention is defined by the claims and is not to be limited to the specific embodiments described and shown.
| Number | Name | Date | Kind |
|---|---|---|---|
| 3679821 | Schroeder | Jul 1972 | A |
| 4177514 | Rupp | Dec 1979 | A |
| 4583164 | Tolle | Apr 1986 | A |
| 4591979 | Iwashita | May 1986 | A |
| 4644461 | Jennings | Feb 1987 | A |
| 4755810 | Knierim | Jul 1988 | A |
| 4814978 | Dennis | Mar 1989 | A |
| 4992857 | Williams | Feb 1991 | A |
| 5045940 | Peters et al. | Sep 1991 | A |
| 5130797 | Murakami et al. | Jul 1992 | A |
| 5146324 | Miller et al. | Sep 1992 | A |
| 5212742 | Normile et al. | May 1993 | A |
| 5225875 | Shapiro et al. | Jul 1993 | A |
| 5233689 | Rhoden et al. | Aug 1993 | A |
| 5267334 | Normille et al. | Nov 1993 | A |
| 5267344 | Nelson, III | Nov 1993 | A |
| 5369744 | Fukushima et al. | Nov 1994 | A |
| 5371896 | Gove et al. | Dec 1994 | A |
| 5596369 | Chau | Jan 1997 | A |
| 5598514 | Purcell et al. | Jan 1997 | A |
| 5608652 | Astle | Mar 1997 | A |
| 5613146 | Gove et al. | Mar 1997 | A |
| 5623311 | Phillips et al. | Apr 1997 | A |
| 5630033 | Purcell et al. | May 1997 | A |
| 5646692 | Bruls | Jul 1997 | A |
| 5657465 | Davidson et al. | Aug 1997 | A |
| 5768429 | Jabbi et al. | Jun 1998 | A |
| 5790881 | Nguyen | Aug 1998 | A |
| 5809538 | Pollmann et al. | Sep 1998 | A |
| 5821886 | Son | Oct 1998 | A |
| 5845083 | Hamadani et al. | Dec 1998 | A |
| 5870310 | Malladi | Feb 1999 | A |
| 5883823 | Ding | Mar 1999 | A |
| 5889949 | Charles | Mar 1999 | A |
| 5898881 | Miura et al. | Apr 1999 | A |
| 5909224 | Fung | Jun 1999 | A |
| 5923375 | Pau | Jul 1999 | A |
| 5954786 | Volkonsky | Sep 1999 | A |
| 5969728 | Dye et al. | Oct 1999 | A |
| 5999220 | Washino | Dec 1999 | A |
| 6035349 | Ha et al. | Mar 2000 | A |
| 6073185 | Meeker | Jun 2000 | A |
| 6088355 | Mills et al. | Jul 2000 | A |
| 6098174 | Baron et al. | Aug 2000 | A |
| 6104470 | Streefkerk et al. | Aug 2000 | A |
| 6144362 | Kawai | Nov 2000 | A |
| 6148109 | Boon et al. | Nov 2000 | A |
| 6157751 | Olson et al. | Dec 2000 | A |
| 6175594 | Strasser et al. | Jan 2001 | B1 |
| 6188799 | Tan et al. | Feb 2001 | B1 |
| 6195389 | Rodriguez et al. | Feb 2001 | B1 |
| 6222883 | Murdock et al. | Apr 2001 | B1 |
| 6269174 | Koba et al. | Jul 2001 | B1 |
| 6272281 | De Vos et al. | Aug 2001 | B1 |
| 6305021 | Kim | Oct 2001 | B1 |
| 6311204 | Mills | Oct 2001 | B1 |
| 6317124 | Reynolds | Nov 2001 | B2 |
| 6356945 | Shaw et al. | Mar 2002 | B1 |
| 6360234 | Jain et al. | Mar 2002 | B2 |
| 6418166 | Wu et al. | Jul 2002 | B1 |
| 6459738 | Wu et al. | Oct 2002 | B1 |
| 6539060 | Lee et al. | Mar 2003 | B1 |
| 6539120 | Sita et al. | Mar 2003 | B1 |
| 6560629 | Harris | May 2003 | B1 |
| 6647062 | Mackinnon | Nov 2003 | B2 |
| 6665346 | Lee et al. | Dec 2003 | B1 |
| 6687788 | Vorbach et al. | Feb 2004 | B2 |
| 6690835 | Brockmeyer et al. | Feb 2004 | B1 |
| 6690836 | Natarajan et al. | Feb 2004 | B2 |
| 6721830 | Vorbach et al. | Apr 2004 | B2 |
| 6751721 | Webb, Jr. et al. | Jun 2004 | B1 |
| 6760478 | Adiletta et al. | Jul 2004 | B1 |
| 6782052 | Sun et al. | Aug 2004 | B2 |
| 6799192 | Handley | Sep 2004 | B1 |
| 6807317 | Mathew et al. | Oct 2004 | B2 |
| 6823443 | Horiyama et al. | Nov 2004 | B2 |
| 6950473 | Kim et al. | Sep 2005 | B2 |
| 6996645 | Wiedenman et al. | Feb 2006 | B1 |
| 7038687 | Booth, Jr. et al. | May 2006 | B2 |
| 7173631 | Anderson | Feb 2007 | B2 |
| 7260148 | Sohm | Aug 2007 | B2 |
| 7277101 | Zeng | Oct 2007 | B2 |
| 7289672 | Sun et al. | Oct 2007 | B2 |
| 7379501 | Lainema | May 2008 | B2 |
| 7403564 | Laksono | Jul 2008 | B2 |
| 7450640 | Kim et al. | Nov 2008 | B2 |
| 7499491 | Lee et al. | Mar 2009 | B2 |
| 7548586 | Mimar | Jun 2009 | B1 |
| 7548596 | Yen et al. | Jun 2009 | B2 |
| 7551671 | Tyldesley et al. | Jun 2009 | B2 |
| 7565077 | Rai et al. | Jul 2009 | B2 |
| 7581182 | Herz | Aug 2009 | B1 |
| 7630097 | Kodama et al. | Dec 2009 | B2 |
| 7689000 | Kazama | Mar 2010 | B2 |
| 7693219 | Yan | Apr 2010 | B2 |
| 7720311 | Sriram | May 2010 | B1 |
| 7721069 | Ramchandran et al. | May 2010 | B2 |
| 7924923 | Lee et al. | Apr 2011 | B2 |
| 8009923 | Li et al. | Aug 2011 | B2 |
| 8369402 | Kobayashi et al. | Feb 2013 | B2 |
| 20010020941 | Reynolds | Sep 2001 | A1 |
| 20010024448 | Takase et al. | Sep 2001 | A1 |
| 20010028353 | Cheng | Oct 2001 | A1 |
| 20010028354 | Cheng et al. | Oct 2001 | A1 |
| 20020015445 | Hashimoto | Feb 2002 | A1 |
| 20020015513 | Ando et al. | Feb 2002 | A1 |
| 20020025001 | Ismaeil et al. | Feb 2002 | A1 |
| 20020041626 | Yoshioka et al. | Apr 2002 | A1 |
| 20020109790 | Mackinnon | Aug 2002 | A1 |
| 20020114394 | Ma | Aug 2002 | A1 |
| 20020118743 | Jiang | Aug 2002 | A1 |
| 20030020835 | Petrescu | Jan 2003 | A1 |
| 20030048361 | Safai | Mar 2003 | A1 |
| 20030078952 | Kim et al. | Apr 2003 | A1 |
| 20030141434 | Ishikawa et al. | Jul 2003 | A1 |
| 20030161400 | Dinerstein et al. | Aug 2003 | A1 |
| 20040057523 | Koto et al. | Mar 2004 | A1 |
| 20040095998 | Luo et al. | May 2004 | A1 |
| 20040100466 | Deering | May 2004 | A1 |
| 20040150841 | Lieberman et al. | Aug 2004 | A1 |
| 20040156435 | Itoh et al. | Aug 2004 | A1 |
| 20040174998 | Youatt et al. | Sep 2004 | A1 |
| 20040181564 | MacInnis et al. | Sep 2004 | A1 |
| 20040181800 | Rakib et al. | Sep 2004 | A1 |
| 20040190613 | Zhu et al. | Sep 2004 | A1 |
| 20040190617 | Shen et al. | Sep 2004 | A1 |
| 20040202245 | Murakami et al. | Oct 2004 | A1 |
| 20040213348 | Kim et al. | Oct 2004 | A1 |
| 20040218626 | Tyldesley et al. | Nov 2004 | A1 |
| 20040218675 | Kim et al. | Nov 2004 | A1 |
| 20040228415 | Wang | Nov 2004 | A1 |
| 20040257434 | Davis et al. | Dec 2004 | A1 |
| 20050008254 | Ouchi et al. | Jan 2005 | A1 |
| 20050033788 | Handley | Feb 2005 | A1 |
| 20050047502 | McGowan | Mar 2005 | A1 |
| 20050066205 | Holmer | Mar 2005 | A1 |
| 20050079914 | Kaido et al. | Apr 2005 | A1 |
| 20050105618 | Booth et al. | May 2005 | A1 |
| 20050123040 | Bjontegard | Jun 2005 | A1 |
| 20050190976 | Todoroki et al. | Sep 2005 | A1 |
| 20050238102 | Lee et al. | Oct 2005 | A1 |
| 20050238103 | Subramaniyan et al. | Oct 2005 | A1 |
| 20050265454 | Muthukrishnan et al. | Dec 2005 | A1 |
| 20050276493 | Xin et al. | Dec 2005 | A1 |
| 20050281337 | Kobayashi et al. | Dec 2005 | A1 |
| 20050286630 | Tong et al. | Dec 2005 | A1 |
| 20060002466 | Park | Jan 2006 | A1 |
| 20060017802 | Yoo et al. | Jan 2006 | A1 |
| 20060056513 | Shen et al. | Mar 2006 | A1 |
| 20060056708 | Shen et al. | Mar 2006 | A1 |
| 20060109910 | Nagarajan | May 2006 | A1 |
| 20060115001 | Wang et al. | Jun 2006 | A1 |
| 20060133501 | Lee et al. | Jun 2006 | A1 |
| 20060133506 | Dang | Jun 2006 | A1 |
| 20060176299 | Subbalakshmi et al. | Aug 2006 | A1 |
| 20060176962 | Arimura et al. | Aug 2006 | A1 |
| 20060203916 | Chandramouly et al. | Sep 2006 | A1 |
| 20070002945 | Kim | Jan 2007 | A1 |
| 20070002950 | Yang | Jan 2007 | A1 |
| 20070036215 | Pan et al. | Feb 2007 | A1 |
| 20070070080 | Graham et al. | Mar 2007 | A1 |
| 20070133689 | Park et al. | Jun 2007 | A1 |
| 20070171981 | Qi | Jul 2007 | A1 |
| 20070217506 | Yang et al. | Sep 2007 | A1 |
| 20070230564 | Chen et al. | Oct 2007 | A1 |
| 20070274389 | Kim et al. | Nov 2007 | A1 |
| 20070286284 | Ito et al. | Dec 2007 | A1 |
| 20070286508 | Le Leannec et al. | Dec 2007 | A1 |
| 20080069203 | Karczewicz et al. | Mar 2008 | A1 |
| 20080117214 | Perani et al. | May 2008 | A1 |
| 20080137726 | Chatterjee et al. | Jun 2008 | A1 |
| 20080151997 | Oguz | Jun 2008 | A1 |
| 20080285444 | Diab et al. | Nov 2008 | A1 |
| 20090060277 | Zhang et al. | Mar 2009 | A1 |
| 20090086827 | Wu et al. | Apr 2009 | A1 |
| 20090116549 | Shen et al. | May 2009 | A1 |
| 20090122864 | Palfner et al. | May 2009 | A1 |
| 20090161763 | Rossignol et al. | Jun 2009 | A1 |
| 20090196350 | Xiong | Aug 2009 | A1 |
| 20090268974 | Takagi | Oct 2009 | A1 |
| 20100034268 | Kusakabe et al. | Feb 2010 | A1 |
| Number | Date | Country |
|---|---|---|
| 1489391 | Apr 2004 | CN |
| 1283640 | Feb 2003 | EP |
| 2348559 | Oct 2000 | GB |
| 04162893 | Jun 1992 | JP |
| 11096138 | Apr 1999 | JP |
| 2001184323 | Jul 2001 | JP |
| 2005354686 | Dec 2005 | JP |
| 2006287315 | Oct 2006 | JP |
| 9827742 | Jun 1998 | WO |
| 0233650 | Apr 2002 | WO |
| 2005001625 | Jun 2003 | WO |
| 2005096168 | Oct 2005 | WO |
| 2006085137 | Aug 2006 | WO |
| Entry |
|---|
| Rohini Krishnan, et al., “Design of a 2D DCT/IDCT Application Specific VLIW Processor Supporting Scaled and Sub-sampled Blocks,” 16th International Conference on VLSI Design, 2003, six pages. |
| US Patent Application Publication No. 2005/0033788, Handley, Pub. Date Feb. 10, 2005. |
| “Ralf Schafer et al.,”, “H.264/AVC”, Dated: Jan. 2003, pp. 1-12. |
| —The Merriam-Webster Dictionary—. 2005 ed. Springfield, MA: Merriam-Webster Inc., 2005. |
| A Single-Chip Video/Audio Codec for Low Bit Rate Application Seongmo Park, Seongmin Kim, Igkyun Kim, Kyungjin Byun, Jin Jong Cha, and Hanjin Cho, ETRI Journal, vol. 22, No. 1, Mar. 2000, pp. 20-29. |
| Advanced Video Coding for generic audiovisual services, Telecommunication Standardization Sector of ITU, Dated Mar. 2005, pp. 1-343. |
| Andrew Gibson, “H.264 Video Encoding Standard”, year Aug. 2002, pp. 1-98, Queens University Kingston Ontario Canada. |
| Chen, Hao et al., “Adaptive FMO Selection Strategy for Error Resilient H.264 Coding” International Conference on Audio, Language and Image Processing, 2008. ICALIP 2008, Jul. 7-9, 2008, pp. 868-872. |
| Hannuksela, Miska et al., “Random Access Using Isolated Regions”, IEEE 2003, pp. III-841 to III to 844. |
| Iwasaki, I.; Naganuma, J.; Nitta, K.; Yoshitome, T.; Ogura, M.; Nakajima, Y.; Tashiro, Y.; Onishi, T.; Ikeda, M.; Endo, M., “Single-chip MPEG-2 422P@HL CODEC LSI with multi-chip configuration for large scale processing beyond HDTV level,” Design, Automation and Test in Europe Conference and Exhibition, Mar. 2003. |
| Jamil-Ur-Rehman and Dr. Zhang Ye, “Efficient Techniques for Signaling Intra Prediction modes of H.264/Mpeg-4 Part 10”, Proceedings of the First International Conference on Innovative Computing, Information and Control, ICICIC, Year 2006, pp. 1-4. |
| Mizuno, M. et al.; “A 1.5-W single-chip MPEG-2 MP@ML video encoder with low power motion estimation and clocking,” Solid-State Circuits, IEEE Journal of, vol. 32, No. 11, pp. 18-7-1816, Nov. 1997. |
| Realization of Low-Bit—Ratio Video Encoder Using Mpact Media Processor; Iwasaki, Junichi et al.; 1997. |
| Shih-Hao Wang et al.; “A platform-based MPEG-4 advanced video coding (AVC) decoder with block level pipelining,” Information, Communications and Signal Processing, 2003 and the Fourth Pacific Rim Conference on Multimedia Proceedings of the 2003 Joint Conference of the Fourth International Conference on, vol. 1, No., p. 51-55 vol. 1, Dec. 2003. |
| Tourapis et al., Proc. Of SPIE Conf. Vis. Comm. And Img. Processing, vol. 3, pp. 1365-1373, Jun. '00. |
| Tu, C., Liang, J., and Tran, T. “Adaptive Runlength Coding”, in—IEEE Signal Processing Letters—vol. 10, No. 3, pp. 61-6. Mar. 2003. |
| Tung-Chien Chen; Yu-Wen Huang; Liang-Gee Chen, “Analysis and design of macroblock pipelining for H.264/AVC VLSI architecture,” Circuits and Systems, 2004. ISCAS '04. Proceedings of the 2004 International Symposium on, vol. 2, No., pp. 11-273-6 vol. 2, May 23-26, 2004. |
| Zheng, et al., Inter. Conf. Info. Systems, Analysis and Synthesis, SCI 2001-ISAS 2001, vol. 13, 2001. |
| Jong, et al., “Accuracy Improvement and Cost Reduction of 3-Step Search Block Matching Algorithm for Video Coding”, Feb. 1, 1994, IEEE Transaction on Circuits and Systems for Video Technology, vol. 4 No. 1, pp. 88-90, JP000439487. |
| Kadono Shinya, et. Al. Revised edition H.264/AVC Textbook, Impress R&D, Jan. 1, 2006, pp. 131-133. |
| Number | Date | Country | |
|---|---|---|---|
| 20080050036 A1 | Feb 2008 | US |
