The present invention relates to matrix multiply units. More particularly, the invention relates to a system and method to implement a matrix multiply unit of a broadband processor.
Recent multimedia CPUs operate in parallel on 128b vectors, partitioned into 8-16b elements. Exemplars of these designs are described in Craig Hansen. MicroUnity's MediaProcessor Architecture. IEEE Micro, 16(4):34-41, August 1996, and Keith Diefendorff. Pentium III=Pentium II+SSE. Microprocessor Report, 13(3):1, 6-11, March 1999. These designs perform arithmetic operations on values partitioned into vectors, such as addition and multiplication. The operations are performed by functional units in which the hardware employed to perform the operation, an adder for an add operation, or a multiplier for a multiply operation is in turn partitioned so as to perform vector operations of the specified element size. Vector adds need only AND carries between elements, but vector multiplies idle all but a single stripe one element wide through the product array. Thus, a 128b×128b multiplier, when performing a vector multiplication on 8b operands, only employs the resources of an 8b×128b multiplier, leaving resources of the size of a 120b×128b multiplier idle, or performing a mathematically trivial operation, such as multiplication by zero.
As the hardware resources for a multiplier capable of performing a 128b×128b are considerably larger than that of a 128b+128b adder, the lower utilization of the multiplier when performing vector multiplications of a smaller element size is of considerable concern in designing an efficient processor. While one approach to designing an efficient multiplier is to limit the size of the multiplier to a smaller strip which can perform vector multiplications only of small elements in a single pipeline flow, the present invention instead aims to make efficient use of a large 128b×128b multiplier array pipeline by performing a vector-matrix product.
A vector-matrix multiplier unit fully utilizes a 128×128b data path for operand sizes from 8 to 128b and operand types including signed, unsigned or complex, and fixed-, floating-point, polynomial, or Galois-field while maintaining full internal precision.
The present invention may include a method of performing multiplication with integrated polynomial multiplication capability comprising the steps of: receiving the multiplier operand, the multiplicand operand, and the polynomial specification signal; calculating a product, the product being one of an integer product or a polynomial product, depending on the polynomial specification signal; and transmitting the result.
In accordance with a method of the present invention, the step of calculating the product may include producing partial products and calculating one of an integer product or polynomial product based on the partial products, and wherein the step of producing the partial product may modify a calculation of x1, x2 and negate selection signals using an enhanced Booth encoder, depending on the polynomial specification signal.
The method of the present invention may further comprise the steps of calculating x1, x2 and negate selection signals and recoding the x1, x2 and negate selection signals into x1pass, x1force, x2pass, x2force signals; combining x1pass, x1force, x2pass, x2force signals with multiplicand x1 and multiplicand x2 signals producing a partial product; and inhibiting the value of the x1force, x2pass and x2force signals at certain multiplexor locations to produce an alternate partial product.
The method of the present invention, wherein the step of calculating the product may include producing partial products, and may further include the step of adding partial products in a partial products summation tree, in which the carry output of each full adder is inhibited to perform polynomial multiplication, depending on the polynomial specification signal.
The present invention may further include a method of performing matrix multiplication with partitioning capability comprising the steps of: receiving a multiplier operand, a multiplicand operand, and a partitioning specification signal; calculating partitioned product, depending on a partitioning specification signal; and transmitting the calculated partitioned product.
The method of the present invention, wherein the step of calculating a partitioned product may further comprise the step of producing a carry-save result with no additions for the least significant four bits of an 8b partial product
In accordance with the method of the present invention, the step of calculating a partitioned product may further comprise the step of obtaining a partial product computation in which separate head and tail bits are produced at a partition boundary between operands, or fused together into a single 8b partial product.
The present invention may comprise a system of performing multiplication with integrated polynomial multiplication capability comprising: a receiver which receive a multiplier operand, a multiplicand operand, and a polynomial specification signal; a multiplier that calculates an integer product or polynomial product, depending on a polynomial specification signal; and a transmitter that transmits one of the integer product or the polynomial product calculated by the calculator.
In the system of the present invention, the multiplier may include a portion which produces partial products and a portion which calculates one of an integer product or polynomial product based on the partial products, and wherein the portion that produces partial products may include an enhanced Booth encoder that modifies a calculation of x1, x2 and negate selection signals, depending on the polynomial specification signal.
The present invention may further include a system of performing matrix multiplication with partitioning capability comprising: a receiver which receives a multiplier operand, a multiplicand operand, and a partitioning specification signal; a multiplier that computes partitioned product results, depending on a partitioning specification signal; and a transmitter which transmits the partitioned product results computed by the multiplier.
The accompanying drawings, which are incorporated in and form a part of the specification, serve to explain exemplary principles of the invention. In the drawings:
The exemplary implementation is intended to integrate the greatest possible range of function into a generalized multiplier. The first goal was to handle all of the following operand formats:
The second goal was to provide all functions on these data types within reach of a generalized 128×128b multiplier, including:
In a 128b×128b vector-matrix product, one operand is a 128b vector, consisting of 128b/size elements of the specified size, and the other operand is a matrix of 128b/size by 128b/size elements. The vector-matrix product is formed by summing the products of the vector operand with 128/size vector “slices” of the matrix operand, producing a 128b vector result. As a product generally has a representation that is twice the size of equal-sized operands, and the representation of a addition has one more bit than the size of equal-sized operands, the sum of the products is larger than the operands, and the representation of the result is compressed by extracting a subset of the bits of the result, and rounding the subset to reflect less-significant bits removed from the result.
The summation of products and partial products in vector-matrix multiplication uses a technique called carry-save addition, in which these products are represented in a form that uses greater than the minimum representable bits, so that carries do not need to propagate from one end of the operand to the other in an addition operation. Commonly, the representation doubles the number of bits, so that carries need not propagate at all, but are saved for a later stage. Thus, an adder that sums two such redundant values receives four bits and produces two bits per non-redundant bit position and is named a 4:2 adder. Other common forms of carry-save adders are 3:2 adders (full adders), which have well-known hardware representation, and are building blocks for 4:2 and 5:2 adders.
Media and signal-processing applications perform conventional arithmetic processing, but may also involve dealing with error-correction, in which rounded results do not have appropriate numerical properties, but an alternate arithmetic forms, known as polynomial, finite-field, and Galois-field arithmetic do. A fundamental component of these alternate forms are polynomial arithmetic, which is much like conventional unsigned arithmetic processing except that all carries are suppressed. It is a goal of the present invention to perform polynomial multiplication as well as conventional multiplication using an integrated design. The present invention combines these two capabilities together to reduce the amount of hardware that would otherwise be required to provide these capabilities in separate hardware.
Floating-point formats combine sign, exponent and significand information into a single operand, producing a representation that greatly expands the range of values that the operand can signify. The representation of a result need not be larger than that of the operands, as the exponent allows for a large range, and the significand can be normalized and rounded, so as to produce the closest representable value to the ideal result. The present invention, because the vector-matrix product sums several scalar products together before scaling and rounding the result, can produce results that are more precise than prior art designs.
Matrices are too wide to fit into a register operand—e.g. 8b matrix has 16×16=256 elements, equivalent to 16 128b registers. An SRAM array of eight 2048b entries is embedded in the multiplier to hold the most recently used matrices. The mechanisms employed in the present invention for providing the wide operand are described in U.S. Pat. No. 6,295,599, “System and method for providing a wide operand architecture.” The mechanisms for addressing and filling this embedded matrix cache are identical to the ones used for switch and table caches in the companion Crossbar and Table Lookup units, so much of the infrastructure can be shared. DSP and multimedia code often multiplies one or two matrices many times by a long sequence of vectors. Examples include 4×4 floating-point graphical transform, discrete cosine transform, Reed-Solomon error correction, and AES (Rijndael) encryption.
Fortunately, matrices that change rapidly are often redundant enough to be specified by few registers. For example, the convolve operation is multiplication by a matrix that can be constructed by the Multiplicand Multiplexor from two register operations, as explained below. The Multiplicand Multiplexor also assists the vector operations emuladd, emulsum, escaladd above by forming matrices with one or two non-zero stripes.
The third goal was to perform internal calculations without loss of data—that is, as if intermediate values were computed with unlimited precision. For integer operations, this means that products are computed with double the initial operand width and careful sign extension, and that sums of products keep track of sign and overflow bits that increase in each adder stage.
For floating point operations, one approach would be to convert to integer and then follow the integer discipline, but hardware cost is prohibitive. A more sophisticated strategy is to design each stage of the shift/add tree to forward exactly the results needed to the next stage, but defer the transistor cost of full width to the latest possible stage, as is described in below.
Integrated matrix operations and fully precise processing are a surprisingly compatible and economic mix. The internal accumulations have latency that is independent of width, since they are done in redundant form, and the full width in the last stage is amortized over many fused operations. Using a simple full-width integer verification model as the gold standard of correctness eases logic testing of the hardware.
The fourth goal was to support the full range of mathematical techniques for optimizing accuracy of conversion to each of the above (limited precision) operand formats. These include rounding, saturation, scaling, field extraction, conversions, overflow and other exceptions, NANs, denormalized numbers, etc. These refinements simplify the development of software that complies with standards for algorithms such as MPEG-4 and broadband modems.
At the top of
16 parallel ByteMul blocks 201/202 multiply 8b Booth-encoded multipliers with 128b multiplicand vectors and produce 128b partitioned carry-save product strips. These strips are then combined in a 4-stage binary tree of alignment multiplexors 203, 205, 206, 208, 210 and 4:2 adders 204, 207, 209, 211 to reach the full 128b sum of products.
The Doublet Multiplexor 203 aligns byte product strips that are then added into doublet strips comprising 16b partial products. Each floating-point format contains enough exponent bits to allow left shifting significand products up to 7b without overflow, aligning to 8b granularity with the Bit Align Multiplexor 205. Later multiplexors 206, 208, 210, are therefore 8b-granular and require no more than 8 inputs. The Quadlet Multiplexor 206 aligns all strips to 32b boundaries, so no further horizontal shifting of products occur, except for a small fraction of signals that shift by 32, 64b at the 64b or 128b stages. This reduces the number of multiplexor inputs required in the next stage, the Octlet Multiplexor 208. The Octlet multiplexor 208 aligns all strips to 64b boundaries, reducing the number of multiplexor inputs required in the final summation stage, the Hexlet Multiplexor 210. The result of the final 4:2 addition 211 is then accumulated, the leading zero location is anticipated, and a final carry-propagation addition and rounding is performed by block 212. The Multiplexor Control and FP exponent path 213 computes control signals for the Multiplexors, dependent on the values of floating-point exponents, where present, and the specified size of the operands.
The Multiplicand Format block 216 selects either a wide operand from the Wide Operand Memory 215, or a Multiplicand constructed by (possibly redundant) selections from the Multiplicand Mux 214. The Multiplicand Format block 216 produces multiplicands for each ByteMul block 201/202, with circuits located as closely as possible to the ByteMul blocks to minimize the total amount of wiring.
In
The present design product bits 402 rearranges carry-input bits for negated partial products (pp) to produce the lower 4b of carry-save results with no addition, and rearranges sign-extension bits to partition the multiplier at byte boundaries. Special partial product values PPR and PPL are computed at the locations designated r and l in the product bits 402. The lower half of the partial product bits are shaded and designated as head partial product bits, the upper half of the partial product bits are designated as tail partial product bits.
The present design product bits 403 are further rearranged into rectangular layout with the byte boundary partitioned along the diagonal by 2:1 multiplexors 410 and 411. Here the head partial product bits of one operand are shown adjacent to the tail partial product bits of an adjacent operand. Depending on whether these operands are to be fused together, either a single fused result is generated, or separate head and tail results are generated, using the 2:1 multiplexors 410 and 411. The partial products are summed using full adders designated FA for the head bits 412 and tail bits 413.
In order to compute polynomial or Galois products, the full adders in
Fusing Gates 305 in the PPL and PPR boxes in
The number of full adders listed in the first three stages of
A key feature of the reduction (addition) stages is that the products are aligned with respect to a common reference point, so that values that need to be added are therefore equally aligned. This reduces the complexity of forming a sum of floating-point products considerably, as the partial products can be independently aligned. Prior art multiply-add units aligned the addend to the product, but this method would create great complexity in computing the sum of two, four, or eight products as in the present invention.
Further, the reduction stages progressively align the products and partial products as the values propagate through the reduction, which reduces the number of inputs on the multiplexors required to align the operands. The present invention first aligns 16b partial products to 8b alignment, where 8b alignment signifies that the shifting is by zero to seven bits, so that further alignment shifting will be by multiplies of eight bits. This initial alignment takes advantage of excess precision available in the 16b partial product representation to avoid the need to extend the partial product size, made available by the observation that floating-point formats, by devoting a portion of the representation to sign and exponent, leave sufficient zero space in the multiplier and multiplicand to assure seven bits of zeroes in the 16b partial product. Further alignments to 32b alignment preceed the accumulation of 32b partial products, and 64b preceeds the 64b partial product accumulation, and 128b preceeds the 128b partial product accumulation. Overaligning products increases the effective size of the partial results, while underaligning products increases the number of multiplexor inputs at a reduction stage. Thus, the amount of alignment is selected to minimize the amount of hardware.
When two floating-point products are added with exponents that are far enough apart that the products do not overlap or meet at a byte boundary, a pair of carry-save results are generated, with the implicit assumption that intervening bytes will consist of “sign extension” bytes that have the sum and carry bits set to one and zero respectively. In this way, the number of bits needed to represent the sum of two floating-point products is no greater than that of the products themselves.
When four floating-point products are added, however, the sign-extended least product may need to be added to all three greater products. To preserve accuracy in 64b and 128b stages, the number of full adders per bit is triple the integer amount—24 and 12 instead of 8 and 4. For 16b floating-point matrix multiplication, the 64b and 128b reduce stages actually sums eight 16b products without loss, because the 96b width provides overflow margin for the 88b full integer width of these products. Thus, in the latter stages, the 16b floating-point products are converted to fixed-point by aligning the results by the exponent sum.
The Byte Multiplier strip is key to the partitioning of the vector-matrix array. The design evolved through a few stages. A prior art approach was to compute an array of byte products, and then combine them in a doubling tree. At each stage three of the four products overlap, so a 6:2 redundant adder is needed.
For this reason we designed the Byte multiply stage to compute “strips” of products. For example, if the multiplicands are 16b each, the fuse multiplexors in
Each 8b unit in the Byte Multiplier strip really does a byte multiply add in the time of a multiply, by using slack at inputs in tail summation to add in the early head bits of the next byte. This extra work eliminates the need for a 6:2 adder at the next stage.
Booth byte multiply was used in the exemplary embodiment, because 5:2 is the largest Wallace tree that degenerates to a serial array adder, simplifying the partition multiplexing of high order and sign bits. The enhanced Booth circuitry of
Preservation of internal accuracy enables ideal signal-noise ratios, and is especially valuable in algorithms such as Newton-Raphson or sigma-delta sensitive to zero crossings. When large products cancel, two different addition orders may differ at every bit, so precision simplifies logic verification.
The effective internal bandwidth of a lossless matrix multiplier, as referred to conventional vector units, is very high. A 16×16 matrix at 8b precision has 8+8+16 bits of multiply operands and 3×32 bits of add operands at each of 256 entries, or 32K bits total. That amounts to effective internal bandwidth of 12.8 Tbits at 400 MHz. 16 and 32b lossless floating point is about the same bandwidth, because of the expansion of adder width. The wire counts between stages in
The exemplary embodiment demonstrates that the functional goals of the vector-matrix multiplier could be met with remarkably little logical overhead relative to more conventional multi-precision multipliers: the number of 4:2 stages is the same, the byte multiplicand delay is increased by a single 2:1 multiplexor, the Multiplicand Multiplexor is overlapped with Booth encode. The additional width of full internal precision is deferred to later stages.
Although the invention herein has been described as primarily a hardware implementation for illustration purposes, those of skill will recognize that the invention may be performed by one or more computational devices which capable of performing the calculation of the disclosed hardware devices. Moreover one of skill will recognize that the method of the present invention may encoded or embodied in software programs that may be carried on a recordable medium including digital recording medium such as an optical disc (such as a CD or DVD), magnetic disk or tape, and may be transmitted by a wireline or wireless communications devices in packetized or non-packetized form.
Having fully described a preferred embodiment of the invention and various alternatives, those skilled in the art will recognize, given the teachings herein, that numerous alternatives and equivalents exist which do not depart from the invention. It is therefore intended that the invention not be limited by the foregoing description, but only by the appended claims.
This application is a Continuation of U.S. application Ser. No. 12/330,962, filed Dec. 9, 2008, which is a Divisional of U.S. patent application Ser. No. 10/233,779, filed Sep. 4, 2002, now U.S. Pat. No. 7,483,935, claiming the benefit of U.S. Provisional Application No. 60/317,427, filed Sep. 4, 2001, the entire contents of each of which are hereby incorporated by reference.
Number | Date | Country | |
---|---|---|---|
60317427 | Sep 2001 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 10233779 | Sep 2002 | US |
Child | 12330962 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 12330962 | Dec 2008 | US |
Child | 13462648 | US |