Patent Application
20140258624
This invention relates generally to processor architectures. More particularly, this invention relates to operating a processor with an operation cache.
Instruction latency minimization is an integral part of high-performance processor design. Double digit clock cycles are typically required for an operation like integer division, which computes a quotient and a remainder. Simpler integer operations execute in the low single digits of clock cycles. There is an ongoing need to improve processing latency to enhance overall processor performance.
A method includes simultaneously applying operands to a computation engine and a cache. The operands are compared to stored operands in the cache to selectively identify a match and a miss. The cache outputs a stored value in the event of a match. The computation engine outputs a computed value in the event of a miss.
A processor includes a computation engine to produce a computed value for a set of operands. A cache stores the set of operands and the computed value. The cache is configured to selectively identify a match and a miss for a new set of operands. In the event of a match the computed value is supplied by the cache and a computation engine operation is aborted. In the event of a miss a new computed value for the new set of operands is computed by the computation engine and is stored in the cache.
A non-transitory computer readable storage medium comprises executable instructions to define a processor with a computation engine to produce a computed value for a set of operands and a cache to store the set of operands and the computed value. The cache is configured to selectively identify a match and a miss for a new set of operands. In the event of a match the computed value is supplied by the cache and a computation engine operation is aborted. In the event of a miss a new computed value for the new set of operands is computed by the computation engine and is stored in the cache.
The invention is more fully appreciated in connection with the following detailed description taken in conjunction with the accompanying drawings, in which:
Like reference numerals refer to corresponding parts throughout the several views of the drawings.
The performance benefits of the invention can be illustrated with an example. Consider the following pseudo code fragment:
The division (DIV) instruction computes the quotient of 9/3 and stores the result out to memory. Later it is loaded from memory in a different procedure. This sequence is repeated many times and frequently the DIV instruction's operands are a numerator of 9 and a denominator of 3. As a result, it often computes a result of 3. During execution, the DIV takes more than a dozen clock cycles to compute, resulting in poor performance in this stretch of code. It is not possible to speed up the divide latency by more than a few cycles without hurting processor frequency, resulting in diminishing returns on performance. This is because divide algorithms must handle all possible inputs, resulting in various checking and normalization steps before the actual division can be started and some further overhead before the result can be forwarded to a waiting instruction. Further, even though the divide always produces the same result, the compiler cannot deduce this at compile time, due to issues like aliasing, nor can it eliminate the store word (SW) and load word (LW) pair if they lie in different translation modules, making it even more imperative to have low divide latency.
The division instruction frequently has the same input operands during execution. Thus, although it may take many cycles to compute the result the first time a given pair of inputs is seen by the divide engine, the result of the division operation is placed in an operation cache. The next time the multiply/divide unit is called upon to perform a division, the operation cache is checked in parallel with initiating the division operation. If the result is in the cache, the results are supplied from the cache and the actual long-latency operation is cancelled.
An ordinary cache is a memory that stores the recent results of accessing a larger memory. Later accesses can be satisfied from the cache, saving latency as well as bandwidth and power of the larger memory. Such a cache is written when a store instruction is seen and read when a load instruction is seen. An operation cache is written when a particular set of (opcode, operands) is seen and read when the same operation is seen in the future. Thus, an operation cache exhibits the same property of temporal locality as an ordinary cache. Ordinary caches usually also exploit spatial locality. When a load instruction reads a particular memory location, the cache controller brings in not just the requested word, but also the contents of adjacent locations within the same cache line. Since it is empirically probable that future load instructions will want to access those nearby locations, access latency is reduced. Operation caches can exhibit a similar useful property. It is simple to compute the remainder of a division at the same time the quotient is requested. The operation cache can then save both the quotient and remainder and if a future instruction requests the remainder, it can be provided without repeating the computation.
The same sort of look ahead is applicable to multiplication, since most programs request the low word of the product but sometimes request the high word also. It usually costs just one extra clock cycle to compute the high word after the low word is generated. On an instruction set that has separate instructions for high and low parts of a multiply, such an operation cache allows the same performance as an instruction set with accumulators, yet retains all the considerable benefits of eliminating accumulators from the instruction set.
Consider a very simple operation cache that is being used to cache the results of ADD and SUB instructions. An entry in the cache would have four fields: an Opcode field, two operands and a result. If the program were to encounter the ADD instruction with input operands 5 and 3, an entry (5, 3; ADD: 8) may be created because when the Opcode ADD is applied to 5 and 3, the result is 8. Subsequently, the cache can satisfy future computations that try to ADD 5 and 3 and produce 8 without doing the addition. Next, suppose the program encounters a SUB instruction with input operands 5 and 3. The result of 2 would be computed and a second entry would be created in another row of the operation cache, with the fields (5, 3; SUB: 2). Note that this SUB instruction was a miss in the cache, not a hit, because this was the first time 5-3 was attempted by the program. So the SUB takes a long latency to compute because it is a MISS in the cache.
Instead, if we were to augment the operation cache so that every time it encounters an ADD instruction, it also computes the SUB of those same input operands, we could alter the entries to store a second Opcode and a second result. In other words, the first ADD 5,3 would place (5, 3; ADD: 8, SUB: 2) into one entry of the cache. This entry is saying that when 5 and 3 are ADDed, the result is 8, when they are SUBtracted, the result is 2. Note that this entry is populated when the ADD instruction is seen, the SUB has not yet been seen. Thus, one is speculating that if the program encounters the addition of 2 numbers, it might encounter the subtraction of the same two numbers later. If and when a SUB instruction with operands 5 and 3 is later encountered, the result can be supplied from the cache and an actual subtraction need not be performed. This is an improvement over the basic operation cache described in the previous paragraph, since the operation cache is able to store multiple Opcodes and Results for the same set of input Operands.
Unlike ordinary caches, an operation cache need not be very large to be effective. For example, a 4-entry fully-associative cache works well for quite a few programs. If the operation cache is used to store floating-point results, a larger cache is needed to be effective.
While various embodiments of the invention have been described above, it should be understood that they have been presented by way of example, and not limitation. It will be apparent to persons skilled in the relevant computer arts that various changes in form and detail can be made therein without departing from the scope of the invention. For example, in addition to using hardware (e.g., within or coupled to a Central Processing Unit (“CPU”), microprocessor, microcontroller, digital signal processor, processor core, System on chip (“SOC”), or any other device), implementations may also be embodied in software (e.g., computer readable code, program code, and/or instructions disposed in any form, such as source, object or machine language) disposed, for example, in a computer usable (e.g., readable) medium configured to store the software. Such software can enable, for example, the function, fabrication, modeling, simulation, description and/or testing of the apparatus and methods described herein. For example, this can be accomplished through the use of general programming languages (e.g., C, C++), hardware description languages (HDL) including Verilog HDL, VHDL, and so on, or other available programs. Such software can be disposed in any known non-transitory computer usable medium such as semiconductor, magnetic disk, or optical disc (e.g., CD-ROM, DVD-ROM, etc.). It is understood that a CPU, processor core, microcontroller, or other suitable electronic hardware element may be employed to enable functionality specified in software.
It is understood that the apparatus and method described herein may be included in a semiconductor intellectual property core, such as a microprocessor core (e.g., embodied in HDL) and transformed to hardware in the production of integrated circuits. Additionally, the apparatus and methods described herein may be embodied as a combination of hardware and software. Thus, the present invention should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.
