1. Field of the Invention
The present invention generally relates to computer systems, and more particularly to a memory hierarchy for a computer system that includes large cache structures having different latencies across the cache arrays.
2. Description of the Related Art
The basic structure of a conventional computer system includes one or more processing units which are connected to various peripheral devices (including input/output devices such as a display monitor, keyboard, and permanent storage device), a memory device such as random access memory (RAM) that is used by the processing units to carry out program instructions and store operand data, and firmware which seeks out and loads an operating system from one of the peripherals (usually the permanent memory device) whenever the computer is first turned on. The processing units typically communicate with the peripheral devices by means of a generalized interconnect or bus. A computer system may have many additional components such as various adapters or controllers, and serial, parallel and universal bus ports for connection to, e.g., modems, printers or network interfaces.
In a symmetric multi-processor (SMP) computer, all of the processing units are generally identical, that is, they all use a common set or subset of instructions and protocols to operate, and generally have the same architecture. A typical architecture includes a processor core having a plurality of registers and execution units, which carry out program instructions in order to operate the computer. The processing unit can also have one or more caches, such as an instruction cache and a data cache, which are implemented using high speed memory devices. Caches are commonly used to temporarily store values that might be repeatedly accessed by a processor, in order to speed up performance by avoiding the longer step of loading the values from a main memory device. These caches are referred to as “on-board” when they are integrally packaged with the processor core on a single integrated chip.
A processing unit can include additional caches, such as a level 2 (L2) cache which may support on-board (level 1) instruction and data caches. An L2 cache acts as an intermediary between the main (system) memory and the on-board caches, and can store a much larger amount of information than the on-board caches, but at a longer access penalty.
A cache has many blocks which individually store the various instruction and data values. The blocks in any cache are divided into groups of blocks called sets or congruence classes. A set is the collection of cache blocks that a given memory block can reside in. For any given memory block, there is a unique set in the cache that the block can be mapped into, according to preset mapping functions. The number of blocks in a set is referred to as the associativity of the cache, e.g. 2-way set associative means that for any given memory block there are two blocks in the cache that the memory block can be mapped into; however, several different blocks in main memory can be mapped to any given set. A 1-way set associative cache is direct mapped, that is, there is only one cache block that can contain a particular memory block. A cache is said to be fully associative if a memory block can occupy any cache block, i.e., there is one congruence class, and the address tag is the full address of the memory block.
An exemplary cache line (block) includes an address tag field, a state bit field, an inclusivity bit field, and a value field for storing the actual instruction or data. The state bit field and inclusivity bit fields are used to maintain cache coherency in a multiprocessor computer system (to indicate the validity of the value stored in the cache). The address tag is usually a subset of the full address of the corresponding memory block. A compare match of an incoming address with one of the tags within the address tag field indicates a cache “hit.” The collection of all of the address tags in a cache (and sometimes the state bit and inclusivity bit fields) is referred to as a directory, and the collection of all of the value fields is the cache entry array.
When all of the blocks in a congruence class for a given cache are full and that cache receives a request, whether a “read” or “write,” to a memory location that maps into the full congruence class, the cache must make one of the blocks in that class available for the new operation. The cache chooses a block by one of a number of means known to those skilled in the art (least recently used (LRU), random, pseudo-LRU, etc.). If the data in the chosen block has been modified, that data is written to the next lowest level in the memory hierarchy which may be another cache (in the case of the L1 or on-board cache) or main memory (in the case of an L2 cache). By the principle of inclusion, the lower level of the hierarchy will already have a block available to hold the written modified data. If the data in the chosen block has not been modified, the value in that block is simply abandoned and not written to the next lowest level in the hierarchy. This process of freeing up a block from one level of the cache hierarchy is known as an eviction. At the end of this process, the cache no longer holds a copy of the evicted block. When a device such as the CPU or system bus needs to know if a particular cache line is located in a given cache, it can perform a “snoop” request to see if the address is in the directory for that cache.
As microprocessor computing power grows, it becomes more critical for caches to correspondingly grow in size in order to avoid processing bottlenecks that arise from memory latencies. However, large cache structures can introduce or exacerbate other problems, such as bandwidth and connectivity. Some high-performance computer systems address these issues by dividing the cache array and directory into two or more slices, and allowing multiple access/command ports. One example of such a sliced cache structure is shown in
Each cache array slice 14a, 14b is further divided into four sectors, that is, a given cache line is distributed across all four sectors of a slice. In this example, each cache line is 128 bytes longs, and the digit pairs in each sector represent the beginning byte number (in hexadecimal) for an 8-byte word of the line, e.g., “00” refers to the first 8-byte word in the cache line (bytes 00, 01, 02, 03, 04, 05, 06 and 07), and “08” refers to the second 8-byte word in the cache line (bytes 08, 09, 0A, 0B, 0C, 0D, 0E and 0F). Thus, each sector contains 32 noncontiguous bytes of a given cache line. All of the sectors are in a single latency domain but only 32 bytes are output in a given cycle, so it takes four cycles to output a complete 128-byte cache line, with the entire cache array (all sectors) powered up during each of the four cycles.
Each L2 controller slice 16a, 16b has its own read claim (RC), cast out (CO) and snoop (SN) machines. Each controller slice further has its own directory arbiter 20a, 20b which handles conflicts between these machines and load requests from the processor core. The directory arbiters are connected respectively to cache arbiters 22a, 22b which control the flow of merge data coming from elsewhere in the memory hierarchy (e.g., system memory) using separate command ports. Merge flow logic in each cache slice receives 32 bytes in a given cycle from four 8-byte fabric busses that are connected to system memory and various peripheral devices.
While the use of sliced cache arrays can improve cache bandwidth, there are still serious problems with power consumption, wiring topology, differential latencies, and recoverability, especially when the design scales to larger cache sizes. As designs grow the cache size by placing larger numbers of cache array macros, the latency to the farthest array becomes multiple clock cycles away from the core compared to the closest cache array. Thus, the prior art mechanism wherein all arrays' access times are in the same clock cycle becomes temporally wasteful, because the close arrays must be slowed to match the farthest arrays' access time. Although transmission speed can be increased by providing special wiring (wider/faster), such wiring increases the expense of the design and uses valuable wiring resources, and these problems are compounded in designs requiring large busses for two cache slices. Even in the case of a load hit, there can still be a significant delay in accessing and transmitting the requested cache line, due to the physical layout of the cache and processor core. It would, therefore, be desirable to devise an improved cache structure which could reduce latencies associated with a sizeable growth of the cache, particularly latencies arising from load hits. It would be further advantageous if the cache structure could maintain superior directory bandwidth, and still afford a high degree of recoverability in the case of a defect in the array.
It is therefore one object of the present invention to provide an improved cache memory for a computer system.
It is another object of the present invention to provide such a cache memory which is highly scalable to allow large cache arrays without significantly increasing cache latency.
It is yet another object of the present invention to provide a cache array topology for large cache structures which takes into consideration different latencies associated with different cache sectors.
The foregoing objects are achieved in a method of operating a cache memory by logically partitioning a cache array having a single access/command port into at least two slices, using a first cache directory to access the first cache array slice while using a second cache directory to access the second cache array slice, and managing accesses from the first and second cache directories using a single cache arbiter which controls the single access/command port of the cache array and manages movement of data inside the cache quadrant and over a single data bus back to the core. In the illustrative embodiment, the first cache directory has a first directory arbiter to handle conflicting access requests within the first cache array slice, the second cache directory has a second directory arbiter to handle conflicting access requests within the second cache array slice, and the first and second directory arbiters communicate with the cache arbiter. An address tag associated with a load request is transmitted from the processor core to either the first or second directory with a designated bit that associates the address tag with only one of the cache array slices whose corresponding directory determines whether the address tag matches a currently valid cache entry. The cache array may be arranged with rows and columns of cache sectors, wherein a given cache line is spread across sectors in different rows and columns, with at least one portion of the given cache line being located in a first column having a first latency and another portion of the given cache line being located in a second column having a second latency greater than the first latency. The cache array outputs different sectors of the given cache line in successive clock cycles based on the latency of a given sector.
The above as well as additional objectives, features, and advantages of the present invention will become apparent in the following detailed written description.
The present invention may be better understood, and its numerous objects, features, and advantages made apparent to those skilled in the art by referencing the accompanying drawings.
The use of the same reference symbols in different drawings indicates similar or identical items.
With reference now to the figures, and in particular with reference to
Processing unit 30 may be part of a larger computer system which includes various conventional elements (not shown), such as firmware or read-only memory (ROM) and main or random-access memory (RAM) coupled to a peripheral component interconnect (PCI) local bus using a PCI host bridge. The PCI host bridge can provide a low latency path through which processors 32a and 32b may access PCI devices mapped anywhere within bus memory or I/O address spaces. The PCI host bridge also provides a high bandwidth path to allow the PCI devices to access the main memory. Such PCI devices might include a local area network (LAN) adapter, a small computer system interface (SCSI) adapter providing access to a permanent storage device (e.g., a hard disk drive which stores an operating system and program files), an expansion bus bridge with user input devices such as a keyboard and graphical pointer (mouse), an audio adapter, or a graphics adapter. Service processors (not shown) can be connected to processor cores 32a, 32b via a JTAG interface or other external service port, and a processor bridge (not shown) can optionally be used to interconnect additional processor groups.
The L2 cache portion 34 of the processing subsystem is divided into four directory slice pairs 34a, 34b, 34c and 34d (for a total of 8 slices: A0, B0, C0, D0, A1, B1, C1, D1). Each slice pair has four megabytes (MB) of memory, and each processor has exclusive use of four of the L2 cache slices, i.e., the L2 cache is 8 MB of private memory per core. In the depicted embodiment, core 32a uses slice pairs 34a and 34b, and core 32b uses slice pairs 34c and 34d. The individual slices are selected by an address hashing algorithm residing in cores 32a, 32b. The same hashing algorithm is used by FBC 38 to route snoop traffic to the L2 slices. For example, bits 55 and 56 of the address can be decoded to route to slice A when addr(55:56)=00, to slice B when addr(55:56)=01, to slice C when addr(55:56)=10, and to slice D when addr(55:56)=11.
Each private 4 MB cache is logically partitioned to have two directory slices and two cache array slices (e.g., A and B). The 4 MB data cache is further partitioned into four domains or sectors (sect0,1,2,3) where the first 32 bytes of a cache line go in sect0 and the last 32 bytes of the cache line go in sect3. These sectors are oriented such that all four sectors sect0,1,2,3 (collectively referred to as quadrant 0) are physically located together in a corner of the chip, as explained further below. In this example, the cache line size is 128 bytes, L2 cache 34 is fully inclusive of the L1 data and instruction caches, and is 8-way set associative. The cache array data and directory array are protected by error correction code (ECC) having single-bit correction and double-bit detection (SBC/DBD) capability. A least-recently used (LRU) algorithm is provided and may be enhanced as explained further below.
L2 slices 34a, 34b, 34c and 34d generally handle all cacheable load/store instructions, data and instruction prefetches, zero-out instructions (e.g., the “DCBZ” PowerPC™ instruction), and synchronization instructions. NCUs 36a and 36b handle all other communication with the processor cores, such as cache-inhibited load/store instructions and cache operations (excluding zero-out and similar instructions).
The L2 cache array is spread out over four separate quadrants 52a, 52b, 52c and 52d. Each of these quadrants contains the sectors as explained above, e.g., L2 quadrant 52a (quad0) contains all four sectors sect0,1,2,3 of a given cache line in L2 slice A0, and also contains all four sectors sect0,1,2,3 of a given cache line in L2 slice B0.
Referring now to
The sectors of L2 quadrant 52a are arranged in this manner to facilitate pipelining of the cache output to core 32a when the cache is found to contain a currently valid value requested by the core (a load hit). A load request sent from core 32a is received by L2 controller 44a, which has a sliced directory. The load address is delivered to either the left (A) or right (B) directory slice based on the setting of a designated bit in the address field (e.g., addr(56)), and is also delivered to a latch that feeds the address to all rows of L2 quadrant 52a. If directory slice A finds a valid matching address, it sends an enable signal (LateSel) to the slice A cache (the top and bottom rows), and if directory slice B finds a valid matching address, it sends an enable signal to the slice B cache (the middle two rows). Each L2 directory slice has its own read claim (RC), cast out (CO) and snoop (SN) machines. While the sliced directory effectively has two separate ports, the cache is unified with a single access/command port and a single cache arbiter. The directory arbiters handle access for the RC, CO and SN machines within their respective slices, while the unified cache arbiter handles requests for cache access by the RC, CO and SN machines across both slices. The directory arbiter and cache arbiter also coordinate their access for certain core requests where the directory and cache need access together.
As the address and control information propagates across the cache domains from right to left, cache controller 44a selectively enables the appropriate cache domains for the read access in successive clock cycles based on their latency. As the cache arrays are read, the data propagates back from right to left over the reload data bus back to the core 32a (multiplexers are used to select between the A and B slices for the output). The horizontal (left/right) boundary 54b between sectors 1 and 2 thus represents a cycle boundary in this embodiment. The horizontal (left/right) boundary 54a between sector 0 and the core represents another cycle boundary.
The cache arbiter sends only one control signal per cycle, but it can schedule different tasks for different sectors to be carried out simultaneously or in an overlapping manner (such as different stores to be written to the cache). Only the sectors that are involved with a store operation need to be powered up during a given command sequence.
The timing of the output of L2 cache quadrant 52a is illustrated in
This physical layout allows for pipelining of the cache output in a manner that takes advantage of the differential latencies of the cache line sectors. Furthermore, by providing a unified cache with output pipelining, processing unit 30 significantly reduces overall load hit latency without the need for more expensive cache constructions. The performance gain may be further enhanced by using the faster wires available in physical design for the control signal from the core to the L2 controller and then out to the cache arrays as well as for output from the cache to the core. By designating these paths to use wires constructed of a premium metal which are up to 16 times the width and 16 times the speed of the smallest wires used on the chip, the designer dedicates the fastest wire resource to one of the most sensitive performance areas (L2 latency). These features result in a highly scalable design which keeps load hit latency low in spite of a large growth in cache size.
The unified cache structure for two directory slices provides a power advantage to the circuits by only having to build the large supporting quad dataflow structure once (as opposed to the prior art structure where the dataflow/cache was built once per directory slice). Also, by partitioning the cache quad into individual sectors, the L2 control for store operations only needs to enable the cache sector that the store is targeting, as opposed to the prior art which would cause all caches to light up.
Returning to
The read data from FBC 38 is pipelined for a given sector using eight 8-byte fabric busses and eight multiplexers. Each multiplexer has two fabric bus inputs, one from interleave bus A (ILVA) and one from interleave bus B (ILVB). The output of a given multiplexer is connected to an input line of one of the merge flow circuits 58, e.g., the two rightmost merge flow circuits 58 receive the output of the multiplexers whose inputs are designated for the first sector of a cache line, i.e., the first 32 bytes to be stored in the arrays labeled 00, 08, 10 and 18 (“ILVA_Byte00_data” and “ILVB_Byte00_data”), while the two leftmost merge flow circuits 58 receive the output of the multiplexers whose inputs are designated for the last sector of the cache line, i.e., the last 32 bytes to be stored in the arrays labeled 60, 68, 70 and 78 (“ILVA_Byte60_data” and “ILVB_Byte60_data”). Bus interleaving is scheduled by FBC 38 to avoid conflicts at each of the 8-byte multiplexers (the notation of “A” and “B” interleave busses is unrelated to the notation of “A” and “B” for the cache slices).
The timing of the read data for a load or store miss is illustrated in
Referring now to
For a store operation, the store data from the core is held in a store queue array 60, whose output is connected to another input of multiplexer 64 and similarly propagates to the cache arrays via multiplexer 70. Store commands from the cache arbiter to the merge flow circuits within a given sector can again overlap, i.e., sending a second store command while earlier store data is being merged. In addition, the cache arbiter can initiate commands to have different sectors processing different store operations at the same time. For store hits, the cache arbiter only needs to activate the sector(s) that contain the store data (e.g., an 8-byte store to address 00 only needs to affect sector0). Stores can be collected and batch processed by controller 44a. The vertical (top/bottom) boundary 56 between the top two rows and the bottom two rows thus represents a cycle boundary for latching the merge flow data. If the store operation misses the cache and the congruence class is already full, a cache line must be evicted. In this case, the evicted cache line passes through an ECC check 72 and a cast-out/push/intervention (CPI) array 74 on its way to system memory.
Those skilled in the art will appreciate that the vertical bus structure of the L2 cache quads is generally provided for the merge flow maintenance (load misses, and stores), while the horizontal bus structure of the L2 cache quads is generally provided for reload (load hits).
While the processing unit embodiment of
This construction facilitates partial utilization of the cache (a “half-good” mode) whenever a portion of the cache becomes unusable. In the example shown in
In the example of
In this recoverability embodiment, it is preferable to utilize the cache eviction algorithm to effectuate the disabling of the defective rows. The eviction algorithm may be a least-recently used (LRU) or pseudo-LRU algorithm which allocates a cache block of the congruence class for eviction based on how recently or frequently that block has been accessed. The LRU logic, which resides in controller 44a′, can include configuration bits or flags set by software to identify defective ways and prevent allocation of any blocks in those ways.
In this manner, if a defect occurs in the cache, the cache can continue operations with full directory access and all control queues available while only reducing the size of the cache by half (or less), instead of disabling an entire cache slice. This approach is particularly beneficial for certain technical applications which take advantage of queue size.
Although the invention has been described with reference to specific embodiments, this description is not meant to be construed in a limiting sense. Various modifications of the disclosed embodiments, as well as alternative embodiments of the invention, will become apparent to persons skilled in the art upon reference to the description of the invention. For example, while the invention is particularly useful for larger caches, it is more generally applicable to any type of memory hierarchy having one or more caches, including non-uniform memory access (NUMA) structures. It is therefore contemplated that such modifications can be made without departing from the spirit or scope of the present invention as defined in the appended claims.
This application is related to U.S. patent application Ser. No. ______ entitled “L2 CACHE ARRAY TOPOLOGY FOR LARGE CACHE WITH DIFFERENT LATENCY DOMAINS” (attorney docket no. AUS920041039US1) filed concurrently herewith, U.S. patent application Ser. No. ______ entitled “SYSTEM BUS STRUCTURE FOR LARGE L2 CACHE ARRAY TOPOLOGY WITH DIFFERENT LATENCY DOMAINS” (attorney docket no. AUS920041040US1) filed concurrently herewith, and U.S. patent application Ser. No. ______ entitled “HALF-GOOD MODE FOR LARGE L2 CACHE ARRAY TOPOLOGY WITH DIFFERENT LATENCY DOMAINS” (attorney docket no. AUS920041041US1) filed concurrently herewith, each of which is hereby incorporated.