A computer cache typically consists of a data cache, containing copies of data from a larger, slower, and/or more remote main memory, and a tag array, containing information relating to each “line” of data in the data cache. In general, a cache line is the smallest amount of data that can be transferred separately to and from the main memory. The tag data typically contains at least the location in the main memory to which the cache line corresponds, and status data such as the ownership of a cache line in a multi-user system, and a validity state comprising coherency/consistency data such as exclusively owned, shared, modified, or stale. With the large size of some current or proposed computer systems, the size of the main memory address stored in the tag can be very much the largest part of the tag, and can be comparable in size to the data cache line to which it refers.
In some forms of cache, the tag array is stored in faster memory than the data cache. Fast memory is expensive, and to make effective use of its speed must be close to the processor using it, often on the same chip. As a result, there is pressure to maintain a high ratio of data cache size to tag size. However, very large cache lines are inefficient, because they frequently involve moving quantities of data that are not actually wanted.
It has therefore been proposed to use a “sectored cache” or “buddy cache” in which a single tag entry applies to a “block” of the data cache containing several cache lines known as “sectors” or “buddies.” The buddies within a cache block typically correspond to consecutive lines of the main memory, but can be independently owned and have different validity statuses. Thus, for a cache block containing N buddies, the tag entry contains N sets of ownership and validity data, but only one main memory address, resulting in considerable reduction in tag size as compared with N independent cache lines. The performance of the cache (in terms of hit rate and latency) is typically intermediate between N independent cache lines and one cache line N times the size, depending on the usage pattern in a specific use.
Latency is in some situations limited, because in many configurations the tag entry can only be rewritten as a whole, so that a transaction affecting one buddy must be queued pending updating of the tag entry to reflect a transaction affecting another buddy.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention.
In the drawings:
Reference will now be made in detail to various embodiments of the present invention, examples of which are illustrated in the accompanying drawings.
Referring initially to
The clients 12 are in communication with one or more servers 20, which comprise main memory 22, which may be a computer readable storage medium in the form of a large amount of volatile memory such as DRAM memory or non-volatile memory containing data that the clients 12 can access. Merely by way of example, the plurality of clients 12 may be in one cell 23 of a multiprocessor computer system, and the server 20 may be in the same or another cell 25 of the same multiprocessor computer system. Accesses from the clients 12 to the server 20 may then pass through nodes 24, 26 connecting their respective cells to a fabric 28 between the cells.
One or more caches may be provided between the client processors 14 and the server memory 22, to reduce the load on the server 20 and speed up access when a client 12 repeatedly accesses the same information from server memory 22. Merely by way of example, a lowest-level (that is to say, furthest from the client processor, and typically largest and slowest) cache 30 in the client cell 23 may be provided at the node 24, and may be shared by the clients 12.
Referring now to
The tag array 34 and the data array 32 may be part of the same physical memory device, or different devices. Typically in a sectored cache 30, the tag array 34 is in a smaller but faster memory than the data array 32. In large modern computer systems 10, the length of the main memory address 44 can be comparable to the size of the cache sectors 38, and there can thus be significant savings in having only one main memory address 44 for an entire block 36 of data sectors 38, which can compensate for the loss of flexibility because the sectors 38 within a block 34 must be in a fixed, or at least very concisely describable, relationship, typically consecutive sectors of main memory 22.
The cache 30 may be a partly associative cache, in which the blocks 36 are grouped, each group of blocks (see group 148. in
Referring now to
The cache controller 50 may also comprise a processor 54, and computer readable storage medium 56, such as ROM or a hard disk, containing computer readable instructions to the processor 54 to carry out the functions of the cache controller.
Each entry in the pending allocation table 52 may comprise an identifier for a cache transaction to which it relates, the index of the cache block to which the write is pending, and the contents of the tag block 40 as proposed to be rewritten. For practical reasons, the tag block 40 may be writable only as a whole, so that if two writes are pending at the same time, it would be possible for the second write to reverse or otherwise overwrite the first write.
The cache controller 50 is configured so that, when writing to the tag block 40, the changed data is broadcast to the PAT 52 to update data representing any later pending writes to the tag block. Then, when the later pending writes are written to the tag block 40, changed data that has been received from the broadcasts is included. Thus, the later write refreshes, rather than obliterating, the earlier write.
Referring now to
Where the memory access request can be immediately completed, for example, a read request that is a cache hit, it may be processed immediately.
Where the memory access request cannot be immediately completed and would alter a cache tag entry, in step 62 the cache controller 50 creates an entry in the PAT 52 representing the current state of the cache tag entry, by copying the existing entry from the relevant tag block 40. The cache controller 50 may at this time update the PAT entry with as much as is already certain about the proposed tag write, or may not update the PAT entry until a later stage.
In step 64, the cache controller 50 writes a changed entry to the tag block 40, and sends out a broadcast to the PAT specifying the alteration.
In step 66, the cache controller 50 identifies and updates any still pending current PAT entries relating to the same cache tag entry. Then, when in a subsequent iteration of step 62 the other entries are written from the PAT to the tag block 40, the earlier change is included in the later write to the tag block 40, and is confirmed rather than overwritten. This procedure can speed up the second write by several clocks, because it saves the second write having to wait for the first write to complete and then read the current state of tag block 40 before creating its own write.
Where there is more than one cache block, a single PAT 52 may serve all, or a logical group, of the cache blocks. A broadcast is then applied only to pending transactions for the same block to which the broadcast change applied. The PAT 52 may be stored in content addressable memory (CAM), and the index 42 of the cache block 36, 40 to which an entry in the PAT 52 relates may be addressable content.
Each broadcast may contain only the updated data for the specific sector 46 to which the underlying transaction relates, and an identification of that sector. The data can then be substituted in the PAT 52 for the previous data for that sector 46. Where that approach is used, co-pending tag writes for the same sector may be inhibited.
Referring now to
The buffer 72 may also store currently pending and recently completed cache tag writes.
Where a pending write is supplied from the buffer 72, that can reduce the risk of a client 12 that requests a lookup being supplied with data that is stale before the requesting client has used it, because of the pending write. In the other instances mentioned, time is saved because the second requester does not need to wait for the earlier transaction to complete, and then carry out a tag lookup, which can take several clock cycles. The size of the buffer may be limited so that searching the buffer does not create more delay than it saves, and so as to limit the risk of the buffer itself containing stale data.
Referring to
If the comparison in step 82 succeeds, in step 88 the original cache lookup is voided, and the data from the buffer 72 is supplied to the client 12. In this embodiment the buffer 72 is not, updated in step 88. Where motivations for using the buffer 72 include those mentioned above, it may be more beneficial to allow old transactions to be discarded from the buffer even if they are still being used.
Referring now to
The tag control block 200 includes a tag pipe 202, which contains requests for writes to the tag array 34, and a pending allocation table (PAT) 152, which may be similar in construction and function to the PAT 52 shown in
The tag array 132 is so configured that in normal operation individual tag blocks 140 can be written or overwritten, but that parts of a tag block 140 cannot be written or overwritten separately.
The PAT 152 and the tag pipe 202 feed writes into a Tag Write FIFO 204, from which they are actually written to the tag array 134. The tag pipe 202 can also send non-writing cache tag lookup requests directly to the tag array 134, and can update a Not Recently Used register 206, which tracks how recently each cache block 136, 140 has been used, and can identify suitable blocks for replacement by newly-retrieved data. The tag pipe 202 has a forwarding FIFO 172 that contains tag writes waiting to be passed to the tag write FIFO 204, and may also contain recent past tag writes and the results of recent lookup requests. The tag pipe 202 also comprises a comparator 174 that can compare cache tag lookup requests with entries in the forwarding FIFO 172. The tag pipe 202 also coordinates with a data pipe 210 to ensure that writes to the cache data array 132 are properly synchronized with writes to the tag array 134. The tag pipe 202 also communicates with a Fabric Abstraction Block 212 that converts the memory addresses 144 used in the cache tag and elsewhere within the cell 23 into a form that will be meaningful when sent across the fabric 28 to another cell 25.
The Pending Allocation Table 152 contains, in an example, 48 lines and serves the entire tag array 134. Each line contains status bits indicating whether the line is pending, completed, or invalid, the index of the tag block 140 to which it relates (which may be an index for a group 148 and a way 136, 140 within that group), and the proposed text of the tag block 140. The PAT 152 is a content addressable memory in which the index is addressable content.
Referring now to
In step 304, the comparator 174 compares the request with the contents of forwarding FIFO 172. If the comparison returns a hit, in step 306 cache controller 50 retrieves the tag information from FIFO 172. If the comparison failed, in step 308 cache controller 50 does a cache lookup to see whether that sector of data is in cache 132. If there is a cache hit, in step 310 the cache controller 50 reads the tag information from the relevant tag block 140, and in step 312 may add the tag information just read to FIFO 172. In step 314, using the tag data step 306 or 310, the cache controller retrieves the requested data sector from cache 132 and returns it to the requester 12, and updates the NRU register 206 for the cache block in question. The process then returns to step 302 to await the next read request.
If the cache lookup in step 308, returned a miss, in step 316 the process determines whether a cache block has been allocated to the missing data (which may happen if another sector in the same block is already cached). This may be done in the same cache lookup as step 304 and 308, but is shown separately for logical clarity.
If no cache space has been allocated to the memory block in question, in step 318 the process allocates a cache way 136, 140. If all ways in the relevant group are already occupied, the cache controller 50 uses the NRU 206 to eject the least recently used way. The cache controller 50 then configures the tag block 140 to show that block allocated to the block of main memory 22 containing the requested sector of data, but with all sectors in the cache block invalid. If a cache block has been allocated to a data block including the requested sector, in step 320 the process identifies the block and reads the existing tag entry 140. As explained below, the NRU register 206 may be updated at this stage.
/From either step 318 or 320, the process proceeds to step 322, and creates a PAT entry corresponding to the current state of the tag entry 140. if the PAT 152 is full, step 322 overwrites a completed or otherwise invalid line. If every line in the PAT 152 is valid and pending, the new process stalls until a line becomes available.
In step 324, the process sends a request over the fabric 28 to the main memory 22 to provide the missing data. There may be a considerable wait, step 326, before the data is received.
In the case of step 320, where the cache block 136, 140 had already been allocated, there may be an earlier read request for the same block that is still pending. That may be the request that originally caused the block to be allocated, or may be a request for a third sector in the same block. Alternatively, a request for a different sector in the same block may be issued later, but for some reason fulfilled by main memory 22 earlier. In any of those cases, while the process shown in
Steps 328 and 330 may happen zero, one, or a plural number of times while step 326 continues to wait.
In step 332, the data requested in step 324 arrives from main memory 22, and is forwarded to the requester 12. In step 334, the data is fed into data pipe 210, and a write request is fed into tag pipe 202. In step 334, the tag data relating to the write are passed from tag pipe 202 to PAT 152, if that has not already been updated, including any status data received from the server 20. For example the server 20 may at this time specify whether user 12 has exclusive or shared ownership of the data sector. As in step 330, the process updates only the tag status sector 146 relating to its own transaction, so that other tag data, including any broadcast updates from step 330, are not affected.
In step 336, the data and the tag data are written to the cache. In the data cache 132, only the new sector 138 is written, but in the tag cache 134 the entire block 140 is written, because that is how the tag cache is constructed. In step 338, the process sends out a CAM write broadcast to the PAT 152, which may become step 330 of another instance of the process, if there is a write to the same tag block 140 still pending.
The PAT line is then marked as completed and invalid, and the process ends.
In the case of a write to cache from a local client 12, for example, a writethrough or writeback of modified data, the write can be added to the pipes 202, 210 immediately, and conflicting transactions can be inhibited or stalled during the short period between the write transaction reading the tag block 140 and writing back the updated tag block 140. Such writes can therefore be completed without using the PAT 152. However, a PAT broadcast (steps 328, 330) is issued when the write takes place, in case there are other transactions pending in the PAT 152 for the same cache block.
Where two cache-miss read requests are received for the same sector, the first request proceeds as shown in
Where a cache block is recalled by server 20 while a write resulting from a cache-miss read is pending, either the transaction is abandoned or (if the server 20 actually supplies the data being recalled) the data may be supplied to the requesting client with an invalid status, but not cached.
Where a cache block is ejected because the cache controller needs more space for a new data block, it is usually undesirable for the ejected block to be one on which a cache-miss read is pending. To reduce the probability of that occurring, the NRU register 206 may be updated at step 318 or 320 to show the block in question as recently used.
Various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention cover the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents.
For example, in
For example, the device 50 has been described as a stand-alone cache controller, but may be part of a one of the other devices in a computing system. The Pending Allocation Table 152 may be several cooperating physical tables, assigned to different clients 12, different parts of cache 30, or in some other way. PAT broadcasts may then be sent only to parts of PAT table 152 to which they are potentially applicable. The cache 30 has been described as a single partially-associative sectored cache, but aspects of the present disclosure may be applied to various other sorts of cache. The skilled reader will understand how the components of computing system 10 may be combined, grouped, or separated differently.
Although various distinct embodiments have been described, the skilled reader will understand how features of different embodiments may be combined.
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/US08/78605 | 10/2/2008 | WO | 00 | 6/20/2011 |