1. Field of the Invention
The method and system relate to assuring that data stored in large shared cache memory in a multi-processor environment and the data of the main memory are either identical or controlled so that stale and current data are not confused with each other.
2. Background Art
In a shared memory multiprocessor of the prior art, with a separate cache memory for each processor, it is possible to have many copies of any one instruction operand: one copy in the main memory and one in each individual cache memory. When one copy of an operand is changed, the other copies of the operand must be changed also, i.e., cache coherence. Cache coherence is the discipline that ensures that changes in the values of shared operands are propagated throughout the system in a timely fashion.
There are three distinct levels of cache coherence:
In both level 2 behavior (where all processes see exactly the same sequence of changes of values for each separate operand ) and level 3 behavior (where different processes may see an operand assume different sequences of values, e.g., non coherent behavior), a program can observe stale data.
In a large shared-memory multiprocessor, providing a system level cache of the recently accessed contents of memory, along with an efficient means to handle system-wide cache coherency, can theoretically be accomplished with a single system cache directory array by requiring the contents of the respective processor level caches to be a subset of the system cache. Unfortunately, when the combined size of the processor caches is sufficiently large, this subset rule can be can become impractical because the resulting size of the system cache required to work effectively becomes too big.
While one possible solution to this is to maintain a single system directory only partially backed by cache data, this proves difficult in practice. This is because the logic must now evaluate which entries have data and which do not when determining a system cache data hit.
As a solution to the problems inherent in maintaining a single system directory only partially backed by cache data, where the logic must evaluate which entries have data and which do not when determining a system cache data hit, we provide a system with two complementary system cache directories. That is, these directories are not associated to a particular processor or processors, but are system cache directories accessible to all of the processors.
These two directories are substantially equal in size and collectively large enough to contain all of the processor cache directory entries, but with only one of these cache directories hosting system-cache data to back the most recent fraction of data accessed by the processors. By this expedient, only the directory known to be backed by system cached data will be evaluated for system cache data hits.
More specifically, the method and system uses two non-overlapping system cache directories. These are substantially equally sized and are referred to herein as superset and subset directories. As used herein, the following terms have the following meanings:
Together the two cache directories are sufficient in size to contain the entries of all underlying processor cache directories, together with the system cache.
The total cache is as large as practical given such factors as fast access times and reasonable chip area, and map the most recent fraction of the system cache directory entries into the superset cache.
The two caches collectively behave both as a data cache and a system wide coherency cache. As a data cache, the structure is used for recent memory data needed by the processors. As a system wide coherency cache, the structure is used as a system wide coherency controller. The coherency controller maintains a record of which processors in the multi-processor have copies of which portions of memory and in what state they are in.
The system and method described herein is a dual system directory structure that performs the role of system cache, i.e., data, and of system control, i.e., coherency, when the size of the system cache is insufficient to contain the contents of all of the underlying caches. This avoids the complexity of a single structure solution, with its extra MRU/LRU complexity, area, and data backed hit detection. It also avoids the extra cache level overhead on the underlying caches to manage coherency, and even the absence of a system data cache.
An advantage in using these two symmetric directories is that their entries are paired one-to-one, such that once an entry is identified for LRU-out of the superset directory, the respective subset entry to receive this LRU-out entry is immediately identified, without requiring any special logic of its own.
Furthermore, a processor requesting a system lookup requires a scan of the superset directory only, rather than a lookup in a combined directory, which would require additional logic to distinguish between data-backed and data-less system cache entries.
As far as the design is concerned, the superset directory and system cache are collectively what would be considered a traditional system cache. By adding the subset directory to the mix, entries would normally be LRUed out of the subset directory, as well as the respective processors would now be placed in the subset directory and left to persist in the processors. This improves overall performance.
Various aspects of our invention are illustrated in the Figures appended hereto.
The method and system of the invention uses two complementary caches, one cache for data and the other cache for coherency. These two directories are substantially equal in size and collectively large enough to contain all of the processor cache directory entries, but with only one of these cache directories hosting system-cache data to back the most recent fraction of data accessed by the processors. By this expedient, only the directory known to be backed by system cached data will be evaluated for system cache data hits.
The system and method described herein is a dual system directory structure that performs the role of system cache, i.e., data, and system control, i.e., coherency, when the size of the system cache is insufficient to contain the contents of all of the underlying caches.
To summarize the invention, both the superset 221a and 221b and subset 223a and 223b directories collectively manage cache coherency—the superset directory 221a and 221b is also backed by data, and can readily supply it to any requesting processor (this is what is recognized as a “traditional data/coherency cache”).
The subset directory 223a and 223b is a low-overhead means of allowing processors 201a through 201h to retain a line LRU'd out of the traditional/superset cache 221a and 221b. The subset directory 223a and 223b is not tied to the superset cache 221a and 221b, and it is not actually a data cache but is a directory with memory addresses and processor addresses only. As data is LRUed out of a superset directory 221a and 221b, its address is still retained in the subset directory 223a and 223b.
Without the subset directory 223a and 223b, upon LRUing out of the superset cache 221a and 221b, a line is “required” to be invalidated out of every processor 201a through 201h; i.e., if the L2 directory can't manage the coherency of a line, then “no” processor may possess the line. However now, the processors can retain these lines a while longer, as the subset cache 223a and 223b will continue to manage lines that have been LRU'd-out of the superset cache 221a and 221b.
The processors 201a through 201h do not communicate directly with the main memory 231, but rather they communicate through the L2 control logic 211a and 211b, the superset directory 221a and 221b, and the L2 Data Array 227a and 227b.
There is no data bus connected to the subset cache 223a and 223b. Only control lines do an address lookup, and only control lines have the control means to invalidate lines out of the processors.
In order to work efficiently there is provided a system control method or system in place to control resource access, insure cache coherency, etc. This system, structure, or method can be single level or multi-level, and while it is illustrated herein as a single level system, structure, or method, the methods, systems, and structures illustrated herein may be extended to multi-structure systems, methods, and structures.
The system is useful for providing cache coherence to a multiprocessor system having a dual system directory structure having two system cache directories performing the role of system cache for data, and system control for coherency. This is done by writing an entry to a superset cache, and writing an address and a calling processor identifier for the entry to a subset cache, LRUing the entry out of the superset cache and retaining the address and calling processor identifier for the entry in the subset cache. In this way system-cache data is hosted in the subset cache to back the most recent data accessed by the processors. This enables evaluating the subset cache for system cache data hits, as well as maintaining a record in the subset cache of which processors have cached which portions of memory they are actually using.
There are two parts of the system, structure, and method described herein:
If the total system cache 221a, 221b, 223a, 223b can be built sufficiently large, i.e., to manage the system cache 221a and 211b and the system cache coherency 223a and 223b, such as by requiring all processor cache contents to be part of the system cache 221a, 221b, 223a, 223b contents (the “subset rule”) there would be no problem. But, if such a cache is too large for practical use, it becomes necessary to redesign the system, or to not have a system cache, or to have the subset rule, above, limit a significant portion of the of the processor caches available to the associated processors, or to have two separate system cache directories, one to manage the cache and one to manage the cache coherency, with the associate complexity to make it all work.
According to the method, system, and structure described herein, there is provided a superset system cache directory 221a and 223b (where all processor cache contents are part of the system cache contents) in conjunction with a subset system cache directory not backed by system cache, that is, where only the processors or subset caches have data not available in the system cache. In this way the subset cache manages coherency only, both the subset system cache directory and the superset system cache directory being, in total, large enough to handle both system cache coherency, together with a system cache of practical size, where only the superset directory (hosting the most recently accessed entries) has corresponding data in the system cache.
For purposes of illustration, consider a system cache, e.g., cache 115 of
Typical computer cache designs, e.g., cache 115 in
In this context, the most prevalent uses of LRU logic are:
Returning to the system cache, if N entries are insufficient to provide efficient system cache coherency, but, for example, 2*N entries are sufficient to provide efficient system cache coherency, then the system cache is kept to N entries, but the system directory is doubled to 2*N, as illustrated by caches 121, 123 in
This pair of cache directories, that is, the subset and the superset cache directories, performs both updating and installing. When looking up a particular address, both directories are accessed in parallel. In accordance with the invention described herein, the following changes are made:
First, if a new superset directory entry is installed for modification and a hit for the same entry detected in the subset directory, the subset directory entry will be invalidated as well as being invalidated out of the processors identified by the subset directory as hosting the entry.
If a new superset directory is installed, and a hit is detected for the same entry in the subset directory, the entry is invalidated out of the subset directory. Additionally,
If a new superset directory install requires an LRU-out of the superset directory (and consequently the system cache), the entry is still removed from the system cache, but the superset directory entry is not invalidated and is instead migrated into the subset directory.
In the case of data invalidated out of the processors as a result of a subset directory invalidation, whether either from a new superset directory that is installed, and a hit is detected for the same entry in the subset directory, or from a new superset directory install requires an LRU-out of the superset directory, appropriate action is taken, e.g., writing the entry back to main memory if it differs from the main memory copy.
In a preferred exemplification of our invention, the superset and subset directories are matched and no special logic is required to identify the targeted subset directory receiving a migration from the superset directory, that is, if an entry LRU's out of superset [C][A], it migrates into subset [C][A]. For this reason only the superset directory needs or has LRU/MRU logic.
The system and method described herein is a dual system directory structure that performs the role of system cache, i.e., data, and system control, i.e., coherency, when the size of the system cache is insufficient to contain the contents of all of the underlying caches. That is, it is a separate data/coherency cache
While the invention has been described with respect to certain preferred embodiments and exemplifications, it is not intended to limite the scope of the invention thereby, but solely by the claims appended hereto.