Patent Grant
11907130
The present technique relates to the field of data processing.
A data processing apparatus may comprise a cache to store copies of data in, for example, memory, allowing one of more processors to access the data with reduced latency.
Viewed from one example, the present technique provides an apparatus comprising:
Viewed from another example, the present technique provides a method comprising:
Viewed from another example, the present technique provides a computer program comprising instructions which, when executed on a computer, cause the computer to fabricate an apparatus comprising:
Viewed from another example, the present technique provides a computer-readable medium to store the computer program described above. The computer-readable medium can be transitory or non-transitory.
Further aspects, features and advantages of the present technique will be apparent from the following description of examples, which is to be read in conjunction with the accompanying drawings.
Before discussing example implementations with reference to the accompanying figures, the following description of example implementations and associated advantages is provided.
In accordance with one example configuration there is provided an apparatus comprising a cache comprising a plurality of cache entries. A cache is a storage structure (also referred to herein as storage circuitry) configured to store data—for example, a cache may store copies of data that are also stored in a memory, such that one or more processors can access the copies of the data with a lower latency than would be associated with accessing the data in memory. Thus, each cache entry of the cache may be capable of storing data, and may be associated with a memory address (e.g. this could be a virtual address, a physical address or an intermediate physical address).
The apparatus also comprises cache access circuitry responsive to a cache access request to perform, based on a target memory address associated with the cache access request, a cache lookup operation. For example, the cache access request may be a request to access (e.g. load or store) data in the cache, and the target memory address may identify which data is to be accessed in response to the request.
The apparatus of the present technique also comprises tracking circuitry to track pending requests to modify cache entries of the cache.
When the cache access circuitry performs the cache lookup operation, it may detect a “hit” condition if an entry associated with the target memory address is determined to be present in the cache. On the other hand, the cache access circuitry may detect a “miss” condition if it is determined that there is no entry in the cache that is associated with the target memory address. Typically, if a hit is detected, the requested data is accessed in the cache (e.g. if the cache access request is a load request, the requested data may be read from the identified cache entry, whereas if the cache access request is a store request, data associated with the cache access request may be stored in the identified cache entry). If a miss is detected, a cache refill operation may be performed, to bring the requested data into the cache (for example).
An additional lookup (e.g. one or more lookup operations performed in addition to the cache lookup) of the tracking circuitry can also be performed, e.g. in order to determine whether any of the pending requests tracked by the tracking circuitry are to modify cache entries associated with the target memory address. However, such an additional lookup can be fairly power consuming, for example due to the need to toggle relatively expensive comparison logic associated with the tracking circuitry. Hence, it would be advantageous to be able to reduce the power consumption and latency associated with performing cache lookup operations.
To address this, the apparatus of the present technique comprises prediction circuitry, which is responsive to the cache access request to make a prediction of whether the pending requests tracked by the tracking circuitry include a pending request to modify a cache entry associated with the target memory address. The cache access circuitry is responsive to the cache access request to determine, based on the prediction, whether to perform an additional lookup of the tracking circuitry.
Determining whether to perform the additional lookup in dependence on the prediction made by the prediction circuitry reduces the power consumption of the system, by making it possible to avoid the additional lookup in at least some of the situations in which the lookup is not needed. Moreover, the performance of the apparatus can also be improved, by avoiding the additional latency associated with performing the additional lookup in such situations.
In some examples, the tracking circuitry comprises at least one of:
In this example, one or both of the store buffer and the cache refill control circuitry (also referred to herein as cache refill logic) may be provided (e.g. the tracking circuitry could include one or both of these structures). The store buffer tracks store operations (e.g. which may have been issued by processing circuitry executing store instructions), where store operations are requests to store data to one or more entries in the cache. The cache refill logic may control the allocation of data to the cache—for example, when a cache access misses (e.g. the requested data is not present in the cache), the cache refill logic may be responsible for selecting a victim entry in the cache to invalidate (evict), and bringing a copy of the requested data from memory into the cache.
In some examples, the tracking circuitry comprises the store buffer, the prediction circuitry comprises prediction data storage circuitry to store prediction data for use in making the prediction, and the prediction circuitry is responsive to a store request being issued to the store buffer, the store request being associated with a given memory address, to update the prediction data to indicate that a store buffer lookup should be performed, as the additional lookup, in response to a subsequent cache access request associated with the given memory address.
There are many ways in which the prediction circuitry could predict whether the store buffer tracks a store operation to store data to a cache entry associated with a given memory address, but in this example the prediction circuitry tracks store requests issued to the store buffer, and maintains prediction data that is updated each time a store request is issued.
In some examples, the tracking circuitry comprises the store buffer, the prediction circuitry comprises prediction data storage circuitry to store prediction data for use in making the prediction, and the prediction circuitry is responsive to misses being detected following lookups, based on a given memory address, in both the cache and the store buffer to update the prediction data to indicate that a store buffer lookup need not be performed, as the additional lookup, in response to a subsequent access request associated with the given memory address.
This update to the prediction data helps to maintain accurate prediction data, thus improving the accuracy of the predictions made by the prediction circuitry and reducing the number of additional lookups of the store buffer that are performed.
In some examples, each cache entry is configured to store a block of data, and the cache access request specifies a subset of the block of data stored in the cache entry associated with the target address. In these examples, the tracking circuitry comprises the store buffer, and the prediction circuitry is configured to make a further prediction, in response to predicting that the pending store operations tracked by the store buffer include a pending request to store the target data to the cache entry associated with the target memory address, of whether the target data encompasses the subset specified by the cache access request. In these examples, the cache access circuitry is configured to determine, based on the prediction and the further prediction, whether to perform an additional lookup of the tracking circuitry.
In this example, each cache entry stores a block of data of a predetermined size (e.g. the amount of data stored in a single cache entry is sometimes referred to as a cache line). Cache access requests—including the store operations tracked by the store buffer—can request the entire block of data stored in a cache entry—for example, the subset could be the entire block of data. However, in this example cache access requests may also be permitted to request access to a portion (some but not all, a proper subset) of the block of data—for example, a portion of the target memory address may identify the portion of the block of data to be accessed. When it is determined that the store buffer contains a store operation associated with the target memory address, it can be useful to make a further prediction of whether the subset of the block of data identified by the store operation encompasses the subset specified by the cache access request. This further prediction (in addition to the original prediction) can then be used to determine whether to perform the additional lookup.
In some examples, the tracking circuitry comprises the cache refill control circuitry the additional lookup, the prediction circuitry comprises prediction data storage circuitry to store prediction data for use in making the prediction, and the prediction circuitry is responsive to a cache miss being detected following a lookup, based on a given memory address, in the cache to update the prediction data to indicate that a cache refill control circuitry lookup should be performed, as the additional lookup, in response to a subsequent cache access request associated with the given memory address as the target memory address.
A cache refill operation may be triggered when a cache miss is detected following a cache lookup operation; hence, in this example, the prediction data is updated when a cache miss is detected, such that the prediction data can track cache refill operations which have been triggered.
In some examples, the tracking circuitry comprises the cache refill control circuitry, the prediction circuitry comprises prediction data storage circuitry to store prediction data for use in making the prediction, and the prediction circuitry is responsive to a cache hit being detected following the lookup, based on a given memory address, in the cache to update the prediction data to indicate that a cache refill control circuitry lookup need not be performed, as the additional lookup, in response to a subsequent cache access request associated with the given memory address as the target memory address.
If a cache refill operation is triggered by a cache miss associated with a particular memory address, a subsequent hit associated with the same memory address may indicate that the cache refill operation has completed. Hence, it can be helpful—as in this example—to update the prediction data in response to detecting a cache hit.
In some examples, the cache comprises a set-associative cache, and the apparatus comprises cache way prediction circuitry responsive to the cache access request to predict which of a plurality of ways stores data associated with the target memory address, the cache way prediction circuitry comprising the prediction circuitry.
A cache way tracker may be a structure looked up by memory address (or by a portion of the memory address) that provides information on cache hits (e.g. whether a given block of data is present in the cache and, if so, in which way). In this example of the present technique, the functionality of the cache way tracker is extended to additionally provide information as to whether the additional lookup needs to be performed in response to the cache access request. This is a particularly advantageous implementation of the present technique, because it makes use of a structure (the cache way tracker) which may already be provided, hence reducing the circuit area (and, hence, power consumption) required to implement the present technique.
In some examples, the cache way prediction circuitry comprises prediction data storage circuitry to store, for corresponding address information, prediction data indicative of:
In this way, the prediction data can be used both to predict which way of the cache stores data associated with a given memory address, and to predict whether the additional lookup should be performed.
In some examples, the pending requests to modify cache entries to the cache comprise at least one of:
Concepts described herein may be embodied in computer-readable code for fabrication of an apparatus that embodies the described concepts. For example, the computer-readable code can be used at one or more stages of a semiconductor design and fabrication process, including an electronic design automation (EDA) stage, to fabricate an integrated circuit comprising the apparatus embodying the concepts. The above computer-readable code may additionally or alternatively enable the definition, modelling, simulation, verification and/or testing of an apparatus embodying the concepts described herein.
For example, the computer-readable code for fabrication of an apparatus embodying the concepts described herein can be embodied in code defining a hardware description language (HDL) representation of the concepts. For example, the code may define a register-transfer-level (RTL) abstraction of one or more logic circuits for defining an apparatus embodying the concepts. The code may be define a HDL representation of the one or more logic circuits embodying the apparatus in Verilog, SystemVerilog, Chisel, or VHDL (Very High-Speed Integrated Circuit Hardware Description Language) as well as intermediate representations such as FIRRTL. Computer-readable code may provide definitions embodying the concept using system-level modelling languages such as SystemC and SystemVerilog or other behavioural representations of the concepts that can be interpreted by a computer to enable simulation, functional and/or formal verification, and testing of the concepts.
Additionally or alternatively, the computer-readable code may embody computer-readable representations of one or more netlists. The one or more netlists may be generated by applying one or more logic synthesis processes to an RTL representation. Alternatively or additionally, the one or more logic synthesis processes can generate from the computer-readable code a bitstream to be loaded into a field programmable gate array (FPGA) to configure the FPGA to embody the described concepts. The FPGA may be deployed for the purposes of verification and test of the concepts prior to fabrication in an integrated circuit or the FPGA may be deployed in a product directly.
The computer-readable code may comprise a mix of code representations for fabrication of an apparatus, for example including a mix of one or more of an RTL representation, a netlist representation, or another computer-readable definition to be used in a semiconductor design and fabrication process to fabricate an apparatus embodying the invention. Alternatively or additionally, the concept may be defined in a combination of a computer-readable definition to be used in a semiconductor design and fabrication process to fabricate an apparatus and computer-readable code defining instructions which are to be executed by the defined apparatus once fabricated.
Such computer-readable code can be disposed in any known transitory computer-readable medium (such as wired or wireless transmission of code over a network) or non-transitory computer-readable medium such as semiconductor, magnetic disk, or optical disc. An integrated circuit fabricated using the computer-readable code may comprise components such as one or more of a central processing unit, graphics processing unit, neural processing unit, digital signal processor or other components that individually or collectively embody the concept.
Particular examples will now be described with reference to the figures.
Based on the decoded instructions, the instruction decoder generates control signals to control the processing circuitry 4 to perform data processing operations represented by the instructions. Operands for the data processing operations are obtained from registers (selected as source registers), in a register file 10 comprising a certain number of registers. Results of the operations are written back to registers 10 (selected as destination registers) by the processing circuitry 4. For load/store instructions requesting that data from a cache or memory is loaded to the registers 10 or that data in the registers 10 is stored to a cache or memory, a load/store unit 12 may be instructed to carry out the load/store operation.
In addition to the instruction cache 7, a level 1 data cache 14 is provided to store copies of a subset of the data in memory. When performing an access (e.g. a read or write access), the load/store unit 12 may access the data in the cache instead of (or as well as) accessing it in the memory. Further levels of cache may be provided in addition to the level 1 caches (e.g. the memory system may have a hierarchical arrangement). Moreover, it is not essential for the instruction cache and the data cache to be separate—a shared cache storing both data and instructions can be provided.
It will be appreciated that
Accesses (e.g. loads and stores) to the cache are controlled by cache access circuitry 20. In particular, the cache access circuitry is responsive to access requests (e.g. read requests and/or write requests) issued by the load/store unit 12 to perform a cache lookup operation—hence, the cache access circuitry 20 is an example of cache access circuitry responsive to a cache access request to perform, based on a target memory address associated with the cache access request, a cache lookup operation. Each access request specifies a target memory address identifying a location in memory (e.g. the target memory address thus identifies the data to be accessed), and the target memory address could be a physical address (PA), a virtual address (VA) or an intermediate physical address (IPA), for example. The cache lookup operation performed by the cache access circuitry is to determine whether there is an entry in the cache which corresponds to the target memory address. If it is determined that a valid entry corresponding to the target memory address is present in the cache, this is considered to be a “hit”. On the other hand, if it is determined that there is no valid entry in the cache corresponding to the target memory address, this is considered to be a “miss”.
When a hit is detected following the cache lookup operation, the access can be performed in the cache (e.g. data can be read from or written to the identified cache entry). When a miss is detected, the data needs to be brought into the cache before this access can be performed—cache refill logic 22 is provided for the purpose. The cache refill logic is circuitry which is responsible for, in response to a cache miss being detected, performing a cache linefill operation to bring the requested data from the memory into the cache. In particular, the linefill operation involves:
While the linefill operation is in progress, the cache refill logic tracks pending evictions of data from the cache and pending allocations of data to the cache; hence, the cache refill logic is an example of tracking circuitry to track pending requests to modify cache entries of the cache.
A store buffer 24 is also provided. The store buffer comprises circuitry which tracks pending store operations (also referred to herein as store requests or write requests) to store data in the cache. Accordingly, the store buffer is another example of tracking circuitry to track pending requests to modify cache entries of the cache.
Since the tracking circuitry (e.g. the cache refill logic 22 and the store buffer 24) tracks pending requests to modify entries in the cache 16, additional lookups of these structures may also be performed in response to a cache access request.
Because of the possibility of a pending store request overwriting or updating the data being accessed in response to the cache access request, it can be helpful to identify such a pending store request with an additional lookup in the store buffer. If this additional lookup misses (e.g. it is determined that none of the pending store requests tracked by the store buffer is associated with the target memory address), the cache access circuitry may act in the same way as if the additional lookup had not been performed. For example, as shown in the bottom row of the figure, the cache access circuitry may respond to a cache hit by accessing the data in the cache, and may respond to a cache miss by initiating a cache linefill operation, before accessing the data in the cache.
The cache access circuitry may act in the same way if the additional lookup hits (e.g. it is determined that at least one of the pending store requests tracked by the store buffer is associated with the target memory address), but it is determined that the data associated with the identified pending store request(s) does not fully encompass the data to be accessed in response to the cache access request. This is shown by the middle row in the figure.
However, if the additional lookup of the store buffer hits, and it is determined that the data associated with a pending store request fully encompasses the data subject to the cache access request, the cache access circuitry may act differently. For example, if the cache lookup also hits, the data may be accessed in the store buffer instead of in the cache. Alternatively, the access request may stall (e.g. to wait until the store request has been completed) or may trigger a fault. Similarly, if the cache lookup misses, the data may be accessed in the store buffer instead of performing a cache linefill operation, or the access request may stall or trigger a fault.
If it is determined, following the additional lookup of the cache refill logic, that there is no pending eviction or allocation associated with the target memory address, the cache access circuitry acts in the same ways as if the additional lookup in the cache refill logic had not been performed (possibly in dependence on another additional lookup in the store buffer, as discussed above). This is shown in the bottom row of the figure.
If it is determined, following the additional lookup of the cache refill logic, that there is a pending allocation associated with the target memory address (in which case, the cache lookup will have resulted in a miss, since the cache refill logic allocates data to the cache in response to a miss), the cache access circuitry may either wait/stall until the allocation has completed (following which the access can be performed in the cache), or it may signal a fault (e.g. rather than initiating another cache linefill operation).
If, on the other hand, it is determined that there is a pending eviction associated with the target memory address (in which case, the cache lookup will have resulted in a hit, since an entry cannot be evicted from the cache if it is not present in the cache), the cache access circuitry may either force a miss (e.g. trigger a linefill operation) or signal a fault.
Accordingly, as demonstrated by
The cache access circuitry 20 can then use the prediction made by the prediction circuitry to determine whether or not to perform one or more additional lookups of the tracking circuitry. For example, the cache access circuitry 20 could determine that an additional lookup should be performed unless the prediction circuitry 28 predicts that there are no pending requests to modify one or more cache lines associated with the target memory address of the cache access request. However, it will be appreciated that the way in which the prediction is used to determine whether to perform the additional lookup is not limited to this example. Note that, if the tracking circuitry 26 includes more than one tracking structure (e.g. the cache refill logic and the store buffer could both be provided), it is possible for the cache access circuitry 20 to determine that one structure should be looked up while the other should not.
Providing the prediction circuitry 28, and determining whether to perform the additional lookup based on the prediction, makes it possible to avoid performing the additional lookup in some of the situations where the result of the additional lookup is unlikely to change the way in which the requested data is accessed in the cache. This helps to reduce the number of times an additional lookup is performed, hence reducing the power consumption of the system as a whole.
The cache 16 of the present technique can be any type of cache, and can have any placement policy. However, in a particular example, the cache 16 is a set-associative cache 32. As shown in
The set-associative placement policy used for such a cache means that when data having a given address needs to be allocated into the cache, it is placed in one of the entries 34 within a particular set 38 that is selected based on an index value 39 computed based on the given address. In some cases the index value 39 may simply be a portion of bits extracted from the given address, while in other examples the index 39 could be based on a more complicated function of the given address (e.g. applying a hash function to the given address to obtain the index value 39). Hence, data associated with a given address cannot be stored in entries 34 of sets 38 other than the set corresponding to the index 39 selected based on the given address. This is useful because it reduces the number of entries of the cache which have to be checked to determine whether the cache stores data associated with a specified target address, but in comparison to a direct-mapped cache (where data for a given address can only be allocated to a single entry 34 selected based on the address), the set-associative placement scheme improves performance as there is flexibility to allocate data for a given address to two or more locations, which reduces the likelihood of thrashing as it means two or more different addresses mapping to the same set can each be cached simultaneously. This contrasts with a fully-associated placement policy, where data can be allocated to any entry in the cache, and with a direct-mapped cache, where data can only be allocated to a particular entry of the cache (e.g. a direct-mapped cache can also be considered to be a 1-way set-associative cache).
Each entry 34 may specify a cache tag value 40 and a data value 42. The data value 42 is the information of interest which is stored in the corresponding cache entry 34. The data value 42 could be data or instructions, or could be address mapping information cached in a translation lookaside buffer, for example. The tag 40 corresponds to a portion of the target address which is not used to generate the index, and is stored alongside the cached data 42 to allow the different addresses which map to the same index to be distinguished from each other when looking up in the cache. Each entry 34 may also store state information associated with the corresponding address, such as a valid indicator indicating whether the data in the corresponding entry 34 is valid, coherency state information (e.g. a dirty bit indicating whether the data value 42 has been modified compared to the corresponding data value in a higher level cache (e.g. L2 or L3 cache) or memory), or replacement policy information for selecting a victim cache entry when an entry needs to be evicted from the cache.
Hence, on a cache access to check whether data associated with a target address is stored in the cache, the index value 39 derived from the target address is used to select a set 38 and each of the tag values 40 in the entries 34 within the selected set 38 are compared with the tag portion of the target address. If any of the read tag values 40 match the tag portion of the target address then the corresponding cache entry 34 having the matching tag 40 stores the data for the requested target address, and that entry can be read or written depending on the type of access being performed. In some examples, an additional portion of the target address (an offset portion) may be used to identify a particular data word within the entry to be accessed. The scenario when one of the tags 40 in the indexed set 38 matches the tag of the target address is called a cache hit.
On the other hand, if none of the tags 40 in the indexed set 38 match the tag of the target address, then this is known as a cache miss, and in this case the information associated with the target address may need to be fetched from a further data store, such as a further level of cache or main memory. If one of the indexed set of caches is invalid, then the invalid entry can be selected for allocating the new data associated with a target address. However, if all of the indexed set of entries are already filled with valid data then one entry 34 of the indexed set 38 can be selected as a victim entry for which the data 42 is to be evicted from the cache to make way for the new information associated with the target address.
In the present technique, the functionality of the cache way predictor 44 can be extended to also provide the functionality of the prediction circuitry 28 described above. This is a particularly advantageous implementation of the present technique, because it makes use of the comparison logic that provided in the cache way predictor to also predict the contents of the tracking circuitry—hence, this approach reduces the circuit area required to provide the functionality of both the prediction circuitry 28 and the cache way predictor (and hence also reduces the cost in terms of the power consumption of the circuitry).
However, it will be appreciated that the prediction circuitry 28 could be a separate structure from the cache way tracker 44, and indeed the cache way tracker 44 need not be provided at all.
Note that whether or not a cache way tracker 44 and/or prediction circuitry 28 are provided does not affect the functional correctness of data processing performed by the apparatus. For example, the result of a cache access will be the same regardless of whether the prediction circuitry and/or the cache way predictor 44 are provided (e.g. the same data will be read from/written to the cache). However, each of these structures can improve the performance and reduce the power consumption of the apparatus as a whole.
The method in Figured 9 also includes a step 64 of determining whether the additional lookup, if performed, resulted a hit. If the additional lookup did result in a hit (“Y”), an alternative response (e.g. signalling an error, forcing a miss and/or stalling) is enacted.
In the present application, the words “configured to . . . ” are used to mean that an element of an apparatus has a configuration able to carry out the defined operation. In this context, a “configuration” means an arrangement or manner of interconnection of hardware or software. For example, the apparatus may have dedicated hardware which provides the defined operation, or a processor or other processing device may be programmed to perform the function. “Configured to” does not imply that the apparatus element needs to be changed in any way in order to provide the defined operation.
Further, the words “comprising at least one of . . . ” in the present application are used to mean that any one of the following options or any combination of the following options is included. For example, “at least one of: A; B and C” is intended to mean A or B or C or any combination of A, B and C (e.g. A and B or A and C or B and C).
Although illustrative embodiments of the invention have been described in detail herein with reference to the accompanying drawings, it is to be understood that the invention is not limited to those precise embodiments, and that various changes and modifications can be effected therein by one skilled in the art without departing from the scope of the invention as defined by the appended claims.
Examples of the present technique include:
(1) An apparatus comprising:
(2) The apparatus of clause 1, wherein
(3) The apparatus of clause 2, wherein:
(4) The apparatus of clause 2 or clause 3, wherein:
(5) The apparatus of any of clauses 2 to 4, wherein:
(6) The apparatus of any of clauses 2 to 5, wherein:
(8) The apparatus of any preceding clause, wherein:
(9) The apparatus of clause 8, wherein:
(10) The apparatus of any preceding clause, wherein the pending requests to modify cache entries to the cache comprise at least one of:
(11) A method comprising:
(12) A computer program comprising instructions which, when executed on a computer, cause the computer to fabricate an apparatus comprising:
(13) A computer-readable medium to store the computer program of clause 12.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5752069 | Roberts | May 1998 | A |
| 6016533 | Tran | Jan 2000 | A |
| 20140304492 | Abdallah | Oct 2014 | A1 |
| 20150067305 | Olson | Mar 2015 | A1 |
| 20150309792 | Meier | Oct 2015 | A1 |
| 20170123794 | Chen | May 2017 | A1 |
| 20180081590 | Farahani | Mar 2018 | A1 |
| 20180342096 | Peterson | Nov 2018 | A1 |
| 20220358037 | Favor | Nov 2022 | A1 |
| Entry |
|---|
| Akkary et al. “Checkpoint Processing and Recovery: An Efficient Scalable Alternative to Reorder Buffers.” Nov.-Dec. 2003. IEEE. IEEE Micro. vol. 3. pp. 11-19. |
| Park et al. “Reducing Design Complexity of the Load/Store Queue.” Dec. 2003. IEEE. MICRO-36. |
| Amir Roth. “A High-Bandwidth Load-Store Unit for Single- and Multi-Threaded Processors.” Jan. 2004. University of Pennsylvania Department of Computer and Information Science Technical Report No. MS-CIS-04-09. |
| Tuck et al. “Scalable Cache Miss Handling for High Memory-Level Parallelism.” Dec. 2006. IEEE. Micro '06. |
| Mathew et al. “Low power L2 cache design using partially tagged bloom filter and hotline check.” Jan. 2013. IEEE. ICCCI-2013. |
| Lee et al. “Write Buffer-Oriented Energy Reduction in the L1 Data Cache for Embedded Systems.” Feb. 2016. IEEE. IEEE Transactions on Very Large Scale Integration (VLSI) Systems. vol. 24. pp. 871-883. |
