The present invention relates to data processing apparatuses. More particularly, the present invention relates to the prefetching of data values in a data processing apparatus.
It is known for a data processing apparatus which executes a sequence of program instructions to be provided with a prefetcher which seeks to retrieve data values from memory for storage in a cache local to an instruction execution unit of the data processing apparatus in advance of those data values being required by the instruction execution unit. The memory latency associated with the retrieval of data values from memory in such data processing apparatuses can be significant, and without such prefetching capability being provided would present a serious performance impediment for the operation of the data processing apparatus.
It is further known for such a prefetcher to dynamically adapt the number of data values which it prefetches into the cache in advance. On the one hand, if the prefetcher does not prefetch sufficiently far in advance of the activities of the processor (instruction execution unit), the processor will cache up with the prefetcher and will seek access to data values in the cache before they have been retrieved from the memory, requiring the processor to wait whilst the corresponding memory accesses complete. On the other hand, if the prefetcher prefetches data values too far in advance, data values will be stored in the cache for a long time before they are required and risk being evicted from the cache by other memory access requests in the interim. The desirable balance between these competing constraints can vary in dependence on the nature of the data processing being carried out and accordingly it is known for the prefetcher to be configured to adapt its prefetch distance (i.e. how far is advance of the processor it operates) dynamically i.e. in the course of operation by data processing apparatus.
Viewed from a first aspect, the present invention provides a data processing apparatus comprising:
an instruction execution unit configured to execute a sequence of program instructions, wherein execution of at least some of the program instructions initiate memory access requests to retrieve data values from a memory;
a cache unit configured to store copies of the data values retrieved from the memory; and
a prefetch unit configured to prefetch the data values from the memory for storage in the cache unit before they are requested by the instruction execution unit by extrapolating a current data value access pattern of the memory access requests to predict future data values which will be requested by the instruction execution unit and prefetching the future data values,
wherein the prefetch unit is configured to perform a miss response comprising increasing a number of the future data values which it prefetches when a memory access request specifies a pending data value which is already subject to prefetching but is not yet stored in the cache unit,
wherein the prefetch unit is configured, in response to an inhibition condition being met, to temporarily inhibit the miss response for an inhibition period.
The prefetch unit according to the present techniques is configured to dynamically adjust its prefetch distance, i.e. the number of future data values for which it initiates a prefetch before those data values are actually requested by memory accesses issued by the instruction execution unit. It should be understood that here the term “data value” should be interpreted as generically covering both instructions and data. This dynamic adjustment is achieved by monitoring the memory access requests received from the instruction execution unit and determining whether they are successfully anticipated by data values which have already been prefetched and stored in the cache unit. In particular, the prefetch unit is configured to adapt the prefetch distance by performing a miss response in which the number of data values which it prefetches is increased when a received memory access request specifies a data value which is already the subject of prefetching, but has not yet been stored in the cache unit. In other words, generally the interpretation in this situation is that the prefetcher has correctly predicted that this data value will be required by a memory access request initiated by the instruction execution unit, but has not initiated the prefetching of this data value sufficiently far in advance for it already to be available in the cache unit by the time that memory access request is received from the instruction execution unit. Accordingly, according to this interpretation, the prefetch unit can act to reduce the likelihood of this occurring in the future by increasing the number of data values which it prefetches, i.e. increasing its prefetch distance, such that the prefetching of a given data value which is predicted to be required by the instruction execution unit is initiated further in advance of its actually being required by the instruction execution unit.
However, the present techniques recognise that it may not always be desirable for the prefetch unit to increase its prefetch distance every time a memory access request is received from the instruction execution unit which specifies a data value which is already subject to prefetching but is not yet stored in the cache. For example, the present techniques recognise that in the course of the data processing activities carried out by the data processing apparatus, situations can occur where increasing the prefetch distance would not necessarily bring about an improvement in data processing performance and may therefore in fact be undesirable. Accordingly, the present techniques provide that the prefetch unit can additionally monitor for an inhibition condition and where this inhibition condition is satisfied, the prefetch unit is configured to temporarily inhibit the usual miss response (i.e. increasing the prefetch distance) for a predetermined inhibition period. This then enables the prefetch unit to identify those situations in which the performance of the data processing apparatus would not be improved by increasing the prefetch distance and to temporarily prevent that usual response.
The inhibition condition may be configured in a number of different ways, but in one embodiment the inhibition condition comprises identification of a mandatory miss condition, wherein the mandatory miss condition is met when it is inevitable that the pending data value specified by the memory access request is not yet stored in the cache unit. Accordingly, in situations where it is inevitable that the pending data value is not yet stored in a cache unit, i.e. the fact that the data value is not yet stored in cache unit could not have been avoided by a different configuration of the prefetch unit, it is then advantageous for the configuration of the prefetch in unit is particular its prefetch distance) not to be altered.
A mandatory miss condition may arise for a number of reasons, but in one embodiment the mandatory miss condition is met when the memory access request is not prefetchable. The fact that the memory access request is not prefetchable thus presents one reason explains why the configuration of the prefetch unit (in particular its prefetch distance) was not at fault, i.e. did not cause the pending data value to not yet be stored in the cache unit.
In some embodiments the prefetch unit is configured to perform a stride check for each memory access request, wherein the stride check determines if the memory access request does extrapolate the current data value access pattern, and wherein memory addresses in the data processing apparatus are administered in memory pages, and wherein the prefetch unit is configured to suppress the stride check in response to a set of memory addresses corresponding to the number of the future data values crossing a page boundary. In order to successfully extrapolate the current data value access pattern of the memory access requests being issued by the instruction execution unit, the prefetch unit may generally be configured to check for each new memory access request if the corresponding new address does match the predicted stride (i.e. data value access pattern extrapolation), but this stride check can be suppressed when a page boundary is crossed to save unnecessary processing where there is a reasonable expectation that the stride check may in any regard not result in a match.
In some embodiments, memory addresses in the data processing apparatus are administered in memory pages and the inhibition condition is met when a set of memory addresses corresponding to the number of the future data values crosses a page boundary. When the number of future data values being prefetched by the prefetch unit crosses a page boundary, this means that a first subset of those data values are in one memory page, whilst a second part of those data values are in a second memory page. Due to the fact that the physical addresses of one memory page may have no correlation with the physical addresses of a second memory page, this presents a situation in which it may well not have been possible for the prefetch unit to have successfully predicted and prefetched the corresponding target data value.
In some embodiments, the prefetch unit is configured such that the inhibition condition is met for a predetermined period after the number of the future data values (i.e. the prefetch distance) has been increased. It has been recognised that, due to the memory access latency, when the prefetch distance is increased the number of memory access requests which are subject to prefetching (and corresponding to a particular program instruction) will then increase before a corresponding change in the content of the cache unit has resulted and there is thus an interim period in which it is advantageous for the miss response (i.e. further increasing the prefetch distance) to be inhibited. Indeed, positive feedback scenarios can be envisaged in which the prefetch distance could be repeatedly increased. Whilst this is generally not a problem in the case of a more simple instruction execution unit, which would be stalled by the first instance in which the pending data value is not yet stored in the cache unit, in the case of a multi-threaded instruction execution unit, say, a greater likelihood exists of such repeated memory access requests relating to data values which are already subject to prefetching but not yet stored in the cache unit and the present mitigate litigate against repeated increased in the prefetch distance occurring as a result.
The duration of the inhibition period can be configured in a variety of ways depending on the particular constraints of the data processing apparatus, but in one embodiment the inhibition period is a multiple of a typical memory latency of the data processing apparatus, the memory latency representing a time taken for a data value to be retrieved from the memory. The inhibition period can therefore be arranged such that an adjustment in the number of future values which the prefetch unit prefetches (i.e. the prefetch distance) cannot be increased until this multiple of the typical memory latency has elapsed. For example, in the situation where the prefetch distance has not been increased because the prefetch distance has only recently been increased, this inhibition period then allows sufficient time for the desired increase in content of the cache unit to result.
The instruction execution unit may take a variety of forms, but in one embodiment, the data processing apparatus comprises plural instruction execution units configured to execute the sequence of program instructions. Further, in some embodiments the instruction execution unit is configured to execute multiple threads in parallel when executing the sequence of program instructions. Indeed, in some such embodiments, the instruction execution unit is configured to operate in a single instruction multiple thread fashion. As mentioned above, some of the problems which the present techniques recognise with respect to increasing the prefetch distance in response to a cache miss in a cache line which is already subject to a prefetch request can become more prevalent in a data processing apparatus which is configured to execute instructions in a more parallel fashion, and multi-core and/or multi-threaded data processing apparatuses represent examples of such a device.
Whilst the prefetch unit may be configured to increase its prefetch distance as described above, it may also be provided with mechanisms for decreasing the prefetch distance, and in one embodiment the prefetch unit is configured to periodically decrease the number of future data values which it prefetches. Accordingly, this provides a counterbalance for the increases in the prefetch distance which can result from the miss response, and as such a dynamic approach can be provided whereby the prefetch distance is periodically decreased and only increased when required. This then allows the system to operate in a configuration which balances the competing constraints of the prefetcher operating sufficiently in advance of the demands of the instruction execution unit whilst also not fetching too far in advance, thus using up more memory bandwidth than is necessary.
In some embodiments the prefetch unit is configured to administer the prefetching of the future data values with respect to a prefetch table, wherein each entry in the prefetch table is indexed by a program counter value indicative of a selected instruction in the sequence of program instructions, and each entry in the prefetch table indicates the current data value access pattern for the selected instruction, and wherein the prefetch unit is configured, in response to the inhibition condition being met, to suppress amendment of at least one entry in the prefetch table. The prefetch unit may maintain various parameters within each entry in the prefetch table to enable it to predict and prefetch data values that will be required by the instruction execution unit, and in response to the inhibition condition, it may be advantageous to leave these parameters unchanged. In other words, the confidence which the prefetch unit has developed in the accuracy of the prefetch table entries need not be changed when the inhibition condition is met.
Viewed from a second aspect the present invention provides a data processing apparatus comprising:
means for executing a sequence of program instructions, wherein execution of at least some of said program instructions initiate memory access requests to retrieve data values from a memory;
means for storing copies of the data values retrieved from the memory; and
Viewed from a third aspect the present invention provided a method of data processing comprising the steps of:
executing a sequence of program instructions, wherein execution of at least some of said program instructions initiate memory access requests to retrieve data values from a memory;
storing copies of the data values retrieved from the memory in a cache;
prefetching the data values from the memory for storage in the cache before they are requested by the executing step by extrapolating a current data value access pattern of the memory access requests to predict future data values which will be requested by the executing step and prefetching the future data values;
performing a miss response comprising increasing a number of the future data values which prefetched when a memory access request specifies a pending data value which is already subject to prefetching but is not yet stored in the cache; and
in response to an inhibition condition being met, temporarily inhibiting the miss response for an inhibition period.
The present invention will be described further, by way of example only, with reference to embodiments thereof as illustrated in the accompanying drawings, in which:
The memory system of the data processing apparatus 10 is arranged in a hierarchical fashion, wherein a level 1 (L1) cache 15, 16 is associated with each processor core 11, 12, whilst the processor cores 11, 12 share a level 2 (L2) cache 17. Beyond the L1 and L2 caches, memory accesses are passed out to external memory 18. There are significant differences in the memory latencies associated with each of the three levels of this memory hierarchy. For example, whilst it only takes approximately one cycle for a memory access request to access the L1 caches 15, 16, it typically takes 10-20 cycles for a memory access request which is passed out to the L2 cache 17, and a memory access request which does not hit in any of the caches and must therefore be passed out to the external memory 18 typically takes of the order of 200 cycles to complete.
Due to the significant memory latency in particular associated with accessing the memory 18, the data processing apparatus 10 is further provided with a prefetch unit 19 associated with the L2 cache 17. This prefetch unit 19 is configured to monitor the memory access requests received by the L2 cache 17 and on the basis of access patterns seen for those memory access requests to generate prefetch transactions which retrieve data values from memory 18 which are expected to be required in the future by one of the cores 11, 12. By causing these data values to be prepopulated in a cache line 20 of the level cache 17, the prefetch unit 19 seeks to hide the large memory latency associated with accessing memory 18 from the processor cores 11, 12.
In order to do this, the prefetch unit 19 must in particular maintain a given “prefetch distance” with respect to the memory access requests being issued by the processor cores 11, 12 by issuing a number of prefetch transactions in advance of the corresponding memory access requests being issued by the cores 11, 12, such that these prefetch transactions have time to complete and populate a cache line 20 before the corresponding data value is required and requested by a memory access request issued by one of the processor cores 11, 12. Accordingly, the prefetch unit 19 is provided with a prefetch table 21 populated with entries corresponding to the memory access requests observed to be received by the L2 cache 17 and allowing the prefetch unit 19 to develop a data value access pattern which it can extrapolate to determine the prefetch transactions which should be issued. More detail of this table 21 will be given below with reference to
The prefetch unit 19 also maintains a list of pending prefetches 22, i.e. a record of the prefetch transactions which it is has issued, but have not yet completed. In other words, as part of monitoring the L2 cache 17, when a prefetch transaction issued by the prefetch unit 19 completes and the corresponding data has been stored in a cache line 20, the corresponding entry in the list of pending prefetches 22 can be deleted. One particular use of the pending prefetches list 22 is to enable the prefetch unit 19 to adapt the prefetch distance it maintains with respect to a given entry in its prefetch table 21. When the prefetch unit 19 observes a memory access request received by the L2 cache 17 which hits in a cache line 20 which is currently in the process of being prefetched (i.e. has a corresponding entry in the pending prefetch list 22) then the prefetch unit 19 generally uses this as a trigger to increase the prefetch distance for that entry in the prefetch table 21, since this may well be an indication that the prefetch unit 19 needs to issue a prefetch transition for this entry in the prefetch table 21 earlier if it is to complete and populate the corresponding cache line 20 before the expected access request from one of the processor cores 11, 12 is received by the L2 cache 17. However, according to the present techniques the prefetch unit 19 will not always increase the prefetch distance in response to this situation, as will be described in more detail with respect to the following figures.
The final part of the entry in the prefetch table 21 is the prefetch distance which the prefetch unit maintains for this entry. This prefetch distance determines how many transactions in advance of the latest memory access request seen in association with this PC value the prefetch unit 19 generates. For example, in the snap shot shown in
The prefetch unit 19 is configured to dynamically adapt the prefetch distance in order to seek to maintain an optimised balance between not prefetching far enough in advance (and thus causing the processor cores 11, 12 to wait while the prefetched transaction corresponding to a memory access request catches up), and prefetching too far in advance which uses unnecessary memory bandwidth and further risks prefetched entries in the cache 17 being evicted before they have been used by the processor cores 11, 12. As a part of this dynamic adaptation, the prefetch unit 19 is generally configured to determine when a memory access request has been received by the L2 cache 17 which is currently in the process of being prefetched (i.e. has a corresponding entry in the pending prefetch list 22) and in this situation to increase the prefetch distance. However, the prefetch unit 19 is, in accordance with the present techniques, additionally configured to temporarily inhibit this response for a predetermined period under certain identified conditions.
One circumstance under which the control unit 30 determines the inhibition condition to be met is the crossing of a page boundary (as discussed above with reference to
Returning to a consideration of step 55, if it is found to be true that the memory access request has resulted in a miss in the L2 cache and a prefetch transaction for the corresponding memory address is currently pending, then the flow proceeds to step 59 where the control unit 30 of the prefetch unit 19 determines if the inhibition condition is currently met (note that at this stage of this embodiment, as defined in box 52 of
Although a particular embodiment has been described herein, it will be appreciated that the invention is not limited thereto and that many modifications and additions thereto may be made within the scope of the invention. For example, various combinations of the features of the following dependent claims could be made with the features of the independent claims without departing from the scope of the present invention.