Patent Application
20190004916
One or more exemplary embodiments discussed herein are generally directed to collecting profile information, and more specifically, are directed to collecting profile information in regard to instructions that are within one or more code regions of a software program.
Central processing unit (CPU) speeds have increased at a much faster rate than dynamic random access memory (DRAM) speeds. While static random access memory SRAM speed has generally kept up with the improvement in CPU speed, the cost of SRAM is substantially greater than that of DRAM. To address this problem, data can be loaded into a cache, so that the data are available when needed by the CPU. While it is now common to employ a cache to reduce the latency for data accessed by a CPU, the CPU can often experience delays while waiting for required data that are not in the cache to be accessed from memory and placed in the cache in response to memory requests.
Data prefetching can reduce this latency problem by loading data into the cache before and in anticipation of a memory request being made for the data by the CPU. However, prefetching can also cause problems if not efficiently implemented. For example, if prefetching loads data into the cache that will not be used by the CPU, the unneeded data can waste cache capacity, preventing data that will be needed from being loaded into the cache, or the prefetched data may displace data in the cache that are being used by the CPU (known as “cache pollution”). Also, if the data are prefetched and cached too early (causing a “capacity miss” because the prefetched data are displaced before a memory request for the data is made), or if the prefetching technique fetches the data too late or even fails to load data that will be needed by the CPU (both resulting in a “cache miss”), the efficiency with which memory requests are filled will be adversely affected.
Although prefetching mechanisms are beneficial in reducing memory latency problems for many applications, the increase in performance provided by prefetching can be greatly reduced, or the use of prefetching techniques can even result in a loss in performance, compared with a system that does not use prefetching mechanisms. This lower than expected performance can result from the interference caused by requests made by a prefetching engine, in regard to the shared resources employed by an application for satisfying normal memory requests. The problem is especially noticeable when prefetching is performed for a plurality of cores in a chip multiprocessor (CMP) where demand and prefetch requests made by different cores can interfere with each other. This potential negative effect has led to microprocessor designs where prefetch techniques are employed that behave with a restrained aggressiveness and use only easily predictable patterns. As a result, the full benefit of prefetching may not be achieved.
A number of techniques have been proposed to better handle the different memory requests made in modern computing systems by differentiating between demand and prefetching requests that use several shared resources (especially caches, but also queues of the various memory components, on-chip interconnect, memory controller, etc.), so as to treat them in a different manner. In many of these proposals, this differentiated treatment is handled dynamically, taking into account runtime information on the behavior of the prefetch requests and their interaction with other memory requests.
To more effectively fine tune the prefetching techniques employed in a specific application, it can be useful to collect various metrics, such as accuracy, lateness, and cache pollution, which define how well prefetching is performing. These statistics are usually obtained using a phase/interval-based profiling technique. Data related to prefetching events are collected during a specific interval of time. At the end of this interval, the desired metrics are computed, and the information gathered is used to select a behavior or configuration modification for the prefetching hardware structures being employed, which will be applied during the next interval of time.
The kind of phase/interval prefetching profiling information that has previously been obtained, apart from having a coarse granularity, would be of little use for systems such as Dynamic Binary Translators, just-in-time (JIT) compilers, or regular compilers, etc., where it would be desirable to employ the profiling information to generate better code or to dynamically modify the behavior of specific hardware components (dynamic memory management techniques, or prefetching throttling, for example) by inserting in the code, directives or hints to the hardware. More useful information might be obtained with instruction level profiling. However, in the case of prefetching statistics, this type of profiling would require quite complex hardware support, especially given the asynchronous nature of the kind of metrics desired, with respect to the memory execution that triggers prefetching. Moreover, for the purpose of applying compiler directed optimizations, it is not necessary to have prefetching statistics associated with specific instructions, but instead, to associate the statistics with specific code regions. The mechanism used for phase/interval prefetching profiling is incapable of collecting prefetching profile information related to asynchronous events, at the finer code region level of granularity.
Accordingly, it would be desirable to collect prefetching profiling data at a code region granularity. The resulting finer grain profiling data would be more useful for binary optimization of prefetching compared to profile data collected for a phase/interval, as noted above. It would also be desirable to employ relatively simple hardware and minimal software overhead to collect prefetching profiling data. This same approach that is used for prefetching profiling at the code region level of granularity might also be employed for colleting profiling data for other types of asynchronous operations.
Various aspects and attendant advantages of one or more exemplary embodiments and modifications thereto will become more readily appreciated as the same becomes better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, wherein:
Exemplary embodiments are illustrated in referenced Figures of the drawings. It is intended that the embodiments and Figures disclosed herein are to be considered illustrative rather than restrictive. No limitation on the scope of the technology and of the claims that follow is to be imputed to the examples shown in the drawings and discussed herein. Further, it should be understood that any feature of one embodiment disclosed herein can be combined with one or more features of any other embodiment that is disclosed, unless otherwise indicated.
The profiling information that is obtained for a phase/interval in this example include: (a) the number of times that the data were prefetched too late to be used, which is accumulated by a late_prefetching_requests_counter 112, using the MSHR; (b) the total number of prefetch memory requests that were issued during the phase/interval, as determined by the count of data with the prefetch_bit set accumulated by a prefetching_requests_counter 114; and, (c) the accumulated count of the prefetched data lines in cache 108, that were used, based on the accumulated count by a useful_prefetching_requests_counter 116. Again, it is emphasized that this prior approach is not able to collect profile data for asynchronous events, on a code region level of granularity.
As noted above, it is not necessary to collect profile data for prefetching on an instruction level. Instead, a compiler can effectively work toward optimizing prefetching based on profile data metrics obtained at a code region level of granularity.
The present approach for collecting profile data at a code region level of granularity is implemented using an exemplary hybrid hardware/software mechanism, such as shown in a schematic block diagram 400 illustrated in
Prefetcher 410 is continually monitoring memory accesses by the processor, searching for predictable patterns, and eventually generates prefetching requests in response to specific memory operations that trigger prefetching engine 412. These prefetching requests are stored in prefetching queue 414 and are subsequently submitted to the memory system. Each prefetching request brings memory blocks to a cache 424, potentially evicting existing useful blocks that were already stored there. The prefetched data will be later requested by a regular memory demand request if the prefetching engine was successful in predicting its choice of prefetched data blocks, or will have wasted cache storage space—if not. All these subsequent events, although related to the original memory operation that triggered the prefetching, take place asynchronously with respect to that operation.
A selector 418 is responsible for issuing memory requests to the cache and for arbitrating between demand and prefetch requests if they are waiting to be issued at the same time. When a memory request is issued an AND gate 420 responds to an IS_PREFETCH signal from selector 418 and to a PROFILE_BIT signal that indicates that the first prefetch request to be issued was generated from a instruction in the desired code region being profiled and determines if the request corresponds to a prefetching request for which events are to be tracked and profiled. If the issued request is a prefetching request that lands in the cache, it is discarded. If the data requested is not in the cache, the request is annotated in MSHR 422 by marking it with a PREFETCH_BIT in response to the DO_PREF_PROFILE signal produced by AND gate 420. When the data arrive and are stored in cache 424, the data are marked with a PREFETCH_BIT derived from the marker in MSHR 422. If the data thus stored in cache 424 are used by the processor, a USED_BIT is set. MSHR 422 also keeps track of the prefetched requests that were fulfilled too late to be used by the processor, so that such profile information can be collected in regard to the desired code region. Note that this type of event occurs when a demand request arrives at MSHR 422 for a data block already requested by an in-flight prefetching request. In this case, the PREFETCH_BIT is cleared.
A way of generating prefetching events only for memory requests and memory blocks associated with the prefetching requests triggered by an instruction from a specific code region is described above. In addition to these types of events, it may be desirable to collect profiling statistics related to other events, synchronous or asynchronous, and at a code region and/or interval/phase level of granularity. The exemplary mechanism shown in
Other exemplary embodiments for determining if an instruction address is within a desired code region are illustrated in
It is contemplated that, at the cost of a higher complexity and hardware costs, profile information can be collected for a plurality of desired code regions at the same time, since any of the above alternative exemplary embodiments for determining whether an instruction is within a desired code region can be applied to make that determination for each of the multiple code regions of interest. It would also be necessary to mark each prefetch request, each entry in the MSHR, each block in the cache, and each event signal not only with a profile prefetch bit, but in addition, with an identifier of the code region in which the instruction giving rise to the event was located. This identifier would thus be used to ensure that profile information being collected is associated with the correct code region. Using n bits for this identifier would enable 2n different code regions to be profiled at the same time. Accordingly, events of interest that result from instructions in each of the multiple desired code regions can readily be identified and accumulated to provide the profiling information for each such code region.
It also should be understood that the desired code region can be a single software instruction, so that the present approach can thus be used to collect profile information at an instruction level of granularity. Further, profile information can simultaneously be collected for multiple single instruction code regions. It is also contemplated that profile information can be collected for relevant events that result from execution of only a subset of 2n instructions in a code region that trigger the prefetcher engine, where n is the number of bits required to mark each prefetch request, entry in intermediate structures, and event signals—just as noted above in the case where profile data are simultaneously collected for a plurality of code regions.
While not required for implementing the present novel approach, a communication module 610 can be included to enable the computer system to communicate with other computing devices or storages over a network, and the communication module may comprise a modem, or an network interface card for coupling to an Ethernet, a token ring, or other type of local area network, or a wide area network, or to the Internet. The communication module enables computer system 600 to upload and/or download data and/or software programs by communicating with other computer systems, or servers, or storage devices.
Bus 604 may also include an input/output bus (not separately shown) for coupling to a display 616 that displays graphics and/or text, a pointing device 614, such as a mouse, trackball, touchpad, or other input device for controlling a cursor and making selections, and a keyboard 612 (or touch screen) for providing input of alphanumeric text and controlling the operation of computer system 600.
Decision block 708 determines if a memory instruction that is within the current desired code region for which profile data are being collected is triggering prefetching. Again, it should be noted that the present approach is not limited to collecting profile information for prefetching. However, the collection of profile data for prefetching event is discussed in the present example, since this application clearly illustrates the ability of this approach for collecting profile information for asynchronous events. If the response in decision block 708 is affirmative, a block 710 adds a profile bit (as a marker) to the prefetching memory request that is loaded into the prefetching queue. As noted in a block 712, when a prefetching memory request that is thus marked with a bit is issued to the memory system, the marker bit is propagated with the request to indicate that the memory request was in response to a prefetching instruction from the desired code region for which the profile data are being collected. Accordingly, in a block 714, all lines of the memory block that is prefetched and loaded into the cache in response to the executed marked prefetching request are also marked to indicate that they were prefetched in response to a prefetch request from within the desired code region. A block 716 indicates that the processor executes the software instructions of the program being run, using the data in the cache, when possible. Block 716 also applies when the response to decision block 708 is negative, i.e., when a memory instruction triggering prefetching was in response to a prefetch request that was not within the desired code region, bypassing the intervening logic between decision block 708 and block 716.
A block 718 detects the occurrence of each desired event, i.e., each different type of profile information that is to be collected, for those events related to the execution of instructions from the code region of interest. For prefetching profile data, the collected information might include, for example, the total number of late prefetches, and the total count of each prefetch providing data actually used by the processor. However, profile information can also be collected for other types of events. In a block 720, the event counter that is collecting profile data for each different type of event is incremented each time that a desired event relating to a marked prefetching request is detected. A block 722 transfers the count(s) accumulated in the event counter(s) as profile data for the desired code region, once the end of the current profiling interval is reached. The transfer of this data can be to a more permanent storage or to a dynamic compiler that can use the data to improve the prefetching performance. As indicated in a block 724, the profile data that are thus collected can be employed to recompile software, either statically or dynamically, to more effectively employ prefetching.
An advantage of one or more embodiments of the present approach is that profile information can be collected for both synchronous and asynchronous events that result from the execution of instructions in a desired code region. In contrast, the prior approach is generally limited to collecting profile information at a phase/interval level of granularity, or in capturing only profile information related to synchronous events—but not profile information for asynchronous events at a code region level of granularity.
Although the concepts disclosed herein have been described in connection with exemplary embodiments for practicing them and modifications thereto, those of ordinary skill in the art will understand that many other modifications can be made thereto within the scope of the claims that follow. Accordingly, it is not intended that the scope of these concepts in any way be limited by the above description, but instead be determined entirely by reference to the claims that follow.
| Number | Date | Country | |
|---|---|---|---|
| Parent | 13993054 | Jun 2013 | US |
| Child | 16026870 | US |
