Patent Application
20140181462
The present invention is generally related to computer systems.
Modern processors reorder memory operations to increase performance. The reason for such reordering might be that some of the operands to some of the memory operations (i.e. addresses for loads and stores, store data for stores) are available at different times, and executing said memory instructions in program order would result in reduced performance and utilization, as the operands to a younger operation might be available earlier than the operands to an older operation.
Such reordering, however, is not always safe. For example, if the machine reorders a younger load to location X ahead of an older store to location X, the load will produce the wrong value, and the machine will operate incorrectly unless it detects such a condition and initiates recovery actions which ultimately result in the younger load effectively executing after the older store.
Some machines reorder operations only after their addresses are known, thereby preventing unsafe re-orderings, but reducing throughput and performance if the address operands are late. For example, a single store with an unresolved address can prevent many younger loads and stores from executing early even though the likelihood of such a conflict is often very low.
The memory reordering unit (MRU) is a unit that detects conflicts between loads and stores that have been reordered by the machine and initiates recovery action when an unsafe reordering is detected. The memory reordering unit is informed of the program order (or ordering constraints) of the loads and stores, and on a violation of order, initiates recovery. The memory reordering unit is either a part of or coupled to a load-store unit, but is not itself involved in the data portion of the execution of loads and stores—it only cares about addresses and sizes to detect overlap conditions that may constitute a violation of order.
In addition to such local violations of order (where a single-threaded program would operate incorrectly), depending on the architecture being implemented, there can also be violations of global order, where the consistency model for the architecture requires that some operations not be observably reordered by CPU cores such that other agents (e.g. device DMA or CPU cores running other threads in a multi-threaded program) can detect that reordering occurred.
The MRU can also detect such violations (potential or actual) of global order if snoops are sent to the MRU as well as to the caches. The MRU can effectively implement Fray's algorithm or variants as required by the consistency model for the architecture.
Typically such a reordering unit is based on physical addresses, as two different memory operations can use different virtual addresses that resolve to the same physical address, and if only virtual addresses are compared, two memory instructions (e.g. a load and a store) may appear not to overlap because the virtual addresses used are different, even though they both actually access the same location because the physical addresses are the same. Physical addresses, however, are known later than virtual addresses, making some overlap comparisons more difficult or introducing additional recovery situations.
It would be advantageous to use virtual addresses in the memory reordering unit and somehow handle the virtual address alias problem (two virtual addresses mapping to the same physical address) and the global ordering problem (if it matters for the architecture) by some other means.
Although virtual aliases have to be handled correctly, they are very rarely used in close proximity, hence the likelihood of an incorrect reordering detection based on virtual addresses is low, but correctness demands that it be detected.
Embodiments of the present invention advantageously utilize the feature whereby physical addresses are known later than virtual addresses. Embodiments of the present invention advantageously utilize virtual addresses in the memory reordering unit and efficiently handle a virtual address alias problem where two virtual addresses mapping to the same physical address, in addition to any global ordering problem.
In one embodiment, the present invention implements a method for detecting an instruction ordering violation in a CPU. The method includes receiving a reordered stream of instructions and detecting whether an ordering violation has occurred by using virtual addresses. The method further includes transferring results of the reordered stream of instructions from a load store buffer into a cache and detecting whether an ordering violation has occurred by using physical addresses. Subsequently, a recovery is initiated upon detection of an ordering violation.
The foregoing is a summary and thus contains, by necessity, simplifications, generalizations and omissions of detail; consequently, those skilled in the art will appreciate that the summary is illustrative only and is not intended to be in any way limiting. Other aspects, inventive features, and advantages of the present invention, as defined solely by the claims, will become apparent in the non-limiting detailed description set forth below.
The present invention is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements.
Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings. While the invention will be described in conjunction with the preferred embodiments, it will be understood that they are not intended to limit the invention to these embodiments. On the contrary, the invention is intended to cover alternatives, modifications and equivalents, which may be included within the spirit and scope of the invention as defined by the appended claims. Furthermore, in the following detailed description of embodiments of the present invention, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be recognized by one of ordinary skill in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail as not to unnecessarily obscure aspects of the embodiments of the present invention.
Some portions of the detailed descriptions, which follow, are presented in terms of procedures, steps, logic blocks, processing, and other symbolic representations of operations on data bits within a computer memory. These descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. A procedure, computer executed step, logic block, process, etc., is here, and generally, conceived to be a self-consistent sequence of steps or instructions leading to a desired result. The steps are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of non-transitory electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated in a computer readable storage medium of a computer system. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.
It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise as apparent from the following discussions, it is appreciated that throughout the present invention, discussions utilizing terms such as “ processing” or “accessing” or “ executing” or “ storing” or “rendering” or the like, refer to the action and processes of a computer system (e.g., computer system 100 of
Embodiments of the present invention advantageously utilize the feature whereby physical addresses are known later than virtual addresses. Embodiments of the present invention advantageously utilize virtual addresses in the memory reordering unit and efficiently handle a virtual address alias problem where two virtual addresses mapping to the same physical address, in addition to any global ordering problem.
The virtual-address based version (VMRU for virtual memory reordering unit) detects ordering violations by using only virtual addresses. As such, it can be very early in the pipeline since it does not need to wait for the TLB results. It compares the virtual addresses and sizes of loads and stores, and taking into account their program order (or otherwise-specified ordering constraints) it performs overlap detection and triggers recovery when a violation of order is detected.
A physical-address based version (PMRU for physical memory reordering unit) does not detect ordering violations per-se, but detects virtual aliases instead. As such, only cache line addresses need to be compared. The purpose of the physical-address version is to detect situations under which the virtual-address based reordering unit may have failed to report an ordering violation due to the use of virtual aliases in close-enough proximity. As such, the PMRU may be conservative (depending on the actual implementation used) as the particular virtual aliases detected may not have resulted in a violation of order. For example, the older store may have stored to some bytes in a cache line while the younger load may have loaded from different bytes in the same cache line.
In one embodiment, it should be noted that unlike the VMRU, which detects precise overlap, the physical-address based version can be implemented such that it does not need byte-overlap comparison and only examines cache line addresses. It can also execute later in the pipeline, once TLB results (physical addresses) are known. In particular, in one embodiment, all it needs to do is prevent the commitment or retirement (depending on the micro-architecture) of the younger operation that may have a virtual alias problem.
This decoupling off the MRU into VMRU and PMRU is particularly advantageous when using a virtually-indexed cache that resolves aliases as if they were misses, as virtual aliases will be detected as part of miss conditions, and the PMRU only needs to be queried when resolving such misses. In fact, if such virtual alias conflicts are resolved by evicting the line and re-fetching it (e.g. from L2), the PMRU only needs to be queried on cache line evictions (whether prompted by a virtual alias detection or not), since an eviction (and re-fetch) is required before a line can be accessed under a different virtual address.
It should be further noted that, in one embodiment, querying the PMRU on evictions caused by coherence snoops also results in detection of violations of global memory order (between different agents) if the architecture in question has any such constraints.
Thus the VMRU detects local violations of order assuming that there are no virtual aliases, while the PMRU detects both potential virtual alias conflicts and potential violations of global order due to local reordering. The VMRU performs precise overlap comparisons, while the PMRU only needs to perform comparisons of cache line addresses.
Additionally, it should be noted that the particular details of the decomposition into VMRU and PMRU depend on the rest of the implementation of the machine, and can be adapted to both transactional and non-transactional micro-architectures, to both in-order and out-of-order micro-architectures, and to scalar, super-scalar, and VLIW micro-architectures.
Process 300 begins in step 301 where a reordered stream of instructions are received. The instructions are annotated with an encoding corresponding to their original order so that the VMRU can determine which have been incorrectly reordered (locally). As described above, the stream of instructions has been reordered in order to exploit instruction level parallelism and increase the efficiency of execution. In step 302, the VMRU functions by detecting whether an ordering violation has occurred by using virtual addresses as described above. In step 303, the results of the reordered stream of instructions are transferred from the load store buffer into the cache. In step 304, the PMRU functions by detecting whether an ordering violation has occurred by using physical addresses as described above. The PMRU is informed when instructions commit or retire and detects ordering violations for not yet committed/retired instructions. And subsequently, in step 305, a recovery is initiated upon detection of an ordering violation (e.g., causing a cache miss, flushing the cache, re-executing the stream of instructions in program order, or the like).
It should be noted that depending on whether the machine can recover from miss-speculation, step 303 may only happen after it is known that no recovery will take place. Additionally, it should be noted that in one implementation where the cache is used to implement the PMRU and it provides good filtering so that we only check evictions in a PMRU, embodiments of the present invention could also be implemented by sending everything to a PMRU, or by performing some other filter.
It should be appreciated that the GPU 410 can be implemented as a discrete component, a discrete graphics card designed to couple to the computer system 400 via a connector (e.g., AGP slot, PCI-Express slot, etc.), a discrete integrated circuit die (e.g., mounted directly on a motherboard), or as an integrated GPU included within the integrated circuit die of a computer system chipset component (not shown). Additionally, a local graphics memory 414 can be included for the GPU 410 for high bandwidth graphics data storage.
The foregoing descriptions of specific embodiments of the present invention have been presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the invention to the precise forms disclosed, and many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and its practical application, to thereby enable others skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the claims appended hereto and their equivalents.
