Patent Application
20080126764
One or more embodiments relate to dynamic binary code optimizations. More specifically, one or more of the embodiments relate to using transactional memory for precise exception handling in aggressive dynamic binary code optimizations.
Dynamic binary optimization takes advantage of information discovered at runtime to improve the performance of binary code. Runtime optimizations include reordering code, eliminating dead code, and otherwise streamlining the original binary code.
Dynamic binary optimization may be performed when translating binary code from one instruction set architecture (ISA) to another. For example, binary code may be dynamically optimized when it is dynamically translated from 32-bit Intel® Architecture (“IA32”) to a 64-bit Intel architecture (“IA64”). Intel architecture refers to ISAs developed by Intel Corporation of Santa Clara, Calif. For convenience, the original binary code is referred to as “source binary code” and the optimized, translated binary code is referred to as “target binary code.”
Often the target binary code is native to and optimized for the machine on which it is being executed. However, the source binary code is often neither native to nor optimized for the machine. Consequently, it is often desirable to perform dynamic optimization when dynamically translating a source binary code to a target binary code. For example, source IA32 code is generally not optimized to take advantage of efficiencies available on machines running IA64 code. Therefore, the IA32 source code may be optimized as well as translated into the target IA64 code. Then, the optimized target IA64 binary code is executed.
Challenges arise when aggressively optimized binary code generates runtime exceptions. Precise handling of these runtime exceptions requires 1) that the order of the exceptions occur in the same order as they would have running the original unoptimized binary code; and 2) that the exception handlers see the same processor states as if the original unoptimized binary code had generated the exceptions.
Straightforward enforcement of precise exception handling may severely constrain dynamic binary code optimizers. For example, a dynamic binary code optimizer may not be able to move a register update down the instruction stream, passing an instruction that may generate an exception, because doing so would change the machine state seen by the exception handler. Also, the optimizer may not be able to reorder two loads that could potentially cause segmentation errors because the reordering changes the order of the possible exceptions raised by the two instructions.
Dynamic binary code optimizers may use transactional memory to provide precise handling of runtime exceptions generated by aggressively optimized binary code. Transactional memory is used to implement transactions—code regions or subregions that when executed, are either atomically committed or atomically aborted.
Transactional memory may be implemented with a variety of memory types. In one embodiment, transactional memory is dedicated memory on a processor. In other embodiments, transactional memory may include on-chip cache memory, off-chip cache memory, dynamic random access memory, memory on storage devices such as, for example, disk drives, or other memory types. In some embodiments transactional memory is limited to the size of a particular physical memory or set of physical memories. In other embodiments, transactional memory is implemented with virtual memory and may be as large as the virtual memory address space permits.
Transactional memory simplifies parallel programming by guaranteeing that transactions appear to execute atomically. As discussed above, a transaction has a property that when it is executed, it is either atomically aborted or atomically committed. In atomic transactions, either all memory operations in the transaction take effect (a “commit”) or none does (an “abort”). Atomicity is provided by version management, which handles simultaneous storage of both new and old data. The new data is used if the transaction commits. The old data is retained if the transaction aborts.
On memory stores, the version management implementing atomicity may be eager or lazy. Eager version management writes a new value to memory without waiting for a commit. The old value is retained separately for recovery in case of an abort. Lazy version management waits until there is a commit before writing new values to memory. Meanwhile, the new value is stored on the side to be used if there is a commit.
Transactional memory provides transaction isolation. There are two scopes of isolation, weak isolation and strong isolation. Weak isolation means that the partially-complete state of a transaction is hidden from other transactions. Strong isolation means that the partially-complete state of a transaction is hidden from all the operations outside the transaction, whether they are in a transaction or not.
Isolation is provided by conflict detection. In weak isolation, conflict detection signals a conflict when the write set of one transaction overlaps the read/write set of another concurrent transaction. In strong isolation, conflict detection signals a conflict when the write set of a transaction overlaps the read/write set of any another concurrent operation—regardless of whether it is a transaction.
The conflict detection for strong or weak isolation may be eager (offending load/store is signaled immediately) or lazy (conflict is detected at the transaction commit time). Therefore, there are at least the following four possible transactional memory implementations for either strong or weak isolations.
Dynamic binary code optimizers may use transactions to provide precise exception handling for aggressive optimizations that the dynamic optimizer otherwise might forego for lack of precise exception handling capability.
An aggressively optimized binary code region that might generate an exception may be converted to a transaction. Then, if the code generates a runtime exception, the transaction is aborted.
In one embodiment, after the abort, original unoptimized binary code and processor state associated with an entry point to the original unoptimized binary code is retrieved. The original unoptimized binary code is then executed and any exceptions will of course occur in the original order. Thus, precise exception handling is provided.
In another embodiment, if an entire aggressively optimized code region is coded as a transaction, the original unoptimized binary code is reoptimized before being executed. The code is reoptimized less aggressively (foregoing at least some reordering optimizations) and is not coded as a transaction. The less aggressively optimized code is then executed and any exceptions will occur in the original order. Precise exception handling is still provided.
Although aborts are expensive in terms of processing time, they are usually rare. Thus, using transactions for precise exception handling typically results in overall net improvement in dynamically optimized binary code performance.
However, it is desirable to minimize the frequency of aborts. When executing parallel programs, the risk of aborts is greater in transactions that share a variable with another transaction or code region that is being executed in a different thread, process or processor core. The risk of aborts is also greater in transactions having loops, especially spin loops. Therefore, in some embodiments, a code region is not converted to a transaction if it has a shared variable or a loop. A shared variable is a variable that, upon execution of a portion of application code, is to be readable or writable by two or more threads, processes or processors. A code region that is not converted to a transaction is then less aggressively optimized.
Referencing
In code discovery 116, the front end 102 identifies blocks or regions of code. In some embodiments, a code region is a basic block of instructions, usually a sequence of non-branching instructions ending in a branch instruction. In code discovery 116, the front end 102 also translates source binary code (for example, IA32 code) 104 directly to target binary code (for example, IA64 code) 10, without first translating to an intermediate representation. This is sometimes referred to as a cold translation. In some embodiments, some limited binary code optimization is performed with the cold translation. Runtime 112 facilitates the execution of the target binary code 110. Runtime 112 also interfaces with the operating system and responds to runtime exceptions.
As translated code is executed, hot region formation 118 identifies frequently executed code regions and flags them for aggressive optimization by backend 108. Front end 102 then translates 120 these frequently executed code regions from binary code to intermediate representation (IR) code 106 and passes the IR code 106 to backend 108. Generally, IR code 106 is passed to backend 108 in groups of code regions. The backend 108 may then select certain code regions from the groups for particular optimizations, such as aggressive optimization inside a transaction.
Backend 108 aggressively optimizes the IR code 106 and translates it to target binary code 110. A runtime 112 facilitates execution of the target binary code 110 and interfaces with transactional memory 114, in one embodiment through an operating system interface (not shown). In another embodiment, a dynamic optimizer runs below the operating system and does not rely on an operating system interface. Backend 108 identifies code regions or subregions to convert into transactions and interfaces through runtime 112 with transactional memory 114 to effect the conversion of the code regions or subregions to transactions.
If a code region is coded as a transaction, then transactional memory 114 stores buffered memory updates 126 associated with code region instructions that have been executed but not yet committed or aborted. Transactional memory 114 also stores processor state 118 associated with an entry point to source binary code 104 that has been converted to a transaction not yet committed or aborted. Although source binary code 104 is not stored in transactional memory 114, it is retained in memory, for example, in main memory or memory other than transactional memory 114.
If an exception is generated by code in a transaction, the transaction is aborted. In one embodiment, stored processor state 124 and source binary code 104 are used to recover from the exception, as discussed below. In another embodiment, if an entire aggressively optimized region is coded as a transaction, source binary code 104 is reoptimized, but less aggressively and without being coded as a transaction. The reoptimized code is then executed to recover from the exception.
Further referencing
As previously stated, front end 102 performs translation from source binary code 104 to IR code 106. In an alternative embodiment, this translation is performed by backend 108. In yet another embodiment, source binary code is not translated to IR code 106 at all. Rather, backend 108 performs aggressive optimization directly on binary code.
The front end 102, the backend 108, and runtime 112 are simultaneously translating, optimizing, and facilitating the execution of application code. As used in this disclosure, simultaneous refers both to a single processor core performing multiple tasks concurrently through timeslicing and to actual parallel execution by the cores of a multi-core processor. In either case, runtime information gleaned from the execution of code is used to dynamically optimize the code.
Referencing
If the optimized code region 232 generates an exception, a transactional memory mechanism aborts the transaction, recovers minimal states (to be discussed below), and transfers control to an abort handler. The flow of control 238C in such case moves from the optimized code region 232 to an unoptimized code region 240, which is then executed.
Although an unoptimized code region 240 is described above, in another embodiment, flow of control 238C moves to a less aggressively optimized code region. For example, the region is optimized in ways that allow precise exception handling without the support of transactions.
Referencing
The dynamic optimizer identifies the original address for the code at the beginning of the transaction (retry_address). (Process block 346). The dynamic optimizer recovers stored processor state (such as, for example, stored processor state 124 of
An exception handler executes the unoptimized code with the following conditions satisfied: 1) the order of further exceptions is the same as that of the original code because the original code is being executed; and 2) the exception handler sees the same processor state as if the original code raised the exception because the original processor state was stored and recovered.
The original unoptimized code is executed or interpreted until the end of the region. (Process block 348). After being executed, the original unoptimized code may, in some embodiments, be reoptimized. In the event the dynamic optimizer is also translating the code region, it is reoptimized and retranslated. (Process block 350). The reoptimization/retranslation uses less aggressive optimization techniques that allow precise handling of exceptions without the support of transactions. Application code execution continues after the code region. (Process block 352).
In an alternative embodiment, if an entire code region is coded as a transaction, rather than being executed, the retrieved original unoptimized code is reoptimized, but less aggressively and without being coded as a transaction. The code is reoptimized using less aggressive techniques that allow precise handling of exceptions without the support of transactions. Then, the less aggressively reoptimized code is executed or interpreted. Application code execution continues after the code region.
Without support from transactional memory, certain optimizations may not be performed because precise exception handling cannot be assured. This is because without transactional memory, there is only a limited ability to buffer old and updated memory state, to checkpoint processor state, and to cancel executed instructions.
For purposes of this discussion, a code region may include both exception instructions (EI) and irreversible instructions (II). “Exception instruction” refers to an instruction that may generate a runtime exception. Examples of exception instructions are loads, stores, and floating point instructions (for example, floating point division).
“Irreversible instruction” refers to an instruction that changes the processor or application state in such a way that, without the use of transactional memory, it may not be possible, at least in some ISAs, to recover a previous processor state to precisely handle an exception. Without transactional memory support, a dynamic optimizer may forego an instruction-reordering optimization that would move an exception instruction past an irreversible instruction. That is because if the exception instruction generates an exception, it may not be possible to recover application or processor states for precise exception handling. Examples of irreversible instructions include store instructions and system call instructions.
However, with transactional memory support, the “irreversible instruction” is not irreversible if it is within a transaction. That is because a transaction may be aborted. With a transaction abort, as described above with reference to
Referencing
Referencing
If the execution of store instruction 458 causes an exception, transaction 430 may be aborted and the exception precisely handled as described above with reference to
Referencing
Referencing
Transactional memory may support other types of aggressive optimizations, for example, dead code removal. Referencing
However, removing dead code instruction 660 would change the processor state prior to floating point division instruction 662, an instruction that may cause an exception. Removing instruction 660 may therefore prevent an exception handler from recovering the processor state associated with unoptimized code 640 immediately prior to instruction 662. Therefore, without the support of transactional memory, a dynamic optimizer may forego removal of the dead code instruction 660.
Referencing
Further referencing
Aborts occur when the write set of a transaction executing in one thread, process, or core conflicts with the read/write set of another transaction that is executing on another thread, process or core. Two complementary techniques are described for using transactions to support parallel programming while minimizing aborts.
First, the dynamic optimizer uses common idiom to identify shared (including volatile) variable accesses. For example if a variable appears in a cmpxchg instruction, it is often an access to a shared variable. Also, if a variable controls a spin loop, the variable is usually a shared variable. The list of shared variable may be identified conservatively and maintained as a set of memory ranges organized as a hash table. This data structure is then used to check for shared variables in code regions being considered for conversion to a transaction. Code regions with a shared variable are then not allowed to be converted to transactions. This technique is further discussed below with reference to
Second, if a transaction aborts due to a conflict being detected, a region coded as a transaction is retranslated without using a transaction to avoid an abort occurring in that code region in the future. The handling of aborts and the retranslating of code regions have already been described above with reference to
Referencing
If the code subregion does have a loop, then the code subregion is not selected as a possible transaction. (Process block 773). If the code subregion does not have a loop, then it is further considered as a possible transaction. A code region may have a loop, but a subregion may include only the code within the loop that is executed each time when the loop iterates. The entire code region is not selected as a possible transaction, but the code subregion is selected as a possible transaction.
Continuing with method 770, it is determined whether the code subregion has a shared variable. (Process block 774). As discussed above, if a variable appears in a cmpxchg instruction or if the variable controls a spin loop, then the variable is likely a shared variable.
In one embodiment, the determination is made conservatively. Sections of code located between accesses to possible shared variables are considered as having a shared variable. These sections of code are identified as a set of memory ranges organized as a hash table.
Referencing
For example, the high order bits of the memory address of the variable “x” are used as a key 886 into the hash table 882. If the pointer 888 associated with the key 886 for the variable x is null, then the variable x is determined not to be a shared variable. If the pointer 888 is not null, then the linked list 890 is traversed to determine if x's memory address matches a memory address range stored in a node 892. If there is no match, then the variable x is determined not to be a shared variable. If there is a match, then the variable x is identified as a shared variable. This is just one of many possible implementations and is given merely as an example.
Further referencing
Code regions without loops or shared variables are evaluated to determine whether to apply a group of aggressive optimizations to the code region. (Process block 778). As used here, aggressive optimization refers to optimizations that depend on transactional memory for precise exception handling, as discussed above with reference to
Referencing
Also stored in memory 915 is a transaction 911, which includes optimized binary code 929 that is an optimized version of the original unoptimized binary code 909. The optimized binary code 929 of transaction 911 does not include any loops. Further, the optimized binary code 929 of transaction 911 does not include any shared variables.
Memory 915 includes an execution module 919 to instruct the processor 901 to execute the transaction 911. Consistent with the methods described with reference to
Memory 915 also includes an optimizer module 917 that applies optimizations to binary code. These optimizations are reflected in the optimized binary code 929 of the transaction 911, which includes aggressive optimization supported by transactional memory for precise exception handling. Original unoptimized binary code 909 includes an ordered sequence 923 of instructions, including a first instruction 921A and a second instruction 921B. The ordered sequence corresponds to the flow of control—that is, the order in which the instructions are to be executed by the processor 901. In the ordered sequence 923 of the original unoptimized binary code 909, the first instruction 921A precedes the second instruction 921B.
The transaction 911 also has an ordered sequence 925 of instructions, including the first instruction 921A and the second instruction 921B. However, in contrast to their order in the ordered sequence 923 of the original unoptimized binary code 909, in the ordered sequence 925 of the transaction 911, the first instruction 921A succeeds the second instruction 921B. This change in sequence reflects a code reordering pursuant to an aggressive optimization having transactional memory support for precise exception handling.
In one embodiment the first instruction 921A is a load instruction and the second instruction 921B is a store instruction. Thus, an exception instruction has been moved downward in the flow of instructions past an irreversible instruction—one that would irretrievably change the processor state of processor 903 absent the transaction 911 for precise exception handling.
In another embodiment, both the first instruction 921A and the second instruction 921B are load instructions. In yet another embodiment, the first instruction 921A is a load instruction and the second instruction is a floating point instruction. Various other possibilities exist, as discussed relative to
In one embodiment, dead code instruction 927 is present in the original unoptimized binary code 909, but is not present in the optimized binary code 929 of transaction 911. This reflects an aggressive optimization supported by transaction 911.
In one embodiment, the optimized binary code 929 of transaction 911 is a translation of the original binary code 909. The original source binary code 909 comprises an ISA that is not native to machine 901. The optimized binary code 929 of transaction comprises an ISA that is native to machine 901. The optimization of the original unoptimized binary code 909 having been applied in connection with the translation to the ISA of the optimized binary code of the transaction 911. In a particular embodiment, the ISA that is not native to machine 901 is 32-bit IA and the ISA that is native to the machine is 64-bit IA.
In one embodiment, instead of original unoptimized binary code 909, memory 915 stores original optimized binary code that is less aggressively optimized than the optimized binary code of transaction 911. In a particular embodiment, the original less aggressively optimized code has had optimizations applied that allow precise exception handling without the support of transactional memory. In contrast, optimized binary code of transaction 911 has had optimizations applied that may not allow precise exception handling without the support of transactional memory.
Any reference in this specification to “one embodiment,” “an embodiment,” “example embodiment,” etc., means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with any embodiment, it is submitted that it is within the purview of one skilled in the art to affect such feature, structure, or characteristic in connection with other ones of the embodiments. Furthermore, for ease of understanding, certain method procedures may have been delineated as separate procedures; however, these separately delineated procedures should not be construed as necessarily order dependent in their performance. That is, some procedures may be able to be performed in an alternative ordering or simultaneously, as one or ordinary skill would appreciate based at least on the teachings provided herein.
Embodiments of the present invention may be described in sufficient detail to enable those skilled in the art to practice the invention. Other embodiments may be utilized, and structural, logical, and intellectual changes may be made without departing from the scope of the present invention. Moreover, it is to be understood that various embodiments of the invention, although different, are not necessarily mutually exclusive. For example, a particular feature, structure, or characteristic described in one embodiment may be included within other embodiments. Accordingly, the detailed description is not to be taken in a limiting sense.
The foregoing embodiments and advantages are merely exemplary and are not to be construed as limiting the present invention. Those skilled in the art can appreciate from the foregoing description that the techniques of the embodiments of the invention can be implemented in a variety of forms. Therefore, while the embodiments of this invention have been described in connection with particular examples thereof, the true scope of the embodiments of the invention should not be so limited since other modifications will become apparent to the skilled practitioner upon a study of the drawings, specification, and following claims.
