Patent Grant
9965277
The present invention is generally related to digital computer systems, more particularly, to a system and method for selecting instructions comprising an instruction sequence.
Processors are required to handle multiple tasks that are either dependent or totally independent. The internal state of such processors usually consists of registers that might hold different values at each particular instant of program execution. At each instant of program execution, the internal state image is called the architecture state of the processor.
When code execution is switched to run another function (e.g., another thread, process or program), then the state of the machine/processor has to be saved so that the new function can utilize the internal registers to build its new state. Once the new function is terminated then its state can be discarded and the state of the previous context will be restored and execution resumes. Such a switch process is called a context switch and usually includes 10's or hundreds of cycles especially with modern architectures that employ large number of registers (e.g., 64, 128, 256) and/or out of order execution.
In thread-aware hardware architectures, it is normal for the hardware to support multiple context states for a limited number of hardware-supported threads. In this case, the hardware duplicates all architecture state elements for each supported thread. This eliminates the need for context switch when executing a new thread. However, this still has multiple draw backs, namely the area, power and complexity of duplicating all architecture state elements (i.e., registers) for each additional thread supported in hardware. In addition, if the number of software threads exceeds the number of explicitly supported hardware threads, then the context switch must still be performed.
This becomes common as parallelism is needed on a fine granularity basis requiring a large number of threads. The hardware thread-aware architectures with duplicate context-state hardware storage do not help non-threaded software code and only reduces the number of context switches for software that is threaded. However, those threads are usually constructed for coarse grain parallelism, and result in heavy software overhead for initiating and synchronizing, leaving fine grain parallelism, such as function calls and loops parallel execution, without efficient threading initiations/auto generation. Such described overheads are accompanied with the difficulty of auto parallelization of such codes using state of the art compiler or user parallelization techniques for non-explicitly/easily parallelized/threaded software codes.
In one embodiment the present invention is implemented as an out of order processor. The processor includes a virtual load store queue for allocating a plurality of loads and a plurality of stores, wherein more loads and more stores can be accommodated beyond an actual physical size of the load store queue of the processor; wherein the processor allocates other instructions besides loads and stores beyond the actual physical size limitation of the load/store queue; and wherein the other instructions can be dispatched and executed even though intervening loads or stores do not have spaces in the load store queue.
The foregoing is a summary and thus contains, by necessity, simplifications, generalizations and omissions of detail; consequently, those skilled in the art will appreciate that the summary is illustrative only and is not intended to be in any way limiting. Other aspects, inventive features, and advantages of the present invention, as defined solely by the claims, will become apparent in the non-limiting detailed description set forth below.
The present invention is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements.
Although the present invention has been described in connection with one embodiment, the invention is not intended to be limited to the specific forms set forth herein. On the contrary, it is intended to cover such alternatives, modifications, and equivalents as can be reasonably included within the scope of the invention as defined by the appended claims.
In the following detailed description, numerous specific details such as specific method orders, structures, elements, and connections have been set forth. It is to be understood however that these and other specific details need not be utilized to practice embodiments of the present invention. In other circumstances, well-known structures, elements, or connections have been omitted, or have not been described in particular detail in order to avoid unnecessarily obscuring this description.
References within the specification to “one embodiment” or “an embodiment” are intended to indicate that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. The appearance of the phrase “in one embodiment” in various places within the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Moreover, various features are described which may be exhibited by some embodiments and not by others. Similarly, various requirements are described which may be requirements for some embodiments but not other embodiments.
Some portions of the detailed descriptions, which follow, are presented in terms of procedures, steps, logic blocks, processing, and other symbolic representations of operations on data bits within a computer memory. These descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. A procedure, computer executed step, logic block, process, etc., is here, and generally, conceived to be a self-consistent sequence of steps or instructions leading to a desired result. The steps are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals of a computer readable storage medium and are capable of being stored, transferred, combined, compared, and otherwise manipulated in a computer system. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.
It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise as apparent from the following discussions, it is appreciated that throughout the present invention, discussions utilizing terms such as “processing” or “accessing” or “writing” or “storing” or “replicating” or the like, refer to the action and processes of a computer system, or similar electronic computing device that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories and other computer readable media into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.
Embodiments of the present invention implement an out of order instruction scheduling process, where instructions within an input instruction sequence are allowed to issue, out of order, as soon as processor resources are available to execute them. Embodiments of the present invention are able to ensure that external agents see instructions execute in order (e.g., memory consistency rules/models). Ensuring instructions visibly execute in order to the external agents thereby ensures error-free program execution. Embodiments of the present invention ensure that the memory hierarchy (e.g., L1 cache, L2 cache, system memory, etc.) of the processor sees a consistent in order execution of the instructions.
Additionally, out of order execution of the loads and the stores leads to speculative execution. When performing speculative execution, the machine needs to recognize speculative errors. In the
The functionality is the same for the stores. Stores are also split into two macroinstructions. The first instruction is a store address and fetch, the second instruction is a store of the data at that address. The split of the stores and two instructions follows the same rules as described below for loads.
The split of the loads into two instructions allows a runtime optimizer to schedule the address calculation and fetch instruction much earlier within a given instruction sequence. This allows easier recovery from memory misses by prefetching the data into a temporary buffer that is separate from the cache hierarchy. The temporary buffer is used in order to guarantee availability of the pre-fetched data on a one to one correspondence between the LA/SA and the LD/SD. The corresponding load data instruction can reissue if there is an aliasing with a prior store that is in the window between the load address and the load data (e.g., if a forwarding case was detected from a previous store), or if there is any fault problem (e.g., page fault) with the address calculation. Additionally, the split of the loads into two instructions can also include duplicating information into the two instructions. Such information can be address information, source information, other additional identifiers, and the like. This duplication allows independent dispatch of LD/SD of the two instructions in absence of the LA/SA.
The load address and fetch instruction can retire from the actual machine retirement window without waiting on the load data to come back, thereby allowing the machine to make forward progress even in the case of a cache miss to that address (e.g., the load address referred to at the beginning of the paragraph). For example, upon a cache miss to that address (e.g., address X), the machine could possibly be stalled for hundreds of cycles waiting for the data to be fetched from the memory hierarchy. By retiring the load address and fetch instruction from the actual machine retirement window without waiting on the load data to come back, the machine can still make forward progress.
It should be noted that the splitting of instructions enables a key advantage of embodiments of the present invention to re-order the LA/SA instructions earlier and further away from LD/SD the instruction sequence to enable earlier dispatch and execution of the loads and the stores.
A more detailed description of the instruction splitting functionality in accordance with one embodiment of the present invention is now described. In one embodiment, the instruction set does not have direct analogue instructions to LA, SA, LD or SD. In such an embodiment, these concepts are realized with a combination of instruction prefixes, LAF, SAF, LASAF and a companion suffix instruction. And a set of instructions that roughly do map onto the LA has LAD and SA has SAD, and a combined LADSAD can be implemented. These concepts can also be implemented as microinstructions within microcode.
a) What is defined here as LAF-prefix+suffix instruction can be described as an ‘LD’.
b) What is defined here as SAF-prefix+suffix instruction can be described as an ‘SD’.
c) What is defined here as LAD instruction can be described as an ‘LA’.
d) What is defined here as SAD instruction can be described as an ‘SA’.
e) Further we have a LASAF instruction and LADSAD instruction. These instructions comprise a combined LAF/SAF-prefix+suffix instruction which could be used to implement semaphore (locked-atomic) operations. It is possible to also define a combined LAD-SAD instruction to again pre-fetch the memory operands, with resultant complexity in hardware.
The LAD instruction initiates a data-prefetch into the execution pipeline. It differs from a normal prefetch in that it loads directly into the execution pipeline affording lower execution latencies than first level caches. In one embodiment, this functionality is implemented by using a fixed storage for the LA-LD pair that can be tagged using the ID link between the LA-LD pair (e.g., the QID number. The LAD instruction calculates an effective memory addresses (e.g., from potentially a complex specification), specify operand size (byte, half word, word, double word, or larger); initiate the memory reference; through the TLB and cache hierarchy. Exceptions (page walk miss, privilege, protections,) are recorded to be reported at LAF+suffix execution OR an alternate implementation could cancel/invalidate the Q entry, forcing the LAF+suffix instruction to re-execute and it to take the exceptions.
The LAD instruction has the general format and operands:
LAF is an instruction prefix, meaning it must be directly coupled (or fused) with a suffix instruction. The suffix instruction can be stand alone. The suffix instruction can be any instruction that has at least one source register. The LAF as a prefix must be coupled. The LAF-prefix changes the nature of the suffix instruction. One or more of its register operands is redefined by the prefix as a memory queue identifier (QID). Further the data associated as being sourced from the register, now is sourced from the memory queue.
A LAF-prefix+suffix instruction may or not have an antecedent LAD instruction. If the QID==0, then the LAF is without an antecedent LAD. If the QID !=0, then the LAF has or had an antecedent LAD. When it is intended to create a split of the load instruction into LA and LD, then the LAF will have a QID!=0 and an LAD will be instantiated with the same QID preceding the LAF (e.g., this basically creates the split LA/LD pair).
When the LAF/suffix executes and has QID=0, the 0 entry of the memory queue is used to do an ‘LA’ operation, memory read, stage data into the memory queue, and then completed by loading the data into the suffix instruction sources and the operation applied combined with potential other sources and the result written to the suffix instructions destination register(s). On the other hand, if the QID!=0, then the memory queue is consulted (lookup) for a matching QID, if present the data is read from the memory queue and the operation of the suffix instruction is applied, and result written to the destinations register. If the matching QID is valid but not complete, the data is stalled until data is available. If the QID is not valid, then the LAF has sufficient enough information (address and data-operand-size) to restart the operation. A matching QID may not be present for a variety of reasons, some of which are:
a) The antecedent LAD never executed, bad coding, or other reason.
b) the execution flow between LAF and the antecedent LAD was broken by exception or interrupt
c) An intervening store-operations aliased with the LAD's address and invalidated its data integrity.
In any of these cases, the LAF prefix+suffix have sufficient information to repeat the LAD (LA) operation. This capability makes our LAD instruction into a hint. The LAD did not have to successfully execute or for that matter to be even be implemented beyond being a NOP for correct code to use it.
The general format and operands of a LAF instruction with a suffix instruction is:
LAF M[ea]
SUFFIX(os) Rt,QID, . . .
The LAF instruction borrows is operand size, QID, and from the encoding of the suffix instruction. If the suffix is a SIMD, it also borrows from the suffix the SIMD-width of the operation. The QID is always encoded in one of the source register specification fields of the suffix instruction. In SMI's particular implementation this is always bits 23:18, but this does not need to be the case.
SAD stands for ‘SA-defused’
SAD is the parallel instruction to a LAD only for stores. It too prefetches data bringing in data to caches for modification. Further it creates a memory-store-queue entry. SAD primary has 2 primary uses:
a) as a prefetch, read for modification of data
b) to keep correct memory ordering and expose and handle potential write-after-read hazards after promoting a load (read) before a store (write)
SAD is a hint instruction. The SAD instruction calculates an effective memory address (from potentially a complex specification), specifies operand size (byte, half word, word, double word, . . . ); initiates memory reference; through TLB, cache/memory hierarchy. Exceptions (page walk miss), privilege, protection) are recorded at SAF+suffix execution to re-execute and it to take the exceptions.
The SAD instruction has the general format and operands:
SAF stands for ‘SA-fused’
SAF is the parallel prefix to the LAF prefix, only for stores. As a prefix it must be directly coupled (or fused) with a suffix instruction. The suffix instruction can be stand alone. The suffix instruction can be any instruction that has at least one target register. The SAF as a prefix must be coupled. The SAF changes the nature of the suffix instruction: one or more of the destination register operands which is normally register-selection index into a memory store queue identifier (QID), and the operation from targeting a register to targeting a memory (more precisely a memory queue entry). As such it changes a register operation into a store memory operation.
The SAF instruction may or may not have an antecedent SAD. If the QID==0, then the SAF is without an antecedent SAD. If the QID !=0, then the SAF has or had an antecedent SAD. When the SAF/suffix executes if the QID==0, the 0 entry of the memory queue is used to do an ‘SA’ operation, memory write, stage data into the memory queue, and then completed by storing the data supplied by the suffix instruction source. On the other hand, if the QID!=0, then the memory queue is consulted (lookup) for a matching QID, if present the data will be written into the memory queue entry when the operation of the suffix instruction is applied. If the matching QID, is valid but not complete, the data is stalled until data is available. If the QID is not valid, then the SAF has sufficient enough information (address and data-operand-size) to restart the operation and complete the memory write operation. A matching QID may not be present for a variety of reasons, some of which are:
a) the antecedent SAD never executed, bad coding, or other reason.
b) the execution flow between SAF and the antecedent SAD was broken by exception or interrupt
In any of these cases, the SAF prefix+suffix have sufficient information to repeat the SAD (SA) operation. This capability makes our SAD instruction into a hint. The SAD did not have to successfully execute or for that matter to be even be implemented beyond being a NOP for correct code to use it.
LASAF is an instruction prefix.
LASAF as a prefix modifies an instruction that has a same register as a source and a destination. LASAF changes such an instruction into an atomic memory reference read/write once operation. One from the load-memory queue and one from the store memory queue are used. There is no antecedent LAD or SAD instruction.
LASAF M[ea3]
ADD QID1,QID2,R1
LASAF creates QID entries in both the load and store memory queue. And would them read memory[ea3] using QID2, add R1 and store the result in store memory QID1, effectuating an atomic read-modify write of M[ea3].
The SMI implementation (if we were to implement this) would require QID1==QID2==0. But we don't want to restrict ourselves to that implementation.
Could we have a LASAD instruction, I think so, but we'd have to snoop all the way into the memory queue's to do it. Invalidate both entries on a snoop on a hit. And then re-execute the load/store on the LASAF.
Example Usage:
A. Saving Register resource after promoting a load far in advance of a use of the data.
Assume the original code is.
One downside of doing this is it keeps the R1 register ‘busy’ waiting for data, and it can not be used for other purposes. The memory queue expands the pool of resources to hold data. So we covert the in LDR into a LAD and a subsequent LAD:
LAD QID,M[ea1]
. . . many instructions
LAF M[ea1]
ADD32 Rt,QID,R2
Since a load-queue entry QID is used R1, is freed to be used for other purposes.
Or load Rt with the difference of Rt-QID, or if QID not present then reload data from M[ea1] subtract R2 from it, and place result in Rt.
It should be noted that with the above described implementation it is not necessary for the memory address calculation to match between the 2 loads. If the two address calculations differ, and the QID is still valid there is most likely a bad programmer. The OS (in our example 32), also does not have to match between the LAD and LAF. More data than necessary may be read by the LAD, in which case the least-significant bytes of the data read will be used. Or more data may be required by the LAF+suffix than the LAD read, in which case the least-significant bytes read by the LAD will be used, followed by 0 until the suffix operation is sufficed. Further the address calculation operands do not have to match between the LAD and LAF, although for good coding they should get the same resultant effective address.
B. Saving execution cycles, by morphing a normal ALU register-register operation into an ALU register memory operation.
Here we are using the special QID=0 (%0) just to change the normal register-register ADD instruction into a memory reference. Since LAF is a prefix and must be directly coupled to the ADD, there is no instruction in between. The QID=0 is always available for usage, as it is immediately used.
Another issue is that we want to promote the load (LDR) above a store (STR) which may or may not alias with the address of the load. Alias: some or all of the data address by ea1 is the same as ea2.
To safely do this safely (generate correct code), we need the tools to do it. Here we use both LAD and SAD instructions and their respective LAF and SAF prefix+suffix instructions, and to be able to preserve execution order and generate correct code.
LOOP:
In the above code both the LAD and the SAD have to be promoted and keep the same ordering. What can happen? At each point a, b, c, d an alternate is indicated.
a1) interrupt, invalidates the SAD, the subsequent SAF will have to re-execute
a2) LAD aliases with SAD, invalidates the LAD or rather won't be inserted into the memory queue
b1) interrupt, invalidated the SAD and LAD
b2) SAF aliases with the LAD, and invalidates the LAD
b3) SAF either uses the still valid SAD, or re-executes.
c1) interrupt, invalidates the LAD,
c2) if still valid LAF uses LAD's data, otherwise re-executes.
c3) loops, do to the magic of hardware, a combination of tagging with IP and execution sequence ID, and the QID, LAD/SAD/LAF/SAF are properly managed.
In the above described descriptions of LA/SA and LD/SD, the LA and SA relative program order positions are used to enforce order for forwarding purposes. In another embodiment, the LD/SD relative program order positions can be used to enforce order for forwarding purposes (e.g., as described below).
As illustrated by step 401, an objective of embodiment of the present invention as to find stores that forward to a load upon an address match between that store and that load. In step 402, the closest earlier store (e.g., in machine order) forwards to the load.
In step 403, the actual ages are updated for LA/SA when LD/SD is allocated in machine order. The LA/SA actual ages are assigned the same value as the LD/SD ages. The LD/SD maintains the actual ages and enforces the original program order semantics.
Steps 404-407 show the rules for maintaining program sequential semantics while supporting speculative execution. The steps 404-407 are shown as being arranged horizontally with each other to indicate that the mechanisms that implement these rules function simultaneously. As shown in step 404, if a store has an actual age but the load has not yet obtained an actual age, then the store is earlier than the load. As shown in step 405, if a load has an actual age but the store has not yet obtained an actual age, then the load is earlier than the store. As shown in step 406, if either the load or the store has obtained an actual age, then a virtual identifier (VID) will be used to find out which is earlier (e.g., in some embodiments the QID that is associated with the load/store instructions represents the VID). As shown in step 407, if both a load and a store have obtained actual ages, then the actual age is used to find out which is the earlier.
It should be noted that algorithm described by the
It should be noted that the VID table functions by keeping track of the associations between the LA/SA and LD/SD instructions by storing the LA/SA corresponding machine ID and machine resources that correspond to each VID unique identifier. It should also be noted that the term “VID” is synonymous with the term “QID” as described in the discussion of
An example of operation of the
The allocation pointer now moves to 6. The store actual age of V2 SA (#5) now updates the V3 LA load Q entry (because V2 SA is the store of record that has stamped to forward to this load). V4 SA now dispatches and compares with the load initial age, and because V4 is larger than V3 LA, it does not forward. Allocation pointer now moves to 11. At the time of allocation of V3 LD, it updates the load Q entry #4 with the actual age of V3 LD (#7). V1 SA #11 is now dispatched. Since V3 LA #1 now has an actual age but not V1 SA #11, then the load is earlier than the store, and thus no forwarding is possible.
The prediction table is for detecting cases where the default assumption has been incorrect. The default assumption is that no store forwards to a load. Once forwarding is detected for a load store pair the program counter of the load store pair is recorded so that the load will always wait for that store address to be dispatched and address calculated to find out if that load address matches that store address and thus needs to forward from it.
It should be noted that in one embodiment, the feature described herein, wherein the LD/SD is allowed to dispatch in absence of the LA/SA, facilitates reordering of LA/SA ahead of a branch or within a branch scope in a given sequence of instructions. If the LA and SA were skipped over as a result of a branch, or they were ignored as a result of having caused a memory exception, the LD and SD can still function correctly because they include the necessary information to dispatch twice: first as an LA/SA, and second as an LD/SD. In such case, the first dispatch of the LD/SD is performing the address calculation (e.g., load address). Subsequently, the same LD/SD can dispatch again to fulfill the consuming part of the load or store (e.g., load data). This mechanism can be referred to as a “dual dispatch” of the load and store instructions.
It should be noted that, in one embodiment, the dual dispatch of the LD/SD happens when the corresponding defused LA/SA is non-existent (e.g., as is the case with a fused LD/SD), or if the LA/SA was skipped over as a result of a branch, or they were ignored as a result of having caused a memory exception, or the like.
The above described dual dispatch functionality ensures LD/SD executes correctly independent of the lost, ignored or skipped LA/SA. The benefit provided by the above described feature is that prefetching of the data specified by the load/store can start earlier in the program order (e.g., reducing latency) by scheduling the LA/SA earlier, even in the presence of branches, potential faults, exceptions, or the like.
Both V1 SA of machine ID 2 and V2 SA of machine ID 3 are now dispatched. They compare with V1 LA of ID 10 and because V1 LA of ID 10 has no machine age (its corresponding LD has not been allocated yet), while both V1 SA of machine ID 2 and V2 SA of machine ID 3 have actual age, then it is known that both V1 and V2 stores are earlier/older than V1. Then the latest of these two stores (V2) can forward to V1 of ID 10. SA (V2) #11 is now dispatched. Since V1 LA and V2 SA do not have an actual age, their VID's are used for comparison, and no forwarding is detected. The allocation pointer now moves to 16. V4 SA of ID 16 is now dispatched and it compares with V1 LA of ID 10 and since the V1 LA has an actual age but the V4 SA does not, then the V4 SA is later than the V1 LA. Thus no forwarding from this store to this earlier load is possible.
The allocation pointer now moves to 11. V3 LA gets the actual age of 7. V1 SA #10 V2 SA #11 are dispatched. V3 LA is dispatched and it compares its address with the store Q entries and finds a match across V1 SA, V2 SA and V4 SA and V2 SA #11. Since V3 LA has its actual age of 7, it compares its actual age with the closest store age to it, which is age 5, belonging to V2 SA, and thus that load will forward from this store and be marked such in the load Q.
Process 800 begins in step 801, where a store instruction is split into an SA and SD. As described earlier, the SA instruction maintains semantics with the SD instruction to allow dual dispatch in the event that there is no match in the VID table between the split SA and the just allocated SD. In step 802, SA is reordered to an earlier machine visible program order and that SA is tracked using a VID table to retain the original SD program order. In step 803, upon dispatch of the SA, a check is made against all loads in the load queue for address match against the SA.
In step 804, upon an address match, the program order of the matching loads is compared against the program order of the SA by using the VID numbers of the loads and the SA, or using the actual ages of the loads and the stores. This is the process that was diagrammed earlier in the discussion of the
In step 805, for loads that are later in the program order, the store will check to see if the loads have been forwarded to by other stores. In step 806, if so, the store checks a stamp of the store that previously forwarded to this load to see if that store was earlier in program order than itself. In step 807, if so, the store checks a stamp of the store that previously forwarded to this load to see if that store was earlier in program order than itself. In step 808, if not, the store does not forward to this load.
In step 901, a load instruction is split into an LA and LD in the manner described above. In step 902, the LA is reordered to an earlier machine visible program order and is tracked using the VID table as described above. Instead 903, the LA is checked against all stores in the store queue for address match against the load.
In 904, upon an address match, compare the program order of the matching load against the program order of the store by using the VID numbers of the load and the store, or using the actual ages of the load and the store. This is the process that was diagrammed earlier in the discussion of the
As loads retire out of the load queue, the load dispatch window moves to subsequent instructions in the sequence and will include more allocated loads to be considered for dispatch equivalent to the number of loads that have retired from the load queue. In this diagram, the load dispatch window will move from left to right.
In one embodiment, the load dispatch window will always include the number of loads that equal the number of entries in the load queue. No loads at any time can be dispatched outside the load dispatch window. Other instructions in the scheduler window besides loads (e.g., Sub, Add etc.) can dispatch. All loads within the load dispatch window can dispatch whenever they are ready.
The
It should be noted that in distributed load/store queues, there is a problem with respect to allocated load/stores in that their address is unknown at allocation time. Because of this, it is only known at out of order dispatch time which of the distributed queues a given load or store will occupy.
Embodiments of the present invention can employ three different solutions for the distributed load/store queue to avoid deadlocks with out of order dispatches:
1. Cause a miss-prediction and flush at the earliest load/store that deadlocked (have no space to dispatch to the load/store buffer) and start dispatching load/stores either in order for a period of time, or by conservative allocation where each load/store allocates space in all distributed queues. Once the address of that load/store is known (at dispatch time) thus the particular load queue which will receive that load/store is known, it can de-allocate the reserved spaces in the other queues.
2. An extension of the load/store queue is created and is used to allocate deadlocked loads/stores to that extension queue in program order from the point of the load/store that caused the deadlock (
3. Dynamic dispatch window sizing, where the sum of the un-dispatched loads outside the continuity window should be less than or equal to the number of free unreserved spaces in the that particular load queue (e.g.,
Booking ratio of the reserve is 3. The booking ratio is the number of in order loads that compete for each of the reserved spaces. In this example, only the first two in order un-dispatched loads (scanning the in-order continuity window from the left to right) can dispatch to the reserve portion (assuming 2 entries of the queue were assigned to reserve). Hence, the number of virtual entries=(Booking ratio−1)*number of reserve entries=(3−1)*2=4.
With respect to the in order continuity window sizing, the number of loads at any time (counting from the oldest to the youngest) that have not dispatched to an entry (captured space) in the load queues plus the number of dispatched loads to the reserve space must be less than or equal to (the booking ratio*the number of reserve entries). In this case, the number of loads must be less than or equal to 3. The booking ratio is a design configurable performance metric that determines what is the accepted (occupancy VS booking) ratio of the reserved space. This is exercised in case the earliest un-dispatched loads cannot find a queue space to dispatch to outside the reserved entries. In such case, those loads starting from the earliest (oldest) load will compete for the reserved space, the booking ratio determines how many loads will wait to occupy each reserved entry, the reserved entries are always assigned first to the oldest un-dispatched load and once that load retires the next oldest load can occupy the entry (the booking ratio determines the number of those loads that occupy the reserved entries one after the other starting from the oldest dispatched).
It should be noted that in one embodiment, loads from the in order continuity window of each queue can dispatch to the reserved space of that queue when there is no space left in the unreserved portion of that queue (starting from the oldest load in order). It should be also noted that in one embodiment, loads outside the in order continuity window of either queue and within the dynamic dispatch window of that queue cannot dispatch to the reserved portion of that queue.
It should be noted also that as long as there is space in the unreserved portion of the queue, any load within the whole dynamic dispatch window of that queue can dispatch out of order to any entry of the unreserved portion of any of the distributed queues. The sizes of both the in order continuity window and the dynamic dispatch window of either queue is adjusted each cycle to reflect their size limitations stated in the equations provided above after each load dispatch or retirement.
This mechanism also serves also as a coherency scheme for the memory architecture among those engines/cores/processors. This scheme starts by an address request from one of the address calculation units in one fragment/core/processor. For example, assume the address is requested by fragment 1 (e.g., 1211). It can obtain and calculate its address using address registers that belong to its own fragment and or from registers across other fragments using the address interconnect bus 1200. After calculating the address it creates the reference address of either 32-bit address or 64-bit address that is used to access caches and memory. This address is usually fragmented into a tag field and a set and line fields. This particular fragment/engine/core will store the address into its load store buffer and/or L1 and/or L2 address arrays 1202, at the same time it will create a compressed version of the tag (with smaller number of bits than the original tag field of the address) by using a compression technique.
Moreover, the different fragments/engines/cores/processors will use the set field or a subset of the set field as an index to identify which fragment/core/processor the address is maintained in. This indexing of the fragments by the address set field bits ensures exclusiveness of ownership of the address in a particular fragment/core/engine even though the memory data that corresponds to that address can live in another or multiple other fragments/engines/cores/processors. Even though the address CAM/tag arrays 1202/1206 are shown in each fragment to be coupled with the data arrays 1207, they might be only coupled in physical proximity of placement and layout or even by the fact that both belongs to a particular engine/core/processor, but there is no relation between addresses kept in the address arrays and the data in the data arrays inside one fragment.
Embodiments of the present invention implement a distributed load store ordering system. The system is distributed across multiple fragments. Within a fragment, local data dependency checking is performed by that fragment. This is because the fragment only loads and stores within the store retirement buffer of that particular fragment. This limits the need of having to look to other fragments to maintain data coherency. In this manner, data dependencies within a fragment are locally enforced.
With respect to data consistency, the store dispatch gate enforces store retirement in accordance with strict in-program order memory consistency rules. Stores arrive out of order at the load store buffers. Loads arrive out of order also at the load store buffers. Concurrently, the out of order loads and stores are forwarded to the store retirement buffers for processing. It should be noted that although stores are retired in order within a given fragment, as they go to the store dispatch gate they can be out of order from the multiple fragments. The store dispatch gate enforces a policy that ensures that even though stores may reside across store retirement buffers out of order, and even though the buffers may forward stores to the store dispatch gate out of order with respect to other buffers' stores, the dispatch gate ensures that they are forwarded to fragment memory strictly in order. This is because the store dispatch gate has a global view of stores retiring, and only allows stores to leave to the global visible side of the memory in order across all the fragments, e.g., globally. In this manner, the store dispatch gate functions as a global observer to ensure that stores ultimately return to memory in order, across all fragments.
Stores snoop the caches for address matches to maintain coherency. If thread/core X load reads from a cache line, it marks the portion of the cache line from which it loaded data. Upon another thread/core Y store snooping the caches, if any such store overlaps that cache line portion, a miss-predict is caused for that load of thread/core X.
One solution for filtering these snoops is to track the load queue entries' references. In this case stores do not need to snoop the load queue. If the store has a match with the access mask, that load queue entry as obtained from the reference tracker will cause that load entry to miss predict.
In another solution (where there is no reference tracker), if the store has a match with the access mask, that store address will snoop the load queue entries and will cause the matched load entry to miss predict.
With both solutions, once a load is reading from a cache line, it sets the respective access mask bit. When that load retires, it resets that bit.
In the
In this manner, the access mask ensures the memory consistency rules are correctly implemented. Memory consistency rules dictates that stores update memory in order and loads read from memory in order for this semaphore to work across the two cores/threads. Thus, the code executed by core 1 and core 2, where they both access the memory locations “flag” and “data”, will be executed correctly.
As described above, memory consistency rules dictate that stores update memory in order and loads reefer memory in order in order that the two cores/threads communicate properly. In the bottom right-hand side of
With an out of order architecture where loads and stores execute out of order, a lock based memory consistency model can be used to ensure the two entities (e.g., core 1 and core 2) maintain in order memory consistency semantics. This is shown through the use of an access mask, a thread ID register, and the tracker register. The lock is set by setting the corresponding access mask bit of any load within the critical section of the code. If any access from another thread/core to that cache line word happens, the lock will prevent that access. In one embodiment, this can be implemented by treating the access as a miss. When the lock is cleared, accesses to that word are allowed.
Alternatively, a transactional-based method can be used to maintain in order memory consistency semantics. With the transactional-based method, atomicity is set by setting the corresponding access mask bit of any load within a transaction. If any access from another thread/core or parallel transaction to that cache line word happens while the mask bit is set it will signal the load queue entry corresponding to that load (e.g., via the tracker) to be miss-predicted/flushed or retried with its dependent instructions. The access mask also tracks thread/core. The mask bit will be cleared when that transaction is concluded. The thread ID register is used to track which thread is accessing which word of a unified store queue entry.
It should be noted that if this multi-core segmented subsystem is a part of a larger cluster where there are external processors/cores/clusters with shared memory subsystems. In this case, the load's belonging to the other external processors/cores/clusters would proceed and would not be prevented from loading from any memory location not paying attention if that memory location is part of a transactional access. However, all loads will mark the access mask to notify future stores that are part of a transaction.
Snooping stores coming from other processors compare their addresses to the mask. If a store sees the address it is trying to store to is marked in the access mask from another thread load (a load that is part of a transaction), then the store will cause that load to be miss predicted. Otherwise, the mark will be cleared upon that load retiring (e.g., thereby completing the transaction).
As described above, memory consistency rules dictates that stores update memory in order and loads reads from memory in order so that the cores/threads communicate properly. In the bottom right-hand side of
In the
A store is senior when all loads and stores before it in the same thread have been executed. A thread that receives a forward from another thread cannot retire loads/stores independently. Threads have to miss predict conditionally in case other threads from which they receive forwarding have miss predicted. A particular load can forward from the same thread forwarding store or a from a different thread senior store if there is no store forwarding to it within the same thread.
With the
Additionally, it should be noted that a thread cannot retire loads/stores independently. The thread has to load miss predict when another thread from which it received a forwarding store miss predicts or flushes.
The store retirement/reorder buffer (SRB) can operate in two implementations, a retirement implementation and a reorder implementation.
In a retirement implementation, stores are loaded into the SRB from the store queue in original program order at retirement of stores, such that stores that are earlier in original program order are at the top of the SRB. A subsequent load can then look for address matches (e.g., using address CAM), and forward from the matching entry in the SRB/store cache. In cases where there are two or more address matches, the priority encoder can locate the correct forwarding entry by scanning for the first one. This saves a trip to memory and allows the machine to make forward progress. If a load is dispatched and the store that forwards to it has already retired to the SRB/store cache, that load forwards from the SRB/store cache and records the pairing relationship in the prediction table. To detect the case where a load is dispatched before the store that forwards to it is retired to the SRB/store cache, the load has to create an address mask where it marks its own address. This can be implemented in different ways (e.g., the
As discussed above,
In
As loads are dispatched, they check the prediction table to see if they are paired with a corresponding store. If the load is paired and that particular store has already dispatched, the load will forward from that store queue entry number as recorded in the prediction table. If the store has not been dispatched yet, then the load will register its load queue entry number in the prediction table and will mark itself in the load queue to wait for the store data to be forwarded. When the store is dispatched later, it checks the prediction table to obtain the load queue entry number and forward to that load.
Once forwarding is detected for a load store pair, the PC and the addresses of the load store pair are recorded so that the address match is verified. If the address matches, the load will not dispatch until the store data is dispatched and the load will be marked to forward from it. The prediction threshold is used to set a confidence level in the forwarding relationship between load store pairs.
The store retirement/reorder buffer (SRB) can operate in two implementations, a retirement implementation and a reorder implementation.
In the
1. The load does not dispatch, it waits until all earlier stores have dispatched before it dispatches itself.
2. The load dispatches and marks its address in the access mask of the cache (as shown in
It should be noted that priority encoder functions as described above to locate the correct forwarding entry.
It should be noted that the load will be able to check the SRB for a matching address against a previous store because all the stores in SRB will not be committed to external cache/store cache architecturally visible state (leave the SRB storage to visible memory) till all the instructions in the trace including the mentioned load had reached the trace commit state (e.g., all become non speculative and trace as a whole is ready to commit).
The store retirement/reorder buffer functionally enables speculative execution. The results of speculative execution can be saved in the store retirement/reorder buffer until speculative outcomes are known. The speculative results are not visible architecturally. Once speculative state is committed, stores can be committed to the store cache. Before committing the state, any exceptions or loads and stores that need to be retried will signal an exception or a miss-predicts that will prevent the state commit. Forwarding miss-predictions between stores and corresponding loads can be fixed (e.g., by causing a miss-prediction that flushes the machine at the miss=forwarding load point, or the like).
Additional descriptions of the SRB functionality can be found in U.S. patent application Ser. No. 13/360,024, filed Jan. 27, 2012, “HARDWARE ACCELERATION COMPONENTS FOR TRANSLATING GUEST INSTRUCTIONS TO NATIVE INSTRUCTIONS”, by Mohammad Abdallah.
For purposes of explanation, the foregoing description refers to specific embodiments that are not intended to be exhaustive or to limit the current invention. Many modifications and variations are possible consistent with the above teachings. Embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, so as to enable others skilled in the art to best utilize the invention and its various embodiments with various modifications as may be suited to their particular uses.
This application is a continuation of, and claims benefit to, International Application Number PCT/US2013/045734, titled “A VIRTUAL LOAD STORE QUEUE HAVING A DYNAMIC DISPATCH WINDOW WITH A UNIFIED STRUCTURE” by Mohammad A. Abdallah, filed Jun. 13, 2013, which in turn claims the benefit commonly assigned U.S. Provisional Patent Application Ser. No. 61/660,544, titled “A VIRTUAL LOAD STORE QUEUE HAVING A DYNAMIC DISPATCH WINDOW WITH A UNIFIED STRUCTURE” by Mohammad A. Abdallah, filed on Jun. 15, 2012, and which is incorporated herein in its entirety.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4342080 | Grondalski | Jul 1982 | A |
| 5319766 | Thaller et al. | Jun 1994 | A |
| 5339443 | Lockwood | Aug 1994 | A |
| 5367656 | Ryan | Nov 1994 | A |
| 5487156 | Popescu et al. | Jan 1996 | A |
| 5553265 | Abato et al. | Sep 1996 | A |
| 5553266 | Metzger et al. | Sep 1996 | A |
| 5625837 | Popescu et al. | Apr 1997 | A |
| 5748987 | Craig | May 1998 | A |
| 5751983 | Abramson et al. | May 1998 | A |
| 5778245 | Papworth et al. | Jul 1998 | A |
| 5788245 | Hada | Aug 1998 | A |
| 5943686 | Arimilli et al. | Aug 1999 | A |
| 6006324 | Tran et al. | Dec 1999 | A |
| 6006326 | Panwar et al. | Dec 1999 | A |
| 6058472 | Panwar et al. | May 2000 | A |
| 6185660 | Mulla et al. | Feb 2001 | B1 |
| 6298435 | Chan | Oct 2001 | B1 |
| 6349381 | Tremblay | Feb 2002 | B1 |
| 6470444 | Sheaffer | Oct 2002 | B1 |
| 6487639 | Lipasti | Nov 2002 | B1 |
| 6591342 | Akkary et al. | Jul 2003 | B1 |
| 6631460 | Morris et al. | Oct 2003 | B1 |
| 6694421 | Yoaz et al. | Feb 2004 | B2 |
| 6738896 | Webb, Jr. | May 2004 | B1 |
| 6782461 | Lam | Aug 2004 | B2 |
| 6883086 | Dundas | Apr 2005 | B2 |
| 6920546 | Gochman et al. | Jul 2005 | B2 |
| 7174428 | Hily et al. | Feb 2007 | B2 |
| 7340567 | Chou et al. | Mar 2008 | B1 |
| 7343477 | Thatipelli et al. | Mar 2008 | B1 |
| 7380071 | Blumrich et al. | May 2008 | B2 |
| 7392369 | Baxter et al. | Jun 2008 | B2 |
| 7418552 | Akkary et al. | Aug 2008 | B2 |
| 7502895 | Jones et al. | Mar 2009 | B2 |
| 7600098 | Chou | Oct 2009 | B1 |
| 7606998 | Asher et al. | Oct 2009 | B2 |
| 7703098 | Moir et al. | Apr 2010 | B1 |
| 7890725 | Praun et al. | Feb 2011 | B2 |
| 8006073 | Ali | Aug 2011 | B1 |
| 8041928 | Burky et al. | Oct 2011 | B2 |
| 8135917 | Blumrich et al. | Mar 2012 | B2 |
| 8185700 | Madriles et al. | May 2012 | B2 |
| 8301844 | Steely, Jr. et al. | Oct 2012 | B2 |
| 8321635 | Ish | Nov 2012 | B2 |
| 9043363 | Dragojevic et al. | May 2015 | B2 |
| 9092343 | Yu et al. | Jul 2015 | B2 |
| 9244837 | Bhoria et al. | Jan 2016 | B2 |
| 9455048 | Berckmann et al. | Sep 2016 | B2 |
| 20010034827 | Mukherjee et al. | Oct 2001 | A1 |
| 20010042179 | Ciavaglia et al. | Nov 2001 | A1 |
| 20020087810 | Boatright et al. | Jul 2002 | A1 |
| 20020141426 | Tanaka et al. | Oct 2002 | A1 |
| 20020188817 | Norden et al. | Dec 2002 | A1 |
| 20020199063 | Chaudhry et al. | Dec 2002 | A1 |
| 20030065909 | Jourdan | Apr 2003 | A1 |
| 20030097304 | Hunt | May 2003 | A1 |
| 20030163660 | Lam | Aug 2003 | A1 |
| 20030172198 | Tiwary et al. | Sep 2003 | A1 |
| 20030196075 | Akkary et al. | Oct 2003 | A1 |
| 20030208665 | Peir et al. | Nov 2003 | A1 |
| 20030217251 | Jourdan et al. | Nov 2003 | A1 |
| 20040034757 | Gochman et al. | Feb 2004 | A1 |
| 20040044847 | Ray et al. | Mar 2004 | A1 |
| 20040103256 | Flores et al. | May 2004 | A1 |
| 20040117573 | Sutanto et al. | Jun 2004 | A1 |
| 20040123058 | Hum et al. | Jun 2004 | A1 |
| 20040123078 | Hum et al. | Jun 2004 | A1 |
| 20040243790 | Soltis et al. | Dec 2004 | A1 |
| 20050071573 | Dodson et al. | Mar 2005 | A1 |
| 20050097304 | Luick | May 2005 | A1 |
| 20050138339 | Hily et al. | Jun 2005 | A1 |
| 20050154832 | Steely, Jr. et al. | Jul 2005 | A1 |
| 20050228971 | Samra | Oct 2005 | A1 |
| 20060026371 | Chrysos et al. | Feb 2006 | A1 |
| 20060026594 | Yoshida et al. | Feb 2006 | A1 |
| 20060064551 | Sutanto et al. | Mar 2006 | A1 |
| 20060064554 | Fridella et al. | Mar 2006 | A1 |
| 20060224836 | Blumrich et al. | Oct 2006 | A1 |
| 20060224838 | Blumrich et al. | Oct 2006 | A1 |
| 20070174599 | Dowling | Jul 2007 | A1 |
| 20070204135 | Jiang et al. | Aug 2007 | A1 |
| 20070288694 | Ghai et al. | Dec 2007 | A1 |
| 20070288725 | Luick | Dec 2007 | A1 |
| 20080082794 | Yu et al. | Apr 2008 | A1 |
| 20080134159 | Guo et al. | Jun 2008 | A1 |
| 20080244130 | Jenkins et al. | Oct 2008 | A1 |
| 20090013135 | Burger et al. | Jan 2009 | A1 |
| 20090031087 | Gaither et al. | Jan 2009 | A1 |
| 20090063782 | Toussi | Mar 2009 | A1 |
| 20090083478 | Kunimatsu et al. | Mar 2009 | A1 |
| 20090172255 | Yeh et al. | Jul 2009 | A1 |
| 20090182968 | Arimilli et al. | Jul 2009 | A1 |
| 20090198972 | Nystad et al. | Aug 2009 | A1 |
| 20090210679 | Tsai et al. | Aug 2009 | A1 |
| 20090240873 | Yu et al. | Sep 2009 | A1 |
| 20090282202 | Krimer et al. | Nov 2009 | A1 |
| 20090282225 | Caprioli et al. | Nov 2009 | A1 |
| 20090307434 | Sivaramakrishnan et al. | Dec 2009 | A1 |
| 20100011166 | Yu et al. | Jan 2010 | A1 |
| 20100017804 | Gupta et al. | Jan 2010 | A1 |
| 20100064111 | Kunimatsu et al. | Mar 2010 | A1 |
| 20100095151 | Kobayashi et al. | Apr 2010 | A1 |
| 20100145969 | Wang et al. | Jun 2010 | A1 |
| 20100161948 | Abdallah | Jun 2010 | A1 |
| 20100268882 | Cargnoni et al. | Oct 2010 | A1 |
| 20100274972 | Babayan et al. | Oct 2010 | A1 |
| 20100281220 | Cantin et al. | Nov 2010 | A1 |
| 20100293347 | Luttrell | Nov 2010 | A1 |
| 20100299484 | Hooker et al. | Nov 2010 | A1 |
| 20100306477 | Luttrell | Dec 2010 | A1 |
| 20100306507 | Day et al. | Dec 2010 | A1 |
| 20110040955 | Hooker et al. | Feb 2011 | A1 |
| 20110047350 | Geissler et al. | Feb 2011 | A1 |
| 20110078697 | Smittle et al. | Mar 2011 | A1 |
| 20110119470 | Ohmacht | May 2011 | A1 |
| 20110138393 | Venkumahanti et al. | Jun 2011 | A1 |
| 20110161632 | Sha et al. | Jun 2011 | A1 |
| 20110185159 | Bishop et al. | Jul 2011 | A1 |
| 20110225297 | Archer et al. | Sep 2011 | A1 |
| 20120110280 | Bryant | May 2012 | A1 |
| 20120117323 | Cypher et al. | May 2012 | A1 |
| 20120117332 | Ish | May 2012 | A1 |
| 20120117335 | Bryant | May 2012 | A1 |
| 20120124587 | Li et al. | May 2012 | A1 |
| 20120137077 | Shah et al. | May 2012 | A1 |
| 20120198174 | Nellans et al. | Aug 2012 | A1 |
| 20130024661 | Abdallah | Jan 2013 | A1 |
| 20140095809 | Moll et al. | Apr 2014 | A1 |
| 20150095588 | Abdallah | Apr 2015 | A1 |
| 20150100734 | Abdallah | Apr 2015 | A1 |
| 20150100766 | Abdallah | Apr 2015 | A1 |
| 20150134934 | Abdallah | May 2015 | A1 |
| 20150199272 | Goel et al. | Jul 2015 | A1 |
| 20150205605 | Abdallah | Jul 2015 | A1 |
| 20150211021 | De | Jul 2015 | A1 |
| 20160154817 | Mason, Jr. et al. | Jun 2016 | A1 |
| Number | Date | Country |
|---|---|---|
| 1241752 | Jan 2000 | CN |
| 1264866 | Aug 2000 | CN |
| 1307700 | Aug 2001 | CN |
| 1095117 | Nov 2002 | CN |
| 1383511 | Dec 2002 | CN |
| 1728087 | Feb 2006 | CN |
| 1732433 | Feb 2006 | CN |
| 101013402 | Aug 2007 | CN |
| 101055546 | Oct 2007 | CN |
| 101449250 | Jun 2009 | CN |
| 101449256 | Jun 2009 | CN |
| 101501659 | Aug 2009 | CN |
| 101542430 | Sep 2009 | CN |
| 101627365 | Jan 2010 | CN |
| 101689143 | Mar 2010 | CN |
| 101447911 | Dec 2010 | CN |
| 101901132 | Dec 2010 | CN |
| 101944068 | Jan 2011 | CN |
| 0871109 | Oct 1998 | EP |
| 0871109 | Jun 2003 | EP |
| 1612661 | Jan 2006 | EP |
| 3727887 | Dec 2005 | JP |
| 20090110920 | Oct 2009 | KR |
| 20110050725 | May 2011 | KR |
| 200511023 | Mar 2005 | TW |
| 200815992 | Apr 2008 | TW |
| 200837568 | Sep 2008 | TW |
| 201042544 | Dec 2010 | TW |
| 201124842 | Jul 2011 | TW |
| 9802804 | Jan 1998 | WO |
| 0150252 | Jul 2001 | WO |
| 0150256 | Jul 2001 | WO |
| 0163240 | Aug 2001 | WO |
| 2008061154 | May 2008 | WO |
| 2011107170 | Sep 2011 | WO |
| Entry |
|---|
| Castro F., et al.,“Memory Disambiguation Hardware: a Review,” Journal of Computer Science & Technology, Oct. 1, 2008, pp. 132-139. |
| Decision to grant a patent from foreign counterpart Korean Application No. 10-2015-7000654, dated Oct. 27, 2016, 2 pages. |
| Extended European Search Report for Application No. EP13803443, dated Nov. 22, 2016, 9 pages. |
| Extended European Search Report for Application No. EP13803692, dated Nov. 22, 2016, 8 pages. |
| Extended European Search Report for Application No. EP13804852, dated Nov. 28, 2016, 7 pages. |
| Extended European Search Report for Application No. EP13805130, dated Nov. 22, 2016, 9 pages. |
| Final Office Action from U.S. Appl. No. 14/559,740, dated Jan. 19, 2017, 23 pages. |
| International Preliminary Report on Patentability for Application No. PCT/US2013/045020, dated Dec. 24, 2014, 7 pages. |
| International Preliminary Report on Patentability for Application No. PCT/US2013/045261, dated Dec. 24, 2014, 8 pages. |
| International Preliminary Report on Patentability for Application No. PCT/US2013/045470, dated Dec. 24, 2014, 6 pages. |
| International Preliminary Report on Patentability for Application No. PCT/US2013/045734, dated Dec. 24, 2014, 7 pages. |
| International Search Report and Written Opinion for Application No. PCT/US2013/045020, dated Oct. 28, 2013, 8 pages. |
| International Search Report and Written Opinion for Application No. PCT/US2013/045261, dated Jan. 29, 2014, 9 pages. |
| International Search Report and Written Opinion for Application No. PCT/US2013/045470, dated Jul. 22, 2013, 7 pages. |
| International Search Report and Written Opinion for Application No. PCT/US2013/045734, dated Feb. 4, 2014, 8 pages. |
| Non-Final Office Action from U.S. Appl. No. 14/559,740, dated Oct. 31, 2016, 21 pages. |
| Non-Final Office Action from U.S. Appl. No. 14/569,537, dated Jun. 29, 2016, 61 pages. |
| Non-final Office Action from U.S. Appl. No. 14/569,554, dated Jun. 29, 2016, 52 pages. |
| Notice of Allowance and Search Report from foreign counterpart Taiwan Patent Application No. 102121094, dated Jan. 26, 2017, 4 pages. |
| Notice of Allowance and Search Report from foreign counterpart Taiwan Patent Application No. 102121096, dated Jan. 26, 2017, 4 pages. |
| Notice of Allowance from counterpart Taiwan Patent Application No. 102121092, dated Jan. 18, 2017, 3 pages. |
| Notification of Reason for Refusal from foreign counterpart Korean Patent Application No. 1020157000653, dated Jan. 26, 2017, 8 pages. |
| Notification of Reason for Refusal from foreign counterpart Korean Patent Application No. 1020157000653, dated Jul. 29, 2016, 4 pages. |
| Notification of Reason for Refusal from foreign counterpart Korean Patent Application No. 20157000751, dated Jan. 20, 2016, 4 pages. |
| Notification of Reason for Refusal from foreign counterpart Korean Patent Application No. 20157000751, dated Jul. 29, 2016, 7 pages. |
| Notification of Reasons for Refusal from foreign counterpart Korean Application No. 10-2015-7000654, dated Jan. 18, 2016, 5 pages. |
| Notification of Reasons for Refusal from foreign counterpart Korean Application No. 10-2015-7000654, dated Jul. 4, 2016, 5 pages. |
| Notification of Reasons for Refusal from foreign counterpart Korean Application No. 10-2015-7000695, dated Aug. 23, 2016, 5 pages. |
| Notification of Reasons for Refusal from foreign counterpart Korean Application No. 10-2015-7000695, dated Feb. 28, 2017, 8 pages. |
| Office action and search report from foreign counterpart China Patent Application No. CN201380043000, dated Oct. 28, 2016, 11 pages. |
| Office Action from foreign counterpart China Patent Application No. 201380043001.8, dated Aug. 22, 2016, 10 pages. |
| Slosiar R., et al.,“MD3Q: A distributed queueing protocol with full channel capacity re-use and guarantee of bandwidth,” Computer Networks and ISDN Systems, Mar. 1, 1994, vol. 26 (6-8), pp. 799-815. |
| Extended European Search Report for Application No. EP13804863, dated Nov. 22, 2016, 10 pages. |
| Final Office Action from U.S. Appl. No. 14/569,537, dated May 18, 2017, 78 pages. |
| First Office Action and Search Report for foreign counterpart China Application No. 201380042996.6, dated Aug. 26, 2016, 27 pages. |
| Grant of Patent for foreign counterpart Korean Application No. 10-2015-7000652, dated Feb. 28, 2017, 2 pages. |
| Hammond L., et al., “Transactional Memory Coherence and Consistency,” Proceedings of the 31st Annual International Symposium on Computer Architecture (ISCA), 2004, pp. 102-113 (http://csl.stanford.edu/˜christos/publications/2004.tcc.isca.pdf). |
| International Preliminary Report on Patentability for Application No. PCT/US2013/045497, dated Dec. 24, 2014, 8 pages. |
| International Search Report for PCT Application No. PCT/US2013/045497, dated Aug. 30, 2013. |
| Non-final Office Action from U.S. Appl. No. 14/563,583, dated Jul. 6, 2016, 54 pages. |
| Notice of Allowance from U.S. Appl. No. 14/559,740, dated May 22, 2017, 19 pages. |
| Notice of Preliminary Rejection from foreign counterpart Korean Patent Application No. 10-2015-7000751, dated Apr. 5, 2017, 9 pages. |
| Notification of Reason for Refusal for foreign counterpart Korean Application No. 10-2015-7000652, dated Aug. 23, 2016, 11 pages. |
| Office Action and Search Report from foreign counterpart Taiwan Patent Application No. 102121102, dated May 18, 2017, 9 pages. |
| Second Office Action from foreign counterpart China Patent Application No. 201380042996.6, dated May 3, 2017, 13 pages. |
| Second Office Action with English translation from foreign counterpart Chinese Patent Application No. 201380043001, dated Apr. 1, 2017, 8 pages. |
| U.S. Appl. No. 13/360,024, filed Jan. 27, 2012, “Hardware Acceleration Components for Translating Guest Instructions to Native Instructions”, by Mohammad Abdallah, pp. 1-54. |
| Written Opinion for Application No. PCT/US2013/045497, dated Aug. 30, 2013, 6 pages. |
| Anonymous: “Memory disambiguation—Wikipedia,” Apr. 24, 2011, Retrieved from the Internet: URL: https://en.wikipedia.org/w/index.php?title=Memorydisambiguation&oldid=425585684. |
| Decision to Grant a Patent for Korean Patent Application No. 20157000752, dated Jul. 29, 2016, 2 pages. |
| Extended European Search Report for Application No. EP13803665, dated Nov. 22, 2016, 9 pages. |
| Extended European Search Report for Application No. EP13803753, dated Oct. 31, 2016, 8 pages. |
| Extended European Search Report for Application No. EP13804462, dated Nov. 29, 2016, 8 pages. |
| Extended European Search Report for Application No. EP13805015, dated Nov. 11, 2016, 8 pages. |
| Final Office Action from U.S. Appl. No. 14/567,699, dated Feb. 28, 2017, 12 pages. |
| First Office Action and Search Report from foreign counterpart Chinese Patent Application No. 201380042998.5, dated Jun. 28, 2017, 16 pages. |
| First Office Action and Search Report from foreign counterpart Chinese Patent Application No. 201380042999, dated Dec. 19, 2016, 13 pages. |
| International Preliminary Report on Patentability for Application No. PCT/US2013/045008, dated Dec. 24, 2014, 6 pages. |
| International Preliminary Report on Patentability for Application No. PCT/US2013/045722, dated Dec. 24, 2014, 7 pages. |
| International Preliminary Report on Patentability for Application No. PCT/US2013/045730, dated Dec. 24, 2014, 7 pages. |
| International Preliminary Report on Patentability for Application No. PCT/US2013/045863, dated Dec. 24, 2014, 6 pages. |
| International Search Report and Written Opinion for Application No. PCT/US2013/045008, dated Oct. 7, 2013, 7 pages. |
| International Search Report and Written Opinion for Application No. PCT/US2013/045722, dated Dec. 2, 2013, 8 pages. |
| International Search Report and Written Opinion for Application No. PCT/US2013/045730, dated Sep. 27, 2013, 8 pages. |
| International Search Report and Written Opinion for Application No. PCT/US2013/045863, dated Sep. 27, 2013, 7 pages. |
| Non-Final Office Action from U.S. Appl. No. 14/567,699, dated Nov. 15, 2016, 15 pages. |
| Non-Final Office Action from U.S. Appl. No. 14/567,731, dated Nov. 2, 2016, 12 pages. |
| Non-Final Office Action from U.S. Appl. No. 14/569,543, dated Nov. 21, 2016, 13 pages. |
| Non-Final Office Action from U.S. Appl. No. 14/569,551, dated Nov. 22, 2016, 10 pages. |
| Notice of Allowance from foreign counterpart Korean Patent Application No. 10-2015-7000751, dated Aug. 30, 2017, 3 pages. |
| Notice of Allowance from U.S. Appl. No. 14/567,699, dated May 19, 2017, 8 pages. |
| Notice of Allowance from U.S. Appl. No. 14/567,731, dated Jan. 31, 2017, 7 pages. |
| Notice of Allowance from U.S. Appl. No. 14/567,731, dated Jul. 19, 2017, 17 pages. |
| Notice of Allowance from U.S. Appl. No. 14/569,543, dated Aug. 24, 2017, 19 pages. |
| Notice of Allowance from U.S. Application No. 14/569,543, dated Feb. 15, 2017, 11 pages. |
| Notice of Allowance from U.S. Appl. No. 14/569,551, dated Aug. 3, 2017, 15 pages. |
| Notice of Allowance from U.S. Appl. No. 14/569,551, dated Feb. 10, 2017, 9 pages. |
| Notice of Preliminary Rejection from foreign counterpart Korean Patent Application No. 10-2015-7000708, dated Apr. 28, 2017, 10 pages. |
| Notice of Preliminary Rejection from foreign counterpart Korean Patent Application No. 20157000647, dated Apr. 28, 2017, 9 pages. |
| Notice on Grant of Patent from foreign counterpart China Patent Application No. 20138004300T5, dated Apr. 6, 2017, 4 pages. |
| Notification of Reason for Refusal from foreign counterpart Korean Patent Application No. 20157000647, dated Aug. 30, 2016, 5 pages. |
| Notification of Reason for Refusal from foreign counterpart Korean Patent Application No. 20157000752, dated Jan. 20, 2016, 7 pages. |
| Notification of Reason for Refusal from foreign counterpart Korean Patent Application No. KR1020157000708, dated Apr. 28, 2017, 10 pages. |
| Notification of Reason for Refusal from foreign counterpart Korean Patent Application No. KR1020157000708, dated Aug. 30, 2016, 6 pages. |
| Notification of Reason for Refusal from Korean Application No. 1020157000678, dated Feb. 28, 2017, 12 pages. |
| Office Action and Search report from foreign counterpart China Patent Application No. 201380043007.5, dated Sep. 9, 2016, 9 pages. |
| Office Action and Search Report from foreign counterpart Taiwan Patent Application No. 102121100, dated Nov. 22, 2016, 10 pages. (English translation only available). |
| Office Action and Search Report from foreign counterpart Taiwan Patent Application No. 20130121088, dated Nov. 28, 2016, 6 pages. |
| Office Action from foreign counterpart Korean Patent Application No. 20157000678, dated Aug. 30, 2016, 4 pages. |
| Second Office Action from foreign counterpart Chinese Patent Application No. 201380042999.X, dated Aug. 25, 2017, 11 pages. |
| Stone S.S., et al., “Address-Indexed Memory Disambiguation and Store-to-Load Forwarding,” Proceedings of the 38th International Symposium on Microarchitecture, Nov. 16, 2005, pp. 171-182. |
| Wenisch T.F., et al.,“Mechanisms for Store-Wait-Free multiprocessors,”Proceedings of the 34th Annual International Symposium on Computer Architecture, 2007, pp. 266-277. |
| First Office Action and Search Report from foreign counterpart China Patent Application No. 201380043003.7, dated Sep. 8, 2017, 9 pages. (Translation available only for office action). |
| First Office Action and Search Report from foreign counterpart China Patent Application No. 201380043004.1, dated Jul. 31, 2017, 22 pages. |
| First Office Action and Search Report from foreign counterpart China Patent Application No. 201380043005.6, dated Jul. 31, 2017, 19 pages. |
| Non-Final Office Action from U.S. Appl. No. 14/560,974, dated Sep. 28, 2017, 108 pages. |
| Non-final Office Action from U.S. Appl. No. 14/563,583, dated Sep. 25, 2017, 92 pages. |
| Non-Final Office Action from U.S. Appl. No. 14/569,537, dated Sep. 22, 2017, 90 pages. |
| Non-final Office Action from U.S. Appl. No. 14/569,554, dated Sep. 25, 2017, 90 pages. |
| Second Office Action from foreign counterpart Chinese Patent Application No. 201380042999.X, dated Aug. 25, 2017, 27 pages. |
| Advisory Action from U.S. Appl. No. 14/560,974, dated Jun. 9, 2017, 3 pages. |
| Allowance Decision of Examination from foreign counterpart Taiwan Patent Application No. 102121087, dated Aug. 8, 2017, 3 pages. |
| Extended European Search Report for Application No. EP13804226, dated Oct. 31, 2016, 8 pages. |
| Final Office Action from U.S. Appl. No. 14/560,974, dated Mar. 17, 2017, 68 pages. |
| Final Office Action from U.S. Appl. No. 14/563,583, dated Jun. 2, 2017, 59 pages. |
| Final Office Action from U.S. Appl. No. 14/569,554, dated Jun. 2, 2017, 71 pages. |
| First Office Action from foreign counterpart Chinese Patent Application No. 201380043002, dated Jul. 5, 2016, 12 pages. |
| International Preliminary Report on Patentability for Application No. PCT/US2013/045193, dated Dec. 24, 2014, 8 pages. |
| International Search Report and Written Opinion for Application No. PCT/US2013/045193, dated Jan. 22, 2014, 9 pages. |
| Microsoft Computer Dictionary [online], 5th edition, Microsoft Press 2002 [retrieved on May 23, 2017]. Retrieved from the Internet URL:http://ieeexplore.ieee.org/document/4683369/>https://www.linkedtech.com/downloads/pcdictionary.pdf, p. 38, ISBN 0-7356-1495-4. |
| Non-Final Office Action from U.S. Appl. No. 14/560,974, dated Aug. 30, 2016, 67 pages. |
| Notice of Preliminary Rejection from foreign counterpart Korean Patent Application No. 10-2015-7000693, dated May 26, 2017, 9 pages. |
| Notification of Reason for Refusal from foreign counterpart Korean Patent Application No. 20157000693, dated Aug. 23, 2016, 10 pages. |
| Office Action and Search Report from foreign counterpart Taiwan Patent Application No. 102121087, dated Nov. 9, 2016, 11 pages (Translation only available). |
| Second Office Action from foreign counterpart Chinese Patent Application No. 201380043002, dated Mar. 9, 2017, 39 pages. |
| Shirur, et al., “Performance Analysis of 8-Bit Pipelined Asynchronous Processor Core,” Design and Reuse, retrieved from https://www.design-reuse.com/articles/35659/8-bit-pipelined-asynchronous-processor-core.html on May 24, 2017, 18 pages. |
| Tremblay, et al., “Improving Resource Utilization in an Multiple Asynchronous ALU DSP architecture,” IEEE Explore, Nov. 21, 2008, pp. 25-28, retrieved from http://ieeexplore.ieee.org/document/4683369/ on May 23, 2017. |
| Notice of Allowance from foreign counterpart Korean Patent Application No. 10-2015-7000693, dated Nov. 29, 2017, 3 pages. |
| Notice of Allowance from U.S. Appl. No. 14/567,699, dated Nov. 28, 2017, 50 pages. |
| Notice of Allowance from U.S. Appl. No. 14/567,731, dated Feb. 14, 2018, 50 pages. |
| Office Action and Search Report from foreign counterpart Taiwan Patent Application No. 106106345, dated Jan. 9, 2018, 14 pages. |
| Third Office Action from foreign counterpart Chinese Patent Application No. 201380042999.X, dated Dec. 21, 2017, 18 pages. |
| Notice of Allowance and Search Report from Foreign Counterpart Taiwan Patent Application No. 106106113, dated Feb. 26, 2018, 4 pages. (Translation available only for Search Report). |
| Notice of Allowance from foreign counterpart Chinese Patent Application No. 201380043001.8, dated Feb. 24, 2018, 4 pages. |
| Notice of Allowance from U.S. Appl. No. 14/569,551, dated Mar. 14, 2018, 42 pages. |
| Office Action and Search Report from foreign counterpart Taiwan Patent Application No. 106106344, dated Mar. 7, 2018, 24 pages. (Translation available only for Office Action). |
| Number | Date | Country | |
|---|---|---|---|
| 20150095618 A1 | Apr 2015 | US |
| Number | Date | Country | |
|---|---|---|---|
| 61660544 | Jun 2012 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/US2013/045734 | Jun 2013 | US |
| Child | 14567797 | US |
