Patent Application
20230305962
Various example embodiments relate generally to computer systems and, more particularly but not exclusively, to processors of computer systems.
Computer systems utilize various types of processors to perform various functions in various contexts.
In at least some example embodiments, an apparatus includes a decode unit and a micro-operations cache, wherein the decode unit is configured to decode a marker instruction of a program into a set of micro-operations wherein the marker instruction is configured to mark a beginning of an execution sequence of the program, and wherein the micro-operations cache is configured to store, in a cache line of the micro-operations cache, the set of micro-operations and store, for the set of micro-operations, an indication of a location of the set of micro-operations in the micro-operations cache. In at least some example embodiments, the indication of the location of the set of micro-operations in the micro-operations cache includes an identifier of the cache line that includes the set of micro-operations and an offset of the set of micro-operations from a beginning of the cache line. In at least some example embodiments, the identifier of the cache line includes a memory address of a parent instruction of a micro-operation at the beginning of the cache line. In at least some example embodiments, the cache line is stored in a first cache of the micro-operations cache, and the indication of the location of the set of micro-operations in the micro-operations cache is stored in a second cache of the micro-operations cache. In at least some example embodiments, the micro-operations cache includes a first cache configured to store the cache line and a second cache configured to store the indication of the location of the set of micro-operations in the micro-operations cache. In at least some example embodiments, the micro-operations cache is configured to add, based on detection of the marker instruction during execution of the program and based on a determination that the marker instruction is a subsequent instruction decoded for the cache line, an entry to the second cache for the marker instruction. In at least some example embodiments, a key of the entry is a memory address of the marker instruction, and the entry includes the indication of the location of the set of micro-operations in the micro-operations cache. In at least some example embodiments, the micro-operations cache is configured to perform, based on a memory address of the marker instruction, a lookup for the marker instruction in the micro-operations cache. In at least some example embodiments, the lookup for the marker instruction in the micro-operations cache includes performing, contemporaneously, a lookup for the marker instruction in the first cache and a lookup for the marker instruction in the second cache. In at least some example embodiments, the lookup for the marker instruction in the micro-operations cache includes performing a lookup for the marker instruction in the second cache based on a determination that a lookup for the marker instruction in the first cache is a miss. In at least some example embodiments, the micro-operations cache is configured to determine, based on a determination that the marker instruction is a hit on the second cache, an identifier of the cache line that includes the set of micro-operations and an offset of the set of micro-operations of the marker instruction in the cache line. In at least some example embodiments, the micro-operations cache is configured to access, based on the identifier of the cache line and the offset of the set of micro-operations of the marker instruction in the cache line, a group of micro-operations including the set of micro-operations and any micro-operations that follow the set of micro-operations in the cache line and supply, to an execution unit, the group of micro-operations. In at least some example embodiments, the micro-operations cache is configured to perform, during a first execution clock cycle and based on an identifier of the marker instruction, a lookup for the marker instruction in the first cache and a lookup for the marker instruction in the second cache. In at least some example embodiments, the micro-operations cache is configured to receive, from the first cache based on the lookup for the marker instruction in the first cache, an indication as to whether the lookup for the instruction in the first cache is a hit or a miss and receive, from the second cache based on the lookup for the marker instruction in the second cache, an indication as to whether the lookup for the marker instruction in the second cache is a hit or a miss. In at least some example embodiments, the micro-operations cache is configured to receive, from the first cache based on a determination that the lookup for the marker instruction in the first cache is a hit, an indication of the cache line that includes the set of micro-operations for the marker instruction. In at least some example embodiments, the micro-operations cache is configured to receive, from the second cache based on a determination that the lookup for the marker instruction in the second cache is a hit, an identifier of the cache line in the first cache and an offset of the set of micro-operations of the marker instruction in the cache line. In at least some example embodiments, the micro-operations cache is configured to block, during a second execution clock cycle based on a determination that the lookup for the marker instruction in the first cache is a miss and the lookup for the marker instruction in the second cache is a hit, a lookup in the second cache based on the identifier of the cache line in the first cache. In at least some example embodiments, the micro-operations cache is configured to perform, during a second execution clock cycle based on a determination that the lookup for the marker instruction in the first cache is a miss and the lookup for the marker instruction in the second cache is a hit, a lookup in the first cache based on the identifier of the cache line in the first cache. In at least some example embodiments, the micro-operations cache is configured to access, based on the identifier of the cache line in the first cache and the offset of the set of micro-operations of the marker instruction in the cache line, a group of micro-operations including the set of micro-operations and any micro-operations that follow the set of micro-operations in the cache line and supply, to an execution unit, the group of micro-operations. In at least some example embodiments, the first cache includes a tag array and a data array, and the second cache includes a target vector array and a target vector pointer array. In at least some example embodiments, the micro-operations cache is configured to store, as an entry in the target vector array based on detection of the marker instruction in the program, a memory address of the marker instruction and store, in the target vector pointer array for the entry in the target vector array, the indication of the location of the set of micro-operations in the micro-operations cache. In at least some example embodiments, the micro-operations cache is configured to perform a lookup for the marker instruction in the tag array and perform, based on a determination that the lookup for the marker instruction is a miss in the tag array, a lookup for the marker instruction in the target vector array. In at least some example embodiments, the micro-operations cache is configured to perform a lookup for the marker instruction in the tag array and in the target vector array within the same execution clock cycle. In at least some example embodiments, the micro-operations cache is configured to perform a lookup for the marker instruction in the target vector array. In at least some example embodiments, the lookup for the marker instruction in the target vector array is performed contemporaneously with a lookup of the marker instruction in the tag array or in response to a determination that a lookup of the marker instruction in the tag array is a miss. In at least some example embodiments, the micro-operations cache is configured to determine, based on a determination that the lookup of the marker instruction in the target vector array is a hit on an entry in the target vector array and from an entry in the target vector pointer array that corresponds to the entry in the target vector array, the indication of the location of the set of micro-operations in the micro-operations cache. In at least some example embodiments, the micro-operations cache is configured to access, from the data array based on the indication of the location of the set of micro-operations in the micro-operations cache, a group of micro-operations including the set of micro-operations and any micro-operations that follow the set of micro-operations in the cache line and supply, to an execution unit, the group of micro-operations. In at least some example embodiments, the micro-operations cache is configured to perform, based on a pipelined implementation of the micro-operations cache that uses a sequence of execution clock cycles, a lookup for the marker instruction, wherein, when the lookup for the marker instruction is initiated in execution clock cycle T, the cache line for the marker instruction is delivered in execution clock cycle T+N, wherein N is a number of execution clock cycles in the sequence of execution clock cycles needed to complete a lookup operation in the micro-operations cache.
In at least some example embodiments, a non-transitory computer-readable medium stores computer program code configured to cause an apparatus to decode, by a decode unit, a marker instruction of a program into a set of micro-operations, wherein the marker instruction is configured to mark a beginning of an execution sequence of the program, store, by a micro-operations cache in a cache line of the micro-operations cache, the set of micro-operations, and store, by the micro-operations cache for the set of micro-operations, an indication of a location of the set of micro-operations in the micro-operations cache. In at least some example embodiments, the indication of the location of the set of micro-operations in the micro-operations cache includes an identifier of the cache line that includes the set of micro-operations and an offset of the set of micro-operations from a beginning of the cache line. In at least some example embodiments, the identifier of the cache line includes a memory address of a parent instruction of a micro-operation at the beginning of the cache line. In at least some example embodiments, the cache line is stored in a first cache of the micro-operations cache, and the indication of the location of the set of micro-operations in the micro-operations cache is stored in a second cache of the micro-operations cache. In at least some example embodiments, the micro-operations cache includes a first cache configured to store the cache line and a second cache configured to store the indication of the location of the set of micro-operations in the micro-operations cache. In at least some example embodiments, the computer program code is configured to cause the micro-operations cache to add, based on detection of the marker instruction during execution of the program and based on a determination that the marker instruction is a subsequent instruction decoded for the cache line, an entry to the second cache for the marker instruction. In at least some example embodiments, a key of the entry is a memory address of the marker instruction, and the entry includes the indication of the location of the set of micro-operations in the micro-operations cache. In at least some example embodiments, the computer program code is configured to cause the micro-operations cache to perform, based on a memory address of the marker instruction, a lookup for the marker instruction in the micro-operations cache. In at least some example embodiments, the lookup for the marker instruction in the micro-operations cache includes performing, contemporaneously, a lookup for the marker instruction in the first cache and a lookup for the marker instruction in the second cache. In at least some example embodiments, the lookup for the marker instruction in the micro-operations cache includes performing a lookup for the marker instruction in the second cache based on a determination that a lookup for the marker instruction in the first cache is a miss. In at least some example embodiments, the computer program code is configured to cause the micro-operations cache to determine, based on a determination that the marker instruction is a hit on the second cache, an identifier of the cache line that includes the set of micro-operations and an offset of the set of micro-operations of the marker instruction in the cache line. In at least some example embodiments, the computer program code is configured to cause the micro-operations cache to access, based on the identifier of the cache line and the offset of the set of micro-operations of the marker instruction in the cache line, a group of micro-operations including the set of micro-operations and any micro-operations that follow the set of micro-operations in the cache line and supply, to an execution unit, the group of micro-operations. In at least some example embodiments, the computer program code is configured to cause the micro-operations cache to perform, during a first execution clock cycle and based on an identifier of the marker instruction, a lookup for the marker instruction in the first cache and a lookup for the marker instruction in the second cache. In at least some example embodiments, the computer program code is configured to cause the micro-operations cache to receive, from the first cache based on the lookup for the marker instruction in the first cache, an indication as to whether the lookup for the instruction in the first cache is a hit or a miss and receive, from the second cache based on the lookup for the marker instruction in the second cache, an indication as to whether the lookup for the marker instruction in the second cache is a hit or a miss. In at least some example embodiments, the computer program code is configured to cause the micro-operations cache to receive, from the first cache based on a determination that the lookup for the marker instruction in the first cache is a hit, an indication of the cache line that includes the set of micro-operations for the marker instruction. In at least some example embodiments, the computer program code is configured to cause the micro-operations cache to receive, from the second cache based on a determination that the lookup for the marker instruction in the second cache is a hit, an identifier of the cache line in the first cache and an offset of the set of micro-operations of the marker instruction in the cache line. In at least some example embodiments, the computer program code is configured to cause the micro-operations cache to block, during a second execution clock cycle based on a determination that the lookup for the marker instruction in the first cache is a miss and the lookup for the marker instruction in the second cache is a hit, a lookup in the second cache based on the identifier of the cache line in the first cache. In at least some example embodiments, the computer program code is configured to cause the micro-operations cache to perform, during a second execution clock cycle based on a determination that the lookup for the marker instruction in the first cache is a miss and the lookup for the marker instruction in the second cache is a hit, a lookup in the first cache based on the identifier of the cache line in the first cache. In at least some example embodiments, the computer program code is configured to cause the micro-operations cache to access, based on the identifier of the cache line in the first cache and the offset of the set of micro-operations of the marker instruction in the cache line, a group of micro-operations including the set of micro-operations and any micro-operations that follow the set of micro-operations in the cache line and supply, to an execution unit, the group of micro-operations. In at least some example embodiments, the first cache includes a tag array and a data array, and the second cache includes a target vector array and a target vector pointer array In at least some example embodiments, the computer program code is configured to cause the micro-operations cache to store, as an entry in the target vector array based on detection of the marker instruction in the program, a memory address of the marker instruction and store, in the target vector pointer array for the entry in the target vector array, the indication of the location of the set of micro-operations in the micro-operations cache. In at least some example embodiments, the computer program code is configured to cause the micro-operations cache to perform a lookup for the marker instruction in the tag array and perform, based on a determination that the lookup for the marker instruction is a miss in the tag array, a lookup for the marker instruction in the target vector array. In at least some example embodiments, the computer program code is configured to cause the micro-operations cache to perform a lookup for the marker instruction in the tag array and in the target vector array within the same execution clock cycle. In at least some example embodiments, the computer program code is configured to cause the micro-operations cache to perform a lookup for the marker instruction in the target vector array. In at least some example embodiments, the lookup for the marker instruction in the target vector array is performed contemporaneously with a lookup of the marker instruction in the tag array or in response to a determination that a lookup of the marker instruction in the tag array is a miss. In at least some example embodiments, the computer program code is configured to cause the micro-operations cache to determine, based on a determination that the lookup of the marker instruction in the target vector array is a hit on an entry in the target vector array and from an entry in the target vector pointer array that corresponds to the entry in the target vector array, the indication of the location of the set of micro-operations in the micro-operations cache. In at least some example embodiments, the computer program code is configured to cause the micro-operations cache to access, from the data array based on the indication of the location of the set of micro-operations in the micro-operations cache, a group of micro-operations including the set of micro-operations and any micro-operations that follow the set of micro-operations in the cache line and supply, to an execution unit, the group of micro-operations. In at least some example embodiments, the computer program code is configured to cause the micro-operations cache to perform, based on a pipelined implementation of the micro-operations cache that uses a sequence of execution clock cycles, a lookup for the marker instruction, wherein, when the lookup for the marker instruction is initiated in execution clock cycle T, the cache line for the marker instruction is delivered in execution clock cycle T+N, wherein N is a number of execution clock cycles in the sequence of execution clock cycles needed to complete a lookup operation in the micro-operations cache.
In at least some example embodiments, a method includes decoding, by a decode unit, a marker instruction of a program into a set of micro-operations, wherein the marker instruction is configured to mark a beginning of an execution sequence of the program, storing, by a micro-operations cache in a cache line of the micro-operations cache, the set of micro-operations, and storing, by the micro-operations cache for the set of micro-operations, an indication of a location of the set of micro-operations in the micro-operations cache. In at least some example embodiments, the indication of the location of the set of micro-operations in the micro-operations cache includes an identifier of the cache line that includes the set of micro-operations and an offset of the set of micro-operations from a beginning of the cache line. In at least some example embodiments, the identifier of the cache line includes a memory address of a parent instruction of a micro-operation at the beginning of the cache line. In at least some example embodiments, the cache line is stored in a first cache of the micro-operations cache, and the indication of the location of the set of micro-operations in the micro-operations cache is stored in a second cache of the micro-operations cache. In at least some example embodiments, the micro-operations cache includes a first cache configured to store the cache line and a second cache configured to store the indication of the location of the set of micro-operations in the micro-operations cache. In at least some example embodiments, the method includes adding, by the micro-operations cache based on detection of the marker instruction during execution of the program and based on a determination that the marker instruction is a subsequent instruction decoded for the cache line, an entry to the second cache for the marker instruction. In at least some example embodiments, a key of the entry is a memory address of the marker instruction, and the entry includes the indication of the location of the set of micro-operations in the micro-operations cache. In at least some example embodiments, the method includes performing, by the micro-operations cache based on a memory address of the marker instruction, a lookup for the marker instruction in the micro-operations cache. In at least some example embodiments, the lookup for the marker instruction in the micro-operations cache includes performing, contemporaneously, a lookup for the marker instruction in the first cache and a lookup for the marker instruction in the second cache. In at least some example embodiments, the lookup for the marker instruction in the micro-operations cache includes performing a lookup for the marker instruction in the second cache based on a determination that a lookup for the marker instruction in the first cache is a miss. In at least some example embodiments, the method includes determining, by the micro-operations cache based on a determination that the marker instruction is a hit on the second cache, an identifier of the cache line that includes the set of micro-operations and an offset of the set of micro-operations of the marker instruction in the cache line. In at least some example embodiments, the method includes accessing, by the micro-operations cache based on the identifier of the cache line and the offset of the set of micro-operations of the marker instruction in the cache line, a group of micro-operations including the set of micro-operations and any micro-operations that follow the set of micro-operations in the cache line and supplying, by the micro-operations cache to an execution unit, the group of micro-operations. In at least some example embodiments, the method includes performing, by the micro-operations cache during a first execution clock cycle and based on an identifier of the marker instruction, a lookup for the marker instruction in the first cache and a lookup for the marker instruction in the second cache. In at least some example embodiments, the method includes receiving, by the micro-operations cache from the first cache based on the lookup for the marker instruction in the first cache, an indication as to whether the lookup for the instruction in the first cache is a hit or a miss and receiving, from the second cache based on the lookup for the marker instruction in the second cache, an indication as to whether the lookup for the marker instruction in the second cache is a hit or a miss. In at least some example embodiments, the method includes receiving, by the micro-operations cache from the first cache based on a determination that the lookup for the marker instruction in the first cache is a hit, an indication of the cache line that includes the set of micro-operations for the marker instruction. In at least some example embodiments, the method includes receiving, by the micro-operations cache from the second cache based on a determination that the lookup for the marker instruction in the second cache is a hit, an identifier of the cache line in the first cache and an offset of the set of micro-operations of the marker instruction in the cache line. In at least some example embodiments, the method includes blocking, during a second execution clock cycle based on a determination that the lookup for the marker instruction in the first cache is a miss and the lookup for the marker instruction in the second cache is a hit, a lookup in the second cache based on the identifier of the cache line in the first cache. In at least some example embodiments, the method includes performing, by the micro-operations cache during a second execution clock cycle based on a determination that the lookup for the marker instruction in the first cache is a miss and the lookup for the marker instruction in the second cache is a hit, a lookup in the first cache based on the identifier of the cache line in the first cache. In at least some example embodiments, the method includes accessing, by the micro-operations cache based on the identifier of the cache line in the first cache and the offset of the set of micro-operations of the marker instruction in the cache line, a group of micro-operations including the set of micro-operations and any micro-operations that follow the set of micro-operations in the cache line and supplying, by the micro-operations cache to an execution unit, the group of micro-operations. In at least some example embodiments, the first cache includes a tag array and a data array, and the second cache includes a target vector array and a target vector pointer array. In at least some example embodiments, the method includes storing, by the micro-operations cache as an entry in the target vector array based on detection of the marker instruction in the program, a memory address of the marker instruction and storing, by the micro-operations cache in the target vector pointer array for the entry in the target vector array, the indication of the location of the set of micro-operations in the micro-operations cache. In at least some example embodiments, the method includes performing, by the micro-operations cache, a lookup for the marker instruction in the tag array and performing, by the micro-operations cache based on a determination that the lookup for the marker instruction is a miss in the tag array, a lookup for the marker instruction in the target vector array. In at least some example embodiments, the method includes performing, by the micro-operations cache, a lookup for the marker instruction in the tag array and in the target vector array within the same execution clock cycle. In at least some example embodiments, the method includes performing, by the micro-operations cache, a lookup for the marker instruction in the target vector array. In at least some example embodiments, the lookup for the marker instruction in the target vector array is performed contemporaneously with a lookup of the marker instruction in the tag array or in response to a determination that a lookup of the marker instruction in the tag array is a miss. In at least some example embodiments, the method includes determining, by the micro-operations cache based on a determination that the lookup of the marker instruction in the target vector array is a hit on an entry in the target vector array and from an entry in the target vector pointer array that corresponds to the entry in the target vector array, the indication of the location of the set of micro-operations in the micro-operations cache. In at least some example embodiments, the method includes accessing, by the micro-operations cache from the data array based on the indication of the location of the set of micro-operations in the micro-operations cache, a group of micro-operations including the set of micro-operations and any micro-operations that follow the set of micro-operations in the cache line and supplying, by the micro-operations cache to an execution unit, the group of micro-operations. In at least some example embodiments, the method includes performing, by the micro-operations cache based on a pipelined implementation of the micro-operations cache that uses a sequence of execution clock cycles, a lookup for the marker instruction, wherein, when the lookup for the marker instruction is initiated in execution clock cycle T, the cache line for the marker instruction is delivered in execution clock cycle T+N, wherein N is a number of execution clock cycles in the sequence of execution clock cycles needed to complete a lookup operation in the micro-operations cache.
In at least some example embodiments, an apparatus includes means for decoding, by a decode unit, a marker instruction of a program into a set of micro-operations, wherein the marker instruction is configured to mark a beginning of an execution sequence of the program, means for storing, by a micro-operations cache in a cache line of the micro-operations cache, the set of micro-operations, and means for storing, by the micro-operations cache for the set of micro-operations, an indication of a location of the set of micro-operations in the micro-operations cache. In at least some example embodiments, the indication of the location of the set of micro-operations in the micro-operations cache includes an identifier of the cache line that includes the set of micro-operations and an offset of the set of micro-operations from a beginning of the cache line. In at least some example embodiments, the identifier of the cache line includes a memory address of a parent instruction of a micro-operation at the beginning of the cache line. In at least some example embodiments, the cache line is stored in a first cache of the micro-operations cache, and the indication of the location of the set of micro-operations in the micro-operations cache is stored in a second cache of the micro-operations cache. In at least some example embodiments, the micro-operations cache includes a first cache configured to store the cache line and a second cache configured to store the indication of the location of the set of micro-operations in the micro-operations cache. In at least some example embodiments, the apparatus includes means for adding, by the micro-operations cache based on detection of the marker instruction during execution of the program and based on a determination that the marker instruction is a subsequent instruction decoded for the cache line, an entry to the second cache for the marker instruction. In at least some example embodiments, a key of the entry is a memory address of the marker instruction, and the entry includes the indication of the location of the set of micro-operations in the micro-operations cache. In at least some example embodiments, the apparatus includes means for performing, by the micro-operations cache based on a memory address of the marker instruction, a lookup for the marker instruction in the micro-operations cache. In at least some example embodiments, the lookup for the marker instruction in the micro-operations cache includes performing, contemporaneously, a lookup for the marker instruction in the first cache and a lookup for the marker instruction in the second cache. In at least some example embodiments, the lookup for the marker instruction in the micro-operations cache includes performing a lookup for the marker instruction in the second cache based on a determination that a lookup for the marker instruction in the first cache is a miss. In at least some example embodiments, the apparatus includes means for determining, by the micro-operations cache based on a determination that the marker instruction is a hit on the second cache, an identifier of the cache line that includes the set of micro-operations and an offset of the set of micro-operations of the marker instruction in the cache line. In at least some example embodiments, the apparatus includes means for accessing, by the micro-operations cache based on the identifier of the cache line and the offset of the set of micro-operations of the marker instruction in the cache line, a group of micro-operations including the set of micro-operations and any micro-operations that follow the set of micro-operations in the cache line and means for supplying, to an execution unit, the group of micro-operations. In at least some example embodiments, the apparatus includes means for performing, by the micro-operations cache during a first execution clock cycle and based on an identifier of the marker instruction, a lookup for the marker instruction in the first cache and a lookup for the marker instruction in the second cache. In at least some example embodiments, the apparatus includes means for receiving, by the micro-operations cache from the first cache based on the lookup for the marker instruction in the first cache, an indication as to whether the lookup for the instruction in the first cache is a hit or a miss and means for receiving, from the second cache based on the lookup for the marker instruction in the second cache, an indication as to whether the lookup for the marker instruction in the second cache is a hit or a miss. In at least some example embodiments, the apparatus includes means for receiving, by the micro-operations cache from the first cache based on a determination that the lookup for the marker instruction in the first cache is a hit, an indication of the cache line that includes the set of micro-operations for the marker instruction. In at least some example embodiments, the apparatus includes means for receiving, by the micro-operations cache from the second cache based on a determination that the lookup for the marker instruction in the second cache is a hit, an identifier of the cache line in the first cache and an offset of the set of micro-operations of the marker instruction in the cache line. In at least some example embodiments, the apparatus includes means for blocking, during a second execution clock cycle based on a determination that the lookup for the marker instruction in the first cache is a miss and the lookup for the marker instruction in the second cache is a hit, a lookup in the second cache based on the identifier of the cache line in the first cache. In at least some example embodiments, the apparatus includes means for performing, by the micro-operations cache during a second execution clock cycle based on a determination that the lookup for the marker instruction in the first cache is a miss and the lookup for the marker instruction in the second cache is a hit, a lookup in the first cache based on the identifier of the cache line in the first cache. In at least some example embodiments, the apparatus includes means for accessing, by the micro-operations cache based on the identifier of the cache line in the first cache and the offset of the set of micro-operations of the marker instruction in the cache line, a group of micro-operations including the set of micro-operations and any micro-operations that follow the set of micro-operations in the cache line and means for supplying, by the micro-operations cache to an execution unit, the group of micro-operations. In at least some example embodiments, the first cache includes a tag array and a data array, and the second cache includes a target vector array and a target vector pointer array. In at least some example embodiments, the apparatus includes means for storing, by the micro-operations cache as an entry in the target vector array based on detection of the marker instruction in the program, a memory address of the marker instruction and means for storing, by the micro-operations cache in the target vector pointer array for the entry in the target vector array, the indication of the location of the set of micro-operations in the micro-operations cache. In at least some example embodiments, the apparatus includes means for performing, by the micro-operations cache, a lookup for the marker instruction in the tag array and means for performing, by the micro-operations cache based on a determination that the lookup for the marker instruction is a miss in the tag array, a lookup for the marker instruction in the target vector array. In at least some example embodiments, the apparatus includes means for performing, by the micro-operations cache, a lookup for the marker instruction in the tag array and in the target vector array within the same execution clock cycle. In at least some example embodiments, the apparatus includes means for performing, by the micro-operations cache, a lookup for the marker instruction in the target vector array. In at least some example embodiments, the lookup for the marker instruction in the target vector array is performed contemporaneously with a lookup of the marker instruction in the tag array or in response to a determination that a lookup of the marker instruction in the tag array is a miss. In at least some example embodiments, the apparatus includes means for determining, by the micro-operations cache based on a determination that the lookup of the marker instruction in the target vector array is a hit on an entry in the target vector array and from an entry in the target vector pointer array that corresponds to the entry in the target vector array, the indication of the location of the set of micro-operations in the micro-operations cache. In at least some example embodiments, the apparatus includes means for accessing, by the micro-operations cache from the data array based on the indication of the location of the set of micro-operations in the micro-operations cache, a group of micro-operations including the set of micro-operations and any micro-operations that follow the set of micro-operations in the cache line and means for supplying, by the micro-operations cache to an execution unit, the group of micro-operations. In at least some example embodiments, the apparatus includes means for performing, by the micro-operations cache based on a pipelined implementation of the micro-operations cache that uses a sequence of execution clock cycles, a lookup for the marker instruction, wherein, when the lookup for the marker instruction is initiated in execution clock cycle T, the cache line for the marker instruction is delivered in execution clock cycle T+N, wherein N is a number of execution clock cycles in the sequence of execution clock cycles needed to complete a lookup operation in the micro-operations cache.
The teachings herein can be readily understood by considering the following detailed description in conjunction with the accompanying drawings, in which:
To facilitate understanding, identical reference numerals have been used herein, wherever possible, in order to designate identical elements that are common among the various figures.
Various example embodiments for supporting processor capabilities are presented herein. Various example embodiments may be configured to support a micro-architecture for a micro-operations cache (UC) of a processor. Various example embodiments for supporting a micro-architecture for a UC of a processor may be configured to implement the UC of a processor using a target vector UC (TV-UC). Various example embodiments for supporting a TV-UC for a processor may be configured to support a processor including a TV-UC where the TV-UC includes a micro-operations cache (UC) configured to store a cache line including sets of micro-operations (UOPs) from instructions decoded by the processor and a target vector cache (TVC) configured to store indications of locations of the sets of UOPs in the cache line of the UC. It will be appreciated that these and various other example embodiments and advantages or potential advantages of example embodiments for supporting processor capabilities may be further understood by way of reference to the various figures, which are discussed further below.
The computing system 100 includes a processor 110 and a memory 120. The processor 110 includes an instruction cache (IC) 111 and a micro-operations cache (UC) 112. The high level stages in the pipeline supported by the processor 110 include a fetch stage 130, a decode stage 140, and an execution stage 150.
In the processor 110, the format and encoding of the instructions in a program is determined by the Instruction Set Architecture (ISA) of the processor 110. For example, some well-known ISAs are include x86/x86-64, IA-32/IA-64, MIPS, ARM, and so forth; however, the micro-architecture of a processor cannot execute the instructions of an ISA in their native form because of their complexity. An ISA is designed to offer sophisticated operations which, in turn, also keep the program compact, i.e., reduces the foot print of a program in the memory. It is noted that the optimal footprint of a program in memory is particularly important for optimal use of the IC. A majority of ISAs offer variable-length instructions, which further adds to the complexity of execution. So, at the micro-architectural level of a processor, instructions are represented by fixed length simpler micro-operations (generally referred to as “micro-ops” or “UOPs”). An ISA instruction is broken down into one or more fixed-length UOPs. UOPs perform basic operations on data stored in one or more registers, including transferring data between registers or between registers and external buses, and performing arithmetic and logical operations on registers. For example, an add-register-to-memory ISA instruction performs addition of the value in a register X to the value in a memory location M. The instruction is broken down into a sequence of three separate UOPs as follows: (1) load from M to a register Y, (2) add Y to X, and (3) store X to M.
In the processor 110, execution of a program is based on pipeline which, as indicated above, includes the fetch stage 130, the decode stage 140, and the execution stage 150. The fetch stage 130 retrieves a block of instructions of a program from memory 120 or IC 111. The IC 111 is located on board the processor 110. The IC 111 is generally much smaller in size (e.g., 32 KB, 64 KB, 128 KB, or the like) than the memory 120 and, thus, much faster than the memory 120. The IC 111 caches blocks of instructions fetched from the memory 120. If a set of instructions is repeatedly fetched then those instructions are likely available in the IC 111, so a hit in the IC 111 reduces the time to fetch instructions (as compared with fetching the instructions from the memory 120). The IC 111 is agnostic of syntax and semantics of instructions and caches in units of memory blocks, i.e., all instructions in a certain range of addresses in memory 120. The processor 110 fetches a block of instructions from the memory 120 only if the block is not found in the IC 111. In the IC 111, a memory block is identified by the first memory address in the memory block. In the decode stage 140, instructions fetched during the fetch stage 130 are dynamically decoded by the processor 110 to the native UOPs of the instructions. This dynamic decoding also provides a cleaner separation of the “stable” and “standardized” ISA from the underlying micro-architecture of the processor that is free to define its own UOP set. As a result, a program that has been written for an ISA can run on different micro-architectures supporting that ISA. This has enabled program compatibility between different generations of processors to be easily achieved. For example, different micro-architectures can support the same ISA, but each can define their own native UOP set. The execute stage 150 executes the UOPs supplied by the decode stage 140.
In the processor 110, the fetch stage 130 and the decode stage 140 generally are costly in terms of clock cycles as well as power consumption. So, many modern processors implement another instruction cache, typically referred to as a micro-op cache (UC) or decoded stream buffer (DSB), which stores the already decoded UOPs. This is illustrated as the UC 112 of the processor 110. When the processor 110 needs to execute an instruction and its decoded UOPs already exists in the UC 112, then the UC 112 can directly supply the UOPs to the execution unit. The UC 112 is generally much smaller in size (e.g. 1.5 KB, 2 KB, 3 KB, or the like) than the IC 111 and the memory 120 and, thus, much faster than the IC 111 and the memory 120 (typically operating at the clock speed of the processor 110). A hit in UC 112 eliminates the fetch stage 130 and the decode stage 140, both of which are costly, thereby improving the performance and power budget of the processor 110. An instruction is fetched and decoded only if it is a miss in the UC 112, otherwise the fetch stage 130 and the decode stage 140 can be powered off. It is noted that, although omitted from
The UC 112 stores the UOPs received from the decode stage 140 in smaller sized blocks, but in the sequential order of execution. This means that each branch, conditional or unconditional, makes the processor 110 start with a new UC line even if the current IC line is not yet filled. This simple rule allows high bandwidth fetching from the UC 112 since, once there is a hit in UC 112, then the entire UC line can be supplied to the execution stage 150 without worrying about a change of execution sequence in the middle of a UC line. Herein, unless indicated otherwise, an address of an instruction in memory is referred to as an Instruction Pointer (IP). A UC line is identified by the IP of the parent instruction of the first UOP in the UC line; other than that no correlation exists between the UOPs in a UC line and their corresponding parent instructions, and it is noted that such correlation is not required since the entire UC line is supplied to the execution stage 150. As a result, UOPs in a UC line cannot be looked up by the IPs of their parent instructions.
It will be appreciated that processors generally implement each of the three high-level stages of the instruction pipeline using additional stages. As a result, a pipeline of a processor may be composed of a large number of stages (e.g., 20 or more stages). An example of a processor, for illustrating stages used to implement portions of the instruction pipeline, is presented with respect to
The processor 200 may includes a frontend and a backend. It is noted that while details of the frontend are illustrated, details of the backend have been omitted for purposes of clarity.
The processor 200 includes a level 1 (L1) instruction cache (L1-IC) 210, an instruction fetch unit (IFU) 220, a branch prediction unit (BPU) 230, an instruction length decoder (ILD) 240, an instruction queue (IQ) 250, an instruction decoder (ID) 260, a UOP cache (UC) 270, and an instruction decoder queue (IDQ) 280. It will be appreciated that the IFU 220 and BPU 230 may be considered to form the fetch stage while the ILD 240, IQ 250, ID 260, and IDQ 280 may be considered to form the decode stage.
The L1-IC 210 is a cache that is part of the cache hierarchy of the associated processor and may be further understood by considering the cache hierarchy of processors and the cache arrangement of caches.
In general, a cache is a smaller, faster memory, closer to a processor, which stores copies of the program data or program instructions from frequently accessed memory locations to reduce the average cost of access (time or energy). The data or program instructions are stored in the cache by blocks of contiguous memory locations, typically referred to as cache lines, where each cache line is indexed in the cache by the first memory address in the cache line. Caches benefit from the temporal and spatial locality of memory access patterns in a program, where spatial locality refers to use of relatively close memory locations (i.e., within a cache line) and temporal locality refers to the reuse of specific cache line within a relatively small time duration. Many processors use multiple levels of caches. For example, a common processor architecture might utilize at least three levels (L) of caches, which are typically referred to as L1, L2, and L3. The L1 cache is the smallest and nearest to the processor and, thus, faster than the other cache levels. Typically, the L1 cache is split into two portions: the L1 Instruction Cache (e.g., 32 KB is size, although other sizes may be used) which holds only program instructions and the L1 Data Cache (e.g., 32 KB in size, although other sizes may be used) which holds only program data. The L2 cache (e.g., 256 KB in size, although other sizes may be used) and the L3 cache (e.g., 2 MB in size, although other sizes may be used) are the subsequent levels which are usually unified caches (meaning that they hold both instructions and program data). The L3 cache typically is common for the processor cores in a multi-core processor and, thus, is located outside of the processor cores. It will be appreciated that the cache size and access latency grow according to the levels. If the cache line corresponding to a memory address sought is missing in the L1 cache, then the processor looks up in subsequent levels of caches (e.g., L2 cache, then L3 cache, and so forth). If the memory address is missing in all of the available cache levels, then the processor can access the main memory to retrieve the instruction or data at the memory address. So main memory is accessed only if the memory address is missing in all caches. The missing block, once located, is brought into a cache line in the L1 cache.
In general, a cache is typically organized as set associative array, which can be imagined as M×N matrix. The cache is divided into M sets and each set contains N cache lines. To place a memory block into the cache, its address is typically segregated into 3 fields—tag, index, offset. A memory block is first mapped into a set based on ‘index bits’ derived from the address of the memory block. Then the memory block is placed into a cache line in the set and a ‘tag’ is stored in the cache line. The tag is composed of the bits in the address of the memory block (other than the index bits) that can distinguish between the cache lines sharing the same set. The offset field refers to any address within a cache line. The offset field is composed of a few least significant bits of the address and the number of bits is dependent on the size of the cache line. For example, if the cache line size is 64 B, then the 6 least significant bits of the addresses are the offset bits. Here, the term Instruction Pointer (IP) is used to denote the memory address of an instruction, and the 3 fields of an IP that are used to map a block of instructions into a cache are referred to as IP-tag, IP-index, and IP-offset. If all cache lines in a set are occupied while trying to store a new memory block, then an existing cache line in the set is evicted (a replacement policy picks which cache line to evict) to make way for the new memory block. When the evicted cache line is accessed later, then it will result in a miss in the cache and, thus, will need to be brought back into the cache from the memory hierarchy. Such misses are referred to as conflict misses and repeated conflict misses due to collisions between cache lines sharing the same set is referred to as thrashing.
The IFU 220 is responsible for feeding the processor with instructions to execute, and thus, it is the first component where instructions are processed. The IFU 220 mainly includes the required logic to compute the next fetch address and then fetch the instructions from the L1-IC 210. The instructions are fetched from the L1-IC 210 by the IFU 220 in streams of raw bytes.
The BPU 230 is configured to predict the next fetch address for the IFU 220, because otherwise branch instructions introduce a significant extra level of complexity in fetching streams of instructions, since the correct fetch address cannot be calculated until the branch instruction itself is executed. By default, instructions are processed by a processor sequentially. This sequential execution can be disrupted by the control instructions (e.g., conditional branches, unconditional branches, subroutine calls and subroutine returns, and so forth) to start executing an instruction sequence starting at a new address (the target address). For example, JE (Jump If Equal) is an example of a conditional branch instruction in x86. A conditional branch is data-dependent (e.g., value of a data acts as the condition) and branches to the target address only if the condition is true. For example, instructions such as CALL, RET, and JUMP are examples of a subroutine call, a subroutine return, and an unconditional branch, respectively, in x86. Any control instruction other than conditional branch instruction will switch the execution sequence to the target address specified in the instruction. Herein, the target instruction sequence of a control instruction is referred to generally as a control block. Execution of a program can be viewed as executing a chain of certain control blocks. Herein, an order of execution of control blocks in a program is referred to as a control flow (i.e., flow of control). Conditional branches (e.g., JE) can generate multiple control flows in a program since every such branch is a fork and the execution can go either way on the fork based on the condition of the fork. Control instructions introduce significant extra complexity in fetching streams of instructions, since the correct fetch address after the control instruction cannot be calculated until the backend executes the control instruction itself. For this reason, the frontend of high-performance processors (namely, the BPU 230) predicts the next fetch address and speculatively starts fetching from the predicted address. There are two parts in this prediction. The first is predicting the direction of the branch taken by the control instruction, i.e., taken to the target sequence or not taken. The second part is predicting the target address of a branch. Once the direction of a branch is predicted, then the memory address of the control instruction and its predicted target address is stored in a Branch Target Buffer (BTB), which is a cache organized similar to the set associative array described in the context of L1-IC 210.
The ILD 240 provides a pre-decode phase. The ILD 240 separates the raw byte stream from IFU 220 into a sequence of valid instructions and passes them to the IQ 250. For example, as indicated above, the length of an x86 instruction may vary between 1 B to 15 B and may reside in any byte address in program memory, thus, requiring segregation of the raw byte stream into instructions of variable lengths. Decoding the length of several instructions per cycle adds a level of complexity, since the start addresses have to be speculatively determined. That is, the fact that the starting address of the second instruction is not known until the length of the first instruction is computed, imposes serialization of the length decoding process, and parallelizing this requires determining the length of each instruction before decoding it. The ILD 240 provides complex logic, based on many parallel and speculative computations, to help achieve such parallelization (although this comes at the price of increased power consumption).
The IQ 250 queues the instructions for the instructions decode phase. The IQ 250 queues the instructions, after the ILD 240 separates the instructions from the stream of raw bytes, for use by ID 260 in the instructions decode phase.
The ID 260 provides the instructions decode phase (which also may be referred to as a dynamic translation phase). In this phase, instructions are read from the IQ 250 and translated into subsequent functionally-equivalent UOPs. This translation is performed by one of several decoders in a set of decoders 261 including a complex decoder 261-C and three simple decoders 261-S1-261-S3 (although it will be appreciated that fewer or more instruction decoders may be used). Herein, the ID 260, including the set of decoders 261, also may be referred to as a Micro Instruction Translation Engine (MITE). The resultant UOPs are passed by the ID 260 to the IDQ 280, through which the UOPs may then enter the backend of the processor. For example, in an x86-based processor, simple instructions can translate into 1-4 UOPs and complex instructions can translate into 5 or more UOPs. It will be appreciated that for processors based on other ISAs, instructions may be translated into other numbers of UOPs.
The UC 270, generally speaking, is a UOP cache that is configured to cache UOPs for instructions previously decoded by the MITE, thereby obviating a need for the MITE to re-decode instructions previously decoded by the MITE in order to obtain the associated UOPs (namely, avoiding the L1-IC->IFU->ILD->IQ->MITE decode path). This type of cache may be referred to as an L0 Instruction Cache (L0-IC), which may store blocks of instructions decoded into UOPs, in units of UC lines. UOP caches benefit from the temporal locality of control flows in a program, due to which previously executed instructions are executed again. Before fetching an instruction address from the L1-IC 210, it is first looked up in the L0-IC. If the corresponding UC line exists (meaning a hit) in the L0-IC, then the associated UOPs are directly supplied to the IDQ 280 for further execution, thereby completely avoiding the L1-IC->IFU->ILD->IQ->MITE decoding path. If the corresponding UC line does not exist (meaning a miss) in the L0-IC, then the instruction goes through entire complex decoding cycle through the L1-IC->IFU->ILD-IQ->MITE decoding path. The ability to avoid the L1-IC->IFU->ILD->IQ->MITE decoding path in this manner provide significant advantages, as the decoding process from instructions to UOPs (especially for high performance processors) can be costly in terms of circuitry, power consumption, and time, especially where a single complex instruction may perform several operations. It will be appreciated that, since the backend of a superscalar processor can execute several UOPs per clock cycle (e.g., 6 UOPs per cycle), the rate at which UOPs are supplied from the frontend of the processor to the backend of the processor is a key element of performance which may be achieved by high hit rate in the UC 270.
The IDQ 280 queues UOPs to be provided to the backend of the processor. The UOPs that are queued by the IDQ 280 may include UOPs decoded by the ID 260 (MITE) and UOPs delivered from the UC 270.
The backend of the processor 200, although the details are omitted for purposes of clarity, may include various elements such as a reorder buffer (e.g., configured to receive UOPs from the frontend of the processor 200), a unified reservation station having a set of ports configured to direct UOPs to various chains of elements), various elements or chains of elements configured to support execution of UOPs, or the like, as well as various combinations thereof.
The multi-core processor 300 of
In general, a core is configured to operate as a processor (e.g., similar to the only core of a single core processor). Accordingly, it will be appreciated that each of the cores has its own pipeline (e.g., following the conceptual pipeline of
In general, a cache is a smaller, faster memory, closer to a processor core, which stores copies of the program data or program instructions from frequently used memory locations to reduce the average cost (time or energy). The data or program instructions are stored in the cache by blocks of contiguous memory locations, referred to as cache lines, wherein each cache line is indexed in the cache by the first memory address in the cache line. Caches benefit from the temporal and spatial locality of memory access patterns in a program. Spatial locality refers to use of relatively close memory locations (i.e., within a cache line). Temporal locality refers to the reuse of a specific cache line within a relatively small time duration.
L1 cache is the smallest and nearest to the processor core and, thus, faster than the rest. L1 cache is split into two as follows: an L1 Instruction Cache (e.g., 32 KB size or other suitable size) which holds only program instructions and an L1 Data Cache (e.g., 32K size or other suitable size) which holds only program data. L2 caches (e.g., 256 KB size or other suitable size) and L3 caches (e.g., 2 MB size or other suitable size) are the subsequent levels of caches, which are usually unified caches (meaning that the caches hold both instructions and program data. The L1 Instruction Cache is the IC in
As illustrated in
The data array stores the IC lines, while the tag array is used in order to match IPs into data array entries. The data array is logically organized as a group of S number of sets. Each set consists of N number of IC lines (which also may be referred to as “IC blocks”). The number of IC lines in a set is called the “degree of associativity” of the cache. It is noted that a cache of associativity N is a N-way associative cache, wherein each way is a IC line. A memory block is first mapped into a set Si by its IP and then placed into any IC line Nj in the set Si. To map a memory block into the IC, the IP is partitioned into three fields as illustrated in
The IP-offset field (which also may be referred to as the block offset or, more generally, the offset) includes the K least significant bits of the IP, which are used to identify which bytes inside an IC line are to be accessed. Assuming the size of an IC line is Q bytes, then K=log2(Q) bits in the IP-offset field. Herein, unless indicated otherwise, these K bits are denoted as IP-offset.
The IP-index field (which also may be referred to more generally as the index) includes the M next least significant bits of the IP, which are used to identify the set Si in the IC. For an IC consisting of S sets, M=log2(S) bits are needed in the IP-index field. Herein, unless indicated otherwise, these M bits are denoted as IP-index.
The IP-tag field includes the remaining bits of the IP. Herein, unless indicated otherwise, these bits are denoted as IP-tag.
Different IC lines can map to the same set Si in the IC (they have the same IP-index due to overlapping M bits), so a mechanism is needed to reverse-map IP-indexes to IPs. The tag array serves this purpose. The tag array has the same logical organization as the data array (same number of sets S and associativity N). For each IC line in the data array, the tag array holds some metadata: the IP-tag bits and the state of the IC line (valid, etc.).
To lookup an IP, a set Si in both the data and the tag arrays are accessed using the IP-index part, but to know if an IC line within the set corresponds to the given IP, the IP-tag bits must match to an entry in set Si in tag array. If the IP-tag bits of the j-th entry in set Si match, then the correct data is in the j-th IC line of the corresponding data array in set Si (this is called a cache hit). If no IP-tags is in set Si match in the tag array, then the requested IC line does not reside in the IC (this is a cache miss). In the case of a cache miss, a request to the higher levels of the memory hierarchy must be issued and the processor must wait for the IC line to be installed in the IC before the access can proceed.
As an example, consider an 8-way associative cache with 64 sets with a cache line size of 64 B. Then, each cache line would hold a block of 64 B of instructions. Here K=6 and M=6. If the processor tries to access an instruction at IP 0xf045 (tag=0x1e, index=0x1, offset=0x5) then it looks for the cache line in set 1 bearing the tag 0x1e. If the IC line is found, then the 5th byte in the IC line is retrieved.
The access to the tag and data array can occur in parallel or serially. In
The number of bits in the IP-offset field determines the size of an IC line, i.e., the size of an IC line is log2(number of bits in IP-offset field). The set in the IC is selected based on IP-index and an IC line within the set (i.e., a way in the set) is tagged with the IP-tag. In the example in
The IC suffers from conflict misses when P frequently accessed IC lines map to the same set Si, and the cache associativity N is <P. In that case, one of the valid IC lines in the set Si needs to be evicted out to accommodate a newer IC line. Obviously, the higher the associativity, the less conflict misses the IC will suffer. On the other hand, the more ways the IC has, the bigger the way multiplexor becomes, and this may affect the cycle time of the processor. For example, a modern L1-IC is typically organized as 64 sets×8 ways×64 B per IC line, totaling to maximum capacity of 32 KB of instructions; however, it will be appreciated that various other implementations may be used.
The program 600 of
It is noted that, since the fetching stage is not aware of syntax and semantics of instructions, it is not necessary that an IC line terminate at the boundary of a multi-byte instruction. Thus, an instruction can stride across two IC lines. For simplicity and without the loss of generality, embodiments of IC lines are described with instructions starting and terminating at the boundary of an IC line.
The processor starts fetching from Instr_1, i.e., the first instruction of the program. Assume that Instr_1 to Instr_5 fits into an IC line. This means that all instructions in the IC line share the same IP-Tag and IP-Index. So, when the processor fetches Instr_1 from memory, it caches Instr_1 to Instr_5 in an IC line addressed by IP of Instr_1, which is shown as IC Line 1 in
The processor, after the execution of jump_100, starts fetching from Instr_100. Assume that Instr_100 to Instr_107 fits into an IC line. So, when the processor fetches Instr_100 from memory, it caches Instr_100 to Instr_107 in an IC line addressed by IP of Instr_100, which is shown as IC Line 2 in
The processor, after execution of jump_25, starts fetching from instr_25. Assume that Instr_25 to Instr_29 fits into an IC line. So, when the processor fetches Instr_25 from memory, it caches Instr_25 to Instr_29 in an IC line addressed by IP of Instr_25, which is shown as IC Line 3 in
The processor, after the execution of jump_102, starts fetching from Instr_102. Since Instr_102 shares the same IP-Tag and IP-Index with Instr_100, so the processor hits the IC Line 2. Instr_102 is fetched by its offset from the IC Line 2.
The UC stores the UOPs received from the decode stage in smaller sized lines, but in the sequential order of execution. This means that each branch, conditional or unconditional, makes the processor start with a new UC line even if the current IC line is not yet filled. This simple rule allows high bandwidth fetching from the UC since, once there is a hit in UC, then the entire UC line can be supplied to the execution stage without worrying about a change of execution sequence in the middle of a UC line. Herein, unless indicated otherwise, an address of an instruction in memory is referred to as an Instruction Pointer (IP). A UC line is identified by the IP of the parent instruction of the first UOP in the UC line; other than that no correlation exists between the UOPs in a UC line and their corresponding parent instructions, and it is noted that such correlation is not required since the entire UC line is supplied to the execution stage. As a result, UOPs in a UC line cannot be looked up by the IPs of their parent instructions. UOPs of an instruction cannot span across two UC lines, so a UC line must begin with the first UOP of an instruction.
UC lines are typically organized in an N-way associative array, similar to an IC as in
For each UC line in the data array, the tag array holds some metadata: the IP-tag and IP-offset bits and the state of the UC line (validity, size, and so forth). Note that the IC tag array entry keeps only the IP-tag since the IC lines are aligned by the fixed size of an IC line, so the IP-offset field is always zero for the first address in the IC line; however, the UC tag array entry needs to keep both IP-tag and IP-offset bits since a UC line always starts with the first instruction of an execution sequence and that instruction can be at any offset within an IC line.
To lookup an IP in UC, a set Si in both the data and the tag arrays are accessed using the IP-index part; however, to know if an UC line within the set corresponds to the given IP, the IP-tag and IP-offset bits must match to an entry in set Si in tag array. If the IP-tag bits and IP-offset bits of the j-th entry in set Si match, then the correct data is in the j-th UC line of the corresponding data array in set Si (this is called a UC hit). If no IP-tags in set Si match in the tag array, then the requested UC line does not reside in the UC (this is a UC miss). In the case of a UC miss, a request to fetch and decode the instruction must be issued. The construction of the UC lines in the UC as the instructions of the program 600 are fetched and decoded by the processor is illustrated with respect to
The processor starts execution from Instr_1 of program 600, which is the first instruction of the program. Initially, there are no valid cache lines in the UC (i.e., the UC is empty). Since no UC line is found for the IP of Instr_1, the processor starts fetching and decoding from Instr1 and allocates a new UC line for storing the decoded UOPs. The unconditional jump instruction jump_100 switches the execution sequence to start from instr_100. So, the instructions of the instruction sequence from Instr_1 to jump_100 are decoded and stored in a new UC line, referred to as UC Line 1. The UC Line 1 is identified by the IP of Instr_1. The UC Line 1 is depicted in
The processor, after jump_100, starts execution from Instr_100. Since no UC line is found for the IP of Instr_100, the processor starts fetching and decoding from Instr_100 and allocates a new UC line for storing the decoded UOPs. After decoding Instr_103, the UC line is full. So, the instructions of the instruction sequence from Instr_100 to Instr_103 are decoded and stored in the new UC line, referred to as UC Line 2. The UC Line 2 is identified by IP of Instr_100. The UC Line 2 is depicted in
The processor then starts execution from Instr_104. Since no UC line is found for the IP of Instr_104, the processor starts fetching and decoding from Instr_104 and allocates a new UC line for storing the decoded UOPs. After decoding jump_25, the processor switches the execution sequence to start from Instr_25. So, the instructions of the instruction sequence from Instr_104 to jump_25 are decoded and stored in the new UC line, referred to as UC Line 3. The UC Line 3 is identified by the IP of Instr_104. The UC Line 3 is depicted in
The processor then starts execution from Instr_25. Since no UC line is found for the IP of Instr_25, the processor starts fetching and decoding from Instr_25 and allocates a new UC line for storing the decoded UOPs. After decoding jump_102, the processor switches the execution sequence to start from Instr_102. So, the instructions of the instruction sequence from Instr_25 to jump_102 are decoded and stored in the new UC line, referred to as UC Line 4. The UC Line 4 is identified by IP of Instr_25. The UC Line 4 is depicted in
The processor then starts execution from Instr_102. Since no UC line is found for the IP of Instr_102, the processor starts fetching and decoding from Instr_102 and allocates a new UC line for storing the decoded UOPs. After decoding jump_25, the processor switches the execution sequence to start from Instr_25. So, the instructions of the instruction sequence from Instr_102 to jump_25 are decoded and stored in the new UC line, referred to as UC Line 5. The UC Line 5 is identified by IP of Instr_102. The UC Line 5 is depicted in
The processor then starts execution from Instr_25. The processor already finds the UC Line 4 identified by the IP of Instr_25, so the entire UC Line 4 is directly supplied to the execution unit.
The processor, for the sake of example, then starts executing another sequence starting at instruction Instr_200. Since no UC line is found for the IP of Instr_200, the processor starts fetching and decoding from Instr_200 and allocates a new UC line for storing the decoded UOPs. After decoding jump_103, the processor switches the execution sequence to start from Instr_103. So, the instructions of the instruction sequence from Instr_200 to jump_103 are decoded and stored in the new UC line, referred to as UC Line 6. The UC Line 6 is identified by IP of Instr_200. The UC Line 6 is depicted in
The processor then starts execution from Instr_103. Since no UC line is found for the IP of Instr_103, the processor starts fetching and decoding from Instr_103 and allocates a new UC line for storing the decoded UOPs. After decoding jump_25, the processor switches the execution sequence to start from Instr_25. So, the instructions of the instruction sequence from Instr_103 to jump_25 are decoded and stored in the new UC line, referred to as UC Line 7. The UC Line 7 is identified by IP of Instr_103. The UC Line 7 is depicted in
While a processor is executing a program such as the program 600, the UC suffers from conflict misses when P frequently accessed UC lines map to the same set Si, and the cache associativity N is less than P. In that case, one of the valid UC lines in the set Si needs to be evicted out to accommodate a newer UC line. It will be appreciated that the higher the associativity the less conflict misses the UC will suffer, whereas, on the other hand, the more ways the UC has then the bigger the way multiplexor becomes and this may affect the cycle time of the processor. In the examples of UC Lines depicted in
Various example embodiments are configured to support efficient utilization of a processor cache (e.g. UC, TC, or the like) of a processor by reducing or eliminating duplication of instructions among cache lines of the processor cache of the processor. The reduction or elimination of duplicate instructions among cache lines of a processor cache of a processor may be based on introduction of Target Instructions into an ISA of the processor. The concept of a Target Instruction may be implemented by any ISA (e.g., x86/x86-64, IA-32/IA-64, MIPS, ARM, or the like). The Target Instruction may be inserted into a program before an execution sequence that is the target of a branch (e.g., a conditional branch or an unconditional branch). The Target Instruction may be automatically inserted into the program by a compiler while the compiler (e.g., GCC, LLVM, or the like) is translating the program, which is written in a high level programming language, to the ISA instructions supported by the ISA of the processor. The Target Instruction works as a marker within the program, which may be used by the processor to index into a cache line of a processor cache of the processor not only by the address of its starting instruction but also by the Target Instruction included in the cache line of the processor cache of the processor, thereby enabling the processor to reduce or eliminate duplication of instructions among cache lines in the processor cache of the processor. An example of use of Target Instructions in the program 600 of
The TV-UC 1000 includes a micro-operations cache (UC) 1010 and a target vector cache (TVC) 1020. The TVC 1020 acts as an auxiliary unit of the UC 1010 which aids in elimination of duplicate instructions among the UC lines in UC 1010.
The UC 1010 and TVC 1020 may cooperate to support elimination of duplicate instructions among the UC lines in UC 1010. Whenever a Target Instruction is encountered while decoding and storing UOPs to a UC line in the UC 1010 and the Target Instruction is a subsequent instruction in the UC line (i.e., not the first instruction in the UC line), an entry is added into the TVC 1020 with the IP of the Target Instruction as the key. The TVC entry stores at least the following tuple in addition to its state (e.g., validity): {IP of the UC Line that includes the Target Instruction, Offset of the first UOP of the Target Instruction in the UC Line}. Whenever a first IP is a miss in the UC 1010, the processor looks up the TVC 1020, because the first IP could be the IP of the first instruction of a new sequence (in that case the instruction must be a Target Instruction). If the TVC lookup is hit, then the entry yields a second IP of an already existing UC line and the offset of the first UOP of the instruction within the UC line and, then, the UC is looked up with the second IP. From the UC line identified by the second IP, all UOPs after the offset are supplied to the execution unit. The steps of the process, which are illustrated in
The UC 1010 and TVC 1020 may use the following process to support elimination of duplicate instructions among the UC lines in UC 1010. At clock cycle T, the IP of an instruction (denoted as IP-1, or the first IP) is input into UC 1010 to do a lookup in the UC 1010 (step 1) and is input into TVC 1020 to do a lookup in the TVC 1020 (step 2) in parallel. At the end of clock cycle T, the UC 1010 delivers the following two output(s): (1) an indication as to whether IP-1 is a hit or a miss in the UC 1010 (step 3) and (2) if IP-1 is a hit in the UC 1010, the UC line in the UC 1010 (step 9). At the end of clock cycle T, the TVC 1020 delivers the following two output(s): (1) an indication as to whether IP-1 is a hit or a miss in the TVC 1020 (step 4) and (2) if IP-1 is a hit in the TVC 1020, {IP of UC line, Offset of IP-1 within UC line}. Here, the IP of the UC line is denoted as IP-2 or second IP. If step 3 results is a miss of IP-1 in UC 1010 and step 4 results in a hit of IP-1 in TVC 1020 then, during the next clock cycle (T+1), steps 5 and 6 (signals 5 and 6) are activated. Step (signal) 5 blocks input of IP-2 into TVC 1020, i.e., blocks step 2 in the clock cycle (T+1) since IP-2 must not be looked up in TVC 1020. Step (signal) 6 connects the IP-2 obtained from TVC (in step 7) into the UC, as step 8. In clock cycle (T+1), step 8 inputs the IP-2 into UC 1010, as step 1. Step 1 executed in clock cycle (T+1) results in a hit of the UC line in UC 1010 at the end of clock cycle (T+1). So, at the end of clock cycle (T+1), the UC delivers the UC line in step 9. At clock cycle (T+1), step 10 feeds the offset of IP-1 obtained from TVC 1020 (at step 7 in clock cycle T) to step 9 to supply the UOPs from the UC line to the execution unit. The offset of IP-1 generated at step 10 at the end of clock cycle T may be stored in a register which can be read in clock cycle (T+1) to feed to step 9. For simplicity, this method is not described in
It is noted that, in the above procedure based on TV-UC 1000, the maximum amount of time it will take to supply the UOPs to the execution unit is two clock cycles, as opposed to a much large number of clock cycles where the TV-UC 1000 is not used (e.g., up to 20 or more clock cycles). For example, in the above procedure based on TV-UC 1000, if the IP of an instruction is a hit in UC 1010 then it takes one clock cycle to supply the UOPs to the execution unit. For example, in the above procedure based on TV-UC 1000, if the IP of an instruction is a miss in UC 1010 but is a hit in TVC 1020, then it takes two clock cycles to supply the UOPs to the execution unit.
It is noted that, in at least some example embodiments, the TVC 1020 may be kept switched off during normal operation and then powered up when an IP is a miss in the UC 1010. In this case, it will take three clock cycles to retrieve the UOPs if an IP is a hit in the TVC 1020. In clock cycle T, the IP-1 is input to UC 1010, which results in a miss. At clock cycle (T+1), the TVC 1020 is powered on and IP-1 is input to TVC 1020, which results in a hit. IP-2, i.e., the IP of the UC line pointed to by the TVC entry is input to UC 1010 in clock cycle (T+2). At the end of clock cycle (T+2), the UC 1010 delivers the UC line for IP-2.
It is noted that, in an efficient implementation of TV-UC 1000, access to TV-UC 1000 may be pipelined. In that case, when TVC 1020 is accessed in clock cycle (T+1), the UC 1010 may be accessed for the next UC line. It is also possible to break the operations described in
It will be appreciated that the operation of TV-UC 1000 may be further understood by further considering fetching and decoding of instructions of the program 900 of
The processor starts execution from Instr_1 of program 900, which is the first instruction of program 900. Initially, there are no valid cache lines in the UC (i.e., the UC is empty and the TVC is empty). Since no UC line is found for the IP of Instr_1, the IP is looked up in TVC. Since no entry is found in TVC for the IP, the processor starts fetching and decoding from Instr_1 and allocates a new UC line for storing the decoded UOPs. The unconditional jump instruction jump_100 switches the execution sequence to start from instr_100. So, the instructions of the instruction sequence from Instr_1 to jump_100 are decoded and stored in the new UC line, referred to as UC Line 1. The UC Line 1 is identified by the IP of Instr_1. The UC Line 1 is depicted in
The processor, after jump_100, starts execution from Target_100. Since no UC line is found for the IP of Target_100, the processor starts fetching and decoding from Target_100 and allocates a new UC line for storing the decoded UOPs. After decoding Instr_103, the UC line is full. So, the instructions of the instruction sequence from Target_100 to Instr_103 are decoded and stored in the new UC line, referred to as UC Line 2. The UC Line 2 is identified by IP of Target_100. The UC Line 2 is depicted in
The UC Line 2 contains two intermediate Target instructions: Target_102 and Target_103. So, entries are created in TVC for Target_102 and Target_103, which are shown in
The processor then starts execution from Instr_104. Since no UC line is found for the IP of Instr_104, the processor starts fetching and decoding from Instr_104 and allocates a new UC line for storing the decoded UOPs. After decoding jump_25, the processor switches the execution sequence to start from Instr_25. So, the instructions of the instruction sequence from Instr_104 to jump_25 are decoded and stored in the new UC line, referred to as UC Line 3. The UC Line 3 is identified by the IP of Instr_104. The UC Line 3 is depicted in
The processor then starts execution from Target_25. Since no UC line is found for the IP of Target_25, the processor starts fetching and decoding from Target_25 and allocates a new UC line for storing the decoded UOPs. After decoding jump_102, the processor switches the execution sequence to start from Target_102. So, the instructions of the instruction sequence from Target_25 to jump_102 are decoded and stored in the new UC line, referred to as UC Line 4. The UC Line 4 is identified by IP of Target_25. The UC Line 4 is depicted in
The processor then starts execution from Target_102. Since no UC line is found for the IP of Target_102, the processor looks up the IP in TVC. The TVC entry 1 is a hit, which points to UC Line 2 and the offset of Target_102 in UC Line 2. So, all UOPs in the UC Line 2 after the offset are supplied to the execution unit. Then the processor starts execution from Instr_104. UC Line 3 already exists with the IP of Instr_104 as its identifier. So, all UOPs from the UC Line 3 are supplied to the execution unit. The last UOP in UC Line 3 switches the execution sequence to start from Target_25. The processor already finds the UC Line 4 identified by the IP of Instr_25, so the entire UC Line 4 is directly supplied to the execution unit.
Now, assume that the processor started executing another sequence starting at instruction Target_200. Since no UC line is found for the IP of Target_200, the processor looks up the IP in TVC. Since no entry found in TVC, the processor starts fetching and decoding from Target_200 and allocates a new UC line for storing the decoded UOPs. After decoding jump_103, the processor switches the execution sequence to start from Target_103. So, the instructions of the instruction sequence from Instr_200 to jump_103 are decoded and stored in the new UC line, referred to as UC Line 5. The UC Line 5 is identified by IP of Target_200. The UC Line 5 is depicted in
The processor then starts execution from Target_103. Since no UC line is found for the IP of Target_103, the processor looks up the IP in TVC. The lookup in TVC hits the TVC entry 2, which points to UC Line 2 and the offset of Target_103 in UC Line 1. So all UOPs in the UC Line 2 after the offset are supplied to the execution unit. The processor then starts execution from Instr_104. UC Line 3 already exists with IP of Instr_104 as its identifier. So, all UOPs from the UC Line 3 are supplied to the execution unit.
As is evident from the example, no overlap exists among the UC lines of the TV-UC and, thus, use of the UC is optimal.
It is noted that while the Target Instructions may be used by the decoding stage, they may be ignored by the execution stage.
The TVC 1220 acts as an auxiliary unit of the UC 1210 which aids in elimination of duplicate instructions among the UC lines in UC 1010.
The UC 1210 and TVC 1220 may cooperate to support elimination of duplicate instructions among the UC lines in UC 1210. The UC 1210 can be implemented as N-way set associative Tag Array and Data Array (illustratively, tag array 1211 and data array 1212, respectively). Whenever a Target Instruction is encountered while decoding and storing instructions to a UC line in the UC 1210 and the Target Instruction is a subsequent instruction in the UC line (i.e., not the first instruction in the UC line), the IP of the Target Instruction is added as an entry into the TVC 1220. The TVC 1220 can be implemented as an N-way set associative TV Array and TV Pointer Array (illustratively, TV array 1221 and TV pointer array 1222, respectively). The entry of TV array 1221 includes the IP of a Target Instruction embedded in a UC line. The corresponding entry in the TV pointer array 1222 stores at least the following tuple: {IP of the UC Line that includes the Target Instruction, Offset of the Target Instruction in the UC Line}. When an instruction is a miss in the tag array 1211, the processor looks up the TV array 1221 because this instruction may be the start of a new sequence (so the instruction may be a Target Instruction). If the TV array 1221 is hit then the corresponding entry in TV pointer array 1222 yields the IP of an already existing UC line and the offset of the instruction in the UC line. Then, from the matching UC line, all UOPs after the offset are supplied to the execution unit. An implementation may lookup both tag array 1211 and the TV array 1221 in parallel to avoid additional clock cycles if there is a miss in the tag array 1211. If the IP is a miss in tag array 1211 but a hit in the TV array 1221 then the corresponding TV pointer array 1222 can feed into the tag array 1211 in the next clock cycle. The steps of the process, which are illustrated in
The UC 1210 and TVC 1220 may use the following process to support elimination of duplicate instructions among the UC lines in UC 1210. At clock cycle T, the IP of an instruction (denoted as IP-1, or the first IP) is input into tag array 1211 of UC 1210 to do a lookup in the tag array 1211 of the UC 1210 (step 1) and is input into TV array 1221 of TVC 1020 to do a lookup in the TV array 1221 of the TVC 1220 (step 2) in parallel. At the end of clock cycle T, the UC 1210 delivers the following two output(s): (1) an indication as to whether IP-1 is a hit or a miss in the tag array 1211 (step 3) and (2) if IP-1 is a hit in the tag array 1211, the UC line from the Data Array 1212 (step 9). At the end of clock cycle T, the TVC 1220 delivers the following two output(s): (1) an indication as to whether IP-1 is a hit or a miss in the TV array 1221 (step 4) and (2) if IP-1 is a hit in the TV array 1221, {IP of UC line, Offset of IP-1 within UC line} from the TV pointer array 1222. Here, the IP of the UC line is denoted as IP-2 or second IP. If step 3 results is a miss of IP-1 in UC 1210 and step 4 results in a hit of IP-1 in tag array 1211 then, during the next clock cycle (T+1), steps 5 and 6 (signals 5 and 6) are activated. Step (signal) 5 blocks input of IP-2 into TV array 1221, i.e., blocks step 2 in the clock cycle (T+1). Step (signal 6) connects the IP-2 obtained from TV pointer array 1222 (in step 7) into the UC, as step 8. In clock cycle (T+1), step 8 inputs the IP-2 into the tag array 1211, as step 1. Step 1 executed in clock cycle (T+1) results in a hit of the UC line in the tag array 1211 at the end of clock cycle (T+1). So, at the end of clock cycle (T+1), the data array 1212 delivers the UC line in step 9. At clock cycle (T+1), step 10 feeds the offset of IP-1 obtained from the TV pointer array 1222 (at step 7 in clock cycle T) to step 9 to supply the UOPs from the UC line to the execution unit.
It is noted that, in the above procedure based on TV-UC 1200, the maximum amount of time it will take to supply the UOPs to the execution unit is two clock cycles, as opposed to a much large number of clock cycles where the TV-UC 1200 is not used (e.g., up to 20 or more clock cycles). For example, in the above procedure based on TV-UC 1200, if the IP of an instruction is a hit in UC 1210 then it takes one clock cycle to supply the UOPs to the execution unit. For example, in the above procedure based on TV-UC 1200, if the IP of an instruction is a miss in UC 1210 but is a hit in TVC 1220, then it takes two clock cycles to supply the UOPs to the execution unit.
It is noted that, in at least some example embodiments, the TVC 1220 may be kept switched off during normal operation and then powered up when an IP is a miss in the tag array 1211 of the UC 1210. In this case, it will take three clock cycles to retrieve the UOPs if an IP is a hit in the TV array 1221. In clock cycle T, the IP-1 is input to the tag array 1211 of the UC 1210, which results in a miss. At clock cycle (T+1), the TVC 1220 is powered on and IP-1 is input to the TV array 1221, which results in a hit. IP-2, i.e., the IP of the UC line pointed to by the TV pointer entry is input to UC 1210 in clock cycle (T+2). At the end of clock cycle (T+2), the UC 1210 delivers the UC line for IP-2.
In at least some example embodiments, a delayed invalidation scheme may be employed. Here, consider eviction of a UC line from a way in the UC to accommodate space for a new UC line in the way. The UC line being evicted may include intermediate target instructions. So, the UC line may be referred to by one or more entries in the TVC. Upon eviction of the UC line, such TVC entries need to be invalidated, but generally it is not possible to do so within the same clock cycles of the eviction process. So, at least some example embodiments may employ a “delayed invalidation” scheme as follows.
In at least some example embodiments, a delayed invalidation scheme may be employed as follows. A Tag Array entry includes a unique “Instance ID” allocated to the UC line. Every time the UC line is reallocated, i.e., evicted and allocated, then the Instance ID may be changed (e.g., incremented by one). The TV pointer array entry that refers to the UC line also includes the same Instance ID of the UC line pointed by the entry. When a TV entry is hit then the following checks are made to verify the validity of the TV entry: (1) if the UC line is not found then the TV entry is marked as invalid, (2) if the UC line is found but is not marked as valid then the TV entry is marked as invalid, (3) if the Instance ID of the UC line is not the same as the Instance ID in the TV pointer array entry, then the TV entry is marked as invalid (this would be the case when the UC line was reallocated for the same IP). It is noted that multiple programs may be executing in the same virtual memory space and, thus, a reallocated UC line identified by the same IP may belong to a different program. It is noted that an implementation may use the physical memory address mapped by the IP of a UC line as the Instance ID, because a physical memory address mapped by an IP is always unique.
The computer 1700 includes a processor 1702 (e.g., a central processing unit (CPU), a processor, a processor having a set of processor cores, a processor core of a processor, or the like) and a memory 1704 (e.g., a random access memory, a read only memory, or the like). The processor 1702 and the memory 1704 may be communicatively connected. In at least some example embodiments, the computer 1700 may include at least one processor and at least one memory including computer program code, wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the computer to perform various functions presented herein.
The computer 1700 also may include a cooperating element 1705. The cooperating element 1705 may be a hardware device. The cooperating element 1705 may be a process that can be loaded into the memory 1704 and executed by the processor 1702 to implement various functions presented herein (in which case, for example, the cooperating element 1705 (including associated data structures) can be stored on a non-transitory computer-readable storage medium, such as a storage device or other suitable type of storage element (e.g., a magnetic drive, an optical drive, or the like)).
The computer 1700 also may include one or more input/output devices 1706. The input/output devices 1706 may include one or more of a user input device (e.g., a keyboard, a keypad, a mouse, a microphone, a camera, or the like), a user output device (e.g., a display, a speaker, or the like), one or more network communication devices or elements (e.g., an input port, an output port, a receiver, a transmitter, a transceiver, or the like), one or more storage devices (e.g., a tape drive, a floppy drive, a hard disk drive, a compact disk drive, or the like), or the like, as well as various combinations thereof.
It will be appreciated that computer 1700 may represent a general architecture and functionality suitable for implementing functional elements described herein, portions of functional elements described herein, or the like, as well as various combinations thereof. For example, computer 1700 may provide a general architecture and functionality that is suitable for implementing one or more elements presented herein, such as a node or a portion thereof, a controller or a portion thereof, or the like, as well as various combinations thereof.
It will be appreciated that at least some of the functions presented herein may be implemented in software (e.g., via implementation of software on one or more processors, for executing on a general purpose computer (e.g., via execution by one or more processors) so as to provide a special purpose computer, and the like) and/or may be implemented in hardware (e.g., using a general purpose computer, one or more application specific integrated circuits, and/or any other hardware equivalents).
It will be appreciated that at least some of the functions presented herein may be implemented within hardware, for example, as circuitry that cooperates with the processor to perform various functions. Portions of the functions/elements described herein may be implemented as a computer program product wherein computer instructions, when processed by a computer, adapt the operation of the computer such that the methods and/or techniques described herein are invoked or otherwise provided. Instructions for invoking the various methods may be stored in fixed or removable media (e.g., non-transitory computer-readable media), transmitted via a data stream in a broadcast or other signal bearing medium, and/or stored within a memory within a computing device operating according to the instructions.
It will be appreciated that the term “or” as used herein refers to a non-exclusive “or” unless otherwise indicated (e.g., use of “or else” or “or in the alternative”).
It will be appreciated that, although various embodiments which incorporate the teachings presented herein have been shown and described in detail herein, those skilled in the art can readily devise many other varied embodiments that still incorporate these teachings.
