Patent Application
20100146209
Data memory accesses are one of the single largest components of performance loss in modem microprocessor systems. Currently, Level 1 (L1) data caches in distinct processors on a multi-core chip typically exist as separate coherence units entirely, with no possibility of acting as a single logical memory system, nor do they offer the flexibility of adaptive interleaving since they operate autonomously. Although there has been some prior work done in configuring Level 2 (L2) cache in multi-core processing environments, current multi-core, which include composable lightweight processor (CLP) technologies use fixed L1 data caches that have not been dynamically configurable.
The features of the present disclosure will become more fully apparent from the following description and appended claims, taken in conjunction with the accompanying drawings. Understanding that these drawings depict only several examples in accordance with the disclosure and are, therefore, not to be considered limiting of its scope, the disclosure will be described with additional specificity and detail through use of the accompanying drawings, in which:
a and 3b are diagrams showing two possible configurations for a multi-core processor, illustrating example methods for combining and dynamically reconfiguring independent data caches;
a-5d are diagrams that illustrate four examples of varying degrees of cache interleaving versus cache coherence that can be dynamically configured, all arranged in accordance with the present disclosure.
In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, similar symbols typically identify similar components, unless context dictates otherwise. The illustrative examples described in the detailed description, drawings, and claims are not meant to be limiting. Other examples may be utilized, and other changes may be made, without departing from the spirit or scope of the subject matter presented herein. It will be readily understood that the aspects of the present disclosure, as generally described herein, and illustrated in the Figures, can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations, all of which are explicitly and implicitly contemplated and made part of this disclosure.
The various aspects, features, examples, embodiments or implementations of the invention described herein can be used alone or in various combinations. The methods of the present invention can be implemented by software, hardware or a combination of hardware and software.
The present application is drawn, inter alia, to methods, apparatus, computer programs and systems related to combining independent data caches. The disclosure describes examples of the construction and operation of hardware memory systems that are more flexible, so that a given design can be configured to match the needs of an application, resulting in greater power efficiency and performance.
Various implementations described herein can dynamically aggregate multiple level-one (L1) data caches associated with distinct processors, change the degree of interleaving (e.g., how much consecutive data is mapped to each participating data cache before addresses go on to the next one), and retain the ability to subsequently adjust the number of participating data caches, or the degree of interleaving, when the requirements of an application or computer process change. For example, utilizing a single chip multiprocessor with 32 processors, each with its own 16 KB level one data cache, if an application would work best with a 64 KB level-one data cache (i.e., 64 KB is the size of its primary working set), employing the present systems and methods, four of the processor/caches can be logically grouped together, giving the view of a single logical 64 KB data cache. Thus, the four participating L1 caches can act as a single coherence unit. In addition, the system may determine that it is best to have an interleaving degree, such as 2 cache lines, where addresses map to one cache for 128 bytes (assuming 64 B cache lines), and then to the next cache for the next 128 bytes of the address space, and so on.
At some point, a reconfiguration of the allocation and coherence of the L1 caches may be desirable. For example, the working set may have grown too large for the current configuration, e.g., just under 100 KB. At such point, the system may interrupt the running jobs and add additional processor/data cache combinations. For example, if four more processor/data cache combinations were added, this would bring the logical total of L1 data cache to 128 KB. Since the number of participating caches has changed, the cache lines in the caches now map to different physical L1 cache banks but should preferably be kept coherent. When this example of a reconfiguration occurs, accesses to cache line X (where X is used to designate an arbitrary address) may now be directed to the wrong cache, and X may be modified in another cache bank. As a result, the new cache that should own X “misses” on an attempt to access the L1 cache. Should this occur, the chip-level coherence protocol will act to invalidate the old copy and permit the new cache to hold X and continue. Each individual cache is treated as a separate entity from the coherence protocol's point of view, even when they are configured to cooperate as a single logical unit.
Various implementations for combining independent data caches, including L1 data caches, can be applied to alter the number of banks existing as a single coherence unit, as well as to change the degree of interleaving among the banks participating as a single coherence unit. This permits multiple cache banks in a large distributed microprocessor to dynamically vary the degree of interleaving among cache banks and the coherence interactions among cache banks by writing to control registers. This dynamic capability allows, for example, multiple independent processors that are colluding on a single program to share the multiple level-one data caches, without needing to flush those data caches upon a reconfiguration in which the number of participating cores is changed. Additionally, in various example implementations, the degree of interleaving of the data caches may be set to best align the locality access patterns of the running application with the selected hardware configuration itself.
The figures include numbering to designate illustrative components of examples shown within the drawings, including the following: a computer system 100, a processor 101, a system bus 102, an operating system 103, an application 104, a read-only memory 105, a random access memory 106, a disk adapter 107, a disk unit 108, a communications adapter 109, an interface adapter 110, a display adapter 111, a keyboard 112, a mouse 113, a speaker 114, a display monitor 115, L1 data cache 121, fetch unit 201, Instruction Fetch Address Register 202, Instruction Cache (I-Cache) unit 203, Instruction Dispatch Unit (IDU) 204, instruction sequencer 205, instruction window 206, fixed point units 207, load/store units 208, floating point units 209, General Purpose Register (GPR) file 210, Floating Point Register (FPR) file 212, completion unit 214, Bus Interface Unit (BIU) 216, system memory 217, integrated circuit chip 301, processor cores 310-317, individual L1 cache 320-327, threads 330-334, L2 cache 350, cache block 351, data entry 353, tag 355, directory 356, directory entry 357, bit vector 360, bit 361, L2 cache line 362, address of a cache line 363, composed processors 370, 372 and 380, and cache manager 390.
Referring to
Computer system 100 may further include a communications adapter 109 coupled to bus 102. Communications adapter 109 may interconnect bus 102 with an outside network (not shown) thereby allowing computer system 100 to communicate with other similar devices. I/O devices may also be connected to computer system 100 via a user interface adapter 110 and a display adapter 111. Keyboard 112, mouse 113 and speaker 114 may all be interconnected to bus 102 through user interface adapter 110. Data may be inputted to computer system 100 through any of these devices. A display monitor 115 may be connected to system bus 102 by display adapter 111. In this manner, a user is capable of interacting with the computer system 100 through keyboard 112 or mouse 113 and receiving output from computer system 100 via display 115 or speaker 114.
Instructions from I-Cache unit 203 may be outputted to Instruction Dispatch Unit (IDU) 204. IDU 204 may be configured to decode these received instructions. IDU 204 may further comprise an instruction sequencer 205, configured to forward the decoded instructions in an order determined by various algorithms. The out-of-order instructions may be forwarded to one of a plurality of issue queues, or what may be referred to as an “instruction window” 206, where a particular issue in instruction window 206 may be coupled to one or more particular execution units, fixed point units (FXUs) 207, load/store units (LSUs) 208 and floating point units (FPUs) 209. Instruction window 206 includes all instructions that have been fetched but are not yet committed. Each execution unit may execute one or more instructions of a particular class of instructions. For example, FXUs 207 may execute fixed point mathematical and logic operations on source operands, such as adding, subtracting, ANDing, ORing and XORing. FPUs 209 may execute floating point operations on source operands, such as floating point multiplication and division.
As stated above, instructions may be queued in one of a plurality of issue queues in instruction window 206. If an instruction contains a fixed point operation, then that instruction may be issued by an issue queue of instruction window 206 to any of the multiple FXUs 207 to execute the instruction containing the fixed point operation. Further, if an instruction contains a floating point operation, then that instruction may be issued by an issue queue of instruction window 206 to any of the multiple FPUs 209 to execute the instruction containing the floating point operation.
All of the execution units, FXUs 207, FPUs 209, LSUs 208, may be coupled to completion unit 214. Upon executing the received instruction, the execution units, FXUs 207, FPUs 209, LSUs 208, may transmit an indication to completion unit 214 indicating the execution of the received instruction. This information may be stored in a table (not shown) which may then be forwarded to IFU 201. Completion unit 214 may further be coupled to IDU 204. IDU 204 may be configured to transmit to completion unit 214 the status information (e.g., type of instruction, associated thread, etc.) of the instructions being dispatched to instruction window 206. Completion unit 214 may further be configured to track the status of these instructions. For example, completion unit 214 may keep track of when these instructions have been committed. Completion unit 214 may further be coupled to instruction window 206 and further configured to transmit an indication of an instruction being committed to the appropriate issue queue of instruction window 206 that issued the instruction that was committed.
In various implementations, LSUs 208 may be coupled to a L1 data cache 121 by way of a cache configuration manager 221. The cache configuration manager operates to establish the desired interleaving between and among shared L1 cache across multiple processing cores. The cache configuration manager is coupled to local L1 data cache 121 and other L1 data cache, such as via an on-chip network among processor cores 118. For example, as explained further in connection with
In response to a load instruction, LSU 208 inputs information from L1 data cache 121 and copies such information to one or more selected GPR files 210 and/or FPR files 212. If such information is not stored in L1 data cache 121, then L1 data cache 121 inputs through Bus Interface Unit (BIU) 216 such information from system memory 217 connected to system bus 102 (See
a and 3b are diagrams showing two possible configurations for a multi-core processor, illustrating example methods for combining and dynamically reconfiguring independent data caches.
Also shown in
As illustrated in
b shows an example of a reconfiguration operation of the processing cores previously configured as composed processors 370, 372 shown in
Previously, the cache in core “0” or core “4” did not have a copy of X. Thus, when Thread “0” attempts to read X, there is a miss, but it is serviced by L2 cache 350, and X is loaded into the L1 data cache for core “0”. However, X remains (until a happenstance eviction) in the L1 data cache for core “1”, even though Thread “3” 333 does not access X. In this example, the same applies for Thread “2” 332 loading X into the L1 data cache for core “3”.
Example bit vectors for X's L2 directory entry 357 are shown before (e.g., as shown in
One feature of various present implementations for combining independent data caches is that typically, the chip-level cache coherence protocol utilized for combining independent data caches disclosed herein can naturally reconcile the changed cache mappings over time, for example. This is generally the case regardless of the configuration that is chosen for an arbitrary number of threads, whether or not the threads share a cache line. Cache lines in “stale” mapping places will eventually get replaced or invalidated by the cache coherence protocol, and cache lines mapping to new locations will simply miss and fill the cache line in the newly mapped bank, regardless of to where the mapping was before the change of cache mappings. This capability can reduce the need to flush the data cache and/or move cache lines around proactively upon a reconfiguration, making reconfigurations of composed cores simpler than would be without this capability, for example.
If, in step 410, instead of a reconfiguration occurring, a read command (e.g., Read X) 413 and/or a write command (e.g., Write X) 415 is issued, the process goes to step 416. In step 416 (Find Bank Holding X), the cache configuration manager can use the cache block address and the number of cores being composed to apply a hash function that picks the core number to where the block is mapped. This can depend on how many interleaved caches are composed together to form a single logical banked cache. Upon identifying the core number to where the block is mapped, e.g., core B in step 416, the read command (e.g., Read X) 413 and/or write command (e.g., Write X) 415 is sent to core B in step 418 (Send Request to Bank B). For a read command 413, the process then advances to step 420 (Hit?), where the method inquires whether there is a read hit to the L1 cache. If, in step 420, there is a read hit, the process returns to step 410. If, in step 420, there is not a read hit but rather a read miss 423, the process goes to step 422 (Send Message to L2), in which the method sends a message to L2 cache. The process then proceeds to step 424 (Load Shared Copy of X), in which the method loads a shared copy of X before returning to step 410.
For a write command (e.g., Write X) 415, the process, after step 418, goes to step 426 (Hit?), where the method determines whether there is a write hit. If there is a write hit, the process proceeds to step 428 (Writable Copy?) where the method inquires whether there is a writable copy, e.g., if the cache line is in shared (i.e. read-only) state. If there is a writable copy, the process returns to step 410. If there is not a writable copy, the process goes to step 430 (Send Message to L2), in which a message is sent to L2 cache. The process the proceeds to step 432 (Invalidate All Copies in Banks Other than B), in which the method launches an invalidation procedure to invalidate all copies in banks not located in the core to where the block is mapped (e.g., core B). The process then proceeds to step 434 (Send Writable Copy to Bank B), in which a writable copy (and/or permission) is sent to the requesting core before returning to step 410. If, in step 426, there is not a hit but rather a write miss, then the process proceeds to steps 430, 432 and 434, described above. In this example, upon a read/load miss in step 420, the method does not have to perform all of the steps associated with a write miss, but rather can just send a message to the L2 cache in step 422 and a shared copy of X can be loaded in step 424.
a-5d are diagrams that illustrate four examples of varying degrees of cache interleaving (i.e., sharing of cache within a cache domain) and cache coherence that can be dynamically configured in accordance with the present disclosure. In particular, the present ability to independently configure composed multiprocessor domains and cache interleaving and coherence domains is illustrated in
b illustrates an example of a strategy that assumes that large working sets are common, and so the entire array of processing cores are shared as one composed processor 510 and a corresponding cache domain 509 in which the L1 cache of all processing cores 310, 311, 312, 313, 314, 315, 316 and 317 are interleaved. All of associated threads are shared as well, which reduces L2 cache misses, coherence, etc., but there can be extra routing for a processor to get to the cache bank within the interleaved domain that it needs.
It will be appreciated that the examples in
Examples of various implementations for combining independent data caches described herein can be utilized in the TFlex microarchitecture, for example. The TFlex microarchitecture is a Composable Lightweight Processor (CLP) that allows simple cores, which can also be called tiles, to be aggregated together dynamically. TFlex is a fully distributed tiled architecture of 32 cores, with multiple distributed load-store banks, that supports an issue width of up to 64 and an execution window of up to 4096 instructions with up to 512 loads and stores. Since control decisions, instruction issue, and dependence prediction may all happen on different tiles, for example, a distributed protocol for handling efficient dependence prediction should be used.
The TFlex architecture uses the TRIPS Explicit Data Graph Execution (EDGE) instruction set architecture (ISA), which can encode programs as a sequence of blocks that have atomic execution semantics, meaning that control protocols for instruction fetch, completion, and commit can operate on blocks of up to 128 instructions. The TFlex CLP microarchitecture can allow the dynamic aggregation of any number of cores—up to 32 for each individual thread—to find the best configuration under different operating targets: e.g., performance, area efficiency, or energy efficiency. The TFlex microarchitecture has no centralized microarchitectural structures. Structures across participating cores can be partitioned based on address. Each block can be assigned an owner core based on its starting address (PC). Instructions within a block can be partitioned across participating cores based on instruction IDs, and the load-store queue (LSQ) and data caches can be partitioned based on load/store data addresses, for example.
Various implementations for combining independent data caches may be applicable to any architecture with distributed fetch and distributed memory banks. For example, various implementations for combining independent data caches may be adapted and/or configured for use with Core Fusion™ by giving its steering management unit (SMU) the responsibilities of the controller core. In addition, while the block-atomic nature of the ISA used by TFlex generally can simplify at least some components of various implementations of a method for combining independent data caches described herein, this technique can be employed with other ISAs by artificially creating blocks from logical blocks in the program to simplify store completion tracking, for example. TFlex is a particular CLP design that can achieve the composable capability by mapping large, structured instruction blocks across participating cores differently depending on the number of cores that are running a single thread.
A fully composable processor shares no structures physically among the multiple processors. Instead, a CLP utilizes distributed microarchitectural protocols and/or methods to provide the necessary fetch, execution, memory access/disambiguation, and commit capabilities. Full composability may be difficult in conventional ISAs because the atomic units are individual instructions, which require that control decisions be made too frequently to properly coordinate across a distributed processor. Explicit data graph execution (EDGE) architectures, conversely, can reduce the frequency of control decisions by employing block-based program execution and explicit intrablock dataflow semantics to map well to distributed microarchitectures, for example.
Some example methods for combining independent data caches include: providing a plurality of L1 cache banks associated with a plurality of processing cores; and configuring at least two of the plurality of cache banks to operate as a single coherent shared cache bank. The configuring step may include interleaving among the plurality of L1 cache banks operating as a single coherent shared cache. The methods may further include changing the degree of interleaving among the plurality of L1 cache banks operating as a single coherent shared cache. The methods may also further include the step of providing an L2 data cache, with the L2 data cache storing the coherence information for the at least two of the plurality of L1 cache banks. The coherence information stored in the L2 data cache can be in the form of at least one bit vector stored therein.
Some example apparatuses for combining independent data caches includes a memory system having at least two L1 data cache banks each associated with a processing core; and a cache manager for configuring at least two of said L1 data cache banks to operate as a single coherent shared cache bank. The cache manager can be configured to enable interleaving among at least two of the L1 cache banks operating as a single coherent shared cache bank. The cache manager can configure the cache to be capable of changing the degree of interleaving among the at least two of said L1 cache banks operating as a single coherent shared cache bank. Some example apparatus can include an L2 data cache that is adapted to store the configuration for the at least two of said plurality of L1 cache banks. The configuration stored in the L2 data cache can be in the form of at least one bit vector stored therein.
Some examples of multi-core processing arrangements include a plurality of processing cores that each has a processor and an L1 cache associated with the processor. Two or more of the L1 caches can be configurable to be shared with at least a second one of the processing cores. One or more L2 caches can be operatively coupled to the processing cores and the L1 cache or caches. Each L2 cache can be adapted to store coherence information of the configurable L1 cache. The coherence information may include at least one bit vector that may corresponding to each shared cache line. The processing cores and L1 cache can be provided on a single integrated circuit. The L2 cache can be provided on the single integrated circuit with the processing cores and L1 cache, for example.
The foregoing detailed description has set forth various embodiments of the devices and/or processes via the use of block diagrams, flowcharts, and/or examples. Insofar as such block diagrams, flowcharts, and/or examples contain one or more functions and/or operations, it will be understood by those within the art that each function and/or operation within such block diagrams, flowcharts, or examples can be implemented, individually and/or collectively, by a wide range of hardware, software, firmware, or virtually any combination thereof. In one embodiment, several portions of the subject matter described herein may be implemented via Application Specific Integrated Circuits (“ASICs”), Field Programmable Gate Arrays (“FPGAs”), digital signal processors (“DSPs”), or other integrated formats. However, those skilled in the art will recognize that some aspects of the embodiments disclosed herein, in whole or in part, can be equivalently implemented in integrated circuits, as one or more computer programs running on one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more processors (e.g., as one or more programs running on one or more microprocessors), as firmware, or as virtually any combination thereof, and that designing the circuitry and/or writing the code for the software and or firmware would be well within the skill of one of skill in the art in light of this disclosure. For example, if a user determines that speed and accuracy are paramount, the user may opt for a mainly hardware and/or firmware vehicle; if flexibility is paramount, the user may opt for a mainly software implementation; or, yet again alternatively, the user may opt for some combination of hardware, software, and/or firmware.
In addition, those skilled in the art will appreciate that the mechanisms of the subject matter described herein are capable of being distributed as a program product in a variety of forms, and that an illustrative embodiment of the subject matter described herein applies regardless of the particular type of signal bearing medium used to actually carry out the distribution. Examples of a signal bearing medium include, but are not limited to, the following: a recordable type medium such as a flexible disk, a hard disk drive, a Compact Disc (“CD”), a Digital Video Disk (“DVD”), a digital tape, a computer memory, etc.; and a transmission type medium such as a digital and/or an analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communications link, a wireless communication link, etc.).
Those skilled in the art will recognize that it is common within the art to describe devices and/or processes in the fashion set forth herein, and thereafter use engineering practices to integrate such described devices and/or processes into data processing systems. That is, at least a portion of the devices and/or processes described herein can be integrated into a data processing system via a reasonable amount of experimentation. Those having skill in the art will recognize that a typical data processing system generally includes one or more of a system unit housing, a video display device, a memory such as volatile and non-volatile memory, processors such as microprocessors and digital signal processors, computational entities such as operating systems, drivers, graphical user interfaces, and applications programs, one or more interaction devices, such as a touch pad or screen, and/or control systems including feedback loops and control motors (e.g., feedback for sensing position and/or velocity; control motors for moving and/or adjusting components and/or quantities). A typical data processing system may be implemented utilizing any suitable commercially available components, such as those typically found in data computing/communication and/or network computing/communication systems.
The herein described subject matter sometimes illustrates different components contained within, or connected with, different other components. It is to be understood that such depicted architectures are merely exemplary, and that in fact many other architectures can be implemented which achieve the same functionality. In a conceptual sense, any arrangement of components to achieve the same functionality is effectively “associated” such that the desired functionality is achieved. Hence, any two components herein combined to achieve a particular functionality can be seen as “associated with” each other such that the desired functionality is achieved, irrespective of architectures or intermedial components. Likewise, any two components so associated can also be viewed as being “operably connected”, or “operably coupled”, to each other to achieve the desired functionality, and any two components capable of being so associated can also be viewed as being “operably couplable”, to each other to achieve the desired functionality. Specific examples of operably couplable include but are not limited to physically mateable and/or physically interacting components and/or wirelessly interactable and/or wirelessly interacting components and/or logically interacting and/or logically interactable components.
With respect to the use of substantially any plural and/or singular terms herein, those having skill in the art can translate from the plural to the singular and/or from the singular to the plural as is appropriate to the context and/or application. The various singular/plural permutations may be expressly set forth herein for sake of clarity.
It will be understood by those within the art that, in general, terms used herein, and especially in the appended claims (e.g., bodies of the appended claims) are generally intended as “open” terms (e.g., the term “including” should be interpreted as “including but not limited to,” the term “having” should be interpreted as “having at least,” the term “includes” should be interpreted as “includes but is not limited to,” etc.). It will be further understood by those within the art that if a specific number of an introduced claim recitation is intended, such an intent will be explicitly recited in the claim, and in the absence of such recitation no such intent is present. For example, as an aid to understanding, the following appended claims may contain usage of the introductory phrases “at least one” and “one or more” to introduce claim recitations. However, the use of such phrases should not be construed to imply that the introduction of a claim recitation by the indefinite articles “a” or “an” limits any particular claim containing such introduced claim recitation to inventions containing only one such recitation, even when the same claim includes the introductory phrases “one or more” or “at least one” and indefinite articles such as “a” or “an” (e.g., “a” and/or “an” should typically be interpreted to mean “at least one” or “one or more”); the same holds true for the use of definite articles used to introduce claim recitations. In addition, even if a specific number of an introduced claim recitation is explicitly recited, those skilled in the art will recognize that such recitation should typically be interpreted to mean at least the recited number (e.g., the bare recitation of “two recitations,” without other modifiers, typically means at least two recitations, or two or more recitations). Furthermore, in those instances where a convention analogous to “at least one of A, B, and C, etc.” is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., “a system having at least one of A, B, and C” would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.). In those instances where a convention analogous to “at least one of A, B, or C, etc.” is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., “a system having at least one of A, B, or C” would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.). It will be further understood by those within the art that virtually any disjunctive word and/or phrase presenting two or more alternative terms, whether in the description, claims, or drawings, should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms. For example, the phrase “A or B” will be understood to include the possibilities of “A” or “B” or “A and B.”
While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.
The invention was made with the U.S. Government support, at least in part, by the Defense Advanced Research Projects Agency, Grant number F33615-03-C-4106. Thus, the U.S. Government may have certain rights to the invention.
