1. Technical Field
The present invention relates in general to data processing and, in particular, retrieving data from a memory hierarchy.
2. Description of the Related Art
A conventional symmetric multiprocessor (SMP) computer system, such as a server computer system, includes multiple processing units all coupled to a system interconnect, which typically comprises one or more address, data and control buses. Coupled to the system interconnect is a system memory, which represents the lowest level of volatile memory in the multiprocessor computer system and which generally is accessible for read and write access by all processing units. In order to reduce access latency to instructions and data residing in the system memory, each processing unit is typically further supported by a respective multi-level cache hierarchy, the lower level(s) of which may be shared by one or more processor cores.
In at least one embodiment, a processor detects during execution of program code whether a load instruction within the program code is associated with a hint. In response to detecting that the load instruction is not associated with a hint, the processor retrieves a full cache line of data from the memory hierarchy into the processor in response to the load instruction. In response to detecting that the load instruction is associated with a hint, a processor retrieves a partial cache line of data into the processor from the memory hierarchy in response to the load instruction.
With reference now to the figures and, in particular, with reference to
In the depicted embodiment, each processing node 102 is realized as a multi-chip module (MCM) containing four processing units 104a-104d, each preferably realized as a respective integrated circuit. The processing units 104a-104d within each processing node 102 are coupled for communication by a local interconnect 114, which, like system interconnect 110, may be implemented with one or more buses and/or switches.
The devices coupled to each local interconnect 114 include not only processing units 104, but also one or more system memories 108a-108d. Data and instructions residing in system memories 108 can generally be accessed and modified by a processor core in any processing unit 104 in any processing node 102 of data processing system 100. In alternative embodiments of the invention, one or more system memories 108 can be coupled to system interconnect 110 rather than a local interconnect 114.
Those skilled in the art will appreciate that SMP data processing system 100 can include many additional unillustrated components, such as interconnect bridges, non-volatile storage, ports for connection to networks or attached devices, etc. Because such additional components are not necessary for an understanding of the present invention, they are not illustrated in
Referring now to
The operation of each processor core 200 is supported by a multi-level volatile memory hierarchy having at its lowest level shared system memories 108a-108d, and at its upper levels one or more levels of cache memory. In the depicted embodiment, each processing unit 104 includes an integrated memory controller (IMC) 206 that controls read and write access to a respective one of the system memories 108a-108d within its processing node 102 in response to requests received from processor cores 200a-200b and operations snooped on the local interconnect 114.
In the illustrative embodiment, the cache memory hierarchy of processing unit 104 includes a store-through level one (L1) cache 226 (which may be bifurcated into separate L1 instruction and data caches) within each processor core 200 and a level two (L2) cache 230 shared by all processor cores 200a, 200b of the processing unit 104. L2 cache 230 includes an L2 array and directory 234 and a cache controller comprising a master 232 and a snooper 236. Master 232 initiates transactions on local interconnect 114 and system interconnect 110 and accesses L2 array and directory 234 in response to memory access (and other) requests received from the associated processor cores 200a-200b. Snooper 236 snoops operations on local interconnect 114, provides appropriate responses, and performs any accesses to L2 array and directory 234 required by the operations.
Although the illustrated cache hierarchy includes only two levels of cache, those skilled in the art will appreciate that alternative embodiments may include additional levels (L3, L4, L5 etc.) of on-chip or off-chip in-line, lookaside or victim cache, which may be fully inclusive, partially inclusive, or non-inclusive of the contents the upper levels of cache.
Each processing unit 104 further includes an instance of response logic 210, which implements a portion of the distributed coherency signaling mechanism that maintains cache coherency within data processing system 100. In addition, each processing unit 104 includes an instance of interconnect logic 212 for selectively forwarding communications between its local interconnect 114 and system interconnect 110. Finally, each processing unit 104 includes an integrated I/O (input/output) controller 214 supporting the attachment of one or more I/O devices, such as I/O device 216. I/O controller 214 may issue operations on local interconnect 114 and/or system interconnect 110 in response to requests by I/O device 216.
With reference now to
Cache array and directory 300 also includes a cache directory 302 of the contents of cache array 301. As in conventional set associative caches, memory locations in system memories 108 are mapped to particular congruence classes within cache arrays 301 utilizing predetermined index bits within the system memory (real) addresses. The particular cache lines stored within cache array 301 are recorded in cache directory 302, which contains one directory entry for each cache line in cache array 301. As understood by those skilled in the art, each directory entry in cache directory 302 comprises at least a tag field 304, which specifies the particular cache line stored in cache array 300 utilizing a tag portion of the corresponding real address, a LRU (Least Recently Used) field 308 indicating a replacement order for the cache line with respect to other cache lines in the same congruence class, and a state field 306, which indicates the coherency state of the cache line.
According to the present invention, coherency states that may be utilized in state field 306 to indicate state information may be defined by the well-known MESI coherency protocol or a variant thereof. To support caching of partial cache lines in cache array 301, cache directory 302 preferably indicates which granule(s) 307 are cached in the associated entry 305. For example, in one embodiment, each state field 306 includes a granule identifier (GI) 310 that indicates which granule(s) of the cache line identified by the tag in tag field 304 are held in the associated entry 305 of cache array 301. In at least some embodiments, when GI 310 indicates that fewer than all granules of a cache line are held in the associated entry 305, state field 306 preferably indicates a “Partial” coherency state that indicates that less than the complete cache line is held by cache array 301. In such embodiments, the Partial coherency state functions like the Shared coherency state of the MESI protocol, meaning that a full copy of the cache line must be obtained prior to modification of the contents of the cache line.
Referring now to
Prefetch requests 406 cause data from the memory subsystem to be fetched or retrieved into L1 cache 228 and/or L2 cache 230 preferably before the data is needed by LSU 228. The concept of prefetching recognizes that data accesses frequently exhibit spatial locality. Spatial locality suggests that the address of the next memory reference is likely to be near the address of recent memory references. A common manifestation of spatial locality is a sequential data stream, in which data from a block of memory is accessed in a monotonically increasing (or decreasing) sequence such that contiguous cache lines are referenced by at least one instruction. When DPFU 225 detects a sequential data stream (e.g., references to addresses in adjacent cache lines), it is reasonable to predict that future references will be made to addresses in cache lines that are adjacent to the current cache line (the cache line corresponding to currently executing memory references) following the same direction. Accordingly, DPFU 225 generates data prefetch requests 406 to retrieve one or more of these adjacent cache lines before the program actually requires them. As an example, if a program loads an element from a cache line n, and then loads an element from cache line n+1, DPFU 225 may prefetch cache some or all of cache lines n+2 and n+3, anticipating that the program will soon load from those cache lines also.
As further depicted in
With reference now to
As indicated, source code 502 includes one or more memory access constructs, such as load instruction 510 and store instruction 512. Load instruction 510 specifies data to be loaded from a memory hierarchy by a processor, and store instruction specifies data to be stored into the memory hierarchy by the processor. In response to processing source code 502, compiler 500 generates object code 504 containing a load instruction 514 corresponding to load instruction 510 and a store instruction 518 corresponding to store instruction 512. In the exemplary embodiment, load instruction 514 specifies at least one register identifier (e.g., rX) of an architected register into which data is to be loaded and one or more operands (e.g., Op1-OpN) from which the target memory address of the indicated load operation is to be computed. Generally, operands Op1-OpN can be identified, for example, utilizing register identifier(s), memory address(es), direct specification of an immediate operand, and/or an offset. As shown, load instruction 514 may further includes a hint 516 provided by compiler 500 in accordance with the process of
Referring now to
As shown, the process begins at block 520 and then proceeds to block to blocks 522-524, which depict compiler 500 scanning source code 502 until a load construct, such as load instruction 510, which causes data to be retrieved from the memory hierarchy is detected. In response to detection of the load construct, the process proceeds to block 530, which illustrates compiler 500 determining whether more than one granule 307 of the cache line of data targeted by the load construct is subsequently referenced by source code 502, for example, by one or more arithmetic instructions, logical instructions, or memory access instructions (e.g., store instruction 512). If so, the process passes directly to block 534, which is described below. If, however, compiler 500 determines that only one granule 307 of the target cache line of data is subsequently referenced, compiler 500 marks the load instruction 514 corresponding to the detected load construct with a partial cache line (PCL) hint 516, which indicates that only a partial cache line should be retrieved from memory and preferably further indicates which granule(s) 307 should be retrieved. Following block 532, the process proceeds to block 534, which illustrates a determination whether or not the scan of source code 502 is complete. If so, the process terminates at block 540. If not, the process returns to block 522, which has been described.
Although
With reference now to
Referring now to block 610, an execution unit of a processor, such as LSU 228, determines whether the load instruction is marked with a partial cache line (PCL) hint, such as PCL hint 516. If not, the process passes to block 612, which illustrates the execution unit processing the load instruction to generate a demand load request for a full cache line of data. Generating the demand load request includes, for example, computing the target memory address from which data is to be loaded. If, on the other hand, the execution unit of the processor determines at block 610 that the load instruction is marked with a PCL hint, the execution unit (e.g., LSU 228) generates a demand load request for a partial cache line of data in accordance with the PCL hint. Thus, the demand load request includes an indication of a partial cache line comprising one or more granules 307 that is to be fetched from the memory hierarchy. Following either block 612 or block 614, the process proceeds to block 616. Block 616 illustrates the execution unit (e.g., LSU 228) transmitting the demand load request to the memory hierarchy to retrieve a partial or full cache line of data from the memory hierarchy into the processor core 200. Thereafter, the process depicted in
In at least some embodiments, load hint information can be utilized to influence the amount of data retrieved from the memory hierarchy by data prefetching. With reference now to
If prefetch engine 402 determines to not allocate a new sequential data stream at block 636, the process shown in
Referring now to block 640, in response to a determination that the memory access address received at block 501 falls within an existing sequential data stream to which a stream register 408 has been allocated in PRQ 404, prefetch engine 402 updates the state of the stream register 408 allocated to the sequential data stream. For example, prefetch engine 402 may update address field 412 with the memory access address or modify depth field 416 or stride field 418. Following block 640, the process terminates at block 642.
With reference now to
The process depicted in
In the depicted embodiment, if prefetch engine 402 determines at block 654 that the state of partial field 420 indicates a full cache line, prefetch engine 402 generates a data prefetch request 406 for a full cache line at block 656. Alternatively, if prefetch engine 402 determines at block 524 that the state of partial field 420 indicates a partial cache line, prefetch engine 402 generates a data prefetch request 406 for a partial cache line (e.g., a single granule) at block 658. Following either block 656 or block 658, prefetch engine 402 transmits the data prefetch request 406 to the memory hierarchy (e.g., to L2 cache 230 or to IMCs 206) in order to prefetch the target partial or full cache line into cache memory. Thereafter, the process depicted in
As has been described, in at least some embodiments, a processor detects during execution of program code whether a load instruction within the program code is associated with a hint. In response to detecting that the load instruction is not associated with a hint, the processor retrieves a full cache line of data from the memory hierarchy into the processor in response to the load instruction. In response to detecting that the load instruction is associated with a hint, a processor retrieves a partial cache line of data into the processor from the memory hierarchy in response to the load instruction.
While the invention has been particularly shown as described with reference to a preferred embodiment, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention. For example, although aspects of the present invention have been described with respect to a computer system executing program code that directs the functions of the present invention, it should be understood that present invention may alternatively be implemented as a program product for use with a data processing system. Program code defining the functions of the present invention can be delivered to a data processing system via a variety of computer readable media, which include, without limitation, computer readable storage media (e.g., a computer memory, CD-ROM, a floppy diskette or hard disk drive), and communication media, such as digital and analog networks. It should be understood, therefore, that such computer readable media, when carrying or storing computer readable instructions that direct the functions of the present invention, represent alternative embodiments of the present invention.
This invention was made with United States Government support under Agreement No. HR0011-07-9-0002 awarded by DARPA. The Government has certain rights in the invention.
Number | Name | Date | Kind |
---|---|---|---|
4694395 | Young et al. | Sep 1987 | A |
5210842 | Sood | May 1993 | A |
5276850 | Sakaue | Jan 1994 | A |
5418916 | Hall et al. | May 1995 | A |
5555391 | De Subijana et al. | Sep 1996 | A |
5778438 | Merchant | Jul 1998 | A |
5802572 | Patel et al. | Sep 1998 | A |
5893147 | Deng | Apr 1999 | A |
5893151 | Merchant | Apr 1999 | A |
5926829 | Hagersten et al. | Jul 1999 | A |
6058456 | Arimilli et al. | May 2000 | A |
6122729 | Tran | Sep 2000 | A |
6131145 | Matsubara et al. | Oct 2000 | A |
6195735 | Krueger et al. | Feb 2001 | B1 |
6199107 | Dujari | Mar 2001 | B1 |
6216219 | Cai et al. | Apr 2001 | B1 |
6321306 | Arimilli et al. | Nov 2001 | B1 |
6345341 | Arimilli et al. | Feb 2002 | B1 |
6345342 | Arimilli et al. | Feb 2002 | B1 |
6353877 | Duncan et al. | Mar 2002 | B1 |
6356980 | Arimilli et al. | Mar 2002 | B1 |
6360297 | Arimilli et al. | Mar 2002 | B1 |
6446167 | Mayfield et al. | Sep 2002 | B1 |
6460115 | Kahle et al. | Oct 2002 | B1 |
6470427 | Arimilli et al. | Oct 2002 | B1 |
6499085 | Bogin et al. | Dec 2002 | B2 |
6535962 | Mayfield et al. | Mar 2003 | B1 |
6557080 | Burger et al. | Apr 2003 | B1 |
6564302 | Yagi et al. | May 2003 | B1 |
6571319 | Tremblay et al. | May 2003 | B2 |
6571322 | Arimilli et al. | May 2003 | B2 |
6615321 | Arimilli et al. | Sep 2003 | B2 |
6643744 | Cheng | Nov 2003 | B1 |
6647466 | Steely, Jr. | Nov 2003 | B2 |
6681296 | Liao et al. | Jan 2004 | B2 |
6704860 | Moore | Mar 2004 | B1 |
6763433 | Arimilli et al. | Jul 2004 | B1 |
6763434 | Arimilli et al. | Jul 2004 | B2 |
6772288 | Flake et al. | Aug 2004 | B1 |
6772295 | Spencer et al. | Aug 2004 | B2 |
6785772 | Venkumahanti et al. | Aug 2004 | B2 |
6823447 | Hay et al. | Nov 2004 | B2 |
6848071 | Chaudhry et al. | Jan 2005 | B2 |
6957305 | Ray et al. | Oct 2005 | B2 |
6971000 | Sinharoy et al. | Nov 2005 | B1 |
6978351 | Osborne et al. | Dec 2005 | B2 |
7028159 | Matsubara et al. | Apr 2006 | B2 |
7062609 | Trehus et al. | Jun 2006 | B1 |
7065548 | van Oldenborgh et al. | Jun 2006 | B2 |
7188215 | Hooker | Mar 2007 | B2 |
7234040 | Berg et al. | Jun 2007 | B2 |
7237068 | Wallin et al. | Jun 2007 | B2 |
7350032 | Stenstrom | Mar 2008 | B2 |
7380047 | Emma et al. | May 2008 | B2 |
7386701 | Mowry | Jun 2008 | B2 |
7441110 | Puzak et al. | Oct 2008 | B1 |
7533242 | Moll et al. | May 2009 | B1 |
7600143 | Neuman | Oct 2009 | B1 |
7958309 | Arimilli et al. | Jun 2011 | B2 |
8024527 | Arimilli et al. | Sep 2011 | B2 |
20020087801 | Bogin et al. | Jul 2002 | A1 |
20020087809 | Arimilli et al. | Jul 2002 | A1 |
20020092029 | Smith | Jul 2002 | A1 |
20020112124 | Arimilli et al. | Aug 2002 | A1 |
20020133674 | Martin et al. | Sep 2002 | A1 |
20020138698 | Kalla | Sep 2002 | A1 |
20020174253 | Hayter et al. | Nov 2002 | A1 |
20030046356 | Alvarez et al. | Mar 2003 | A1 |
20030084250 | Gaither et al. | May 2003 | A1 |
20030110117 | Saidenberg et al. | Jun 2003 | A1 |
20030159005 | Augsburg et al. | Aug 2003 | A1 |
20030177320 | Sah et al. | Sep 2003 | A1 |
20030208665 | Peir et al. | Nov 2003 | A1 |
20040037276 | Henderson et al. | Feb 2004 | A1 |
20040039879 | Gaither | Feb 2004 | A1 |
20040049615 | Liang | Mar 2004 | A1 |
20040117510 | Arimilli et al. | Jun 2004 | A1 |
20040205298 | Bearden et al. | Oct 2004 | A1 |
20040260879 | Barroso et al. | Dec 2004 | A1 |
20040268051 | Berg et al. | Dec 2004 | A1 |
20050027911 | Hayter et al. | Feb 2005 | A1 |
20050053057 | Deneroff et al. | Mar 2005 | A1 |
20050080994 | Cohen et al. | Apr 2005 | A1 |
20050204113 | Harper et al. | Sep 2005 | A1 |
20050210203 | Stenstrom | Sep 2005 | A1 |
20050240729 | Van Berkel et al. | Oct 2005 | A1 |
20050240736 | Shaw | Oct 2005 | A1 |
20060080511 | Hoover et al. | Apr 2006 | A1 |
20060085600 | Miyashita et al. | Apr 2006 | A1 |
20060173851 | Singh et al. | Aug 2006 | A1 |
20060174228 | Radhakrishnan et al. | Aug 2006 | A1 |
20060179239 | Fluhr et al. | Aug 2006 | A1 |
20060179254 | Clark et al. | Aug 2006 | A1 |
20060184607 | Le Leannec et al. | Aug 2006 | A1 |
20060184746 | Guthrie et al. | Aug 2006 | A1 |
20060184772 | Dooley et al. | Aug 2006 | A1 |
20060212648 | Cox et al. | Sep 2006 | A1 |
20060251092 | Matterne et al. | Nov 2006 | A1 |
20060259707 | Freytag | Nov 2006 | A1 |
20060265552 | Davis et al. | Nov 2006 | A1 |
20070038846 | Kadambi et al. | Feb 2007 | A1 |
20070050592 | Gschwind et al. | Mar 2007 | A1 |
20070058531 | Dierks et al. | Mar 2007 | A1 |
20070079073 | Rosenbluth et al. | Apr 2007 | A1 |
20070083716 | Rajamony et al. | Apr 2007 | A1 |
20070088919 | Shen et al. | Apr 2007 | A1 |
20070094450 | VanderWiel | Apr 2007 | A1 |
20070136374 | Guedalia | Jun 2007 | A1 |
20070168619 | Hutton et al. | Jul 2007 | A1 |
20070214335 | Bellows et al. | Sep 2007 | A1 |
20070220208 | Nomura et al. | Sep 2007 | A1 |
20070294264 | Bayardo, Jr. et al. | Dec 2007 | A1 |
20080056051 | Mayer et al. | Mar 2008 | A1 |
20080086602 | Guthrie et al. | Apr 2008 | A1 |
20080091921 | Abuaiadh et al. | Apr 2008 | A1 |
20080104330 | Deshpande | May 2008 | A1 |
20080183972 | Dieffenderfer | Jul 2008 | A1 |
20080244185 | O'Krafka et al. | Oct 2008 | A1 |
20080256303 | Croxford et al. | Oct 2008 | A1 |
20090198865 | Arimilli et al. | Aug 2009 | A1 |
20090198912 | Arimilli et al. | Aug 2009 | A1 |
20090198914 | Arimilli et al. | Aug 2009 | A1 |
20090198960 | Arimilli et al. | Aug 2009 | A1 |
Number | Date | Country | |
---|---|---|---|
20090198903 A1 | Aug 2009 | US |