Patent Application
20180032429
The field of invention pertains generally to computing systems, and, more specifically, to techniques to allocate regions of a multi-level, multi-technology system memory to appropriate memory access initiators.
A pertinent issue in many computer systems is the use of system memory. Here, as is understood in the art, a computing system operates by executing program code stored in system memory and reading/writing data that the program code operates on from/to system memory. As such, system memory is heavily utilized with many program code and data reads as well as many data writes over the course of the computing system's operation. Finding ways to improve system memory accessing performance is therefore a motivation of computing system engineers.
Currently, the Advanced Configuration and Power Interface (ACPI) provides for a System Locality Information Table (SLIT) that describes distance between nodes in a multi-processor computer system, and, a Static Resource Affinity Table (SRAT) that associates each processor with a block of memory. The SLIT and SRAT are ideally used to couple processors with appropriately distanced memory banks so that desired performance levels for the applications that run on the processors can be achieved.
However, new system memory advances are introducing not only different system memory technologies but also different system memory architectures into a same comprehensive system memory. The current SLIT and SRAT tables do not take into account these specific newer system memory features.
A better understanding of the present invention can be obtained from the following detailed description in conjunction with the following drawings, in which:
One of the ways to improve system memory performance is to have a multi-level system memory.
The use of cache memories for computing systems is well-known. In the case where near memory 113 is used as a cache, near memory 113 is used to store an additional copy of those data items in far memory 114 that are expected to be more frequently called upon by the computing system. By storing the more frequently called upon items in near memory 113, the system memory 112 will be observed as faster because the system will often read items that are being stored in faster near memory 113. For an implementation using a write-back technique, the copy of data items in near memory 113 may contain data that has been updated by the CPU, and is thus more up-to-date than the data in far memory 114. The process of writing back ‘dirty’ cache entries to far memory 114 ensures that such changes are not lost.
According to various embodiments, near memory cache 113 has lower access times than the lower tiered far memory 114 region. For example, the near memory 113 may exhibit reduced access times by having a faster clock speed than the far memory 114. Here, the near memory 113 may be a faster (e.g., lower access time), volatile system memory technology (e.g., high performance dynamic random access memory (DRAM)) and/or SRAM memory cells co-located with the memory controller 116. By contrast, far memory 114 may be either a volatile memory technology implemented with a slower clock speed (e.g., a DRAM component that receives a slower clock) or, e.g., a non volatile memory technology that is slower (e.g., longer access time) than volatile/DRAM memory or whatever technology is used for near memory.
For example, far memory 114 may be comprised of an emerging non volatile random access memory technology such as, to name a few possibilities, a phase change based memory, a three dimensional crosspoint memory, “write-in-place” non volatile main memory devices, memory devices that use chalcogenide, multiple level flash memory, multi-threshold level flash memory, a ferro-electric based memory (e.g., FRAM), a magnetic based memory (e.g., MRAM), a spin transfer torque based memory (e.g., STT-RAM), a resistor based memory (e.g., ReRAM), a Memristor based memory, universal memory, Ge2Sb2Te5 memory, programmable metallization cell memory, amorphous cell memory, Ovshinsky memory, etc. Any of these technologies may be byte addressable so as to be implemented as a main/system memory in a computing system.
Emerging non volatile random access memory technologies typically have some combination of the following: 1) higher storage densities than DRAM (e.g., by being constructed in three-dimensional (3D) circuit structures (e.g., a crosspoint 3D circuit structure)); 2) lower power consumption densities than DRAM (e.g., because they do not need refreshing); and/or, 3) access latency that is slower than DRAM yet still faster than traditional non-volatile memory technologies such as FLASH. The latter characteristic in particular permits various emerging non volatile memory technologies to be used in a main system memory role rather than a traditional mass storage role (which is the traditional architectural location of non volatile storage).
Regardless of whether far memory 114 is composed of a volatile or non volatile memory technology, in various embodiments far memory 114 acts as a true system memory in that it supports finer grained data accesses (e.g., cache lines) rather than larger based “block” or “sector” accesses associated with traditional, non volatile mass storage (e.g., solid state drive (SSD), hard disk drive (HDD)), and/or, otherwise acts as an (e.g., byte) addressable memory that the program code being executed by processor(s) of the CPU operate out of.
Because near memory 113 acts as a cache, near memory 113 may not have formal addressing space. Rather, in some cases, far memory 114 defines the individually addressable memory space of the computing system's main memory. In various embodiments near memory 113 acts as a cache for far memory 114 rather than acting a last level CPU cache. Generally, a CPU cache is optimized for servicing CPU transactions, and will add significant penalties (such as cache snoop overhead and cache eviction flows in the case of cache hit) to other system memory users such as Direct Memory Access (DMA)-capable devices in a Peripheral Control Hub. By contrast, a memory side cache is designed to handle, e.g., all accesses directed to system memory, irrespective of whether they arrive from the CPU, from the Peripheral Control Hub, or from some other device such as display controller.
In various embodiments, system memory may be implemented with dual in-line memory module (DIMM) cards where a single DIMM card has both volatile (e.g., DRAM) and (e.g., emerging) non volatile memory semiconductor chips disposed in it. The DRAM chips effectively act as an on board cache for the non volatile memory chips on the DIMM card. Ideally, the more frequently accessed cache lines of any particular DIMM card will be accessed from that DIMM card's DRAM chips rather than its non volatile memory chips. Given that multiple DIMM cards may be plugged into a working computing system and each DIMM card is only given a section of the system memory addresses made available to the processing cores 117 of the semiconductor chip that the DIMM cards are coupled to, the DRAM chips are acting as a cache for the non volatile memory that they share a DIMM card with rather than as a last level CPU cache.
In other configurations DIMM cards having only DRAM chips may be plugged into a same system memory channel (e.g., a DDR channel) with DIMM cards having only non volatile system memory chips. Ideally, the more frequently used cache lines of the channel are in the DRAM DIMM cards rather than the non volatile memory DIMM cards. Thus, again, because there are typically multiple memory channels coupled to a same semiconductor chip having multiple processing cores, the DRAM chips are acting as a cache for the non volatile memory chips that they share a same channel with rather than as a last level CPU cache.
In yet other possible configurations or implementations, a DRAM device on a DIMM card can act as a memory side cache for a non volatile memory chip that resides on a different DIMM and is plugged into a different channel than the DIMM having the DRAM device. Although the DRAM device may potentially service the entire system memory address space, entries into the DRAM device are based in part from reads performed on the non volatile memory devices and not just evictions from the last level CPU cache. As such the DRAM device can still be characterized as a memory side cache.
In another possible configuration, a memory device such as a DRAM device functioning as near memory 113 may be assembled together with the memory controller 116 and processing cores 117 onto a single semiconductor device or within a same semiconductor package. Far memory 114 may be formed by other devices, such as slower DRAM or non-volatile memory and may be attached to, or integrated in that device.
In still other embodiments, at least some portion of near memory 113 has its own system address space apart from the system addresses that have been assigned to far memory 114 locations. In this case, the portion of near memory 113 that has been allocated its own system memory address space acts, e.g., as a higher priority level of system memory (because it is faster than far memory) rather than as a memory side cache. In other or combined embodiments, some portion of near memory 113 may also act as a last level CPU cache.
In various embodiments when at least a portion of near memory 113 acts as a memory side cache for far memory 114, the memory controller 116 and/or near memory 113 may include local cache information (hereafter referred to as “Metadata”) 120 so that the memory controller 116 can determine whether a cache hit or cache miss has occurred in near memory 113 for any incoming memory request.
In the case of an incoming write request, if there is a cache hit, the memory controller 116 writes the data (e.g., a 64-byte CPU cache line or portion thereof) associated with the request directly over the cached version in near memory 113. Likewise, in the case of a cache miss, in an embodiment, the memory controller 116 also writes the data associated with the request into near memory 113 which may cause the eviction from near memory 113 of another cache line that was previously occupying the near memory 113 location where the new data is written to. However, if the evicted cache line is “dirty” (which means it contains the most recent or up-to-date data for its corresponding system memory address), the evicted cache line will be written back to far memory 114 to preserve its data content.
In the case of an incoming read request, if there is a cache hit, the memory controller 116 responds to the request by reading the version of the cache line from near memory 113 and providing it to the requestor. By contrast, if there is a cache miss, the memory controller 116 reads the requested cache line from far memory 114 and not only provides the cache line to the requestor (e.g., a CPU) but also writes another copy of the cache line into near memory 113. In various embodiments, the amount of data requested from far memory 114 and the amount of data written to near memory 113 will be larger than that requested by the incoming read request. Using a larger data size from far memory or to near memory increases the probability of a cache hit for a subsequent transaction to a nearby memory location.
In general, cache lines may be written to and/or read from near memory and/or far memory at different levels of granularity (e.g., writes and/or reads only occur at cache line granularity (and, e.g., byte addressability for writes/or reads is handled internally within the memory controller), byte granularity (e.g., true byte addressability in which the memory controller writes and/or reads only an identified one or more bytes within a cache line), or granularities in between.) Additionally, note that the size of the cache line maintained within near memory and/or far memory may be larger than the cache line size maintained by CPU level caches.
Different types of near memory caching implementation possibilities exist. Examples include direct mapped, set associative, fully associative. Depending on implementation, the ratio of near memory cache slots to far memory addresses that map to the near memory cache slots may be configurable or fixed.
Note that a wide range of different systems can loosely or directly fit the exemplary architecture of
The multi-processor system of
Each platform 201_1, 201_2 also includes a “slice” of system memory 208_1, 208_2 that is coupled to a memory control hub 204_1, 204_2 within its respective platform's processor 202_1, 202_2. As is known in the art, the storage space of system memory is defined by its address space. Here, as a simple example, system memory component 208_1 may be allocated a first range of system memory addresses and system memory component 208_2 is allocated a second, different range of system memory addresses.
With the understanding that applications running on any CPU core in the system can potentially refer to any system memory address, an application that is running on a CPU core within processor 202_1 may not only refer to instructions and/or data in system memory component 208_1 but may also refer to instructions and/or data in system memory component 208_2. In the case of the latter, a system memory request is sent from processor 202_1 to processor 202_2 over link 212. The memory control hub 204_2 of processor 201_1 services the request (e.g., by reading/writing from/to the system memory address within system memory slice 208_2). In the case of a read request, the instruction/data to be returned is sent from processor 202_2 to processor 202_1 over communication link 212.
As observed in
As just one possible physical implementation of this particular architecture, for instance, first level memory 209_1, 209_2 may be implemented as DRAM devices that are stacked on top of or otherwise integrated in the same semiconductor chip package as their respective processor 202_1, 202_2.
By contrast, second level memory 210_1, 210_2 may reside outside the semiconductor chip of their respective processor 202_1, 202_2. For example, second level memory 201_1, 202_2 may be implemented as DRAM devices disposed on DIMM cards that plug into memory channels that are coupled to their respective processor's memory control hub 204_1, 204_2. Here, the DRAM devices may be given their own system memory address space and therefore act as a second priority region of system memory beneath levels 209_1, 209_2. In this case, the DRAM devices of the second level 210_1, 210_2 being located outside the package of their respective processor 202_1, 202_2 are apt to have longer latencies and will therefore be a slower level of system memory than the first level 209_1, 209_2.
Alternatively, DRAM devices within the second level 210_1, 210_2 may behave as a memory side cache for their respective lower non volatile system memory level 211_1, 211_2. As a further alternative possibility, some portion of the DRAM devices in the second level 210_1, 210_2 may be allocated their own unique system memory address space while another portion of the memory devices in the second level 210_1, 210_2 may be configured to behave as a memory side cache for the lower non volatile system memory level 211_1, 211_2.
In general, the latency of a system memory component from the perspective of a requestor that issues read and/or write requests to the system memory component (such as an application or operating system instance that is executing on a processing core) is a function of the physical distance between the requestor and the memory component and the technology of the physical memory component.
Here,
As such, referring to column 301, note that all system memory components 209_1, 210_1, 211_11, 211_12 that are integrated with the platform 301_1 having processor 302_1 are observed to be faster for an application that executes on processor 302_1 than any of the system memory components 309_2, 310_2, 311_21, 311_22 that are integrated with the other platform 301_2. Likewise, referring to column 302, note that all system memory components 209_2, 210_2, 211_21, 211_22 that are integrated with the platform 301_2 having processor 302_2 are observed to be faster for an application that executes on processor 302_2 than any of the system memory components 309_1, 310_1, 311_11, 311_12 that are integrated with the other platform 301_2.
Here, the observed decrease in performance of a system memory component from an off platform application is largely a consequence of link 212. In various embodiments link 212 may correspond to a large physical distance which significantly adds to the propagation delay time of issued requests. Even in the case, however, where the physical distance associated with link 212 is not appreciably large there may nevertheless exist on average noticeable queuing delays associated with placing traffic on the link 212 or receiving traffic from the link 212. Thus, as a general observation, local system memory components will tend to be faster from the perspective of a requestor than more remote system memory components.
This same general trend is also observable with the observed performance rankings within a same platform. That is, within both platforms, the internal DRAM level 209 is higher than the external DRAM level 210. That is, recall that the internal DRAM 209 was integrated in a same semiconductor chip package as its processor 202 whereas the external DRAM 210 was physically located outside the package. Because reaching the external DRAM 210 requires signaling that traverses a longer physical distance, the internal DRAM 210 will exhibit smaller access times than an external DRAM device on the same platform.
With respect to technology, note that the non volatile memory components 211 are slower than the DRAM memory components 209, 210, and, moreover, that with respect to non volatile memory components 211_1, 211_2, write operations can be noticeably slower than read operations. For example, as depicted in
Although only exemplary, note that reads for an NVRAM technology that does not have a memory side cache (e.g., as represented by box 211_12_R) can be faster than writes to an NVRAM technology having a memory side cache (e.g., as represented by box 211_11_W).
Unlike the NVRAM technology components of
Apart from generally representing latency, a diagram like
Thus, a system can potentially be characterized with two sets of diagrams that demonstrate the general trends observed in
With respect to architecture, note that a non volatile memory component that also has a memory side cache 211_X1 will be comparatively faster than a non volatile memory component that does not have a memory side cache 211_X2_X. That is, reads of a non volatile memory component having a memory side cache will be faster than reads of a non volatile memory component that does not have a memory side cache. Likewise, writes to a non volatile memory component having a memory side cache will be faster than writes to a non volatile memory component that does not have a memory side cache.
Here,
Another portion of the non volatile system memory 211, labeled in
Here, as just an example, level 311 provides twice as much near memory cache space per unit of far memory storage space than does level 312. This arrangement can be achieved, as just one example, by having the DRAM DIMMs provide near memory service only to those non volatile memory DIMMs that are plugged into the same memory channel. By having a first memory channel configured with more DRAM DIMMs than a second memory channel where both memory channels have the same number of non-volatile memory DIMMs (or, alternatively, both channels have the same number of DRAM DIMMs but different numbers of non volatile memory DIMMs), different ratios of near memory cache space to far memory space can be effected. Because level 312 has less normalized cache space than level 311, level 312 will be observed as being slower than level 311 and is therefore placed beneath it in the visual hierarchy of
A second architectural feature is that different near memory cache eviction policies may be instantiated for either of the memory levels 311, 312 of
For example, as depicted in
Likewise, the system memory address space of the slice of system memory 208_2 associated with the first platform 201_2 corresponds to a fifth system address range SAR1 that is allocated to the internal DRAM 209_2 of the second platform 201_2, a sixth system memory address range SAR3 that is allocated to the portion of the external DRAM 210_2 that is allocated unique system memory address space, a seventh system memory address range SAR5 that is allocated to the portion of non volatile memory 211_21 that receives near memory cache service and an eighth system memory address range SAR7 that is allocated to the portion of non volatile memory 211_22 that does not receive near memory cache service.
As observed in
For ease of drawing, neither of
With all the different levels/partitions that the system memory can be broken down into, and all the different performance dependencies (e.g., reads vs. writes) different software applications can be assigned to operate out of the different system memory levels/partitions in accordance with their actual requirements or objectives. For instance, if a first application (e.g., a video streaming application) would better serve its objective by executing faster, then, the first application can be allocated a memory address space that corresponds to a lower latency read time and higher read bandwidth system memory portion, such as the internal and/or external DRAM portions 209, 210 of the same platform that the application executes from (i.e., the higher ranked memory components in
By contrast, if a second application (e.g., an archival data storage application) does not necessarily need to operate with the fastest of speeds, the second application can be allocated a memory address space that corresponds to a higher latency read or write time and lower read or write bandwidth system memory portion, such as one of the non volatile memory portions of its local platform or even a remote platform.
Here, the applications that run on platform 201_1 can, e.g., be ranked in terms of desired performance level.
As such, application X1 is allocated memory address ranges SAR0 and/or SAR2 to cause application X1 to execute out of either or both of the memory components 209_1, 210_1 that have the lowest latency for an application that runs on platform 201_1. By being configured to operate out of the fastest memory available to application X1, application X1 should demonstrate higher performance.
By contrast, application Y1 is allocated memory address ranges SAR4 and/or SAR6 to cause application Y1 to execute out of either or both of the memory components 211_11, 211_12 that have modest latency for an application that runs on platform 201_1. By being configured to operate out of a modest latency memory that is available to application Y1, application Y1 should demonstrate medium performance.
Further still, application Z1 is allocated memory address ranges SAR5 and/or SAR7 to cause application Z1 to execute off platform out of either or both of memory components 209_2, 210_2 which not only reside on platform 201_2 but are also the higher latency memories on platform 201_2. By being configured to operate out of the slowest memory available to application Z1, application Z1 should demonstrate lowest performance.
An analogous configuration is also observed in
Here, an application's execution from a particular platform may actually be implemented by executing its program code on a particular processing core of the platform. As such, the application's software thread and its associated register space is physically realized on the core even though its memory accesses may be directed to some other platform. Multi-threaded applications can execute on a same core, different cores of a same platform or possibly even different cores of different platforms.
In order to configure a computing system such that its applications will execute out of an appropriate one or more levels of system memory, an operating system instance and/or virtual machine monitor will need some visibility into the different system memory levels and their latency relationship with the different processing cores of the system.
It is pertinent to point out, however, that the above configuration examples could be enhanced to contemplate difference speed metrics (such as latency v. bandwidth) or different read and write latencies/bandwidths. Here, system configuration information could contemplates different latencies and bandwidths for both read and writes for the various memory components and configure the various applications to operate out of certain ones of the different memory components whose characteristics were a good fit from a behavior/performance perspective.
Note that the list 501 and overall root complex may take the form of a directory rather than just a collection of lists. For example, each platform entry in the MAI list 501 may act as a higher level directory node that further lists its constituent CPU cores within/beneath it.
Further still, any kind of entity that issues a request to system memory can have its entry or node in the MAI list with further sub nodes listing its constituent parts that can individually issue system memory requests. For example, an I/O control hub node can further list its various PCIe interfaces as sub nodes. Each of the various PCIe interfaces can list the corresponding devices that connected to it as further sub-nodes of the PCIe interface subnodes. Similar structures can be composed for mass storage devices (e.g., disk drives, solid state drives).
Here, any component that can issue a read or write request to system memory (e.g. a network interface, a mass storage device, a CPU core) can be given MAI status and assigned a region of system memory space. As discussed at length above, a CPU core is assigned system memory space for its software to execute out of. Thus, not only may a CPU core be recognized as an MAI entry within the list, but also, e.g., each application that is configured to run on a particular CPU core may be given MAI status and listed in the MAI list 501.
By contrast, I/O devices may or may not execute software but nevertheless may issue system memory read/write requests. For instance, a network interface may stream the data it receives from a network into system memory and/or receive from system memory the data it is streaming into a network. Again, the notion that higher performance components can be allocated higher performance levels of system memory still applies. For example, a first network interface that is coupled to a high bandwidth link may be coupled to a higher performance system memory level while a second network interface that is coupled to a low bandwidth link may be coupled to a lower performance system memory level. An analogous arrangement can be applied with respect to faster performance mass storage devices and slower performance mass storage devices.
Thus, each MAI entry in the MAI list 501 may include some further meta data information that describes or otherwise indicates its performance level so that an operating system instance and/or virtual machine monitor can comprehend the appropriate level of system memory performance that it will need. CPU core entries and/or the applications that run on them can include similar meta data.
A second list 502 lists the different memory access (“MA”) regions or domains within the system memory that can be separately identified. The MAI list 502 of
Again, each node in the MA list 502, besides identifying its specific system memory address range, may include some meta data that describes attributes of itself such as technology type (e.g., DRAM/non volatile), associated access speed and architecture (e.g., 2LM with a specific amount of near memory cache space and cache eviction policy). An OS instance or virtual machine monitor can therefore refer to this information when attempting to configure a certain memory access initiator with a specific memory domain.
The root complex of
Here, a DRAM component having its own address space may present same read and write latency metadata whereas any of the NVRAM components may present substantially different read and write latency data.
Further still, the performance list of
Returning to
The root complex approach described just above may be written to be compatible with any of a number of system and/or component configuration specifications (e.g., Advanced Configuration and Power Interface (ACPI), NVDIMM Firmware Interface Table (NFIT)). Here, again, the root table may be stored in non volatile BIOS and used by configuration software during a configuration operation (e.g., upon boot-up, in response to component addition/removal, etc.). Conceivably, current versions of SLIT and/or SRAT information (discussed in the background) could be expanded to include the attribute features described just above with respect to the root complex of
As observed in
An applications processor or multi-core processor 750 may include one or more general purpose processing cores 715 within its CPU 701, one or more graphical processing units 716, a memory management function 717 (e.g., a memory controller) and an I/O control function 718. The general purpose processing cores 715 typically execute the operating system and application software of the computing system. The graphics processing units 716 typically execute graphics intensive functions to, e.g., generate graphics information that is presented on the display 703. The memory control function 717 interfaces with the system memory 702. The system memory 702 may be a multi-level system memory and the BIOS of the system may contain attributes of the system memory as discussed at length above so that configuration software can configure certain memory access initiators with specific components of the system memory that have an appropriate latency from the perspective of the initiators.
Each of the touchscreen display 703, the communication interfaces 704-707, the GPS interface 708, the sensors 709, the camera 710, and the speaker/microphone codec 713, 714 all can be viewed as various forms of I/O (input and/or output) relative to the overall computing system including, where appropriate, an integrated peripheral device as well (e.g., the camera 710). Depending on implementation, various ones of these I/O components may be integrated on the applications processor/multi-core processor 750 or may be located off the die or outside the package of the applications processor/multi-core processor 750.
Embodiments of the invention may include various processes as set forth above. The processes may be embodied in machine-executable instructions. The instructions can be used to cause a general-purpose or special-purpose processor to perform certain processes. Alternatively, these processes may be performed by specific hardware components that contain hardwired logic for performing the processes, or by any combination of software or instruction programmed computer components or custom hardware components, such as application specific integrated circuits (ASIC), programmable logic devices (PLD), digital signal processors (DSP), or field programmable gate array (FPGA).
Elements of the present invention may also be provided as a machine-readable medium for storing the machine-executable instructions. The machine-readable medium may include, but is not limited to, floppy diskettes, optical disks, CD-ROMs, and magneto-optical disks, FLASH memory, ROMs, RAMs, EPROMs, EEPROMs, magnetic or optical cards, propagation media or other type of media/machine-readable medium suitable for storing electronic instructions. For example, the present invention may be downloaded as a computer program which may be transferred from a remote computer (e.g., a server) to a requesting computer (e.g., a client) by way of data signals embodied in a carrier wave or other propagation medium via a communication link (e.g., a modem or network connection).
In the foregoing specification, the invention has been described with reference to specific exemplary embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention as set forth in the appended claims. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.
