Patent Application
20080005516
The present invention is illustrated by way of example and is not limited by the figures of the accompanying drawings, in which like references indicate similar elements, and in which:
Embodiments of a method, apparatus, and system to manage memory power through high-speed intra-memory data transfer and dynamic memory address remapping are disclosed. In the following description, numerous specific details are set forth. However, it is understood that embodiments may be practiced without these specific details. In other instances, well-known elements, specifications, and protocols have not been discussed in detail in order to avoid obscuring the present invention.
Processor-memory interconnect 100 provides the central processor 102 and other devices access to the system memory 104. A system memory controller 106 controls access to the system memory 104. In one embodiment, the system memory controller is located within the north bridge 108 of a chipset 106 that is coupled to processor-memory interconnect 100. In another embodiment, a system memory controller is located on the same chip as central processor 102 (not shown). Information, instructions, and other data may be stored in system memory 104 for use by central processor 102 as well as many other potential devices. I/O devices, such as I/O devices 114 and 118, are coupled to the south bridge 110 of the chipset 106 through one or more I/O interconnects 116 and 120.
In one embodiment, the computer system in
Furthermore, in this embodiment, power management interconnects (208 and 210) couple the memory controller 200 to memory power management modules 212 and 214. Memory power management module 212 is coupled to DIMM 0 (202) and memory power management module 214 is coupled to DIMM 1. Both memory power management modules (212 and 214) are also connected to a voltage supply (Vcc) that supplies the necessary power to both DIMMs. Memory power management modules 212 and 214 supply each DIMM with the needed power to operate. These modules are also able to limit the power supplied to the DIMMs in lower power states as well as cut off the supply of power to the DIMMs altogether when one needs to be powered down. Thus, in different embodiments, the modules comprise power circuitry, logic, have stored power management software, or have a combination of all three.
In different embodiments, the modules have the ability to control power individually to each DIMM, or to each rank within the DIMM, or to each device located on the DIMM, or to a portion of each device on each DIMM. Each individually powered portion of the entire memory subsystem will be referred to as a node. For a node to be independent, there must be an ability to isolate the power supplied to that node apart from any other portion of the memory subsystem. Thus, in different embodiments, the granularity of the power management control is dependent upon the size of each node. In certain systems, a node may be an entire DIMM, in other systems, a node may be only a portion of an individual device on a DIMM.
In the illustrated embodiment of
To turn off the power to a node, the memory controller 200 sends one or more power management controls to either memory power management module 212 or 214. Once power is turned off to a DIMM node, the data that is stored in the memory space within the DIMM is invalid, thus it is imperative to move any valid data out of the memory space of the node designated for having the power shut off. In a standard environment, the central processor (102 in
In one embodiment, a table of power managed nodes is maintained by the memory controller 200. Each entry of the table has the starting physical address of the node and the size of the node. In this embodiment, an operating system (OS) 216 (or virtual machine manager (VMM) in another embodiment) access to this table and is able to determine whether one or more nodes can be powered down by checking to see if there is enough empty space available in memory outside of the node targeted for power down to span one or more nodes. If there is enough free space available, the OS 216 may initiate a node power down sequence.
In one embodiment, the free pages within memory that, when combined, are able to span one or more memory nodes are scattered throughout memory. Thus, in one embodiment, the node power down sequence begins with the OS 216 gathering free memory pages scattered throughout memory to create one or more physically contiguous blocks equal to at least the size of the node that is to be powered down. The OS 216 locks this contiguous block of locked memory. The OS 216 aligns this locked block of memory at the starting physical address of the target node and informs the memory controller when completed. The OS 216 actually resides within the memory in the system, block 216 is just a representation for ease of explanation.
Once the memory controller is informed by the OS 216 that the block has been created and aligned at the starting physical address of the target node, the memory controller 200 moves all memory pages from the target node to other nodes in the table through high-speed intra-memory data transfer. This is required because although the free memory blocks are contiguous in physical address space, at the actual DIMM level, the blocks are dispersed across banks, ranks, and DIMMs due to address interleaving. The memory controller 200 then aligns the block of locked memory at the boundary of the target node. The memory controller 200 then remaps all addresses by modifying the address translation to reflect the new configuration of operating nodes (i.e. eliminating address translations that would map into the targeted node in physical memory). The target node is then powered off. In one embodiment, the memory controller 200 sends a power down signal to the memory power management module that manages the power delivery to the targeted node.
In one embodiment, the memory controller's movement of all memory pages between the target node and non-target nodes occurs without the knowledge of the OS 216. The OS need not be aware of the physical layout of memory due to interleaved addressing. Rather, the memory controller can reposition and remap memory directly between nodes in the one or more DIMMs with high-speed intra-DIMM and inter-DIMM memory transfers that never are seen by any device or operating system beyond the memory controller. Thus, as far as the OS 216 is concerned, the contiguous block of memory that it created is actually contiguous at the DIMM level as well. The memory controller is the only device that requires knowledge of the interleaving between banks, ranks, and full DIMMs. Furthermore, once the memory pages have been relocated to the non-targeted nodes, the memory controller dynamically modifies the address mapping scheme so that the targeted node is no longer accessible by OS 216, but the non-targeted nodes remain fully accessible and operational. Below,
In this embodiment, an operating system determines that there is at least 1 GB of free memory. The operating system coalesces pages and creates a contiguous 1 GB block of locked memory aligned at the 1 GB boundary (the upper half of memory). Although, from the operating system's point of view, the 1 GB block of locked memory is contiguous, in actual physical memory the 1 GB block spans both DIMMs due to interleaving. In the dual channel mode (symmetric addressing), the block spans both ranks of both DIMMs, which is illustrated in
In one embodiment, a request is sent to the memory controller to power off the target node. In one embodiment, the operating system requests this because it knows there is enough free memory to power off a node. In response to the request, the memory controller relocates all blocks of memory currently in use (white blocks in
The memory controller then remaps addresses by switching the address translation scheme. Additionally, the memory controller switches addressing mode to ‘asymmetric’ to allow for blocks of memory to be independently populated instead of having memory interleaved between DIMMs. Then the memory controller is able to power down DIMM 1 (which is also memory channel 1). Though address remapping has described with reference to only one addressing mode here (symmetric-enhanced), in different embodiments, the remapping scheme may be extended to cover other modes such as symmetric-non-enhanced, asymmetric-enhanced and asymmetric-non-enhanced address mapping.
Furthermore, in one embodiment, the operating system and memory controller can reverse the process. In this embodiment, the operating system may make a determination that the system needs the powered down node to power back up and start normal operation again. Thus, the node is repowered and the process reverses where the memory controller remaps the memory to a symmetric enhanced mode where the addresses are once again interleaved across nodes (banks, ranks, DIMMs, etc.). The memory controller also completes the same high-speed intra or inter-memory transfers to return the data from the modified storage locations to its original locations on the repowered node.
Otherwise, if it is determined that there is enough space in the non-designated space, then processing logic transfers the data from a first memory location in the designated space to a first memory location in the new, non-designated space (processing block 708). In one embodiment, a memory controller transfers the memory locations to free up space in the node to be powered down. In one embodiment, the memory controller is aware of the interleaved addresses and may transfer data between banks, ranks, or DIMMs to physically free up one node within memory at the physical device level. Then processing logic determines whether the designated portion of memory has completed its transfer to the non-designated portion of memory (processing block 710). If the transfer is not complete, processing logic transfers data from the next memory location in the designated space to the next memory location in the new, non-designated space (processing block 712). Block 712 repeats until all data in memory locations within the designated space have been transferred to memory locations in the non-designated space.
Finally, once the transfers have completed, then processing logic powers down the designated portion of memory (processing block 714). Then processing logic changes the address translation scheme to reflect the new locations of data previously located in the designated portion of memory (processing block 716) and the process is finished. In one embodiment, the address translation changes are reflected in
Thus, embodiments of a method, apparatus, and system to manage memory power through high-speed intra-memory data transfer and dynamic memory address remapping are disclosed. These embodiments have been described with reference to specific exemplary embodiments thereof. It will be evident to persons having the benefit of this disclosure that various modifications and changes may be made to these embodiments without departing from the broader spirit and scope of the embodiments described herein. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.
