A virtual memory mechanism is an addressing scheme implemented in hardware or software or both that allows non-contiguous memory to be addressed as if it is contiguous or vice versa. A virtual memory mechanism maps virtual memory pages to physical memory (for example, in random access memory (RAM)) frames. Virtual memory tables are used to record the mapping of virtual memory pages to physical memory frames.
When the operating system (particularly, the kernel) encounters a shortage in physical memory, its virtual memory manager picks a number of virtual memory pages to be swapped out to the swap area in the stable storage (for example, hard disk), thus releasing the physical memory frames occupied by these pages. The decision of which pages to swap out is made by the swapping algorithm and, for example, may be based on the Least Recently Used (LRU)/Least Frequently Used (LFU) heuristic.
When a page is chosen to be swapped out, the virtual memory manager protects this page by setting a special flag in the virtual memory table, and the contents of the page may then be swapped out to slower stable storage. Upon an application's access to any address residing on this virtual memory page, the hardware memory management unit triggers a page fault exception. The virtual memory manager handles the page fault exception by pausing the application process and paging in the memory page from the stable storage. Upon completion of the page transfer, the virtual memory manager resumes the application which may then successfully complete its access to the virtual memory page.
A bidirectional communication pathway is established between the virtual memory manager (and associated paging system) in the operating system and the memory manager of programs running on the device, allowing for interaction, collaboration and increased functionality.
Current swapping algorithms typically use the LRU/LFU heuristic looking at frequency of use. By also taking into account the size and quantity of live objects on the pages from information provided by the memory manager in determining what to swap, overall performance can be affected.
Rather than swapping entire pages out to stable memory, allocated regions of the page (denoted “live objects”) which do not necessarily occupy a full page, may be compacted. Compacting may involve copying a live object which is smaller than a page into a single page with other live objects. This compaction keeps live objects in physical memory rather than swapping them to slower stable memory. By keeping live objects in physical memory, faster access times and more effective use of virtual memory resources are possible. To further maximize use of scarce physical memory, compression can also be used.
Even where stable memory is not present, a virtual increase in the memory space is possible. Live objects may be compacted or compressed (or both) in memory, thus virtually increasing the available memory space
Additionally, to alleviate the costs involved with un-compaction, application pointers may be updated to refer to the new addresses of the live objects in the compacted pages. The pointer updates may be performed by a runtime system, or by a similar process. In the presence of a garbage collector, the pointer updates may conveniently be updated during garbage collection.
This summary is provided to introduce the subject matter of virtual memory compaction and compression using collaboration between a virtual memory manager and a memory manager, which is described below in the Detailed Description. This summary is not intended to identify essential features of the claimed subject matter, nor is it intended for use in determining the scope of the claimed subject matter.
This disclosure describes a collaboration engine to facilitate communication between the memory manager and the virtual memory system in a computer resulting in more effective use of memory resources and improved performance. Additionally, such collaboration facilitates improved compression and compaction of pages in virtual memory. Compaction and compression as used herein also include their counterpart functions of un-compaction and de-compression unless specifically noted.
We now turn to the collaboration engine 100 inside computer system 200 in more depth. The memory manager 102 may contain, for example, a garbage collector module 202 and a memory allocator module 204. The virtual memory manager 104 may contain a compression module 206, a compaction module 208, and a paging module 210. The memory manager 102 is in bidirectional communication 110 with the virtual memory manager 104. The paging module 210 is in communication with the virtual memory 106. Virtual memory 106 may contain physical memory 212 as well as stable storage 216. The physical memory 212 may also contain in-memory virtual swap space 214. Physical memory 212 may comprise random access memory (RAM), non-volatile RAM, static RAM, etc. Stable storage 216 may comprise hard disk drive, optical storage, magnetic storage, etc. The virtual memory 106 is operatively coupled to communicate with the collaboration engine 100, typically (but not limited to) to the paging module 210 in the virtual memory manager 104.
Application code 218A through 218N are operatively coupled with the memory manager 102 (or other parts of the collaboration engine 100) and can communicate bidirectionally.
Although specific details of exemplary methods are described with regard to
At 402, communication is established between the memory manager and the virtual memory manager. This may involve a contracting solution or a callback solution, as discussed in more depth below.
At 404, virtual memory pages 108 for storing live objects within virtual memory 106 are then configured.
At 406, live objects are stored in virtual memory.
At 408, a determination is made whether the virtual memory manager needs to reduce the number of physical frames assigned to virtual pages. When no reduction takes place, no action is taken.
At 410, a reduction in the number of physical frames assigned to virtual pages is imminent, and the virtual memory manager notifies the memory manager.
At 412, the memory manager may compress, or compact, or both, live objects. Compaction or compression or both may also be performed by the virtual memory manager, or by a combination of memory manager and virtual memory manager. When the objects are compacted they are copied from one place to another in memory. Compression, compaction, and the combination are discussed in more detail below with regards to
At 414, the memory manager notifies the virtual memory manager which pages to discard and which pages or parts of pages to evict or to save to stable storage, thus freeing additional pages for use. The contents of physical memory frames corresponding to virtual memory pages that used to contain live objects that have been compacted or compressed can be discarded because the contents can be reconstructed when needed. For physical memory frames that are only sparsely populated with live objects, only the content of the used parts needs to be swapped to stable storage, as the content of the unused parts do not need to be reconstructed if the virtual memory page is accessed in the future.
At 416, the virtual memory manager may now reclaim physical pages which contained live objects now compressed or compacted, or both. Given that these objects are stored in another location in compressed form or compacted form, or a combination form, they may be reclaimed.
At 502, the live objects to be temporarily removed from a virtual memory page are determined. Based on the results, as described above in 412, the objects may be compacted or compressed or both. After that is done, the page with compacted objects may then be swapped out to stable storage, if such is available. Such a swap to stable storage is independent of compression or compaction or both.
The location to which live objects may be swapped includes physical memory 212 (either physical memory 212 or in-memory virtual swap space 214) or stable storage 216. This may occur after communicating with the memory manager 102 to learn the size and likelihood of use of the live object.
The memory manager 102 may gain this information from the garbage collector module's 202 recent trace of live objects in the heap (in the case of managed applications) or (for unmanaged applications) from the user space memory allocator module 204, which knows which parts of the page were allocated by the application 218A-218N.
With this information, a heuristic for deciding which pages to swap can balance between the two factors of recent (or frequent) use and least occupied space. Other heuristics may be used including, but not limited to, looking among all pages not recently (or not frequently) used and picking the one with the least amount of allocated objects.
At 504, the paging module 210 may then swap out virtual memory pages containing live objects as determined previously. The paging module may base swap decisions on a live object's size and likelihood of use of the live object, based on information from the memory allocator.
At 506, application pointers are updated to refer directly to the location of compacted live objects during garbage collection, which is addressed in more depth in
For some runtime systems, the addition of “forwarding pointers” from the old location to the new location can be used to ensure that the program uses the new memory location even while some program pointers may not have been updated. The process of updating pointers from an old memory location to a new memory location may be incorporated into the process of garbage collection, or it may be performed as a separate process.
At 602, live objects within virtual memory pages are located.
At 604, a destination area for storage of compacted objects is designated. The destination area for these objects may be a specially designated memory space, or any free memory location that can later be found when the objects are needed again. The destination area may be an otherwise unused page. Alternatively, the destination area may be the unused portions of a page that already contains live objects or data from prior compression and compaction efforts.
At 606, these located live objects are then copied to the destination area. Copying only live objects to share pages with other live objects frees the pages which originally contained the live objects for use or eviction. By compacting objects away from sparsely populated pages, a net reduction may be achieved in the total number of physical memory frames needed by the program.
At 608, a mapping of the live object's old location and the new location on the compacted page is stored.
At 702, the operating system may activate a pointer updating module of the runtime system. The pointer updating module may be incorporated into the garbage collector module of the runtime system, or it may be an independent component of the runtime system.
At 704, the pointer updating module then updates the application pointers. The application pointers to be updated may be identified using pointer map information also used by a garbage collection module. This update is similar to pointer updating for a standard garbage collection compaction, except that the compaction is done on a page based granularity and the pointer update does not need to complete on time. If the original data is accessed before pointers fix-up is executed, it is still possible to un-compact the data and restore it into the original virtual address space. While it is convenient to incorporate the pointer updating functionality into a garbage collector, other embodiments are possible. For example, the pointer updating module may be component of the runtime system that is completely separate from any garbage collection module.
For runtime systems where forwarding pointers from the old object location to the new object location are used temporarily during object relocation, when the object is being moved to the compacted location, only the forwarding pointer component of the old object location needs to be restored if the old object location is accessed.
When the pointers are updated, the application no longer holds pointers to the old objects, and thus there is no need to un-compact those pages anymore. This aids un-compaction because rather than un-compacting an object by copying it to a new page upon demand, the application may directly access the live object.
At 706, the old page locations are no longer referenced and thus the memory manager module may release those pages.
One implementation assumes a special memory space for the compacted area and a memory region within that special memory space that corresponds to the original page. In this case, the mapping may consist of any suitable structure including, but not limited to, two words and two bytes (original address, new address, original size, and compacted size) for any allocated space (object) that is moved. An alternative implementation performs compaction by first identifying the parts of the page that are occupied, and then copying the contents of these smaller memory areas into unoccupied parts of pages that are not currently destined to be swapped out. In this case, the collaboration with the memory manager or garbage collector may allow fine-grained interleaving of memory areas actively used by the running program and memory areas used for storage of the contents of compacted (swapped out) pages.
The compaction may be executed at some time before the page is swapped-out. Options for this execution include a contracting solution, callback solution, or other suitable means.
In the contracting solution, the virtual memory manager does the compaction itself, given information from the memory manager about which objects are live. The memory manager library will maintain the mapping of all allocated regions of this process. This mapping may be stored in a memory region whose address will be delivered to the virtual memory manager via an initial system call. The virtual memory manager can then simply access this mapping region when needed. The virtual memory manager can then use this mapping to perform the compaction.
Another option to compact is to not change the memory allocation library itself, but change the user program, by linking through the collaboration engine. Memory allocators (e.g., GNU glibc malloc) allow one to define hook functions as part of user program. Those hooks change the regular allocation function. Inside this hook the collaboration engine can record allocation requests such as calls for the regular allocation method, and before returning to the program record the allocated address and the size, as well as all free requests.
In a callback solution, the memory allocator performs the compaction. The virtual memory manager is given the entry of callback routines that decide which pages to compact, perform a compaction, and perform uncompaction. The virtual memory manager may use a callback to indicate to the memory manager that it intends to reclaim a number of physical memory frames in the near future. The memory manager can then decide if it can clear memory pages by compaction or compression or if it will have to let the virtual memory manager reclaim the pages by means of swapping.
Regardless of whether a contracting or callback solution is used, the compacted pages and mapping information need to temporarily be stored somewhere. The information may be stored in virtual memory that may either be backed by physical memory or be swapped out. The information may be made obsolete and subsequently deleted as a consequence of decompression, uncompaction, or pointer fixups.
As shown in
Alternatively,
As shown in 1106 after compression and compaction, memory pages 0-5 are now unused, with live objects A-G in pages 6 and 7. Compared to
The benefits of collaboration between the memory manager and virtual memory manager are now illustrated. Suppose the virtual memory manager indicates to the memory manager a need to reclaim four pages in physical memory 212.
Where six memory pages were initially in use (pages 0-5), after collaboration between the memory manager and virtual memory manager, at 1208 memory pages have been swapped, objects E, F, and A are compacted and placed in in-memory virtual swap space 1212. Live objects C and B have been moved to stable storage 216. After objects are compacted, and possibly compressed, the original page is protected in the virtual memory table as if being swapped out. This is done even though it was not swapped out to a stable storage 216 swap device. The pages that contained live objects C and B were not necessarily directly swapped out to stable storage, but the objects may instead have been compacted into “in-memory virtual swap space” 1206, and the compacted page subsequently swapped out to stable storage. The compacted object's frame is released by putting it on the list of free pages. Sufficient information is recorded to allow the compacted page to be found whenever it is required to be paged back in. One implementation is to record an index to the mapping of the moved space in the virtual memory table. As a result of the collaboration, four physical memory frames are now available for use.
When a page fault occurs as a result of an attempted access of a virtual memory page that was swapped out according to this method (upon an application's read/write to such a swapped out virtual memory page), the page cannot simply be swapped in from the stable storage. Instead, the index to the mapping data structure is followed to find the compacted page, possibly uncompress the data if it was earlier compressed, and copy the data from its compacted location back into a newly allocated frame (with the same offsets as it was stored before). If the compacted page has been swapped to stable storage, the data on the compacted page must be retrieved from stable storage in order to restore the contents of the original page. After restoring the data into this new physical frame, this frame is associated with the appropriate virtual memory page and the application can resume and access the data in a regular fashion. Alternatively, the association of a frame with the appropriate virtual memory page may be done before the data is being restored to its original location in virtual memory. An important benefit is that paging-in incurs memory copies rather than access to a secondary storage device, decreasing access times. Additional benefits may be further gained from collaboration as described next.
Suppose that via the collaboration module, the paging module 210 in
After swapping based on this information through the collaboration engine, the memory is now in a more optimal state for the next round of operations 1316. Live objects F and B are now in the Most Recently Used location 1320 and ready for immediate use. Live objects D and G are available in the Almost Swapped, Cause Exception 1322 location, and may move up to Most Recently Used 1320 or down to Compact and/or Compress Cause Exception 1324 depending on what happens later. Live objects E and C are stored in Compact and/or Compress Cause Exception 1324, and are more quickly accessible than if in stable storage 1326. Live object A has been moved directly to stable storage 1326. Because of the information available in the collaboration engine, rather than moving down through the location categories, each object is moved in one step to a better location category according to expectations. For example, live object A would have normally taken many steps to iterate down through the Most Recently Used 1320, then into Almost Swapped 1322, possibly to Compact and/or Compress 1324, then eventually to Stable Storage 1326. Using the method in this application, the movement takes only one step.
Not only does use of the collaboration engine provide the potential to move live objects into more appropriate, faster memory locations based on demand, it can free up space as illustrated next.
After swapping 1410 the pages are distributed as discussed above with regard to
A possible additional optimization method, appropriate for systems employing stable storage 216 for swapped-out pages (i.e., objects G and A which were stored in stable storage 1310 and 1326, respectively), is to allow swapping out of the pages that hold the compacted (and possibly compressed) objects to stable storage 216. Where the object in the swapped out pages is not used for a while, there is no need to keep the compacted (and possibly compressed) objects in memory. When a memory access initiates a page fault, the handler looks for the compacted (and possibly compressed) object. If that object is swapped out, then the compacted object (and possibly compressed) is first paged-in, and only then is it un-compacted and decompressed (as necessary).
Exemplary computing system 1500 typically includes a variety of computing device-readable media. Computing device-readable media can be any available media that can be accessed by computing system 1500 and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computing device-readable media may comprise computing device storage media and communication media. Computing device storage media include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computing device-readable instructions, data structures, program modules, or other data. Computing device storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computing system 1500. Communication media typically embodies computing device-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of any of the above should also be included within the scope of computing device readable media.
The system memory 1530 includes or is associated with computing device storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) 1531 and random access memory (RAM). A basic input/output system 1533 (BIOS), containing the basic routines that help to transfer information between elements within computing system 1500, such as during start-up, is typically stored in ROM 1531. RAM system memory 1530 typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 1506. By way of example, and not limitation,
The exemplary computing system 1500 may also include other removable/non-removable, volatile/nonvolatile computing device storage media. By way of example only,
The drives and their associated computing device storage media discussed above and illustrated in
The exemplary computing system 1500 may operate in a networked environment using logical connections to one or more remote computing devices, such as a remote computing device 1580. The remote computing device 1580 may be a personal computing device, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to computing system 1500, although only a memory storage device 1581 has been illustrated in
When used in a LAN networking environment, the exemplary computing system 1500 is connected to the LAN 1571 through a network interface or adapter 1570. When used in a WAN networking environment, the exemplary computing system 1500 typically includes a modem 1572 or other means for establishing communications over the WAN 1573, such as the Internet. The modem 1572, which may be internal or external, may be connected to the system bus 1521 via the user input interface 1560, or other appropriate mechanism. In a networked environment, program modules depicted relative to the exemplary computing system 1500, or portions thereof, may be stored in the remote memory storage device. By way of example, and not limitation,
Although exemplary systems and methods have been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the claimed methods, devices, systems, etc.