Patent Application
20130111144
The present invention relates, in general, to memory management, and, in particular, to management of memory in a cache.
A cache is a component that improves hardware and software performance by transparently storing data such that future requests for that data can be served faster. The data in the cache is often either stored elsewhere or would have been computed earlier. A cache is used when the original data is more expensive to fetch (owing to longer access time) or to compute, than the cost of storing it for immediate reading when needed.
According to one embodiment of the present invention, the use of heap memory is optimized by extending a cache implementation with a base class. An instance of a utility class is attached to the base class. The cache implementation is registered to a garbage collector routine. The registration is stored as a reference list in a memory. In response to an unsatisfied cache allocation request, a garbage collection cycle is triggered to check heap occupancy. In response to exceeding a first threshold value, the reference list is traversed for caches to be cleaned based upon a defined space constraint value. The caches are cleaned in accordance with the defined space constraint value.
According to one embodiment of the present invention, a system optimizes use of heap memory. The system comprises a processor and memory connected to the processor, wherein the memory is encoded with instructions and wherein the instructions when executed extend a cache implementation with a base class. The instructions further comprise instructions for attaching an instance of a utility to the base class. The instructions register the cache implementation to a garbage collector routine. The instructions store the registration as a reference list. Responsive to an unsatisfied cache allocation request, the instructions further trigger a garbage collection cycle to check heap occupancy. Responsive to exceeding a first threshold value, the instructions further traverse the reference list for caches to be cleaned based upon a defined space constraint value. The caches are cleaned in accordance with the defined space constraint value.
According to one embodiment of the present invention, a computer program product optimizes use of heap memory. The computer program product comprises a computer readable storage medium having computer readable program code embodied therewith. The computer readable program code is configured to extend a cache implementation with a base class. The computer readable program code attaches an instance of a utility class to the base class. The computer readable program code registers the cache implementation to a garbage collector routine, and stores the registration as a reference list. In response to an unsatisfied cache allocation request, computer readable program code triggers a garbage collection cycle to check heap occupancy. In response to exceeding a first threshold value, computer readable program code traverses the reference list for caches to be cleaned based upon a defined space constraint value. The caches are cleaned in accordance with the defined space constraint value.
The descriptions of the various embodiments of the present invention have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.
Any combination of one or more computer readable media (memory or device) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).
Aspects of the of the present invention are described below with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks
The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
The computer system 10 is of a type that executes under a suitable operating system installed on the computer system 10. The components of the computer system 10 include a computer 12, a keyboard 22, mouse 24, and a video display 20. The computer 12 includes a processor 26, a memory 28, input/output (I/O) interfaces 30 and 32, a video interface 34, and a storage device 36.
The processor 26 is a central processing unit (CPU) that executes the operating system and the computer software executing under the operating system. The memory 28 includes random access memory (RAM) and read-only memory (ROM), and is used under direction of the processor 26.
The video interface 34 is connected to a video display 20 and provides video signals for display thereon. User input to operate the computer 12 is provided from the keyboard 22 and mouse 24. The storage device 36 can include a disk drive or any other suitable storage medium, as discussed above. Each of the components of the computer 12 is connected to an internal bus 40 that includes data, address, and control buses, to allow components of the computer 12 to communicate with each other via the bus 40. The computer system 10 can be connected to one or more other similar computers via an input/output (I/O) interface 32 using a communication channel 38 to a network, represented as the Internet 18. One or more servers 19 may be connected to the computer 12 via a network, such as, the Internet 18. The servers 19 may comprise the same physical arrangement as the computer 12 and may be co-located with or a part of the computer 12.
The computer software may be recorded on a computer readable storage medium, in which case, the computer software program is accessed by the computer system 10 from the storage device 36. Alternatively, the computer software can be accessed directly from the Internet 18 by the computer 12. In either case, a user can interact with the computer system 10 using the keyboard 22 and mouse 24 to operate the programmed computer software executing on the computer 12.
The present invention provides improvements over the prior art. One embodiment of the present invention comprises a framework that provides underlying virtualized run-times, such as Java, having an automatic garbage memory collection facility, with the ability to coordinate with cache implementations such that optimal performance of the application is achieved with the least memory overhead and maximum processor utilization. This framework introduces two classes, a base class named CacheInterface and an instance of a utility class named ReferenceToCache. It is to be understood that the names given these two classes are arbitrary and used, herein, for convenience of description only. The classes could be given any other name, as desired.
The CacheInterface class is an abstract base class which the cache implementations must extend to use certain base attributes and functions. This class provides API functions which the garbage collector (GC) can use to communicate with an object of this class in order to better manage memory and CPU consumption by the application. The extending class should consider:
Class definition:
(1)CONSTRAINT - helps determine the severity of the memory “pressure.”
(2)The ReferenceToCache constructor is where an export happens which is needed to “associate” a newly created cache object and GC.
(3)Start time.
(4)End time.
(5)Subtract end time and start time to obtain the total time spent “adding”/creating a value to a cache.
(6)Similar to “adding” cost computation, but, here, “get” cost is computed for “getting” a value from a cache.
(7)Abstract methods that child classes of each abstract class must implement.
Method Description:
Table 1.1 describes the methods defined in the CacheInterface class.
The public method “get( )” defined in the abstract base class, CacheInterface, is declared as “final.” This prevents other classes that extend from this abstract class from overriding this method implementation. The implementation of this method first computes the total time taken to complete the “getEntry( )” operation. This is done by looking up the system clock before and after invocation of this method. Further, if the value for the given key is absent inside the cache, the method implementation will create and put a new value associated with the key in question into the cache. As part of this operation, the implementation computes the time taken to create and put a new value into the cache. The time taken to add a new value into the cache is also computed by looking up the system clock before creating the new value and after putting the created value into the cache. See pseudo-code implementation of the method in Class Definition, above, for more details.
Field Description: Table 1.2 describes all the fields in the CacheInterface class.
ReferenceToCache Class:
Whenever a CacheInterface object is created, an instance of a utility class (as used herein, ReferenceToCache) is attached to this object. ReferenceToCache provides a way for cache implementations to register the cache with the GC component in the JVM and also provides functions to compute the various costs associated with the different cache operations, as will be discussed in greater detail below. The object is instantiated by making a call into the GC routine. The same object is added to a list of objects maintained by the GC. All the cache statistics are stored in this object and the GC has ready access to the cache and its statistics. The GC, based on its requirements at run-time, can request the caches, via this object, to free up memory held by them.
Class Definition:
(8)Gets and stores the cache reference. Returns reference to the ReferenceToCache class.
(9)Method to compute and store “create” cost.
(10)Method to compute and store “get” cost.
(11)Ratio of time taken to get( ) a value from cache and create a new value in the cache.
(12)Ratio of frequency of get( )s and updates (create + add) into the cache.
(13)Methods that provide information about the cache effectiveness.
Method Description
Table 1.3 describes all the methods defined in the ReferenceToCache class.
Field Description: Table 1.4 describes all the fields in the ReferenceToCache class.
Example implementation of a cache.
The following pseudo code demonstrates a sample Socket connection cache:
(14)A local variable. This is then ‘set’ to a value based on spaceConstraint argument received from caller (GC in this case)..
(15)GC tells how great the memory pressure is. That is conveyed via “spaceConstraint” argument to cleanup( ) method. However, the number of entries to remove is cache implementation specific. The cache implementer can determine how important the cached entries are for performance of the application and then come up with the percentage of entries to be removed.
The percentages listed above are suggestive in nature. The implementation of a cache called ConnectionCache whose implementer decided 40% of the cache can be removed when memory pressure is too much (critical).
Referring to
Referring to
If the heap occupancy of the caches exceeds this acceptable first threshold value, the runtime invokes the cache cleanup procedures for each of the individual cache implementations that have contributed significantly to the memory pressure. One embodiment of the present invention provides the GC component/JVM like runtime with the contribution of each of the software caches registered with it. This can then be compared against the overall memory occupancy by all the caches. This comparison and other calculations done at runtime (such as readVSwriteRatio, getVsCreateRatio, lastRead, lastWrite) can also help detect candidate caches that contribute significantly to the memory pressure and, therefore, can be cleaned. These parameters can also help prioritize the cleanup order of the remaining caches and invoke cleanup procedures on them. However, if the response to decision block 304 is no, the process ends at block 306, as the available memory is considered to be sufficient.
If the response to decision block 304 is yes, a sub-task of invoking the cleanup procedures of the caches that have contributed significantly (significance being previously defined above) to the memory pressure will be undertaken. As part of this sub-task, the reference list is first traversed at block 308 for cleanup based upon defined space constraint values. For example, a “mild” memory pressure exists, so, about 5% of total entries in the cache can be cleaned using, for example, a “least recently used” algorithm. As shown above, further space constraint values can be defined as “medium” at 10%, “severe” at 20%, and “critical” at 40%. Although not shown, it is to be understood that the values and percentages represented, thereby, can be set at the user's option.
At block 310, all caches with a “get” versus “create” cost having a value greater than 1, are cleared. A get versus create cost (in terms of total wall clock time taken for completion) that is greater than 1 indicates that it is faster to create the value than it is to “fetch” the value from the cache. Thus, implementations are better off without caching such values. It is then determined at decision block 311 whether the defined space constraint value has been met. If the response to decision block 311 is yes, the process ends at block 306.
If it is determined at decision block 311 that the space constraint value is not met, the process continues to block 312, wherein all caches whose read (number of times a value is fetched from the cache) vs. write (number of times a value is updated in the cache including its creation) ratio is less than or equal to 1 are also cleared. A ratio less than or equal to 1 indicates that the cached values are not being looked up frequently enough.
It is then determined at decision block 313 whether the defined space constraint value has been met. If the response to decision block 313 is yes, the process ends at block 306. If it is determined at decision block 313 that the space constraint value is not met, the process continues to block 314 wherein all the caches that have not been read within a pre-designated time (second threshold to be decided by the implementer) are cleared.
It is then determined at decision block 315 whether the defined space constraint value has been met. If the response to decision block 315 is yes, the process ends at block 306. If it is determined at decision block 315 that the space constraint value is not met, the process continues to block 316. At block 316, for all the remaining caches, the cleanup method is invoked using an appropriate flag based on the heap occupancy. The process then ends at block 306.
Enhancement using binary dump of heap contents:
A binary dump of heap contents is a facility that generates a dump of all the reachable objects that are on the Java heap; that is, those that are used by the Java application. It shows the objects that are using large amounts of memory on the Java heap. One embodiment of the present invention introduces an additional enhancement on the way memory analysis tools can work. The tool would only need to look for the CacheReference objects and by processing the references can determine how much of the memory is actually held under the caches.
For example, a tool, such as, “Memory Analysis Tool” (MAT), can use the CacheInterface list available with the garbage collector component of the JVM to “display” accurate details of the memory occupancy of the software caches. At present, MAT or any other memory analysis tool cannot identify a cache object from a dump and hence cannot readily provide detailed information about memory occupancy of caches instantiated during the application run. Therefore, the present invention can help identify a cache object from a binary/memory dump, and, thus, closer introspection of the cache object becomes possible.
Once the cache object is examined, more intricate details related to memory occupancy and its effectiveness can be displayed. For example, using the fields and methods previously described above, to retrieve the field values can be used to determine memory occupancy. Cache effectiveness can also be determined by using the same methods and fields. For example, getVsCreateRatio( ) helps determine if the number of “gets” far out-weigh the creates. If yes, then the cache is being used effectively and is needed. Similarly, if time returned by lastRead( ) is huge, then there aren't enough cache lookups happening during the application run. Thus, these values, when looked at in conjunction with occupancy or create cost, can help make intelligent decisions.
The corresponding structures, materials, acts, and equivalents of all elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the present invention has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the invention in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the invention. The embodiment was chosen and described in order to best explain the principles of the invention and the practical application, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated.
Having thus described the invention of the present application in detail and by reference to embodiments thereof, it will be apparent that modifications and variations are possible without departing from the scope of the invention defined in the appended claims.
