Patent Application
20060143389
1. Field
Embodiments of the invention relate to caching. More specifically, embodiments of the invention relate to shared caching with common monitoring.
2. Background
Within a typical virtual machine (VM) each component maintains its own cache infrastructure. As used herein, “component” refers generically to managers, services, and applications that may execute within the VM. Because each component maintains its own cache infrastructure and its own cache implementation, there is no common control of the cache. As the number of VMs in the system becomes arbitrarily large, a memory footprint of each successive VM increases the systems memory footprint proportionally. Additionally, there is no basis for common monitoring and administration nor is it possible to share objects and data between components in different VMs. Moreover, because the cache states are not globally visible, it is not possible to obtain an overview of cooperation and operation of different caches in a particular environment. Moreover, in the event that a VM fails, all information about the cache usage is lost. It would be desirable to develop a flexible system having shared cache usage and common monitoring.
A system and method of common cache management is disclosed. Plural VMs each have a cache infrastructure component used by one or more additional components within the respective VM. An external cache is provided and shared by the components of each of the VMs. In one embodiment, a shared external memory is provided and populated by the VMs in the system with cache state information responsive to caching activity. This permits external monitoring of caching activity in the system.
The invention is illustrated by way of example and not by way of limitation in the figures of the accompanying drawings in which like references indicate similar elements. It should be noted that references to “an” or “one” embodiment in this disclosure are not necessarily to the same embodiment, and such references mean at least one.
In the illustrated embodiment, worker nodes 120 each include a Java virtual machine (“JVM”) 135, one or more internal managers/monitors (e.g., a virtual machine (“VM”) monitor 145, a cache manager 134, and a session manager 155), and a shared memory application programming interface (“API”) 160 all supported within a native wrapper 165. JVMs 135 interpret and execute Java programs 140 while servicing work requests assigned to the particular worker node 120-1, 120-2. Although
During operation of worker nodes 120, the internal managers/monitors (e.g., VM monitor 145, cache manager 134, session manager 155, etc.) update shared monitoring memory 115 with status information. In one embodiment, the status information is logically organized into topic buffers 160A, 160B, and 160C (collectively 160) containing topically related status information from each of worker nodes 120. Each topic buffer 160 may include multiple slots S1-SN, each holding the topically related status information from a respective one of worker nodes 120. Once the status information is stored into shared monitoring memory 115, the status information may be retrieved from shared monitoring memory 115 by network interface 130 and transmitted to management console 110 for display thereon. Using management console 110, an information technology (“IT”) technician can remotely monitor the operational health of AS instance 105 in real-time to ensure AS instance 105 remains in a healthful state. Shared monitoring memory 115 working in concert with management console 110, enables the IT technician to make informed decisions when taking preventative and/or remedial action to effectively maintain and manage an enterprise system.
JVMs 135 interpret Java programs 140 by converting them from an intermediate interpreted language (e.g., Java bytecode) into a native machine language, which is then executed. Java programs 140 may be interpreted and executed by JVMs 135 to provide the business, presentation, and integration logic necessary to process the work requests received at AS instance 105. As the work requests are serviced, sessions are setup and taken down, caching occurs, and memory and processor cycles consumed. Shared monitoring memory 115 provides a mechanism by which these operational characteristics of worker nodes 120, as well as others, may be monitored.
VM monitor 145, cache manager 134, and session manager 155 are generators of status information describing the operational status of various aspects of worker nodes 120. Although only three such generators are illustrated in
Native wrapper 165 provides the runtime environment for JVM 135. In an embodiment where JVM 135 is a JVM compliant with the J2EE standard, native wrapper 165 is often referred to as “JLaunch.” Native wrapper 165 is native machine code (e.g., compiled C++) executed and managed by an operating system (“OS”) supporting AS instance 105. Once launched, native wrapper 165 establishes JVM 135 within itself. In one embodiment, the generators of status information (e.g., VM monitor 145, thread manager 134, session manager 155, etc.) are native code components of native wrapper 165. As such, even in the event of a failure of JVM 135, the generators of the status information can still operate providing updates on the failure status of the particular JVM 135. In other embodiments, a generator of status information may indeed be interpreted and executed on JVM 135, in which case a failure of JVM 135 would also terminate the particular generator.
While processing work requests, connections may be established between a client generating the work request and the particular worker node 120 servicing the work request. While the connection is maintained, a session is established including a series of interactions between the two communication end points (i.e., the worker node and the client). In one embodiment, session manager 155 is responsible for the overall managing and monitoring of these sessions, including setting up and taking down the sessions, generating session status information 171, and reporting session status information 171 to an appropriate one of topic buffers 160. For example, topic buffer 160A may be a “session buffer” assigned to store session related status information. In one embodiment, session manager 155 registers a different slot for each session currently open and active on its corresponding one of worker nodes 120.
In one embodiment, cache manager 134 generates cache status information 173 and reports cache status information 173 to an appropriate topic buffer 160. For example, topic buffer 160B may be a “cache buffer” assigned to store cache related status information.
VM monitor 145 may monitor various internal activities of JVM 135. For example, VM monitor 145 may monitor the work load of JVM 135 and report overload situations into shared monitoring memory 115. VM monitor 145 may further monitor an internal heap of JVM 135 and report memory scarce situations into shared monitoring memory 115. VM monitor 145 may even monitor garbage collecting activity within JVM 135 and report over active garbage collecting situations into shared monitoring memory 115. It should be appreciated that any aspect of worker nodes 120 capable of monitoring may be monitored by a generator of status information and the status information copied into a relevant topic buffer 160 and associated slots S1-SN.
The generators of the status information (e.g., session manager 155, thread manager 134, VM monitor 145, etc.) access shared monitoring memory 115 via shared memory API 160. In one embodiment, shared memory API 160 abstracts access to shared monitoring memory 115 through use of function calls. Each generator of status information that wishes to copy status information into shared monitoring memory 115 makes a “call” to one or more functions published internally to worker nodes 120 by shared memory APIs 160. The generator then passes the generated status information to the called function. In turn, the called function copies the status information into the appropriate slots and topic buffers 160.
In one embodiment, shared monitoring memory 115 is a portion of system memory pre-reserved to store status information. Abstracting access to shared monitoring memory 115 with shared memory APIs 160 insulates and protects the contents of shared monitoring memory 115 from each worker node 120. Should a worker node 120 crash, enter an infinite loop, or otherwise fail, the status information saved into shared monitoring memory 115 may still be protected and preserved from corruption.
VMs 202 include a cache infrastructure component, such as cache manager 234. As noted previously, as used herein “component” generically refers to managers, services, and applications. VM 202 includes one or more additional components, such as component 230 and 232. In one embodiment, a component might be an HTTP service, a session manager, or a business application. In one embodiment, the cache infrastructure component, such as cache manager 234, is shared between all of the additional components (e.g., 230, 232) within the respective VM 202. In one embodiment, cache manager 234 is a core component of a J2EE engine, and worker node 200 is an application server node. As a core component, cache manager 234 is always available to every component sitting on the application server infrastructure.
In one embodiment, cache manager 234 includes a cache management library (CML) 240. In one embodiment, CML 240 is used by additional components 230, 232 within the VMs 202 to access an external cache 204, which is shared among the VMs 202. The CML 240 can be thought of as having two layers: a first layer including a user application programming interface (API) and a cache implementation, both of which are visible to the cache uses such as additional components 230, 232 and a second layer having a set of possible storage plugins and eviction policy plugins. The second layer is not visible to the cache users. Initially, a component such as components 230-1 accesses CML 240-1 to obtain a cache region. CML 240 uses region factory 242-1 to generate a cache region 244-1 for componentA 230-1.
A cache region is basically a facade to allow the component to use external cache 204. The cache region is based on a pluggable architecture, which enhances the flexibility of the architecture layer by allowing the use of various storage plugins 250-1, 254-1 and eviction policy plugins 252-1, 256-1. Storage plugin 250-254 are responsible for persisting cache objects. The storage plugin may persist the data in a database or the file system. Different storage plugin may have different policies for persisting cache object, e.g., write through, write back, or spooling. In one embodiment, only a single storage plugin is used for any particular cache region. Eviction policy plugin 252, 256 are responsible for selecting the least important cache content for removal from the cache once a threshold has been exceeded. Two common threshold parameters are total size of object cached and total number of objects cached. Various eviction policies may be employed, such as, first in first out (FIFO), least frequently used (LFU), etc. In one embodiment, a single eviction policy plugin is bound to a region.
Similarly, component B1232-1 may access CML 240-1 to obtain cache region B 246-1 with associated plugins 254-1 and 256-1 bound thereto. Notably, storage plugin 250-1 and 254-1 need not, but may be the same plugin. Similarly, eviction plugins 252-1 and 256-1 may need not be the same. Thus, for example, region A 244-1 may use write through storage and an LRU eviction policy while region B may use write back storage and a FIFO eviction policy.
After obtaining a cache region, if componentA1 creates an object O1 it wishes to cache, componentA1 230-1 call a PUT <object> method directed to CML 240-1, which via cache region 244-1 places object O1 and an associated key in an area of external cache 204 associated with the cache region 244, more specifically, region 264. Concurrently, cache manager 234-1 populates the caches area 220 of monitoring shared memory 206 with cache state information, such as, information from which hit rate, fill rate, number of cache objects, etc. may be derived. In one embodiment, cache area 220 is analogous to the topic buffer 160B discussed above with respect to
ComponentA2 230-2 needs object O1 and may issue a get O1 command to CML 240-2, which will retrieve object O1 from the external cache 204 via the accesses point provided by region 244-2 notwithstanding that componentA2 did not create object O1. ComponentA2 objects may merely call a GET <object> method with O1, as the argument and the cached object O1 will be returned as indicated in the figure and be shared between components in different VMs reducing the need for expensive recreation of objects and reducing the memory required for caching. The foregoing, of course, assumes that componentA1 230-1 has already created object O1 if O1 is not yet in external cache 204, a cache miss will result from the GET command. ComponentA2 230-2 will then need to create object O1 and may cache it as described above. In response to the cache activity, cache manager 234-2 populates the caches area 220 of monitoring shared memory 206 to maintain an accurate global picture of caching activity in external cache 204.
The cache area 220 of shared memory 206 may be accessed by a start up service 211, which is responsible for starting the worker nodes. That information in the shared memory 206 may be made visible to a management console such as, MMC 208 via a network interface 212. This permits ongoing monitoring of the cache state information independent of the health of the worker nodes 200.
In one embodiment, each cache region has a single storage plugin (e.g., 250-1 of
Time to live (TTL) workers 436 provide automatic eviction from the cache of object that have remained in the cache for a particular amount of time. In one embodiment, that time can be set on a group-by-group basis and is retained as an element in the configuration 472.
In one embodiment, the application 430 requires cache control 434 from the cache region 464. Once application 430 has acquired cache control 434, it can invalidate cached objects through the cache control 434 and can register invalidation listeners 428 in cache control 434. Once registered, invalidation listeners 428 will be notified about invalidation of cached objects. Invalidation includes modification, explicit invalidation or removal. Notification may occur in two ways. First, the application 430 may explicitly invalidate a cached object key through cache control 434. Alternatively, an application 430 modifies or removes a cached object using a cached object key. The cache group or cache facade 440 signals local notification instance 456, which is bound to the cache region 464 about this invalidation event. Local notification instance 456 informs cache control 434, which in turn notifies registered invalidation listeners 428.
Additionally, local notification instance 456 may use notification globality hook 458 to distribute invalidation messages to other nodes in a cluster. Whether a particular notification is distributed globally may depend on the configuration 472 of cache region 464. If the region is configured to only provide a local notification, the globality hook will not be notified about invalidation messages. Conversely, if configuration 472 indicates that invalidation should be made globally visible, the notification globality hook 458 is notified of invalidation messages and propagates them to external nodes.
The application 430 may use the cache groups or cache facade 440 to access a cache, (such as, external cache 204 of
In one embodiment, this cache state information is largely a set of counter values entered by the storage plugin 450 bound to the cache region 464 or obtained from the cache implementation, e.g., NAMES_SIZE. More complex state information may be derived externally. For example, hit rate based on cache hits and gets or cache mutability based on modifications and puts. By making this information globally available an overview of cooperation between various caches in an environment may be obtained. Additionally, cache state information remains available even if the caching VM fails.
Elements of embodiments may also be provided as a machine-readable medium for storing the machine-executable instructions. The machine-readable medium may include, but is not limited to, flash memory, optical disks, CD-ROMs, DVD ROMs, RAMs, EPROMs, EEPROMs, magnetic or optical cards, propagation media or other type of machine-readable media suitable for storing. electronic instructions. For example, embodiments of the invention may be downloaded as a computer program which may be transferred from a remote computer (e.g., a server) to a requesting computer (e.g., a client) by way of data signals embodied in a carrier wave or other propagation medium via a communication link (e.g., a modem or network connection).
It should be appreciated that reference throughout this specification to “one embodiment” or “an embodiment” means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Therefore, it is emphasized and should be appreciated that two or more references to “an embodiment” or “one embodiment” or “an alternative embodiment” in various portions of this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures or characteristics may be combined as suitable in one or more embodiments of the invention.
In the foregoing specification, the invention has been described with reference to specific embodiments thereof. It will, however, be evident that various modifications and changes can be made thereto without departing from the broader spirit and scope of the invention as set forth in the appended claims. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.
