Patent Application
20080005403
In one example in accordance with aspects of the invention, the apparatuses and methods disclosed allow a virtual machine on a device to overcome method cache overloads in dynamic just-in-time (JIT) compilation. The apparatuses and methods dynamically adapt the size of the compiled methods cache according to the effective run-time behavior of running applications on the device. These and other aspects of the invention are disclosed in detail below.
Furthermore, aspects of the invention may be described in the general context of computer-executable instructions, such as modules, being executed by the processor 102 of the device 100. Generally, modules may include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. The memory unit 104 may be comprised of: compiler module 106 including a buffer 108; buffer monitor module 110 including a counter 116; overload detector module 112; and buffer shaper module 114.
In one example, the compiler module 106 includes a just-in-time (JIT) compiler. A JIT compiler is a platform-specific compiler that generates native machine code from bytecode. For example, Java executable programs in bytecode format may be comprised of methods. The bytecode may be provided to a Java virtual machine residing in the memory unit 104 of a device 100. The Java virtual machine may be embedded with a JIT compiler for dynamic compilation of the bytecode. The JIT compiler may compile (e.g., translate) the bytecode into native machine code executable by the processor 102.
The native code corresponding to each method in the Java executable program may be cached in a buffer 108 (e.g., a method-compilation cache) in the compiler module 106. A depiction of the contents of an illustrative method-compilation cache in the buffer 108 is shown in
The buffer monitor module 110 in the memory unit 104 is capable of monitoring for an overload condition in the compiler module 106. In one example, the buffer monitor module 110 intercepts particular method calls associated with the buffer 108. For example, a method call requesting to add the native code of a method to the buffer 108 (e.g., method-compilation cache) may be intercepted by the buffer monitor module 110. Similarly, a method call requesting to remove the native code of a method from the buffer 108 (e.g., method-compilation cache) may be intercepted by the buffer monitor module 110. One skilled in the art will appreciate that the buffer monitor module 110 can intercept particular method calls without affecting the usual operation of the compiler module. For example, the buffer monitor module 110 may simply receive a notification that a particular method call associated with the buffer 108 has been requested.
In accordance with aspects of the invention, the buffer monitor module 110 may be comprised of computer-executable instructions (e.g., firmware, software objects, etc.) and/or logic (e.g., multiplexers, demultiplexers, logic gates, etc.) for monitoring for an overload condition in the compiler module. For example, the buffer monitor module 110 may comprise a counter 116. A counter 116 may be a portion of memory (e.g., a register block) for storing one or more values. For example, a counter 116 may be incremented to keep count of the number of method calls requesting to add information to the buffer 108 (i.e., cache) over a particular period of time. Likewise, a counter 116 may be incremented to keep count of the number of method calls requesting to remove information from the buffer 108 (i.e., cache) over a particular period of time. In some examples, the counter 116 may store two values: one value counting the number of requests to add, and another value counting the number of requests to remove.
The buffer monitor module 110 may receive overload threshold values as an input. In accordance with aspects of the invention, one overload threshold value may correspond to method calls (e.g., the ObjectHeap::compiler_area_allocate_code method from Sun Microsystem's CLDC Hotspot VM implementation) requesting to add information to the buffer 108. Similarly, another overload threshold value may correspond to method calls (e.g., the CompiledMethodCache::Evict method from Sun Microsystem's CLDC Hotspot VM implementation) requesting to remove information from the buffer 108. Although aspects of the invention have been described using Sun Microsystem's CLDC (Connected Limited Device Configuration) Hotspot VM for illustrative purposes, one skilled in the art will appreciate that various aspects of the invention may be embodied using CDC (Connected Device Configuration) and other configurations known to those of skill in the art.
For example, the buffer monitor module 110 may identify an overload condition if the counter value corresponding to requests for adding information to the buffer 108 has a greater value than the corresponding overload threshold value inputted. Likewise, the buffer monitor module 110 may identify an overload condition if the counter value corresponding to requests for removing information from the buffer 108 has a greater value than the corresponding overload threshold value inputted. In another example, the buffer monitor module 110 may output an indication of an overload condition to the overload detector module 112 only if the counter value corresponding to both the request for adding information and the requests for removing information to/from the buffer 108 are outside their respective overload threshold values. The indication outputted may equal binary ‘1’ if the compiler module is in an overload condition and may equal binary ‘0’ if the compiler module is not in an overload condition.
A compiler module in an overload condition may result in the inefficient execution of application code by a virtual machine (e.g., JIT compiler). An overload condition frequently occurs when the size of the buffer 108 is too small relative to its usage by the compiler module. The virtual machine adds native code in the buffer 108, and it also removes (e.g., overwrites) native code when the buffer becomes full. In situations where the virtual machine is repeatedly adding and removing information from the buffer 108, the computing device becomes inefficient (e.g., due to congestion). For example, a graphics-intensive application executing on a mobile terminal with limited memory resources may render the application unusable due to processor delays and insufficient frame rate. Rather, in accordance with aspects of the invention, various modules in the memory unit 104 of the device 100 may identify the overload condition, detect the type of overload condition (e.g., corresponding to transient congestion, permanent congestion, or no congestion) and react appropriately (e.g., increase buffer size, decrease buffer size, not change buffer size).
The indication outputted from the buffer monitor module 110 may be inputted into the overload detector module 112. The overload detector module 112 is capable of determining the type of overload condition occurring in the compiler module based at least in part on the indication inputted from the buffer monitor module 110. The overload detector module 112 may be comprised of computer-executable instructions (e.g., firmware, software objects, etc.) and/or logic (e.g., multiplexers, demultiplexers, logic gates, etc.) for determining the type of overload condition in the compiler module. Some examples of the types of overload conditions include, but are not limited to, a non-performance-critical overload condition, a performance-critical overload condition, and a no overload condition. A non-performance-critical overload condition is characterized by transient manifestation of overload-critical conditions. Meanwhile, a performance-critical overload condition represents permanent presence of overload-critical conditions. The overload detector module 112 is capable of distinguishing between different kinds of conditions by using any of numerous approaches, including but not limited to, count-and-threshold mechanisms (e.g., single threshold alpha-count mechanism, double threshold alpha-count mechanism, etc.). Numerous count-and-threshold mechanisms are known to those of ordinary skill in the art, therefore, each and every approach and variation on these approaches need not be expressly described here. The alpha-count mechanism in accordance with various aspects of the invention is illustrated in detail below.
In accordance with aspects of the invention, the overload detector module 112 generates an indication of the overload condition type and outputs the indication to the buffer shaper module 114. The overload detector module 112 outputs an indication of a non-performance-critical overload condition to the buffer shaper module 114 if the overload detector module detects a non-performance-critical overload condition. Meanwhile, the overload detector module 112 outputs an indication of a performance-critical overload condition to the buffer shaper module 114 if the overload detector module detects a performance-critical overload condition. Moreover, the overload detector module 112 may output an indication of a no overload condition to the buffer shaper module 114 if the overload detector module detects neither a performance-critical overload condition nor a non-performance-critical overload condition. One skilled in the art will appreciate that other overload conditions may be contemplated in accordance with aspects of the invention by one of ordinary skill in the art after review of the entirety disclosed herein.
Depending at least in part on the output from the overload detector module 112, the buffer shaper module 114 may dynamically adjust the size of the buffer 108 in the compiler module 106. The buffer shaper module 114 may be comprised of computer-executable instructions (e.g., firmware, software objects, etc.) and/or logic (e.g., multiplexers, demultiplexers, logic gates, etc.) for dynamically adjusting the size of the buffer 108. The buffer shaper module 114 may increase the size of the buffer 108 if the information (e.g., indication of overload condition type) from the overload detector module 112 indicates that a performance-critical overload condition exists. In contrast, the buffer shaper module 114 may decrease the size of the buffer 108 if the information (e.g., indication of overload condition type) from the overload detector module 112 indicates that a no overload condition (i.e., none of a performance-critical overload condition or a non-performance-critical overload condition) exists. Meanwhile, the buffer shaper module 114 may not change the size (i.e., the size of the buffer remains unchanged) of the buffer 108 if the information (e.g., indication of overload condition type) from the overload detector module 112 indicates that a non-performance-critical overload condition exists. In another example, the buffer shaper module 114 uses a maximum buffer size parameter to prevent the increase of the buffer 108 beyond a certain size. At least one benefit of a maximum buffer size parameter is the safeguard it provides the device 100 from a memory overflow error or similar fault.
In step 206, the type of overload condition is detected (i.e., determined) using a count-and-threshold mechanism (in step 208) that is capable of detecting a plurality of types of overload conditions. Numerous count-and-threshold mechanisms are known to those of ordinary skill in the art, including the alpha-count mechanism. One example of the alpha-count mechanism in accordance with various aspects of the invention is illustrated in detail below. One skilled in the art will appreciate that many variations of this count-and-threshold mechanism exist and are considered to be contemplated in the disclosure herein.
With reference to
The alpha-count mechanism may be implemented using software modules and/or hardware logic. The u(k) in the above alpha-count mechanism corresponds to the indication of overload condition output from the buffer monitor module 110. As explained above, the indication outputted from the buffer monitor module 110 may equal binary ‘1’ if the compiler module is in an overload condition and may equal binary ‘0’ if the compiler module is not in an overload condition. Furthermore, “A” (i.e., delta) is a fixed parameter satisfying the constraint: 0<“Δ”<1. The alpha value may be increased by one upon indication of an overload condition (i.e., u(k)==1) and decreased by “A” (e.g., a value smaller than one) upon indication of no overload condition (i.e., u(k)==0). The alpha-count mechanism may be adjusted through the delta value to increase or decrease the rate at which an alpha threshold value is met or not met. For example, a high delta value may result in a faster recovery time because each indication of a no overload condition reduces the alpha value by more. One skilled in the art will appreciate that the alpha-count mechanism is well-known and beneficial for many reasons. At least one advantage of the alpha-count mechanism is its capability to discriminate between transient phenomena (e.g., non-performance critical overload condition) and persistent/permanent phenomena (e.g., performance critical overload condition).
In step 306, the current alpha value (e.g., a first alpha value) is compared to a predetermined alpha threshold value to detect (in step 308) the type of overload condition in the compiler module 106. Once the alpha threshold value is exceeded, the type of overload condition may be changed from transient (e.g., non-performance critical overload condition) to permanent (e.g., performance critical overload condition). A permanent overload condition corresponds to a state where the buffer monitor module persistently indicates an overload condition.
In step 304, an updated alpha value (e.g., a second alpha value) is calculated using the above alpha-count mechanism. Equation 1 above is used to calculate the updated alpha value based on the alpha value (e.g., the first alpha value) calculated earlier. As with the first alpha value calculated earlier, the updated alpha value (e.g., second alpha value) is compared (in step 306) to a predetermined alpha threshold value to detect (in step 308) the type of overload condition in the compiler module 106. For example, if the alpha value is zero, then an indication corresponding to a no overload condition is generated (in step 212) and outputted to the buffer shaper module 114 to decrease the buffer 108 size (in step 214). In another example, if the alpha value is equal to or above the predetermined alpha threshold value, then an indication corresponding to a performance-critical overload condition is generated (in step 212) and outputted to the buffer shaper module 114 to increase the buffer 108 size (in step 216). Otherwise, the buffer 108 size remains unchanged (in step 218).
One skilled in the art will recognize that the alpha-count mechanism may use an iterative process in calculating an updated alpha value. At least one benefit of the alpha-count mechanism, among other threshold-based heuristics, is its simpler mathematical representation, which makes its implementation more compact and easy. In addition, the computational complexity of this mechanism is relatively small, and thus it allows reducing the overhead that is inherent in some other mechanisms. One skilled in the art will appreciate that although the above example refers to a single-threshold alpha-count mechanism, a double-threshold alpha-count mechanism may also be used.
While it is understood that the invention may be implemented as a method, it may also be implemented in a computer readable medium having computer-executable instructions. Aspects of the invention may be implemented by way of software, hardware, or a combination thereof. Computer-readable media includes any available media that can be accessed by a processor in a computing device or system. Other features of the invention will become apparent from the detailed description when taken in conjunction with the drawings.
The present invention has sometimes been described in terms of preferred and illustrative embodiments thereof. Numerous other embodiments, modifications and variations within the scope and spirit of the appended claims will occur to persons of ordinary skill in the art from a review of this disclosure.
