Patent Grant
7979863
The present disclosure relates generally to systems management and, more particularly, to methods and apparatuses for dynamic CPU resource management.
Systems management involves the supervision and management of information technology resources in an enterprise (or other organization). For example, many systems management software include tools for monitoring and collecting information regarding resource usage, such as CPU usage.
As enterprises grow, their needs for information technology resources can change rapidly, as demands for performance and reliability tend to increase. The typical approach for addressing the growing demands is to consolidate the servers in order to maximize resources available to a user. For example, applications and infrastructure services running on diverse operating systems can be consolidated on a reduced number of high performance servers, or even a single high performance server, running multiple virtual machines.
A virtual machine is typically a software based concept or logical entity that is implemented over a hardware platform and operating system and can use multiple devices (such as memory, processors, other hardware resources, network systems, etc.) to create multiple virtual systems, each of which can run independently as a copy of the operating system. In other words, a virtual machine can be thought of as a computer that operates inside, for example, a server, as an entity separate from other virtual machines. A virtual machine can allow for flexibility across platforms and can provide performance optimization by allowing time-expensive hardware to be shared.
Virtual machine software, such as VMware ESX Server (“ESX”), can be used to consolidate systems in advanced environments. System(s) as herein referred to may include(s) individual computers, servers, computing resources, and/or networks, etc. For example, ESX can provide a virtualization software tool that deploys multiple, secure isolated virtual machines on a single server system where system resources can be allocated to any virtual machine based on need. However, these system resource allocations can only take place statically or manually, and can prove to be problematic. For example, if a network administrator over-provisions resources based on a worst case scenario, under-utilization of system resources most likely results. On the other hand, under-provisioning of resources can be equally problematic.
It is desirable for users, such as network administrators, to have reliable and effective tools for dynamically managing (in addition to monitoring) system resources based on real time requirements of, for example, virtual machines in virtual environments, in order to meet growing business demands.
This application describes methods and apparatuses for dynamic CPU resource management. According to one embodiment of the present disclosure, a method for dynamic CPU resource management comprises collecting CPU usage information for a virtual machine, and dynamically changing CPU shares (for example, as a proportional resource unit used to determine how much processor time a virtual machine should use compared with other running virtual machines). The CPU shares of the virtual machine are dynamically changed. The CPU shares of the virtual machine preferably are increased, if it is determined that the CPU usage of the virtual machine relative to CPU usage of other virtual machines is equal to or higher than the allocated CPU share of the virtual machine. On the other hand, if it is determined that the CPU usage of the virtual machine relative to CPU usage of other virtual machines is substantially lower than the allocated CPU share of the virtual machine, the CPU shares of the virtual machine are decreased.
A method, according to another embodiment, for dynamic CPU resource management includes collecting CPU related information and user defined criteria for one or more virtual machines, determining the CPU status for each of the virtual machines, computing a dynamic priority for each of the virtual machines, and determining whether to increase, decrease, or leave as is the CPU shares for each of the virtual machines.
An apparatus for dynamic CPU resource management, according to one embodiment of the present disclosure, includes means for collecting CPU usage information for a virtual machine, and means for dynamically changing CPU shares of the virtual machine, as needed, based on the CPU usage information for the virtual machine and based on a specified priority of the virtual machine.
According to another embodiment of the present disclosure, an apparatus for dynamic CPU resource management includes means for collecting CPU related information and user defined criteria for one or more virtual machines, means for determining the CPU status for each of the virtual machines, means for computing a dynamic priority for each of the virtual machines, and means for determining whether to increase, decrease, or leave as is the CPU shares for each of the virtual machines.
The methods and apparatuses of this disclosure may be embodied in one or more computer programs stored on a computer readable medium or program storage device and/or transmitted via a computer network or other transmission medium. For example, a computer storage medium including computer executable code for dynamic CPU resource management, according to an embodiment of the present disclosure, includes code for code for collecting CPU related information and user defined criteria for one or more virtual machines, code for determining the CPU status for each of the virtual machines, code for computing a dynamic priority for each of the virtual machines, and code for determining whether to increase, decrease, or leave as is the CPU shares for each of the virtual machines.
The features of the present application can be more readily understood from the following detailed description with reference to the accompanying drawings wherein:
The present disclosure provides tools (in the form of methodologies, apparatuses, and systems) for dynamic CPU resource management (also referred to herein as “DCRM”). DCRM according to this disclosure can be utilized to dynamically manage CPU utilization and allocation in virtual machines located in virtual environments. Based on certain predefined criteria and priorities, DCRM can continuously adjust and balance the CPU utilization across a virtual environment, to meet the resource needs of each virtual machine in the environment, while simultaneously following the user-defined priorities in real time. DCRM may employ dynamic programming techniques to detect possible peak scenarios of CPU utilization based on given criteria. The criteria, such as the number of CPUs, the number of virtual machines, the priority assigned to the virtual machines, available resources on the host system, etc., can be established by the user, such as a network administrator, and/or dynamically determined as time progresses in order to satisfy real time requirements.
The following exemplary embodiments are set forth to aid in an understanding of the subject matter of this disclosure, but are not intended, and should not be construed, to limit in any way the claims which follow thereafter. Therefore, while specific terminology is employed for the sake of clarity in describing some exemplary embodiments, the present disclosure is not intended to be limited to the specific terminology so selected, and it is to be understood that each specific element includes all technical equivalents which operate in a similar manner.
The computer system 100 can include a central processing unit (CPU) 102, program and data storage devices 104, a printer interface 106, a display unit 108, a (LAN) local area network data transmission controller 110, a LAN interface 112, a network controller 114, an internal bus 116, and one or more input devices 118 (for example, a keyboard, mouse etc.). As shown, the system 100 may be connected to a database 120, via a link 122.
Methods for dynamic CPU resource management are discussed below.
According to an exemplary embodiment (
A method for dynamic CPU resource management, according to one embodiment (
A method according to another embodiment for dynamic CPU resource management is illustrated in
A dynamic priority can be computed for a virtual machine, using the user defined criteria. The user defined criteria can comprise management criteria, a virtual machine priority, a virtual machine status upper limit, and a virtual machine status lower limit, for each of the virtual machines. User defined criteria can also comprise an analysis interval, and a poll interval, for each server. The management criteria can comprise information indicating virtual machines to be managed, not managed or monitored. The virtual machine status upper limit is the maximum threshold of the virtual machine for CPU usage. The virtual machine status lower limit is the minimum threshold of the virtual machine for CPU usage. The analysis interval is the period of time during which the CPU related information is collected. The poll interval is the period of time between collections of the CPU related information. The CPU related information is preferably analyzed in real time.
An analytical engine can perform dynamic CPU resource management based on CPU related information and user-defined criteria. An engine service can also be provided to assist the analytical engine in managing the virtual machines. The engine service can be, for example, a service on a Windows platform or a daemon on a Linux platform, and can spawn the analytical engine responsible for managing a server. For example, the engine service can read the discovered server objects, such as the virtual machines running on the server, and then spawn the analytical engine to take care of dynamic CPU resource management.
The analytical engine determines whether, based on the CPU status, a CPU can be added to a virtual machine's CPU affinity.
After the analytical engine determines whether a CPU can be added to the virtual machine's CPU affinity, it determines how many CPU shares to add or remove for the virtual machine.
The virtual machines can reside, according to an embodiment of the present disclosure, in a client-server system with one or more management stations. CPU activity, in the system and in particular for each virtual machine, can be monitored, and reported using a reporting tool. The server can be supplemented with additional instrumentation for providing management stations with a comprehensive, real time view of the server. The management stations can pinpoint the servers' status at various levels, such as at the server level or at the virtual machine level. The status of the server and virtual machines can be shown through the reporting tool. Events can be automatically forwarded to event management components, which can further propagate the events to a central management station.
A user, such as a network administrator, can specify attributes (that is, user defined criteria) and their critical values for individual virtual machines. The user can also define a coefficient for each attribute, a virtual machine status upper limit, and a virtual machine status lower limit, for each of the virtual machines located on a server. The user can also define an analysis interval, and a poll interval, for each server. The virtual machine status upper limit can be the maximum threshold of the virtual machine for CPU usage. This value can be set as a percentage. For example, a value of 100 means that the CPU is being completely used. The virtual machine status lower limit can be the lower threshold of the virtual machine for CPU usage. This value can be set as a percentage expressed between 0 and a maximum threshold. For example, if the CPU usage falls below this value, the system is considered underutilized and a CPU may be removed from the affinity (if it does not make the server become critical) and/or CPU shares may be reduced.
An analysis interval may include the period of time, for example, in seconds, during which CPU related information can be collected for analysis of virtual machine CPU status. A poll interval may include the amount of time, for example, in seconds, between collections of CPU related information. The analysis interval is preferably not less than the poll interval. In such an instance, the analysis interval can be reset. For example, the default value of the analysis interval can be set to twice the amount of the poll interval. If the poll interval is, for example, 60 seconds, the analysis interval can be 120 seconds.
A front end graphical user interface (for example, implemented in Java) can be provided to configure the user-defined criteria. The parameters entered by a user can be stored in the reporting tool. The analytical engine can then read those stored values from the reporting tool and use them in its dynamic reconfiguration calculations.
The specific embodiments described herein are illustrative, and many additional modifications and variations can be introduced on these embodiments without departing from the spirit of the disclosure or from the scope of the appended claims. Elements and/or features of different illustrative embodiments may be combined with each other and/or substituted for each other within the scope of this disclosure and appended claims.
Additional variations may be apparent to one of ordinary skill in the art from reading U.S. provisional application Ser. No. 60/573,688, filed May 21, 2004 and entitled “METHOD AND APPARATUS FOR DYNAMIC CPU RESOURCE MANAGEMENT”, the entire contents of which are incorporated herein by reference.
This application claims the benefit of U.S. provisional application Ser. No. 60/573,688, filed May 21, 2004 and entitled “METHOD AND APPARATUS FOR DYNAMIC CPU RESOURCE MANAGEMENT”.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4253145 | Goldberg | Feb 1981 | A |
| 5742762 | Scholl et al. | Apr 1998 | A |
| 5870559 | Leshem et al. | Feb 1999 | A |
| 5889523 | Wilcox et al. | Mar 1999 | A |
| 6115646 | Fizman et al. | Sep 2000 | A |
| 6122664 | Boukobza et al. | Sep 2000 | A |
| 6145001 | Scholl et al. | Nov 2000 | A |
| 6173306 | Raz et al. | Jan 2001 | B1 |
| 6178529 | Short et al. | Jan 2001 | B1 |
| 6226273 | Busuioc et al. | May 2001 | B1 |
| 6304864 | Liddy et al. | Oct 2001 | B1 |
| 6331858 | Fisher | Dec 2001 | B2 |
| 6430592 | Davison | Aug 2002 | B1 |
| 6505217 | Venkatraman et al. | Jan 2003 | B1 |
| 6530840 | Cuomo et al. | Mar 2003 | B1 |
| 6691176 | Narin et al. | Feb 2004 | B1 |
| 6694419 | Schnee et al. | Feb 2004 | B1 |
| 6738886 | Mendoza et al. | May 2004 | B1 |
| 6742099 | Mendoza et al. | May 2004 | B1 |
| 6745312 | Schnee et al. | Jun 2004 | B1 |
| 6848104 | Van Ee et al. | Jan 2005 | B1 |
| 6851030 | Tremaine | Feb 2005 | B2 |
| 6853738 | Nishigaki et al. | Feb 2005 | B1 |
| 6968441 | Schnee | Nov 2005 | B1 |
| 6986137 | King et al. | Jan 2006 | B1 |
| 7080379 | Brenner et al. | Jul 2006 | B2 |
| 7299468 | Casey et al. | Nov 2007 | B2 |
| 7308687 | Trossman et al. | Dec 2007 | B2 |
| 7673114 | Allen et al. | Mar 2010 | B2 |
| 20010028729 | Nishigaki et al. | Oct 2001 | A1 |
| 20020087611 | Tanaka et al. | Jul 2002 | A1 |
| 20020091702 | Mullins | Jul 2002 | A1 |
| 20020173863 | Imada | Nov 2002 | A1 |
| 20020184171 | McClanahan | Dec 2002 | A1 |
| 20030009543 | Gupta | Jan 2003 | A1 |
| 20030037092 | McCarthy et al. | Feb 2003 | A1 |
| 20030158884 | Alford, Jr. | Aug 2003 | A1 |
| 20030182597 | Coha et al. | Sep 2003 | A1 |
| 20030214525 | Esfahany | Nov 2003 | A1 |
| 20030233571 | Kraus et al. | Dec 2003 | A1 |
| 20040143664 | Usa et al. | Jul 2004 | A1 |
| 20040154018 | Doering et al. | Aug 2004 | A1 |
| 20040221121 | Hamilton, II et al. | Nov 2004 | A1 |
| 20040250248 | Halpern et al. | Dec 2004 | A1 |
| 20050015661 | Vaidyanathan | Jan 2005 | A1 |
| 20050038989 | Esfahany | Feb 2005 | A1 |
| 20050044301 | Vasilevsky et al. | Feb 2005 | A1 |
| 20050081201 | Aguilar, Jr. et al. | Apr 2005 | A1 |
| 20050120160 | Plouffe et al. | Jun 2005 | A1 |
| 20050131941 | Dettinger et al. | Jun 2005 | A1 |
| 20050132362 | Knauerhase et al. | Jun 2005 | A1 |
| 20050262505 | Esfahany et al. | Nov 2005 | A1 |
| 20050289145 | Voegel | Dec 2005 | A1 |
| 20060017969 | Ly et al. | Jan 2006 | A1 |
| 20060020781 | Scarlata et al. | Jan 2006 | A1 |
| 20060069761 | Singh et al. | Mar 2006 | A1 |
| 20060136912 | Robinson et al. | Jun 2006 | A1 |
| 20060242641 | Kinsey et al. | Oct 2006 | A1 |
| 20060265711 | Bantz et al. | Nov 2006 | A1 |
| 20070055647 | Mullins et al. | Mar 2007 | A1 |
| 20070079308 | Chiaramonte et al. | Apr 2007 | A1 |
| 20070094367 | Esfahany et al. | Apr 2007 | A1 |
| 20070106769 | Liu | May 2007 | A1 |
| 20070266136 | Esfahany et al. | Nov 2007 | A1 |
| Number | Date | Country |
|---|---|---|
| WO 0138992 | May 2001 | WO |
| WO 02088938 | Nov 2002 | WO |
| WO 03071424 | Aug 2003 | WO |
| WO 03088046 | Oct 2003 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20050262504 A1 | Nov 2005 | US |
| Number | Date | Country | |
|---|---|---|---|
| 60573688 | May 2004 | US |
