Patent Application
20220027189
The following relates to systems and methods for determining optimal placements of virtual machines (VMs) on hypervisor hosts; and for generating corresponding VM/host placement rules, particularly for virtual and cloud computing environments.
Virtual and cloud computing environments are comprised of one or more physical hypervisor hosts that each run zero or more VMs. These virtual environments are typically managed by a virtual machine manager (VMM) that can organize the hypervisor hosts into one or more groups (often referred to as “clusters”), for performing management functions. Many virtualization technologies allow VMs to be live migrated between hosts with no downtime. Some virtualization technologies leverage the live migration capability by automatically balancing the VM workloads across the hosts comprising a cluster on a periodic basis. Similarly, some virtualization technologies also support the ability to automatically minimize the host footprint of the running VMs to conserve power. These automated load balancing and power saving capabilities typically operate within the scope of a virtual cluster.
Such VM-to-host placements are normally subject to host level resource constraints (e.g. CPU, memory, etc.) as well as static, user-defined VM-VM affinity, VM-VM anti-affinity, VM-host affinity and VM-host anti-affinity rules. These static placement rules can be used for various purposes such as:
Determining placement constraints and placement rules for a given computing environment can be time consuming, particularly when done on an ad hoc basis. It is an object of the following to address at least one of the above concerns.
In one aspect, there is provided a method of determining host assignments for sub-groups of virtual machines (VMs) in a computing environment comprising a plurality of hosts, each host configured for hosting zero or more VMs, the method comprising: determining at least one sub-group of VMs from an overall set of VMs, according to at least one technical or business criterion; and determining, for each sub-group of VMs, a particular set of hosts from the plurality of hosts to be assigned to that sub-group of VMs, based on at least one of: VM-host compatibilities, and existing VM-host placements.
In other aspects there are provided computer readable media and systems configured for performing the method.
Embodiments will now be described by way of example with reference to the appended drawings wherein:
It has been found that existing technologies do not support the ability to automatically determine the placement constraints and generate the corresponding placement rules. The following provides a system and method to address this need. Common use cases for such dynamic VM-host placement constraints and rules are to:
The following systems and methods are also found to be applicable to container technologies (e.g. Docker, Linux Containers) that can run multiple container workloads on container hosts. Containers and container hosts are analogous to the VMs and the hypervisor hosts. Containers also support mobility between container hosts, typically by stopping a container workload on one host, and starting a corresponding container on a different host. This technology is also applicable to routing workloads to the optimal virtual clusters while considering the compatibility and available capacity of the incoming workload and clusters.
In general, the following provides and exemplifies a model of a virtual computing environment, and provides an example of host-based licensing optimization scenario. Also provided are policies for placing VMs on hosts, and policies for optimizing VM sub-groups of an overall set of VMs in a computing environment. The system is configured to determine optimal number of hosts required per VM sub-group, determine optimal set of hosts for each VM sub-group, and deploy placement rules to enforce VM-host affinity placements. The placement affinity rules can be specified (e.g., by codifying) the relationship between the VM sub-group and the host sub-group in the VMM.
Turning now to the figures,
Data is collected from the environment 10 to determine the configuration and capacity of existing hosts 16 and of hosts 16 to be added or removed, and to determine the configuration, allocation, and utilization of existing VMs 18 and VMs 18 to be added or removed. The data collected can also be used to determine the existing VM placements, e.g., as shown schematically in
An example of a host-based licensing scenario is shown in
As illustrated in
The optimized VM placements are determined by the analysis engine 20 and are subject to the VM-host placement policies 22 that constrain the amount of resources that can be consumed by VMs on each host and the VM sub-group optimization policies 24 that dictate how to optimize the VM sub-group placements.
To determine the membership of the VM sub-groups to run on a minimum host footprint (i.e. an optimal or otherwise minimal set of hosts), VMs 18 requiring a host-based software license can be determined through discovery or imported from a configuration management database (CMDB). Also, VMs 18 in the data model are tagged based on their VM sub-group memberships, and VMs 18 can belong to one VM sub-group at a time. If VMs 18 are using more than one software license (e.g. Windows® and SQL server®), the VMs can be grouped onto multiple VM sub-groups (e.g. group with Windows® and SQL server®, and group with Windows® and no SQL server®).
The primary constraint is determined for each VM sub-group by computing the minimum number of hosts required to run the VMs based on each resource constraint being modeled (e.g. CPU overcommit, memory allocation, CPU utilization, memory utilization, etc.). For each resource constraint, the total resource allocations (e.g. virtual CPUs, memory allocations) or total resource utilization (CPU used, memory used, disk I/O activity, network activity) of the VMs in the VM sub-group is computed and compared against the corresponding useable resource capacity of the hosts. The useable host capacity is based on the actual host capacity and the corresponding resource limit specified through the policies. For example, the total CPU allocation for a VM sub-group is the sum of the virtual CPU allocations of the VMs. The useable CPU allocation capacity for a host is the number of CPUs of the host multiplied by the host CPU allocation limit. Similar calculations are performed for other potential resource constraints, the resource constraint that requires the most number of hosts for the VM sub-group is considered to be the primary constraint. If more than one resource constraint requires the same number of hosts, the primary constraint may be determined by considering the fractional hosts required (e.g. if CPU allocation requires 1.5 hosts and memory allocation requires 1.6 hosts, CPU allocation is considered to be the primary constraint).
If the total estimated number of hosts 16 required for all the VM sub-groups exceeds the actual number of hosts 16 (determined at 60), the fair share rule can be used to allocate the number of hosts 16 per VM sub-group, i.e. by allocating the number of hosts for each VM sub-group by pro-rating the available hosts (62).
However, if the estimated number of hosts 16 required is less than the actual number of hosts 16 (as determined at 60), the number of hosts for each VM sub-group is allocated via a permutation stacking analysis (64). The permutation stacking analysis can be performed by first sorting the VM sub-groups from largest to smallest based on the primary constraint. Then, for each group, the permutation analysis is performed by stacking the VMs 18 on the hosts 16 to ensure that the VMs 18 fit. This analysis may find that more hosts 16 are required.
The hosts are then assigned to the determined groups as required to output minimum number of host allocations for each VM sub-group (66).
To illustrate the process flow in
Based on primary resource constraints (e.g. memory), the estimated number of hosts for G1, G2 and G3 are 4, 3, 2, and thus the total number of estimated hosts=9. It may be noted that each group can have different primary constraints.
In this example, the total estimated # hosts=9>actual # hosts=6.
In applying fair share, the # of hosts are allocated to the groups as follows:
G1=4*6/9=2.67;
G2=3*6/9=2; and
G3=2*6/9=1.33.
To allocate hosts, a floor value for each group is determined as follows:
G1=2;
G2=2; and
G3=1.
Next, the sum of allocated hosts is computed, and is=5, so one host is available. The available host to group with the largest remainder (i.e. G1 with 0.67 in this example) is allocated, and the final host allocation is:
G1=3;
G2=2;
G3=1.
The optimal hosts 16 for the VM sub-groups are also determined. This process chooses the best hosts 16 for each VM sub-group, accounts for existing VM-host affinity and VM-host anti-affinity rules, and can favor current placements to minimize volatility in implementing the placement plan, and assigns hosts 16 to a host group associated with a VM-host affinity placement rule.
Using VM-host placement rules for affinity and anti-affinity (70), a VM-host compatibility score is computed (72) for each VM-host pair based on the placement rules. A normalized VM-host compatibility score is computed (74) for each VM-host pair based on the placement rules, and a VM-group-host compatibility score is computed (76) for each group-host pair based on the placement rules.
Using the current VM placements on the hosts 16 (78), a VM-host compatibility score is computed (80) for each VM-host pair based on the current placements. A normalized VM-host compatibility score is computed (82) for each VM-host pair based on the current placements, and a group-host compatibility score is computed (84) for each group-host pair based on the current placements.
The group-host compatibility scores based on the placement rules and the current placements are then used to compute an overall group-host compatibility score (86) for each group-host pair, based on a weighting factor (88) and such scores (76, 84) from the current placements and placement rules.
From the overall group-host compatibility scores (86), a VM sub-group is chosen to process (90). The group-host compatibility metrics and the number of allocated hosts are used to select the optimal host assignments for the group (92). This is done by comparing group-host scores to choose the most suitable hosts for a group of VMs 18. For example, the largest group may be chosen first.
After assigning hosts to a VM sub-group, the process then determines if any group exists with unassigned hosts (94). In this way, the process re-computes group-host compatibility scores (96) based on the remaining groups and hosts until there are no additional unassigned hosts and the process is done. The output is a set of one or more VM sub-groups with optimal host assignments (98).
An example will now be provided, making reference to the tables shown in
The VM-host compatibility scores may also be based on the current VM placements on the hosts. For the current placements, the VM-host compatibility of 100 indicates that the VM is currently placed on the given host and 0 indicates VM is not placed on the given host.
When computing the compatibility scores, as shown by way of example below, when there is not full compatibility (with a score of 100) or complete incompatibility (with a score of zero), any one of a variety of scoring mechanisms can be used to assign a score between zero and 100 for partial compatibility.
For a given VM-host pair, the normalized score is compute as follows:
Normalized score for V(n)-Host(n)=compatibility score of V(n)-Host(n)/sum of scores of V(n)-Host(i=1 to h). For example, the normalized score for V1-Host1=100/(100+0+100)=0.5.
In the example shown in
Turning now to
SUM(Normalized VM-host scores of current group)−SUM(Normalized VM-host scores of other groups); and
Compatibility score for G1-Host1=(NS(V1)+(V2))−(NS(V3)+NS(V4)+NS(V5)+NS(V6)).
In this example, the compatibility score for G1-Host1=(0.5+0.5)−(0+0+0.33+0.33)=0.33.
These group-host scores provide a relative measure for selecting optimal hosts for VM sub-groups to maximize the overall compatibility for all the groups across the available hosts. That is, the group-host scores consider not only the compatibility of that group with that host, but also how compatible other groups are with that host to optimize assignments across the board.
In the example shown in
The group-host compatibility scores based on the current placements are shown in
The overall group-host compatibility scores for this example are shown in
The overall score is then computed as:
Overall score=current placement weight*current placement compatibility score+rules weight*rules compatibility score.
In the example scores shown in
New VM placement rules are deployed for VM-host group placement optimization in order to: replace existing dynamic rules, and optionally move VMs 18 to the optimal hosts 16. The environment 10 can be re-analyzed periodically and placement rules can be replaced as needed (110).
For simplicity and clarity of illustration, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous elements. In addition, numerous specific details are set forth in order to provide a thorough understanding of the examples described herein. However, it will be understood by those of ordinary skill in the art that the examples described herein may be practiced without these specific details. In other instances, well-known methods, procedures and components have not been described in detail so as not to obscure the examples described herein. Also, the description is not to be considered as limiting the scope of the examples described herein.
It will be appreciated that the examples and corresponding diagrams used herein are for illustrative purposes only. Different configurations and terminology can be used without departing from the principles expressed herein. For instance, components and modules can be added, deleted, modified, or arranged with differing connections without departing from these principles.
It will also be appreciated that any module or component exemplified herein that executes instructions may include or otherwise have access to computer readable media such as storage media, computer storage media, or data storage devices (removable and/or non-removable) such as, for example, magnetic disks, optical disks, or tape. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. Examples of computer storage media include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by an application, module, or both. Any such computer storage media may be part of the any component described herein or accessible or connectable thereto. Any application or module herein described may be implemented using computer readable/executable instructions that may be stored or otherwise held by such computer readable media.
The steps or operations in the flow charts and diagrams described herein are just for example. There may be many variations to these steps or operations without departing from the principles discussed above. For instance, the steps may be performed in a differing order, or steps may be added, deleted, or modified.
Although the above principles have been described with reference to certain specific examples, various modifications thereof will be apparent to those skilled in the art as outlined in the appended claims.
The present application is a continuation of U.S. application Ser. No. 16/774,193 filed on Jan. 28, 2020; which is a continuation of U.S. application Ser. No. 15/384,107 filed on Dec. 19, 2016, which is a continuation of PCT Application No. PCT/CA2015/050575 filed on Jun. 22, 2015, which claims priority to U.S. Provisional Application No. 62/015,183 filed on Jun. 20, 2014, which are incorporated herein by reference in their entirety.
| Number | Date | Country | |
|---|---|---|---|
| 62015183 | Jun 2014 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16774193 | Jan 2020 | US |
| Child | 17493096 | US | |
| Parent | 15384107 | Dec 2016 | US |
| Child | 16774193 | US | |
| Parent | PCT/CA2015/050575 | Jun 2015 | US |
| Child | 15384107 | US |
