Patent Application
20160364258
Embodiments of the invention relate to the upgrade of virtualization layers in the context of high availability. More specifically, embodiments of the invention relate to the generation of upgrade campaigns for upgrading the virtualization layers.
The Service Availability (SA) Forum has defined the Software Management Framework (SMF) for orchestrating the upgrade of entities within a compliant system. SMF considers only the entities managed by the Availability Management Framework (AMF) and not the lower layers managed by the Platform Management Service (PLM). Consideration of the lower layers within the upgrade campaign allows a system to maintain high availability, such that compatibility among the different layers is maintained throughout the upgrade campaign. This is especially challenging in case of virtualization when virtual machines (VMs) can automatically migrate from one virtual machine monitor (VMM) to another to protect availability.
Some existing tools automate the management of an Information Technology (IT) infrastructure. However, these tools are not designed to address high availability of a system. These tools are not designed to address the coordination of the upgrade of different entities in a system such that the system services can be maintained during the upgrades. The automation of IT infrastructure management is addressed by the SA Forum SMF, which receives an input of an XML file specifying an upgrade campaign. Based on this input, SMF can then orchestrate the execution of the upgrade campaign. The key role that SMF plays is the coordination of the different upgrade steps according to the provided upgrade campaign so that the availability of services is not jeopardized.
However, the SA Forum SMF does not address the upgrade of lower layer entities. Existing tools that addresses the layered architecture of a single host machine fails to address the coordination of the upgrade of host machines that host migration-enabled VMs. Such VMs can be automatically migrated by the system to maintain high availability, which in the context of upgrade may result in incompatibility.
According to one embodiment, a method performed by a computer system is provided for generating an upgrade campaign for runtime execution on a highly available system. The highly available system includes VMs that are enabled for migration between VMMs. The method comprises the steps of: receiving a source configuration and a target configuration of the highly available system as input; generating a configuration change that disables migration of the VMs to be added, modified and removed in the upgrade campaign, wherein the migration is disabled when each of the VMs is anchored on a hosting VMM selected from the input; and identifying a first set, a second set and a third set of sub-trees from tree views of the source configuration and the target configuration. Each sub-tree contains entities representing a subset of the VMs and the VMMs. Each sub-tree of the first set includes entities tagged for addition only, each sub-tree of the second set includes entities tagged for modification, and each sub-tree of the third set includes entities tagged for removal only. The method further comprises the steps of: generating an upgrade procedure for each sub-tree of the first set, the second set and the third set, with an indication that upgrade procedures of the first set are to be executed first and upgrade procedures for the third set are to be executed last in the upgrade campaign; and generating another configuration change that re-enables the migration of the VMs according to the target configuration.
According to another embodiment, a computer system is provided to generate an upgrade campaign for runtime execution on a highly available system. The highly available system includes VMs that are enabled for migration between VMMs. The computer system comprises a memory to store a source configuration and a target configuration of the highly available system. The computer system further comprises one or more processors coupled to the memory. The one or more processors are adapted to: receive the source configuration and the target configuration as input; generate a configuration change that disables migration of the VMs to be added, modified and removed in the upgrade campaign, wherein the migration is disabled when each of the VMs is anchored on a hosting VMM selected from the input; and identify a first set, a second set and a third set of sub-trees from tree views of the source configuration and the target configuration. Each sub-tree contains entities representing a subset of the VMs and the VMMs. Each sub-tree of the first set includes entities tagged for addition only, each sub-tree of the second set includes entities tagged for modification, and each sub-tree of the third set includes entities tagged for removal only. The one or more processors are further adapted to: generate an upgrade procedure for each sub-tree of the first set, the second set and the third set, with an indication that upgrade procedures of the first set are to be executed first and upgrade procedures for the third set are to be executed last in the upgrade campaign; and generate another configuration change that re-enables the migration of the VMs according to the target configuration.
According to yet another embodiment, a system is provided for generating an upgrade campaign for runtime execution on a highly available system. The highly available system includes VMs that are enabled for migration between VMMs. The system comprises: a receiving module configured to receive a source configuration and a target configuration of the highly available system as input; a first configuration change generating module configured to generate a configuration change that disables migration of the VMs to be added, modified and removed in the upgrade campaign, wherein the migration is disabled when each of the VMs is anchored on a hosting VMM selected from the input; and an identifying module configured to identify a first set, a second set and a third set of sub-trees from tree views of the source configuration and the target configuration. Each sub-tree contains entities representing a subset of the VMs and the VMMs. Each sub-tree of the first set includes entities tagged for addition only, each sub-tree of the second set includes entities tagged for modification, and each sub-tree of the third set includes entities tagged for removal only. The system further comprises an upgrade procedure generating module configured to generate an upgrade procedure for each sub-tree of the first set, the second set and the third set, with an indication that upgrade procedures of the first set are to be executed first and upgrade procedures for the third set are to be executed last in the upgrade campaign; and a second configuration change generating module configured to generate another configuration change that re-enables the migration of the VMs according to the target configuration.
The present invention is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like references indicate similar elements. It should be noted that different references to “an” or “one” embodiment in this disclosure are not necessarily to the same embodiment, and such references mean at least one. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to effect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.
In the following description, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In other instances, well-known circuits, structures and techniques have not been shown in detail in order not to obscure the understanding of this description. It will be appreciated, however, by one skilled in the art, that the invention may be practiced without such specific details. Those of ordinary skill in the art, with the included descriptions, will be able to implement appropriate functionality without undue experimentation.
Embodiments of the invention generate upgrade campaigns for upgrading lower layers of a highly available system. The upgrade campaigns can be executed by an extension of SMF. The input to the upgrade campaign generation is the current (i.e., source) configuration and the target configuration of the lower layers of the system. The upgrade includes addition, modification, and/or removal of various entities of the system.
The term “lower layers” herein refers to all of the layers including and below the middleware defined in the SA Forum. Examples of a lower-layer entity include an operating system (OS), a virtual machine (VM), and a virtual machine monitor (VMM). When applying an upgrade to a lower layer, an upgrade campaign can be executed at runtime to perform version change, base type change, new installation, removal of an entity, among other actions.
The upgrade campaign generation described herein allows a controlled runtime upgrade of lower layers of a system so that high availability of the system is not jeopardized during the upgrade. While the upgrade campaign generation is presented in the context of the SA Forum specifications, the principles described herein can be applied to other systems or frameworks that support highly available services.
In one embodiment, the upgrade campaign generation includes three phases. To render the upgrade predictable and controllable, in the first phase the entities representing the VMs are caught (also referred to as “anchored” or “demobilized”) on selected VMMs; that is, VM migration is disabled. In the second phase entities are upgraded according to a schedule, which orders the execution of upgrade procedures in such a way that the upgrade causes no loss of service. In the last phase the VM migration is re-enabled according to the target configuration.
An example is provided below to explain how a system that supports VMMs and migration-enabled VMs is upgraded at runtime without loss of services provided by an application. In the example below, the suffixes of the names of the VMs and VMMs indicate the upgrade operation to be performed on them. Specifically, ‘-R’ indicates removal, ‘-A’ indicates addition and ‘-M’ indicates modification. Entities to be removed are present in the source configuration only, entities to be added are present in the target configuration only, and entities to be modified are present in both the source and target configurations under the same name.
In one embodiment, application services provided by the system are configured in such a way that for all the application components hosted on VM1-R, VM2-R, VM1-M and VM2-M, there is a redundant application component present on VM3-M. It is understood that other VMs or VMMs may also exist in the system to provide redundancy but they are omitted for simplicity of the description.
In this example, it is assumed that the VMs that provide redundancy (and therefore protecting some other VMs at the application level) share no VMM with those protected VMs. Thus, VMM4-M sponsors VM3-M only. As VM3-M hosts all the redundant application components, VM3-M is not sponsored from any of VMM1-R and VMM1-M.
To prepare for the upgrade of the VMs, in the first phase the VMs are demobilized such that each VM resides on one VMM only. In one embodiment, the VMs' dependencies are modified such that each VM has only one hosting VMM. As a result, each VM is caught on a single VMM and cannot migrate even if that hosting VMM goes out of service. After the demobilization, the VMs and the hosting VMM can be upgraded using an upgrade step (e.g., an embedded upgrade step) and their compatibility can be maintained.
The process of capturing VMs on a single VMM is performed at the beginning of the upgrade campaign for the VMs that are to be removed or modified. VMs that are to be added to the system are not yet present at the beginning of the upgrade campaign, and, therefore, no demobilization is applied to these added VMs. When a VM is added during the upgrade, it is added with a single hosting VMM selected from any of its hosting VMMs present in the target configuration.
A VM that is to be removed or added can be caught on any of its eligible hosting VMMs; however, a VM that is to be modified can only be caught on a VMM that is also to be modified or remains unchanged in the upgrade. That is, modified VMs are not caught on the VMMs that are to be removed or added. This is because the VMMs that are to be removed or added are present only in one of the source and target configurations.
In this example VM2-M and VM1-M are caught on VMM3-M, all of which are to be modified. VMs that are to be removed (VM1-R, VM2-R) are caught on the VMM that is to be removed (VMM1-R). Catching the VMs transforms the source configuration to an intermediate configuration shown in the example of
In the second phase the entities are upgraded according to the source and target configurations. The upgrade of different entities are scheduled such that the execution does not cause any application service outage, even in the special case when all VMs of the source configuration that host application components to provide a given service are removed from the system and replaced by VMs added by the target configuration that host the new application components for that same service. To avoid service outage, the execution of the addition of entities is scheduled first so that these added entities are already able to provide the services when the removal and modification of the entities of the source configuration are performed.
Accordingly, in the example configuration VM1-A, VM2-A and VMM2-A are added first. At this time the VMs are added with one hosting VMM (i.e., VMM2-A in this example).
Next, modification and removal of the entities are performed. The modification of VMM4-M and VM3-M is performed before or after the modification of VMM3-M, VM1-M and VM2-M, but not in parallel.
In the last phase the VM migration is re-enabled. The dependencies for the migration-enabled VM are modified according to the target configuration. In the example, the dependency objects of VM1-A and VM2-A are modified to be sponsored from VMM2-A, and the dependency objects of VM1-M and VM2-M are modified to be sponsored from VMM3-M. The result is the target configuration as shown in the example of
To generate an upgrade campaign that specifies the three-phase execution described above, an upgrade campaign generator receives a source configuration and a target configuration as input. The source and target configurations are compared to identify the upgrade target (i.e., the entities to be added, removed or modified by the upgrade campaign). The configuration models are extended with the information and relations for the upgrade campaign generation. For each migration-enabled VM to be upgraded, an appropriate hosting VMM is selected for that VM. That is, each VM is caught on a single VMM and this relation is added to the tree view of the configuration.
The upgrade campaign generator generates an initialization section of the upgrade campaign. This section, when executed, performs a configuration change to modify the VM dependency information for the VMs to be removed and modified according to the VM-VMM relations set up previously.
Subsequently, the upgrade campaign generator generates the campaign body section as a collection of single-step upgrade procedures. First, a set of upgrade procedures for the added entities is generated. These upgrade procedures are set to one or more first execution levels (e.g., the lowest execution level) to indicate that they are to be executed first. These addition upgrade procedures may be executed in parallel or in sequence. Secondly, a set of upgrade procedures is generated for the modified entities and removed entities that are hosted on the modified entities. These upgrade procedures are set to sequential execution levels so that they are executed in sequence, starting after the completion of the addition upgrade procedures. Thirdly, a set of upgrade procedures for the remaining removed entities is generated. These upgrade procedures are set to execution levels higher than all of the previous upgrade procedures to indicate that they are to be executed last. They may be executed in parallel or in sequence.
Finally, the upgrade campaign generator generates a campaign wrap-up section. This section, when executed, performs a configuration change to modify VM dependencies such that the dependencies match the target configuration; that is, VM migration is re-enabled.
The “single-step upgrade procedure” as used herein can be a single upgrade step as defined by the SMF specification, or, alternatively, an embedded step. An upgrade step generally includes three phases that are executed in sequence: the tear-down phase (T), the reconfiguration phase (R) and the build-up phase (B). An upgrade step is completed when all three phases are completed. For example, in a rolling upgrade that includes three upgrade steps, each step will be executed to completion in sequence as: {T1, R1, B1}; {T2, R2, B2}; {T3, R3, B3}. An embedded step is different in that it reorders this execution sequence by nesting the upgrade steps. When nesting the steps, first the tear down phase of each nested step is executed, then the reconfiguration phase of each nested step is executed, and finally the build-up phase of each nested step is executed. So the three phases are still clearly separated and executed in sequence for the entire embedded step. In the above three-step example, the three nested steps of an embedded step will be executed in sequence as: {T1, T2, T3, R1, R2, R3, B3, B2, B1}. It is noted that the tear-down phases (T1, T2, T3) of the embedded step starts from the outer most nested step (T1) while the build-up phase (B3, B2, B 1) starts from the inner most nested step (B3). Thus, the “single-step upgrade procedure” herein can be a single upgrade step (e.g., {T, R, B}) or, alternatively, an embedded step that includes any number of nested steps (e.g. {T1, T2, . . . , Tn, R1, R2, Rn, Bn, . . . , B2, B1}.
In one embodiment, the upgrade campaign generator is part of the system that executes the upgrade campaign. In an alternative embodiment, the upgrade campaign generator resides in a system different from the system executing the upgrade campaign. Further, a system may start executing an upgrade campaign after the generation of the upgrade campaign is completed, or while the upgrade campaign is being generated.
To identify the upgrade operation for each EE in the system, these tree views of the source and target configurations are compared using the distinguished names (DN) of the EEs as the basis. If a DN of an EE is found in the source configuration but not in the target configuration, that EE is tagged for the REMOVAL operation in the source configuration (as indicated by the suffix ‘It’ in the EE's name). If a DN of an EE is found in the target configuration, but not in the source configuration, that EE is tagged for the ADDITION operation in the target configuration (as indicated by the suffix ‘A’ in the EE's name). If a DN of an EE is found in both the source configuration and the target configuration, then the objects in the PLM information model that represent the EE before and after the upgrade are compared. More specifically, the values of all of the attributes of the object in the source configuration and the object in target configuration are compared. If the attribute values in the source configuration and the target configuration are same, the EE is tagged for NO-OPERATION. Otherwise, it is tagged for the MODIFY operation in both configurations (as indicated by the suffix ‘M’ in the EE's name).
Referring again to
Referring again to
Subsequently, upgrade procedures are generated for the addition of entities (650), modification of entities and removal of their hosted entities (if any) (660), and the remaining removal of entities (670).
For each identified sub-tree, a single-step upgrade procedure is generated. In one embodiment, the execution level of the generated procedures is set to the lowest (e.g., execLevel=0) level for all of the EEs to be added. In an alternative embodiment, the execution level for each sub-tree is incremental, such that the upgrade of different sub-trees is executed sequentially (1104).
Within the single-step upgrade procedure, a separate nested step is generated for each layer of the hosted EEs using the information of the target configuration. Entities are in the same layer of a sub-tree if they have the same distance (i.e., the same number of hops) from the root of the sub-tree. Each of the nested steps includes a sequence of actions for upgrading a layer of the entities in the sub-trees. In one embodiment, sub-tree-height and embedding are set to be the number of levels in the sub-tree (1105), where embedding is used as a down-counter to keep track of the levels of sub-trees that have yet to be processed. All the EEs at the embedding level are selected (1106), and a nested step is created for all of the selected EEs at that embedding level (1107). Then embedding is decremented, and the nested step is added to the upgrade procedure for the sub-tree at the level of (sub-tree-height—embedding) (1108). The operations of blocks 1106-1108 are repeated until embedding reaches zero (1109), at which point the next unprocessed sub-tree (if any) is identified and processed. When all of the sub-trees for ADDITION are processed, the method 1100 is terminated.
For each identified sub-tree, a single-step upgrade procedure is generated. In one embodiment, the execution level for each sub-tree is incremental, such that the upgrade of different sub-trees is executed sequentially (1204).
Within the single-step upgrade procedure, a separate nested step is generated for each layer of the hosted EEs. Each of the nested steps includes a sequence of actions for upgrading a layer of the entities in the sub-trees. For each nested step, the entities taken out of service are the selected EEs in the source configuration, while the entities put back to service are the remaining equivalent entities in the target configuration. The execution level of the upgrade procedures created for the different sub-trees is incrementally increased so that these upgrade procedures will be executed in sequence. The sequential execution ensures that redundant entities are not taken out of service simultaneously.
In one embodiment, sub-tree-height and embedding are set to be the number of levels in the sub-tree (1205), where embedding is used as a down-counter to keep track of the levels of sub-trees that have yet to be processed. All the EEs at the embedding level are selected (1206), and a nested step is created for all of the selected EEs at that embedding level (1207). Then embedding is decremented, and the nested step is added to the upgrade procedure for the sub-tree at the level of (sub-tree-height—embedding) (1208). The operations of blocks 1206-1208 are repeated until embedding reaches zero (1209), at which point the next unprocessed sub-tree (if any) is identified and processed. When all of the sub-trees for MODIFY are processed, the method 1200 is terminated.
For each identified sub-tree, a single-step upgrade procedure is generated. The execution level of the generated procedures is equal to or higher than the execution levels of the upgrade procedures generated for the MODIFY operation. In one embodiment, the execution level of the generated procedures is set to the maximum level for all of the EEs to be removed. In an alternative embodiment, the execution level for each sub-tree is incremental, such that the upgrade of different sub-trees is executed sequentially (1304).
Within the single-step upgrade procedure, a separate nested step is generated for each layer of the hosted EEs using the information of the source configuration. Each of the nested steps includes a sequence of actions for upgrading a layer of the entities in the sub-trees. In one embodiment, sub-tree-height and embedding are set to be the number of levels in the sub-tree (1305), where embedding is used as a down-counter to keep track of the levels of sub-trees that have yet to be processed. All the EEs at the embedding level are selected (1306), and a nested step is created for all of the selected EEs at that embedding level (1307). Then embedding is decremented, and the nested step is added to the upgrade procedure for the sub-tree at the level of (sub-tree-height—embedding) (1308). The operations of blocks 1106-1108 are repeated until embedding reaches zero (1309), at which point the next unprocessed sub-tree (if any) is identified and processed. When all of the sub-trees for REMOVAL are processed, the method 1300 is terminated.
Referring again to
The highly available system (that is, the system that is to execute the resulting upgrade campaign) includes VMs that are enabled for migration between VMMs. In one embodiment, the method 1400 begins with receiving a source configuration and a target configuration of the highly available system as input (1401). A configuration change is generated, which disables migration of the VMs that are to be added, modified and removed in the upgrade campaign (1402). The migration is disabled when each of the VMs is anchored on a hosting VMM selected from the input. Then a first set, a second set and a third set of sub-trees are identified from tree views of the source configuration and the target configuration (1403). Each sub-tree contains entities representing a subset of the VMs and the VMMs. Each sub-tree of the first set includes entities tagged for addition only, each sub-tree of the second set includes entities tagged for modification, and each sub-tree of the third set includes entities tagged for removal only. The method 1400 proceeds to generate an upgrade procedure for each sub-tree of the first set, the second set and the third set, with an indication that upgrade procedures of the first set are to be executed first and upgrade procedures for the third set are to be executed last in the upgrade campaign (1404). Then the method 1400 proceeds to generate another configuration change that re-enables the migration of the VMs according to the target configuration (1405).
The method 1400 may be performed by hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions run on a processing device), or a combination thereof. In one embodiment, the method 1400 may be performed by a system 1500 of
The system 1500 comprises a receiving module 1510 configured to receive a source configuration and a target configuration of the highly available system as input; a first configuration change generating module 1520 configured to generate a configuration change that disables the migration of the VMs that are to be added, modified and removed in the upgrade campaign. The migration is disabled when each of the VMs is anchored on a hosting VMM selected from the input; and an identifying module 1530 configured to identify a first set, a second set and a third set of sub-trees from tree views of the source configuration and the target configuration. Each sub-tree contains entities representing a subset of the VMs and the VMMs. Each sub-tree of the first set includes entities tagged for addition only, each sub-tree of the second set includes entities tagged for modification, and each sub-tree of the third set includes entities tagged for removal only. The system 1500 further comprises an upgrade procedure generating module 1540 configured to generate an upgrade procedure for each sub-tree of the first set, the second set and the third set, with an indication that upgrade procedures of the first set are to be executed first and upgrade procedures for the third set are to be executed last in the upgrade campaign. The system 1500 further comprises a second configuration change generating module 1550 configured to generate another configuration change that re-enables the migration of the VMs according to the target configuration.
The computer system 1600 includes a processing device 1602. The processing device 1602 represents one or more general-purpose processors, each of which can be: a microprocessor, a central processing unit (CPU), a multicore system, or the like. The processing device 1602 may also be one or more special-purpose processing devices such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), network processor, or the like. In one embodiment, the processing device 1602 is adapted or operative to execute the operations of a campaign generation logic 1622 which contains instructions executable by the processing device 1602 to perform the method 600 of
In one embodiment, the processor device 1602 is coupled to one or more memory devices such as: a main memory 1604 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM), etc.), a secondary memory 1618 (e.g., a magnetic data storage device, an optical magnetic data storage device, etc.), and other forms of computer-readable media, which communicate with each other via a bus or interconnect 1630. The memory devices may also include different forms of read-only memories (ROMs), different forms of random access memories (RAMs), static random access memory (SRAM), or any type of media suitable for storing electronic instructions. In one embodiment, the memory devices may store the code and data of the campaign generation logic 1622. In the embodiment of
In one embodiment, the computer system 1600 is adapted or operative to perform the method 1400 of
The computer system 1600 may further include a network interface device 1608. A part or all of the data and code of the campaign generation logic 1622 may be transmitted or received over a network 1620 via the network interface device 1608.
In one embodiment, the campaign generation logic 1622 can be implemented using code and data stored and executed on one or more computer systems (e.g., the computer system 1600). Such computer systems store and transmit (internally and/or with other electronic devices over a network) code (composed of software instructions) and data using computer-readable media, such as non-transitory tangible computer-readable media (e.g., computer-readable storage media such as magnetic disks; optical disks; read only memory; flash memory) and transitory computer-readable transmission media (e.g., electrical, optical, acoustical or other form of propagated signals—such as carrier waves, infrared signals). A non-transitory computer-readable medium of a given computer system typically stores instructions for execution on one or more processors of that computer system.
The operations of the flow diagrams of
It is to be understood that the above description is intended to be illustrative, and not restrictive. Many other embodiments will be apparent to those of skill in the art upon reading and understanding the above description. Accordingly, the specification and drawings are to be regarded in an illustrative sense rather than a restrictive sense.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/IB2014/059505 | 3/6/2014 | WO | 00 |
