Patent Application
20190317867
This invention relates generally to the backup and recovery of virtual machines in a Windows environment, and more particularly to a single unified framework for backing up virtual machines in different configurations and in different environments.
Traditional approaches to Windows hypervisor backups employed volume shadow copy service (VSS) for incremental backups. During a full backup, a VSS snapshot was taken of the volume on which the target virtual machine resides to freeze the virtual machine's state. Once the snapshot was taken, a recovery snapshot and a differencing disk were created for the target virtual machine. The virtual machine was then run on the recovery snapshot, and all writes would be directed to the new differencing disk.
This and other traditional approaches to backup of virtual machines were heavily dependent on the underlying system design. Windows hypervisors provided the user with various configurations to store and use virtual machines. These included a stand-alone local storage-based host; a failover cluster with various hosts having common cluster shared volumes (CSVs); a standalone or failover cluster with an SMB share to store and use virtual machines; or a mix of the foregoing. To handle the various configurations, different writers, different providers, etc. were required, which resulted in different backup architectures and solutions for different configurations. Most approaches also required the active participation from all hosts in the case of a cluster. This resulted in numerous issues. One issue was reliability. The VSS writers were not very reliable and large environments, leading to backup failures, and the virtual machines ran on differencing disks all the time which impacted overall performance. Another issue was scalability. Because the VSS writers on the host were involved in the backups, parallel backups could not be performed, thus leading to serialized backups. Moreover, because the VSS snapshots were taken at the volume level, even if only a single virtual machine was backed up all the virtual machines on the volume were impacted. Additionally, the virtual machines running on differencing disks led to space overhead. A further issue was performance. Backup agents had to be installed on every host because the VSS backup writers were involved, which adversely impacted availability; a lot of data had to be copied; and it took a long time to create volume level snapshots.
To address some of these issues, resilient change tracking (RCT), which is a native Microsoft change block tracker (CBT), was introduced in Windows Server 2016. It made incremental backups faster and easier without the necessity of writing and maintaining different CBT mechanisms by creating for every virtual hard disk files that tracked the changed blocks for each virtual hard disk chain. Instead of having to backup an entire virtual hard disk file, or having to traverse the whole file in order to obtain at the changed blocks, the backup software could query the files and get a list of blocks that had changed and backup only the changed blocks during incremental backups.
While RCT addressed some of the issues with VSS, it did not address all of them. For instance, different workflows were still required for different hypervisor configurations. For example, virtual machines stored on a CSV volume and on a SMB share on the same host cannot be backed up using the same configuration. This required the involvement of their hosts.
It is desirable to provide methods and systems for backing up virtual machines in different hypervisor configurations that address the foregoing and other problems of known backup approaches. In particular, it is desirable to provide a unified backup approach for virtual machines in a user environment that enables machines that are local as well as those that are highly available (CSV and SMB) to be backed up, and that effectively performs load-balancing without requiring the active participation of all the hosts in the user environment. It is to these ends that the present invention is directed.
As will be described, the invention provides a system and method employing a single unified framework process for backing up virtual machines in any of a variety of different configurations and environments by adapting the capabilities and functions of many of the windows-based RCT capabilities for use in a novel sequence of operations comprising the inventive framework. This is done in a manner that insures that a majority of the operations can be performed from a single host of a hypervisor cluster so as not to unduly hinder the availability of other hosts of the cluster, and without requiring different workflows for different configurations.
A workflow process in accordance with the invention can be used to back up machines that are local as well as part of a highly available CSV or SMB group. The workflow process effectively performs load-balancing without requiring the active participation of all of the hosts in the user environment. Furthermore, a majority of the operations can be performed from a single host in the user environment, which results in a number of performance and availability advantages.
In essence, the invention comprises formulating a new unified backup and recovery process comprising a sequence of operations which leverages capabilities of RCT functionality to expand the backup and recovery process to handle virtual machine targets in different environments and on different types of storage. One key factor contributing to this includes affording one configuration that is capable of backing up all virtual machines residing on different storage such as CSV and SMB. Another key factor comprises creating snapshots that are limited to each virtual machine, rather than requiring volume level snapshots as is done in traditional configurations. This affords faster snapshots and frees up resources on other hosts for other operations. A further key factor is the data is sent to the remote backup server not as a file, but as a data stream. This obviates the need to maintain any chains or structures identifying parent-child disks on the server.
As shown in
Once the process of collecting virtual machine data points for all targets has been completed, at 64 the process for checkpointing the targets is performed from a single host. This may be done by using Windows APIs to obtain virtual machine-level snapshots for the target virtual machines, which is in contrast to traditional approaches that required volume level snapshots. Obtaining virtual machine-level snapshots as checkpoints for virtual machine targets, rather than volume level snapshots, substantially reduces the downtime for the targets since individual virtual machines are occupied for only a fraction of the time required for volume level snapshots that necessitated the entire target set to be occupied for the duration of the checkpointing operation.
Once checkpoints have been obtained, at 66 hosts are identified for a backup rollover process, and the entire load of the targets' rollover is balanced among the identified hosts. The identified hosts (proxy hosts) act as open points from where the rollover process for the corresponding targets is initiated. For a full backup, the underlying disks are sent over to the backup server as a data stream. For an incremental backup, the data from the last backup of the target (RCT ID) that identifies extents (location, offset and length) of the chunks of data which may have changed since the last backup may be sent to the remote server. The corresponding locations where the data would be written (merged on top of backup data already present) is identified, and the changed data may be streamed to the remote server where it may be merged into a single disc with the already present data.
Once the rollover data process is complete on all nodes, at 68 the primary node may convert a checkpoint created at 66 into a reference point. This may be done using the initially created snapshot and a RCT ID which is created to track subsequent changes to the target following backup. The reference point is required to track changes following the backup process so that the next backup only backs up incremental data changes of the targets.
After the successful creation of a checkpoint at 68, the primary node at 70 validates the backup and creates a snap-view comprising metadata to store various information data points and properties of the targets and the current backup process. This metadata enables access to the last backup information of a target, and may be used while performing recovery from the backup as well as performing the next incremental backup of the target.
As may be appreciated from the foregoing, a unified backup and recovery process in accordance with the invention which leverages RCT functionality and simplifies and enhances the backup and recovery process and affords a number of advantages over traditional backup and recovery processes. In particular, it permits one configuration to backup all the virtual machines residing on different storage in a virtual machine environment, rather than requiring different backup configurations for different types of storage. Additionally, it permits utilization of snapshots per individual virtual machine, as compared to volume level snapshots which were required in traditional configurations. This affords faster snapshots and frees up resources on other hosts. Moreover, as data is sent to the remote server as a stream rather than a file, is a necessary to maintain change of parent-child disks on the server. This further simplifies the backup and recovery process.
While the foregoing has been with respect to particular embodiments of the invention, it will be appreciated by those skilled in the art the changes to these embodiments may be made without departing from the principles and the spirit of the invention, the scope of which is defined by the appended claims.
