Virtual machines (VMs) may be used to execute a variety of applications at a computing device. For example, VMs may execute database workloads, file sharing workloads, file server workloads, and web server workloads. One or more workloads executed by a VM may be a mission-critical workload at an enterprise. Frequently backing up such a VM may be important to maintain data redundancy at the enterprise. When a VM is shared by computing devices, backup methodologies in certain environments may not be supported, since the VM may incur modifications from multiple computing devices.
The present disclosure describes backup methods to achieve fast and complete backups (i.e., “express-full”) backups of a virtual machine that is shared between multiple computing devices in a cluster. As an example, each computing device in the cluster may modify the shared virtual machine via a direct input/output (I/O) transaction, bypassing a file-system stack. The backup methods of the present disclosure may reduce an amount of data transferred during a backup operation and may enable granular recovery at a backup device (e.g., a backup server). For example, the backup methods may enable express-full backups of Hyper-V virtual machines in a cluster shared volume (CSV) environment.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
In a particular embodiment, a computer-implemented method includes creating a first snapshot of at least one virtual machine (VM) at a first time. The first snapshot is created at a computing device of a cluster of computing devices configured to share the at least one virtual machine. The first snapshot is transmitted to a backup device such as a backup server. The method includes creating a second snapshot of the at least one virtual machine at a second time and determining a set of changed data blocks associated with a difference between the second snapshot and the first snapshot. The set of changed blocks is transmitted to the backup device. In one embodiment, the first snapshot is created at a virtual machine level and the second snapshot is created at a computing device level. For example, the at least one VM may include a volume filter that tracks changes of one or more volumes of the at least one VM after the first snapshot is created.
In another particular embodiment, a computer-implemented method includes creating a first snapshot of a virtual machine (VM) comprising a virtual hard drive (VHD). A first differencing virtual hard drive captures modifications to the virtual hard drive after the first snapshot is created. The first snapshot is created at a computing device of a cluster of computing devices, where the cluster is configured to share the virtual machine. The method includes creating a shadow copy of the virtual hard drive and transmitting a copy of the virtual hard drive and the first differencing virtual hard drive to a backup device.
In another particular embodiment, a computer-readable medium is disclosed that includes instructions that are executable by a computing device. The computing device generates a start transaction message indicating that a file of a virtual machine shared via a cluster shared volume (CSV) is open for a direct input/output (IO) transaction. The computing device sets a dirty flag of the virtual machine in response to the start transaction message. The computing device generates one or more bitmasks (e.g., direct IO bitmasks) identifying blocks of the file modified during the direct IO transaction. In a particular embodiment, a File System Filter driver (e.g., a CSV filter) generates at least one of the bitmasks. The computing device sends the one or more bitmasks to a backup device records one or more changes to the virtual machine based on the one or more bitmasks. The computing device generates an end transaction message indicating that the direct IO transaction is complete and clears the dirty flag in response to the end transaction message.
Referring to
The first computing device 108 is configured to share a first Hyper-V virtual machine (VM) 120 that includes a first virtual hard drive (VHD) 122. To illustrate, the first computing device 108 may include a parent partition configured to execute a host operating system, and the first Hyper-V VM 120 may be executed by the host operating system at a child partition of the first computing device 102 (see
The first computing device 108 creates a second snapshot of the first Hyper-V VM 120 at a second time (e.g., a time after the creation of the first snapshot 124). A set of changed data blocks 126 is associated with a difference between the second snapshot and the first snapshot 124. Thus, the first computing device 108 may store the first snapshot 124 taken at the first time to determine the set of changed data blocks 126. Alternatively, the first computing device 108 may use network communication and the VHD snapshot 128 at the backup device 116 to determine the set of changed data blocks 126. In one embodiment, an application at the first computing device 108 (e.g., a backup application) invokes an application programming interface (API) to determine the set of changed data blocks 126 from the SAN 106. For example, the API may determine a start offset and an end offset for each changed block in the set of changed data blocks 126. In one embodiment, the first computing device 108 may no longer store the first snapshot 124 created at the first time after the set of changed data blocks 126 is determined (e.g., to save storage space at the first computing device 108). The first computing device 108 may store the second snapshot to determine another set of changed data blocks associated with a difference between the second snapshot and a third snapshot taken at a third time.
The first computing device 108 transmits the set of changed data blocks 126 to the backup device 116. The backup device 116 is configured to update the VHD snapshot 128 of the first Hyper-V VM 120 based on the set of changed data blocks 126 to generate another VHD snapshot 130 of the first Hyper-V VM 120. In one embodiment, the backup device 116 may no longer store the VHD snapshot 128 of the first Hyper-V VM 120 upon generation of the VHD snapshot 130 of the first Hyper-V VM 120 (e.g., to save storage space at the backup device 116). In the embodiment illustrated, the amount of data transmitted as the set of changed data blocks 126 is less than the amount of data transmitted as the first snapshot 124. As such, the set of changed data blocks 126 may represent an express-full backup of the first VHD 122 of the first Hyper-V VM 120. As a result, the amount of data transferred from the first computing device 108 to the backup device 116 via the network 118 may be reduced while still maintaining a full backup of the first VHD 122 at the backup device 116.
In one embodiment, the first computing device 108 creates a third snapshot of the first Hyper-V VM 120 at a third time (e.g., a time after the creation of the second snapshot). A second set of changed data blocks is associated with a difference between the third snapshot and the second snapshot. The first computing device 108 transmits the second set of changed data blocks to the backup device 116. The backup device 116 is configured to update the VHD snapshot 130 of the first Hyper-V VM 120 based on the second set of changed data blocks to generate another VHD snapshot of the first Hyper-V VM 120. In one embodiment, the backup device 116 may no longer store the VHD snapshot 130 of the first Hyper-V VM 120 upon generation of the updated VHD snapshot of the first Hyper-V VM 120 (e.g., to save storage space at the backup device 116). Thus, the first computing device 108 may perform periodic backups (e.g., at the second time, at the third time, etc.) to maintain a recent copy of the first VHD 122 at the backup device 116. The time interval between backups may be fixed or may be variable. More frequent backups (e.g., transfers of sets of changed data blocks) may allow the backup device 116 to maintain a more recent copy of the first VHD 122 but may use more computing resources (e.g., at the first computing device 108 and at the backup device 116) and more bandwidth of the network 118. As such, the time interval between backups may be adjusted to balance utilization of computing resources and network resources with backup maintenance of the first VHD 122 at the backup device 116.
Each computing device of the cluster of computing devices 102 may share the first Hyper-V VM 120. That is, each of the computing devices 108, 110, 112, and 114 may own the first Hyper-V VM 120 at different times. When the second computing device 110 is the owner, the first Hyper-V VM 120 may be migrated to the second computing device 110 (e.g., as illustrated by the first Hyper-V VM 132). Similarly, when the third computing device 112 is the owner, the first Hyper-V VM 120 may be migrated to the third computing device 112 (e.g., as illustrated by the first Hyper-V VM 146), and when the fourth computing device 114 is the owner, the first Hyper-V VM 120 may be migrated to the fourth computing device 114 (e.g., as illustrated by the first Hyper-V VM 156). The second computing device 110 is configured to share a first Hyper-V virtual machine (VM) 132 that includes a first virtual hard drive (VHD) 134. At a first time, a first snapshot 138 of the first Hyper-V VM 132 is created. It should be noted that the first snapshot 138 associated with the second computing device 110 may be created at the same time as the first snapshot 124 associated with the first computing device 108. Alternatively, the first snapshot 138 associated with the second computing device 110 may be created at a different time. Thus, the first time associated with the first computing device 108 may be the same as the first time associated with the second computing device 110 or may be different from the first time associated with the second computing device 110. In a particular embodiment, the first snapshot 138 is created at a VM level (e.g., at the level of the first Hyper-V VM 132). Initial snapshots may be created and transmitted for each of a plurality of VMs at the second computing device 110.
The second computing device 110 communicates the first snapshot 138 to the backup device 116 via the network 118. The second computing device 110 creates a second snapshot of the first Hyper-V VM 132 at a second time (e.g., after the creation of the first snapshot 138). The second snapshot may be created at a computing device level (e.g., at the level of the second computing device 110). Thus, subsequent snapshots at the second computing device 110 may include information for each of a plurality of VMs at the second computing device 110. The second computing device 110 communicates the first snapshot 138 to the backup device 116 via the network 118. The backup device 116 stores data associated with the first snapshot 138 as the VHD snapshot 128 of the first Hyper-V VM 132 at the first time.
In the embodiment illustrated in
The second computing device 110 transmits the set of changed data blocks 140 to the backup device 116. The backup device 116 is configured to update the VHD snapshot 128 of the first Hyper-V VM based on the set of changed data blocks 140 to generate another VHD snapshot 130 of the first Hyper-V VM 132 at the second time. The backup device 116 may no longer store the VHD snapshot 128 upon generation of the VHD snapshot 130 (e.g., to save storage space at the backup device 116). In the embodiment illustrated, the amount of data transmitted as the set of changed data blocks 140 is less than the amount of data transmitted as the first snapshot 138. As such, the set of changed data blocks 140 may represent an express-full backup of the first VHD 134 of the first Hyper-V VM 132. As a result, the amount of data transferred from the second computing device 110 to the backup device 116 via the network 118 may be reduced while still maintaining a full backup of the first VHD 134 at the backup device 116. Furthermore, once initial snapshots are transmitted to the backup device 116, multiple VMs may be backed up at the host level of the second computing device 110 without taking individualized snapshots of each VM at the second computing device 110.
The third computing device 112 is configured to share a first Hyper-V VM 146 that includes a first VHD 148. The third computing device 112 includes shadow copy logic 142 and modification tracking logic 144. The modification tracking logic 144 creates a first snapshot of the first Hyper-V VM 146. A first differencing virtual hard drive 152 captures modifications to the first VHD 148 made after the first snapshot is created. The shadow copy logic 142 creates a shadow copy of the first VHD 148. A copy 150 of the first VHD 148 is transmitted to the backup device 116 via the network 118 and the shadow copy of the first VHD 148 is stored at the third computing device 112 (e.g., as a local read-only backup image of the first VHD 148). The modification tracking logic 144 creates a second snapshot of the first Hyper-V VM 146. A second differencing virtual hard drive (not shown) captures modifications to the first VHD 148 after the second snapshot is created. The first differencing virtual hard drive 152 is transmitted to the backup device 116 via the network 118.
In a particular embodiment, the shadow copy is a read-only writer-involved copy of the first VHD 148. The backup device 116 may merge the copy 150 of the first VHD 148 with the first differencing VHD 152 to generate an updated copy of the first VHD 148. In one embodiment, the modification tracking logic 144 creates a third snapshot of the first Hyper-V VM 146. A third differencing VHD (not shown) captures modifications to the first VHD 148 after the third snapshot is created. The third differencing VHD may be transmitted to the backup device 116 via the network 118. The backup device 116 may be configured to selectively merge the copy 150 of the first VHD 148 with the first differencing VHD 152 to generate an interim copy of the first VHD 148. The backup device 116 may selectively merge the copy 150 of the first VHD 148 with the first differencing VHD 152 and the second differencing VHD to generate an updated copy of the first VHD 148. The backup device 116 may thus support granular recovery of the first VHD 148.
The fourth computing device 114 is configured to share a first Hyper-V VM 156 that includes a first VHD 158. The fourth computing device 114 includes direct input/output (IO) logic 154 configured to generate a start transaction message and an end transaction message. Further, in the embodiment illustrated in
In operation, the direct IO logic 154 generates a start transaction message that indicates that a file of a virtual machine 168 shared via the CSV 104 is open for a direct IO transaction. For example, the virtual machine 168 may be a cluster shared copy of the first Hyper-V VM 156 and a file at the first Hyper-V VM 156 may be open in direct IO mode. The direct IO logic 154 sets a dirty flag of the virtual machine in response to the start transaction message and generates an end transaction message that indicates that the direct IO transaction is complete. The one or more direct IO bitmasks 162 identify blocks of the file that have been modified during the direct IO transaction.
In the embodiment illustrated in
In a particular embodiment, the CSV filter 155 sends the one or more bitmasks 162 to the backup device 116 using a file system control (fsctl) message. Further, the CSV filter 155 may periodically send the one or more bitmasks 162 to the backup device 116 based on a user-defined update period (e.g., every sixty seconds). The backup device 116 may store a backup copy of the virtual machine 168 shared via the CSV 104, and the CSV filter 155 may be coupled to an owning computing device of the virtual machine 168 (e.g., coupled to the fourth computing device 114) or may be coupled to each computing device of the cluster of computing devices 102 (e.g., coupled to a system/host volume at each of the computing devices 108, 110, 112, and 114). The express-full backup system 100 of
Referring to
The computing device 108 includes physical hardware 204 (e.g., one or more processors and one or more storage elements) and a Hyper-V Hypervisor 206. The Hyper-V Hypervisor 206 is configured to manage a parent partition 208 and one or more child partitions. In the embodiment illustrated in
A virtualized partition (e.g., the child partitions 210, 212) may not have access to physical processor(s) at the computing device 108 and may not handle real interrupts. Instead, the first child partition 210 and the second child partition 212 may have a virtual view of the processor(s) and may run in a guest virtual address space. Depending on configuration, the hypervisor 206 may not use an entire virtual address space at the computing device 108. The hypervisor 206 may instead expose a subset of the address space of the processor(s) to each of the child partitions 210, 212. The hypervisor 206 may handle interrupts to the processor(s) and may redirect the interrupts to the appropriate child partition using a logical Synthetic Interrupt Controller (SynIC). Address translation between various guest virtual address spaces may be hardware accelerated by using an IO Memory Management Unit (IOMMU) that operates independently of memory management hardware used by the physical processor(s).
The child partitions 210, 212 may not have direct access to the physical hardware 204 of the computing device 108. Instead, the child partitions 210, 212 may each have a virtual view of the physical hardware 204 (e.g., in terms of virtual devices). A request to the virtual devices may be redirected via a VMBus to devices in the parent partition 208 that manages the requests. The VMBus may be a logical channel that enables inter-partition communication (e.g., communication between the parent partition 208 and the child partitions 210, 212). A response may also be redirected via the VMBus. If the devices in the parent partition 208 are also virtual devices, the response may be redirected further within the parent partition 208 in order to gain access to the physical hardware 204. The parent partition 208 may execute a Virtualization Service Provider (VSP), connected to the VMBus, to handle device access requests from the child partitions 210, 212. Child partition virtual devices may internally execute a Virtualization Service Client (VSC) to redirect requests to VSPs in the parent partition 208 via the VMBus. The access process may be transparent to the guest operating systems 216, 218.
In a particular embodiment, a host operating system at a computing device may support multiple virtual machines, where each virtual machine has a different operating system than the host operating system. When backing up the virtual machines, it may be preferable to perform the backups at a host level instead of at a guest level. For example, backing up virtual machines using a backup application executing at the host level may be faster than backing up individual virtual machines using a backup application at each of the individual virtual machines. When virtual machines are in a cluster shared environment, backup operations for the virtual machines may be modified to maintain data integrity and data concurrency across multiple copies of the virtual machines.
The method 300 includes, at a computing device of a cluster of computing devices configured to share at least one virtual machine, creating a first snapshot of at least one virtual machine at a first time, at 302. For example, in
The method 300 also includes transmitting the first snapshot to a backup device, at 304. For example, in
The method 300 includes determining a set of changed data blocks associated with a difference between the second snapshot and the first snapshot, at 308. In a particular embodiment, an API may be invoked to determine the set of changed data blocks from a SAN, where each changed data block has an associated start offset and end offset. The API may be provided by a host operating system at the computing device or may be provided by a third party (e.g., a vendor of a storage area network). For example, in
The method 300 also includes transmitting the changed data blocks to the backup device, at 310. For example, in
It will be appreciated that the method 300 of
The method 400 includes, at a computing device of a cluster of computing devices configured to share at least one virtual machine, creating a first snapshot of at least one virtual machine at a first time, at 402. In a particular embodiment, the first snapshot is taken at a virtual machine level (e.g., the first snapshot may include data of the at least one virtual machine). For example, in
The method 400 also includes transmitting the first snapshot to a backup device, at 404. For example, in
The method 400 includes determining a set of changed data blocks based on a difference between the second snapshot and the first snapshot, at 408. In a particular embodiment, a volume filter at the at least one virtual machine may be queried for a volume bit map that identifies the set of changed data blocks. The volume filter may be installed by a backup application executing at a host level of the computing device. For example, in
The method 400 also includes transmitting the changed data blocks to the backup device, at 410. In a particular embodiment, the set of changed data blocks is used to overwrite an existing copy of the virtual machine at the backup device. For example, in
It will be appreciated that the method 400 of
The method 500 includes, at a computing device of a cluster of computing devices configured to share at least one virtual machine, creating a first snapshot of the at least one virtual machine, at 502. The virtual machine includes a virtual hard drive (VHD) and creating the first snapshot generates a first differencing VHD to indicate modifications to the VHD after the first snapshot is created. For example, in
The method 500 also includes creating a shadow copy of the VHD, at 504. In a particular embodiment, the shadow copy is a read-only writer-involved copy of the VHD. For example, in
The method 500 further includes transmitting a copy of the VHD to a backup device, at 506. For example, in
The method 500 includes creating a second snapshot of the at least one virtual machine to generate a second differencing VHD, at 508. For example, in
The method 500 also includes transmitting the first differencing VHD to the backup device, at 510. The backup device can selectively merge differencing VHDs with copies of VHDs to generate updated copies of the VHD. For example, in
It will be appreciated that the method 500 of
In a particular embodiment, CSV technology used to share virtual machines may provide change-tracking ability for various types of input/output (IO) at the virtual machine except direct input/output (IO). For example, direct IO at the virtual machine may bypass change-tracking in an attempt to increase IO speed.
The method 600 includes generating a start transaction message indicating that a file of a virtual machine is open for a direct input/output (IO) transaction, at 602. The virtual machine is accessible to a cluster of computing devices via a cluster shared volume (CSV). For example, in
The method 600 also includes setting a dirty flag of the virtual machine in response to the start transaction message, at 604. For example, in
The method 600 further includes generating one or more bitmasks (e.g., using a CSV filter) that identify blocks of the file that are modified during the direct 10 transaction, at 606. For example, the one or more direct IO bitmasks 162 of
The method 600 includes sending the one or more bitmasks to a backup device, at 608. The backup device updates the virtual machine based on the received bitmasks. For example, the CSV filter 155 of
The method 600 further includes generating an end transaction message indicating that the direct IO transaction is complete, at 610. For example, in
The method 600 includes clearing the dirty flag of the virtual machine in response to the end transaction message, at 612. For example, in
It will be appreciated that the method 600 of
In a particular embodiment, a CSV filter coupled to a system volume of each computing device of a cluster maintains separate contexts (e.g., bitmasks) for each computing device. To reduce a number of file updates, the CSV filter may implement reference counting on start and end direct IO fsctls. For example, a bitmask may not be saved until the reference count has reached zero. In another particular embodiment where the CSV filter is coupled to just the owning node, when ownership of a virtual machine is transferred, dismounting of the virtual machine causes tear-down (e.g., deallocation) of the CSV filter. During tear-down, existing bitmasks and metadata may be saved so that the bitmasks and metadata can be migrated to a new owner of the virtual machine.
The computing device 710 includes at least one processor 720 and a system memory 730. Depending on the configuration and type of computing device, the system memory 730 may be volatile (such as random access memory or “RAM”), non-volatile (such as read-only memory or “ROM,” flash memory, and similar memory devices that maintain stored data even when power is not provided), or some combination of the two. The system memory 730 typically includes an operating system 731, one or more application platforms 732, one or more applications 733, and program data. In an illustrative embodiment, the system memory 730 further includes shadow copy logic 734, modification tracking logic 735, direct IO logic 736, and a CSV filter 737. For example, one or more of the shadow copy logic 734, the modification tracking logic 735, and the direct IO logic 736 may be present in a backup software application at the computing device 710. In an illustrative embodiment, the shadow copy logic 734 includes the shadow copy logic 142 of
The computing device 710 may also have additional features or functionality. For example, the computing device 710 may also include removable and/or non-removable additional data storage devices such as magnetic disks, optical disks, tape, and standard-sized or flash memory cards. Such additional storage is illustrated in
The computing device 710 may also have input device(s) 760, such as a keyboard, mouse, pen, voice input device, touch input device, etc. Output device(s) 770, such as a display, speakers, printer, etc. may also be included. The computing device 710 also contains one or more communication connections 780 that allow the computing device 710 to communicate with other computing devices 790 over a wired or a wireless network. In an illustrative embodiment, the other computing devices 790 are communicatively coupled to the computing device 710 via a SAN 782. For example, the SAN 782 may be the SAN 106 of
It will be appreciated that not all of the components or devices illustrated in
The illustrations of the embodiments described herein are intended to provide a general understanding of the structure of the various embodiments. The illustrations are not intended to serve as a complete description of all of the elements and features of apparatus and systems that utilize the structures or methods described herein. Many other embodiments may be apparent to those of skill in the art upon reviewing the disclosure. Other embodiments may be utilized and derived from the disclosure, such that structural and logical substitutions and changes may be made without departing from the scope of the disclosure. Accordingly, the disclosure and the figures are to be regarded as illustrative rather than restrictive.
Those of skill would further appreciate that the various illustrative logical blocks, configurations, modules, and process steps or instructions described in connection with the embodiments disclosed herein may be implemented as electronic hardware or computer software. Various illustrative components, blocks, configurations, modules, or steps have been described generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.
The steps of a method described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in computer readable media, such as random access memory (RAM), flash memory, read only memory (ROM), registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor or the processor and the storage medium may reside as discrete components in a computing device or computer system.
Although specific embodiments have been illustrated and described herein, it should be appreciated that any subsequent arrangement designed to achieve the same or similar purpose may be substituted for the specific embodiments shown. This disclosure is intended to cover any and all subsequent adaptations or variations of various embodiments.
The Abstract of the Disclosure is provided with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, various features may be grouped together or described in a single embodiment for the purpose of streamlining the disclosure. This disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter may be directed to less than all of the features of any of the disclosed embodiments.
The previous description of the embodiments is provided to enable a person skilled in the art to make or use the embodiments. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope possible consistent with the principles and novel features as defined by the following claims.