Computing systems and associated networks have revolutionized the way human beings work, play, and communicate, heralding in what is now termed the “information age”. Data management is thus an important field in modern times. One aspect of data management is change tracking. For instance, it is often helpful to be able to distinguish what portions of data have changed between two instances in time.
As an example, when backing up a storage system, a copy of the storage system is written to a backup site. The next time the storage system is backed up, rather than copy again the entire storage system, only a changed subset of the storage system is backed up. Accordingly, to perform this incremental backup, determining which portions of the storage system have changed is a prerequisite. Furthermore, when recovering a storage system to a particular logical time (e.g., as during a recovery), change tracking allows the recovery system to determine which portions of the data are consistent with the state of the storage system at that particular logical time.
The subject matter claimed herein is not limited to embodiments that solve any disadvantages or that operate only in environments such as those described above. Rather, this background is only provided to illustrate one exemplary technology area where some embodiments described herein may be practiced.
At least some embodiments described herein relate to tracking changes amongst unit portions of a storage system. As an example, the unit portions might be files in a file system, or blocks in block-based storage system. For each at least some of the unit portions of the storage system, a logical time identifier is associated with unit portion and is included within a logical time identifier structure.
When writing to a particular write portion that includes one or more unit portions of the storage system, the logical time identifier is updated for any changed unit portions within the write portion. Furthermore, once the appropriate logical time identifier(s) has changed, the mechanism calculates redundancy data, such as a checksum, of a group of one or more logical time identifiers associated with the one or more portions of the write portion. The write portion of the storage system is written. In addition, the corresponding redundancy data for the group of logical time identifiers associated with that write portion is written to the logical time identifier structure.
Later, for a given write portion, the redundancy data is verified to be consistent or inconsistent with the group of one or more logical time identifiers associated with the write portion. If the redundancy data is not consistent, then a current logical time identifier is assigned to each of the logical time identifiers. Accordingly, inconsistent write portions are treated as recently written to. During incremental backup, the logical time identifiers are used to determine which unit portions have changed, and thus to determine which unit portions need to be backed up. Since inconsistent redundancy data for a write portion results in the logical time identifiers for the entire write portion receiving the current logical time, this causes the appearance to the backup system that all unit portions of that write portion have been newly written to. Accordingly, the backup system causes the entire write portion to be backed up, even though one or more of its unit portions might not have changed. While this might perhaps result in more backup data being transferred than absolutely necessary in the rare case that the redundancy data loses consistency, it protects against data inconsistency when such cases occur.
This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
In order to describe the manner in which the above-recited and other advantages and features can be obtained, a more particular description of various embodiments will be rendered by reference to the appended drawings. Understanding that these drawings depict only sample embodiments and are not therefore to be considered to be limiting of the scope of the invention, the embodiments will be described and explained with additional specificity and detail through the use of the accompanying drawings in which:
At least some embodiments described herein relate to tracking changes amongst unit portions of a storage system. As an example, the unit portions might be files in a file system, or blocks in block-based storage system. For each at least some of the unit portions of the storage system, a logical time identifier is associated with unit portion and is included within a logical time identifier structure.
When writing to a particular write portion that includes one or more unit portions of the storage system, the logical time identifier is updated for any changed unit portions within the write portion. Furthermore, once the appropriate logical time identifier(s) has changed, the mechanism calculates redundancy data, such as a checksum, of a group of one or more logical time identifiers associated with the one or more portions of the write portion. The write portion of the storage system is written. In addition, the corresponding redundancy data for the group of logical time identifiers associated with that write portion is written to the logical time identifier structure.
Later, for a given write portion, the redundancy data is verified to be consistent or inconsistent with the group of one or more logical time identifiers associated with the write portion. If the redundancy data is not consistent, then a current logical time identifier is assigned to each of the logical time identifiers. Accordingly, inconsistent write portions are treated as recently written to. During incremental backup, the logical time identifiers are used to determine which unit portions have changed, and thus to determine which unit portions need to be backed up. Since inconsistent redundancy data for a write portion results in the logical time identifiers for the entire write portion receiving the current logical time, this causes the appearance to the backup system that all unit portions of the write portion have been newly written to. Accordingly, the backup system causes the entire write portion to be backed up, even though one or more of the unit portions might not have changed. While this might perhaps result in more backup data being transferred than absolutely necessary in the rare case that the redundancy data loses consistency, it protects against data inconsistency when such cases occur.
Some introductory discussion of a computing system will be described with respect to
Computing systems are now increasingly taking a wide variety of forms. Computing systems may, for example, be handheld devices, appliances, laptop computers, desktop computers, mainframes, distributed computing systems, datacenters, or even devices that have not conventionally been considered a computing system, such as wearables (e.g., glasses). In this description and in the claims, the term “computing system” is defined broadly as including any device or system (or combination thereof) that includes at least one physical and tangible processor, and a physical and tangible memory capable of having thereon computer-executable instructions that may be executed by a processor. The memory may take any form and may depend on the nature and form of the computing system. A computing system may be distributed over a network environment and may include multiple constituent computing systems.
As illustrated in
In the description that follows, embodiments are described with reference to acts that are performed by one or more computing systems. If such acts are implemented in software, one or more processors (of the associated computing system that performs the act) direct the operation of the computing system in response to having executed computer-executable instructions. For example, such computer-executable instructions may be embodied on one or more computer-readable media that form a computer program product. An example of such an operation involves the manipulation of data. The computer-executable instructions (and the manipulated data) may be stored in the memory 104 of the computing system 100. Computing system 100 may also contain communication channels 108 that allow the computing system 100 to communicate with other computing systems over, for example, network 110.
Embodiments described herein may comprise or utilize a special purpose or general-purpose computing system including computer hardware, such as, for example, one or more processors and system memory, as discussed in greater detail below. Embodiments described herein also include physical and other computer-readable media for carrying or storing computer-executable instructions and/or data structures. Such computer-readable media can be any available media that can be accessed by a general purpose or special purpose computing system. Computer-readable media that store computer-executable instructions are physical storage media. Computer-readable media that carry computer-executable instructions are transmission media. Thus, by way of example, and not limitation, embodiments of the invention can comprise at least two distinctly different kinds of computer-readable media: storage media and transmission media.
Computer-readable storage media includes RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other physical and tangible storage medium which can be used to store desired program code means in the form of computer-executable instructions or data structures and which can be accessed by a general purpose or special purpose computing system.
A “network” is defined as one or more data links that enable the transport of electronic data between computing systems and/or modules and/or other electronic devices. When information is transferred or provided over a network or another communications connection (either hardwired, wireless, or a combination of hardwired or wireless) to a computing system, the computing system properly views the connection as a transmission medium. Transmissions media can include a network and/or data links which can be used to carry desired program code means in the form of computer-executable instructions or data structures and which can be accessed by a general purpose or special purpose computing system. Combinations of the above should also be included within the scope of computer-readable media.
Further, upon reaching various computing system components, program code means in the form of computer-executable instructions or data structures can be transferred automatically from transmission media to storage media (or vice versa). For example, computer-executable instructions or data structures received over a network or data link can be buffered in RAM within a network interface module (e.g., a “NIC”), and then eventually transferred to computing system RAM and/or to less volatile storage media at a computing system. Thus, it should be understood that storage media can be included in computing system components that also (or even primarily) utilize transmission media.
Computer-executable instructions comprise, for example, instructions and data which, when executed at a processor, cause a general purpose computing system, special purpose computing system, or special purpose processing device to perform a certain function or group of functions. The computer executable instructions may be, for example, binaries or even instructions that undergo some translation (such as compilation) before direct execution by the processors, such as intermediate format instructions such as assembly language, or even source code. Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the described features or acts described above. Rather, the described features and acts are disclosed as example forms of implementing the claims.
Those skilled in the art will appreciate that the invention may be practiced in network computing environments with many types of computing system configurations, including, personal computers, desktop computers, laptop computers, message processors, hand-held devices, multi-processor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile telephones, PDAs, pagers, routers, switches, datacenters, wearables (such as glasses) and the like. The invention may also be practiced in distributed system environments where local and remote computing systems, which are linked (either by hardwired data links, wireless data links, or by a combination of hardwired and wireless data links) through a network, both perform tasks. In a distributed system environment, program modules may be located in both local and remote memory storage devices.
More specifically,
As an example,
The storage system 400A also includes write portions 411A through 413A. In accordance with the broadest aspect described herein, the write portions may include one or more unit portions, and represent how the unit portions may be written into the storage system. For instance, if the storage system 400A is a file system, then typically, one file is written at a time. Accordingly, the write portions might contain just one unit portion (i.e., one file) each. However, if the storage system 400A is a block-based storage system, then typically the blocks are written one page at a time, with one page encompassing multiple blocks. In the illustrated example, the write portions 411A through 413A are each represented as encompassing three unit portions each. This is again for purposes of simplicity and clarity in describing the example. In a typical storage system, there may be millions or billions of unit portions and write portions, and each write portion can potentially encompass different numbers of unit portions.
The logical time identifier structure 400B includes logical time identifiers 401B through 409B associated with each of the unit portions 401A through 409A, respectively.
In the particular example of
Referring again to
With the logical time identifiers 401A through 409A being zero'ed out in
In this tree structure, if a write portion does not have an associated group of logical time identifiers, then the logical time identifiers are assumed to be a default value represented by a root node in the tree. If any of the group of logical time identifier structures for a write portion is different than the default value represented at the root node, then that group of logical time identifier structures will have its own child node. If the group of logical time identifiers are the same for a given write portion, then the child node will simply hold that value. If each of the group of logical time identifiers are not the same for the write portion, then the child node will contain the redundancy data for the series of logical time identifiers.
For instance, referring to the example of
Moving onto the writing operation 221 of the phase 220, when writing to a particular write portion that includes unit portion(s) of the storage system, the system updates the logical time identifiers for any of the unit portion(s) of the write portion that have changed (act 320), and calculates the redundancy data (e.g., a checksum) associated with a logical time identifier(s) associated with the unit portion(s) of the write portion (act 321). The system writes the write portion (act 322) and also writes the associated redundancy data (act 323). In some embodiments, the redundancy data is written (act 323) in parallel with the updating of the logical time identifiers (act 320). Accordingly, if the redundancy data is later found to be inconsistent with the logical time identifiers (act 324), then a power failure has likely occurred, and the logical time identifiers may be treated as corrupt. This is why the logical time identifiers are, in that case, marked with the current logical time as measured by the application using the data.
For instance, referring to the state 500B of
Next, referring to state 500C of
Next, referring to state 500D of
Referring back to
During a verify operation 222, it might be verified after the write operation 221 that the redundancy data is not consistent with the group of one or more logical time identifiers (act 324). If the redundancy data is not consistent, then a write operation 221 is performed with respect to the current logical time 3 for all of the unit portions in the corresponding write portion.
For instance, referring to state 500E of
In other words, if the redundancy data (e.g., the checksum) is incorrect, the entire associated write portion is treated as though it has the latest sequence number. The next time the write portion is updated, the current logical time is explicitly stored for each logical time identifier for that write portion. This means that if the page becomes corrupt somehow, when the user requests the set of changes from time t (which is less than the current time), the user will see that everything represented by this part of the logical time identifier structure) has changed, even though only some or even none of the unit portions of this write portion have changed. This is much safer than assuming nothing changed (since it is unknown what changed).
Next, referring to the state 500G of
Referring again to
Since inconsistent redundancy data for a write portion results in the logical time identifiers for the entire write portion receiving the current logical time, this causes the appearance to the backup system that all unit portions of that write portion have been newly written to. Accordingly, the backup system causes the entire write portion to be backed up, even though one or more of the unit portions might not have changed. While this might perhaps result in more backup data being transferred than absolutely necessary in the rare case that the redundancy data loses consistency, it protects consistency of the actual data when such cases occur
Accordingly, the principles described herein provide an efficient mechanism for keeping track of changes between arbitrary versions of a storage system. The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.
Number | Name | Date | Kind |
---|---|---|---|
5721915 | Sockut et al. | Feb 1998 | A |
6411964 | Iyer et al. | Jun 2002 | B1 |
6795966 | Lim et al. | Sep 2004 | B1 |
6981114 | Wu et al. | Dec 2005 | B1 |
7162662 | Svarcas et al. | Jan 2007 | B1 |
7249262 | Hauck | Jul 2007 | B2 |
7328366 | Michelman | Feb 2008 | B2 |
7447854 | Cannon | Nov 2008 | B1 |
7516286 | Dalal et al. | Apr 2009 | B1 |
7519858 | Korlepara | Apr 2009 | B2 |
7574459 | Sen | Aug 2009 | B2 |
7860837 | Ali | Dec 2010 | B2 |
7865470 | Fries | Jan 2011 | B2 |
8055613 | Mu | Nov 2011 | B1 |
8145601 | Zizys | Mar 2012 | B2 |
8335902 | Feathergill | Dec 2012 | B1 |
8356013 | Fachan et al. | Jan 2013 | B2 |
8356148 | Popovski et al. | Jan 2013 | B2 |
8443166 | Czezatke et al. | May 2013 | B2 |
8463747 | Wahlert | Jun 2013 | B2 |
8463749 | Zizys | Jun 2013 | B2 |
8468609 | Leggette | Jun 2013 | B2 |
8538919 | Nielsen et al. | Sep 2013 | B1 |
8560855 | Resch | Oct 2013 | B2 |
8712970 | Sim-Tang | Apr 2014 | B1 |
8726127 | Grube | May 2014 | B2 |
8726273 | Le | May 2014 | B2 |
8751515 | Xing et al. | Jun 2014 | B1 |
8782086 | Resch | Jul 2014 | B2 |
8788773 | Goodman et al. | Jul 2014 | B2 |
8805788 | Gross et al. | Aug 2014 | B2 |
8813204 | Leggette | Aug 2014 | B2 |
8966341 | Grube | Feb 2015 | B2 |
9081714 | Grube | Jul 2015 | B2 |
9081715 | Grube | Jul 2015 | B2 |
9086994 | Resch | Jul 2015 | B2 |
9130957 | Yamaura | Sep 2015 | B2 |
9276980 | Chan | Mar 2016 | B2 |
9430272 | Bezbaruah | Aug 2016 | B2 |
20080126445 | Michelman | May 2008 | A1 |
20080140963 | Thomason et al. | Jun 2008 | A1 |
20100049929 | Nagarkar et al. | Feb 2010 | A1 |
20100228913 | Czezatke et al. | Sep 2010 | A1 |
20100262585 | Rosikiewicz et al. | Oct 2010 | A1 |
20100274980 | Stringham | Oct 2010 | A1 |
20110167196 | Scales et al. | Jul 2011 | A1 |
20120072659 | Wade et al. | Mar 2012 | A1 |
20130067179 | Paleologu et al. | Mar 2013 | A1 |
20130254479 | Czezatke et al. | Sep 2013 | A1 |
20130290782 | Chen et al. | Oct 2013 | A1 |
20140025913 | Fuente et al. | Jan 2014 | A1 |
20140040572 | Kotagiri et al. | Feb 2014 | A1 |
20140164722 | Garthwaite et al. | Jun 2014 | A1 |
20140236892 | Blyler et al. | Aug 2014 | A1 |
20140250347 | Grube | Sep 2014 | A1 |
20140337666 | Resch | Nov 2014 | A1 |
20140337932 | Leggette | Nov 2014 | A1 |
Number | Date | Country |
---|---|---|
103946807 | Jul 2014 | CN |
Entry |
---|
Werneburg, Ken, “VMware vSphere® Replication 5.5 Overview”, In Technical White Paper, Nov. 16, 2013, 14 pages. |
“Advanced Restore—Virtual Server Agent for VMware”, Retrieved on: Nov. 7, 2014, Available at: http://documentation.commvault.com/commvault/v10/article?p=products/vs—vmware/restore—adv.htm, pp. 1-32. |
Sinofsky, Steven, “Building the Next Generation File System for Windows: ReFS”, Published on: Jan. 17, 2012, Available at: http://blogs.msdn.com/b/b8/archive/2012/01/16/building-the-next-generation-file-system-for-windows-refs.aspx, pp. 1-45. |
“What's New in Hyper-V for Windows Server 2012”, Published on: Feb. 29, 2012, Available at: http://technet.microsoft.com/en-in/library/hh831410.aspx, pp. 1-6. |
Ramdas, Aashish, “Resynchronization of Virtual Machines in Hyper-V Replica”, Published on: May 10, 2013, Available at: http://blogslechnet.com/b/virtualization/archive/2013/05/10/resynchronization-of-virtual-machines-in-hyper-v-replica.aspx, pp. 1-5. |
“Changed Block Tracking (CBT) on Virtual Machines (1020128)”, Retrieved on: Nov. 7, 2014, Available at: http://kb.vunware.com/selfservice/microsites/search.do?language=en—US&cmd=displayKC&externalId=1020128, pp. 1-2. |
“High Availability and Data Protection with EMC Isilon Scale-Out NAS”, In White Paper, Nov. 2013, 36 pages. |
Park, et al., “Fast and Space-Efficient Virtual Machine Checkpointing”, In Proceedings of ACM SIGPLAN/SIBOPS International Conference o Virtual Execution Environments, Mar. 9, 2011, pp. 75-85. |
Garg, et al., “Checkpoint-Restart for a Network of Virtual Machines”, 2013, IEEE, 8 pages. |
Office Action dated Dec. 3, 2015 cited in U.S. Appl. No. 14/573,976 (Copy Attached) pp. 1-20. |
“International Search Report and Written Opinion Issued in PCT Application No. PCT/US2015/067761”, Mailed Date: May 24, 2016, 12 Pages. |
“International Search Report and Written Opinion issued in PCT Application No. PCT/US2015/063565”, Mailed Date: Jun. 30, 2016, 20 pages. |
Notice of Allowance dated Jun. 7, 2016 cited in U.S. Appl. No. 14/573,976 (Copy Attached). |
Number | Date | Country | |
---|---|---|---|
20160203052 A1 | Jul 2016 | US |