Patent Grant
9514000
As the growth of enterprise data accelerates, organizations struggle to find more efficient ways to manage this data. One emerging approach to controlling data growth is data deduplication. Data deduplication technologies are especially relevant to backups because, despite careful use of differential and incremental backup strategies, a large fraction of backups consists of duplicate data.
When a new computing device (including, without limitations, desktop computers, laptops, tablets, smart phones, servers, Network-Attached Storage (NAS), for example) is issued/sold, it typically has an operating system (e.g., Windows or Mac OS), and a number of applications. In this respect, newly-issued or sold computing devices may be very similar one to another. For example, in the enterprise market, a computing device newly-issued by the IT department may have an operating system such as Microsoft Windows, some database software, an email client and a productivity suite. As a lot of data is typically already stored on a new or newly-configured computing device, the initial full backup thereof is costly, both in terms of time and bandwidth.
Deduplication, also called “dedupe,” removes duplicate information as data is stored, backed up, or archived. The dedupe process may be carried out at the file level, where duplicate files are replaced with a marker pointing to one copy of the file, and/or at the sub-file or byte level, where duplicate bytes of data are removed and replaced by references, resulting in a significant decrease in storage capacity requirements.
Data deduplication is a technology that reduces data volume by identifying and eliminating redundant data. Early technologies for single-instance storage, based on file-grain deduplication, have largely disappeared in favor of data block-based deduplication, in which files are represented as multiple blocks. Each data block of a file is compared to known data blocks. If a data block has been previously stored, the data block is simply referenced rather than stored again. Each data block, stored only once, may then be compressed using encoding technologies.
For example, notice that a reference (BLKID2) to data block 2 is present in the Reference Files of both backup clients 1 and 3. The pool of unique blocks 102, however, need not store two instances of data block 2. Indeed, the pool of unique data blocks 102 may be configured to comprise a single instance of each unique block referenced by the references within the Reference Files. One or more of the blocks within the pool of unique data blocks 102 may be referred to once in the Reference Files and one or more of the blocks within the pool of unique data blocks 102 may be referenced or represented in more than one Reference File. For example, both Reference File1 and Reference File3, associated with backup clients1 and backup client 3, store a reference to block B2, a single instance of which is stored in the pool of unique data blocks 102. A single Reference File may be created and suitably updated at each backup client in the client side shown in
The server side may also store or have access to a Reference File, as shown at 104. The Reference File 104 on the server side, however, may be configured somewhat differently than the Reference Files in each of the backup clients. While the Reference Files of each backup client may comprise a reference (e.g., BKLIDs) to each data block backed up, the server-side Reference File may be configured to comprise both the reference to the data block (e.g., the BLKID), but also a pointer or other reference to the location (e.g., offset) within the pool of unique data blocks 102 for each represented data block. The location of each data block within the pool of unique data blocks enables the backup server to readily retrieve data blocks from the pool of unique data blocks 102 at will.
According to one embodiment, the backup clients Reference Files and the server-side Reference Files may be kept synchronized, such that each BLKID in the backup clients' Reference Files has a corresponding entry in the server Reference File 104. If such is no longer the case, the backup client Reference File may be rebuilt by re-scanning the client computing device and re-generating the backup client Reference File.
Prior to assigning a computing device to a new employee, IT departments may equip the computing device with an operating system, one or more database applications, a browser, an email client and a productivity suite. New computing devices to be shipped to customers may also be similarly configured. This initial configuration may be scheduled for an initial back-up, either before the computing device is delivered to its intended recipient or afterwards. The initial backup of such computing device may be quite lengthy, as it is a full backup. Indeed, the initial backup may backup not only back up the operating system of the computing device, but also any files and programs supplied with the computing device, with the understanding that later backups will most likely be incremental backups that only backup data not previously represented in the first backup. This either delays the delivery of the computing device or places the responsibility for the first backup in the hands of the recipient of the computing device, which may not be optimal. Moreover, such initial, full backup may be quite resource-intensive (e.g., processor cycles, bandwidth and storage) and may degrade the performance of the computing device until the first backup is completed.
One embodiment pre-populates a backup Reference File with references to data blocks (e.g., BLKIDs in one embodiment), such that the pre-populated Reference File is identical or similar to the Reference File of one or more similarly-equipped computing devices. This pre-population of the backup Reference File saves time and bandwidth upon first backup. For example, during the first backup, the computing device can check the initial Reference File, determine that no or little data has changed, and backup only the additional data indicated by the Reference File. According to one embodiment, the Reference File need not be generated from scratch each time a computing device is pressed into service, as a suitable pre-populated Reference File (i.e., one whose references more or less accurately represent the data stored on the computing device) may already exist. That pre-populated Reference File may be selected amongst a plurality of pre-existing and pre-populated Reference Files.
Thereafter, when one of the computing devices 302-310 are purchased, delivered or otherwise put into use, the corresponding Reference File may be simply stored on the purchased computing device. As the Reference File in the newly-purchased computing device comprises references (e.g., BLKIDs) to at least some of the data stored in the computing device and as the pool of unique data blocks 102 already stores an exemplar of each data block referred to by the references in the pre-loaded Reference File, there is no need for an initial, full backup of the computing device, as such has already, in effect, been carried out. The act of creating, copying and storing the pre-populated Reference File, in effect, carries out the initial, full backup of the computing device without any data blocks having to be sent over the network to the pool of unique data blocks 102.
It is possible that the computing device stores some data that is not represented in the data blocks referred to by the references in the Reference File. In that case, however, the next backup may pick up that data in the form of one or more data blocks and suitably update the computing device's reference file with corresponding BLKIDs or other references to that data.
Thereafter, after the computing device is delivered to its intended recipient and/or when new data blocks are created, only references to new data blocks that are unrepresented by corresponding references (e.g., BLKIDs) in the pre-populated Reference File need be added to the pre-populated Reference File. The backup service may then check whether the Server Reference File 104 comprises a corresponding BLKID therein. If, the server Reference File 104 does, in fact, comprise an entry corresponding to the new block ID, the corresponding data block need not be transmitted to the backup server, as an exemplar of this data block is already present in the pool of unique data blocks 102. If, however, the backup service checks the server Reference File 104 and finds no corresponding reference to a new data block to be backed up, a reference to the new data block may be added to both the client-side computing device's Reference File and to the server-side Reference File 104, together with a pointer to the location in the pool of unique data blocks where such new data block may be found. Care may be taken, at every step, to ensure the integrity and synchronism between the Reference Files of the clients and the server Reference File. According to one embodiment, the pool of unique data blocks may be configured as a Universal Block Pool (UBP).
Because the initial configuration of many computing devices, both in the Enterprise space and in the consumer market, is very similar, machine to machine, the act of storing a selected and pre-populated Reference File in a backup client obviates the need for the otherwise-required initial, full backup. Therefore, the first backup of any such computing device or other processing device or machine is likely to be very similar to the first backup of any other identical or similar machine. Indeed, the same data blocks will be created as a result of the first scan and backup, the same Reference File will be constructed, and many of the same blocks will be sent to the server for storage for listing in the pool of unique data blocks. One embodiment, therefore, allows an approximation of an initial, full backup to be pre-stored on the computing device, in the form of a Reference File, without sending any data blocks to the pool of unique blocks 102 (or far fewer than would otherwise be the case). The first “real” backup thereafter only requires incremental changes to the pre-populated Reference File and the sending of a very limited number of previously unrepresented data blocks to the remote server. In fact, it is possible that one or more of the new data blocks created is identical to a data block that has already been backed up in the past. If the unique identifier of the newly-created data block is present in the server-side Reference File 104, then the corresponding data block need not be sent over the network to the pool of unique blocks 102, as an exemplar thereof is already present therein.
One embodiment, therefore, comprises selecting and/or pre-populating the Reference File in the computing device (e.g., the backup client), which pre-populated Reference File contains references to data blocks corresponding to, for example, the OS, applications and/or data files). Since the new machine is similar to other machines whose contents have already been backed up to the server, there is no need to transmit the scanned blocks of the new machine to the backup server, as such blocks are already there. There is no need to construct the Reference File, as other, similarly-constituted machines have already constructed a reference that serves acceptably well for this new machine in this instance. For example, if an IT department is directed to ready seven new computing devices having the same configuration as computing device 306 in
According to one embodiment, the pre-populated Reference File that is provided on the computing device may contain one or more references to blocks that are unrepresented on the computing device. This only marginally increases the size of the pre-populated Reference File and does not affect future incremental backups. In one embodiment, the one or more references are associated with programs or updates that are anticipated to be installed on the computing device so that when those programs or updates are installed in the future, the next backup operation is faster or even unnecessary. Similarly, the computing device may store data blocks that are unrepresented by corresponding references in the provided Reference File. Such differences in the references in the Reference File or in unrepresented data blocks on the computing device may be the result of small configuration differences from one similar computing device to the next. Any such differences may be ignored or picked up upon the first or any subsequent incremental backups.
As differences between similarly-constituted machines are likely to be small, and as any subsequent (first, real) backup is likely to be quite small, only the incremental changes (the delta between the data blocks of this particular machine and other similar machines) need be taken into account. That is, upon first backup, the pre-populated Reference File need be updated only with references to data blocks that are unrepresented in the pre-selected and pre-populated Reference File. The data blocks corresponding to those added references may then be sent to the server. Alternately, the first and subsequent backups may each create a separate version of the Reference File. The creation of a new Reference File does not require the underlying data blocks pointed to by the constituent references within the Reference File to be re-sent to the pool of unique data blocks, provided an exemplar thereof is already present therein. In this manner, it is only those data blocks that are unrepresented (by a corresponding BLKID, for example) in the selected and pre-populated Reference File that are added to the Reference File.
Accordingly, the degree of similarity between the computing devices from which the pre-populated Reference File or files were created and the computing device on which a first backup is carried out may drive the length of the first backup and the number of previously unrepresented blocks that are sent to the backup server for storage in the pool of unique data blocks. Very similar computing devices will finish their first backup very quickly, as the pre-populated Reference File may have contained references to all or almost all of the data blocks on the computing device. Computing devices that contain a greater number of blocks, the references to which are not in the pre-populated Reference File, may take a comparatively longer period of time to complete and may cause a comparatively greater number of blocks to be sent to the backup server for inclusion in the pool of unique blocks stored therein or accessible thereto. It is understood that the server Reference File is updated each time the pool of unique blocks is updated.
Should any client-side Reference File become corrupted, it may simply be reconstructed by re-scanning the computing device and re-populating a new Reference File. Alternatively, the pre-populated Reference File (a copy of which may have been kept in a secure location) may be used to pre-seed the backup process, thereby speeding up the re-construction of the corrupted Reference File. Such copy of the pre-populated reference file may be stored on the computing device itself, on removable media or downloaded from a network location, for example.
According to one embodiment, several pre-populated Reference Files may be prepared in advance, one for each “type” of new computing device/software combination or configuration and the appropriate references (e.g., pointers) may be pre-populated into the Reference File of each different “type” of machine/software combination. Storing the pre-existing and pre-populated Reference File in the computing device effectively carries out what would have been the initial, full backup of the computing device, without sending any data blocks of the computing device to the backup server. Significantly, this saves time and bandwidth upon first backup. Thereafter, the first backup of the computing device may be carried out after the computing device has been delivered to its intended user and data blocks not previously present on the computing device have been generated. This first backup is not an initial, full backup, as the initial, full backup has effectively been carried out by the storing of the pre-existing and pre-populated Reference File in the mass storage of the computing device. Indeed, this first backup may be an incremental backup that only backs up the differences between the similar reference machine/software combination (as represented by the selected pre-populated Reference File stored therein) and the new computing device/software combination and any new data blocks that may have been created after delivery of the computing device to its intended recipient and/or after use of the computing device. That amount of data and the resultant number of data blocks that need be sent to the backup server over the computer network during such first, incremental backup is likely very small, as most users only create a few MB of data each day. As a result, the first, incremental backup may be carried out faster than it otherwise might have been, had a full initial backup have been required. It is of note that embodiments may be implemented within the context of or in conjunction with the dedupe operations described above or entirely separate and independent thereof.
According to one embodiment, the first backup is an incremental backup to back up only those data blocks to which a reference is not present in the selected pre-populated Reference File. As described above, such an incremental back may be initiated without an initial full backup being carried out. Indeed, prior to the first backup being initiated, the backup server comprises, stores or otherwise has access to a pool of unique data blocks or other store that comprises the data blocks referenced by one or more references in the selected pre-populated Reference File stored in the computing device. Effectively, storing the selected pre-populated Reference File backs up at least some of the data blocks of the computing device to the backup server without sending any of the data blocks of the computing device to the backup server over the network.
According to one embodiment, the pre-populated Reference File may be selected from a plurality of pre-existing and pre-populated Reference Files depending upon a configuration of the computing device. The one or more references to at least some of the data blocks comprise references to data blocks of, for example, an operating system, application programs and/or user or other data stored on the computing device. The first backup, in this manner, may add fewer references to data blocks of the computing device to the Reference File than are already present in the Reference File. Storing the selected pre-populated Reference File may advantageously be carried out prior to use of the computing device by the intended user thereof. Indeed, initiating the first backup of the computing device may be carried out after delivery of the computing device to its intended user, after data blocks not previously present in the computing device have been created in the computing device. During the first backup, according to one embodiment, only the data blocks corresponding to the added references may be sent to the backup server over the computer network.
While certain embodiments of the disclosure have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the disclosure. Indeed, the novel methods, devices and systems described herein may be embodied in a variety of other forms. Furthermore, various omissions, substitutions and changes in the form of the methods and systems described herein may be made without departing from the spirit of the disclosure. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the disclosure. For example, those skilled in the art will appreciate that in various embodiments, the actual physical and logical structures may differ from those shown in the figures. Depending on the embodiment, certain steps described in the example above may be removed, others may be added. Also, the features and attributes of the specific embodiments disclosed above may be combined in different ways to form additional embodiments, all of which fall within the scope of the present disclosure. Although the present disclosure provides certain preferred embodiments and applications, other embodiments that are apparent to those of ordinary skill in the art, including embodiments which do not provide all of the features and advantages set forth herein, are also within the scope of this disclosure.
This application claims benefit of U.S. Provisional Patent Application Ser. No. 61/934,355 entitled “BACKUP OF BASELINE INSTALLATION” filed on Jan. 31, 2014, the disclosure of which is incorporated by reference herein in its entirety.
| Number | Name | Date | Kind |
|---|---|---|---|
| 6499054 | Hesselink et al. | Dec 2002 | B1 |
| 6732158 | Hesselink et al. | May 2004 | B1 |
| 7120692 | Hesselink et al. | Oct 2006 | B2 |
| 7454443 | Ram et al. | Nov 2008 | B2 |
| 7467187 | Hesselink et al. | Dec 2008 | B2 |
| 7546353 | Hesselink et al. | Jun 2009 | B2 |
| 7587467 | Hesselink et al. | Sep 2009 | B2 |
| 7600036 | Hesselink et al. | Oct 2009 | B2 |
| 7788404 | Hesselink et al. | Aug 2010 | B2 |
| 7917628 | Hesselink et al. | Mar 2011 | B2 |
| 7934251 | Hesselink et al. | Apr 2011 | B2 |
| 8004791 | Szeremeta et al. | Aug 2011 | B2 |
| 8209540 | Brouwer et al. | Jun 2012 | B2 |
| 8255661 | Karr et al. | Aug 2012 | B2 |
| 8285965 | Karr et al. | Oct 2012 | B2 |
| 8341117 | Ram et al. | Dec 2012 | B2 |
| 8341275 | Hesselink et al. | Dec 2012 | B1 |
| 8352567 | Hesselink et al. | Jan 2013 | B2 |
| 8458131 | Bindal et al. | Jun 2013 | B2 |
| 8526798 | Hesselink | Sep 2013 | B2 |
| 8631284 | Stevens | Jan 2014 | B2 |
| 8646054 | Karr et al. | Feb 2014 | B1 |
| 8661507 | Hesselink et al. | Feb 2014 | B1 |
| 8688797 | Hesselink et al. | Apr 2014 | B2 |
| 9009429 | Kapanipathi | Apr 2015 | B2 |
| 9015122 | Harrison | Apr 2015 | B2 |
| 20050086241 | Ram | Apr 2005 | A1 |
| 20050144195 | Hesselink et al. | Jun 2005 | A1 |
| 20050144200 | Hesselink et al. | Jun 2005 | A1 |
| 20070027937 | McGrattan | Feb 2007 | A1 |
| 20080250085 | Gray et al. | Oct 2008 | A1 |
| 20090006640 | Brouwer et al. | Jan 2009 | A1 |
| 20100211983 | Chou | Aug 2010 | A1 |
| 20100250885 | Nakata | Sep 2010 | A1 |
| 20110213754 | Bindal et al. | Sep 2011 | A1 |
| 20120036041 | Hesselink | Feb 2012 | A1 |
| 20120089578 | Lam | Apr 2012 | A1 |
| 20120109894 | Kishi | May 2012 | A1 |
| 20120233417 | Kalach et al. | Sep 2012 | A1 |
| 20130212401 | Lin | Aug 2013 | A1 |
| 20130266137 | Blankenbeckler et al. | Oct 2013 | A1 |
| 20130268749 | Blankenbeckler et al. | Oct 2013 | A1 |
| 20130268759 | Blankenbeckler et al. | Oct 2013 | A1 |
| 20130268771 | Blankenbeckler et al. | Oct 2013 | A1 |
| 20150242428 | Niles | Aug 2015 | A1 |
| Number | Date | Country |
|---|---|---|
| 2005022321 | Mar 2005 | WO |
| 2010113167 | Oct 2010 | WO |
| 2012030383 | Mar 2012 | WO |
| Entry |
|---|
| International Search Report and Written Opinion dated Apr. 24, 2015 from related PCT Serial No. PCT/US2015/013534, 9 pages. |
| Number | Date | Country | |
|---|---|---|---|
| 20150220403 A1 | Aug 2015 | US |
| Number | Date | Country | |
|---|---|---|---|
| 61934355 | Jan 2014 | US |
