Patent Grant
11016859
Any and all applications for which a foreign or domestic priority claim is identified in the Application Data Sheet, or any correction thereto, are hereby incorporated by reference into this application under 37 CFR 1.57.
Embodiments of the invention relate to data management and, in particular, to systems and methods for reducing duplicate data in a computer system.
With the increased reliance on computer systems to store critical information, the importance of protecting information has grown. As a result, certain storage systems receive an identification of a file to protect and then create one or more secondary copies, such as backup files, containing the contents of the file. These secondary copies can then later be used to restore the original data should anything happen to the original data.
In corporate environments, protecting information is generally part of a routine process that is performed for many computer systems within an organization. For example, a company might back up critical computing systems related to e-commerce such as databases, file servers, web servers, and so on as part of a daily, weekly, or monthly maintenance schedule. The company may also protect computing systems used by each of its employees, such as those used by an accounting department, marketing department, engineering department, and so forth.
Although each computer system contains certain unique information, many systems may contain very similar information. For example, although a computing system used by a marketing employee and a computing system used by an engineering employee will generally contain unique information created by each employee in the course of their work, both computing systems will likely have the same operating system installed, with thousands of identical or similar files used by the operating system. Similarly, both computing systems will likely have at least some similar application programs installed, such as a word processor, spreadsheet, Internet browser, and so on. Both systems may also have similar corporate information. For example, each employee may have an electronic copy of an employee manual distributed by the company. Information other than files may also be identical or similar between systems. For example, user settings and preferences may have similar default values on each system and application programs may contain similar templates on each system that are stored as application-specific information. As another example, several employees may have received a copy of the same email, and the email may be stored in each employee's electronic mailbox.
As a result of the amount of redundant information in an organization, secondary copies of an organization's information are often very large and can require the purchase of expensive storage devices and storage media. The restoration of data in the event of data loss is also slowed by the large size of the secondary copies. As the size of secondary copies increases, locating and restoring information requires more actions to be taken. For example, it may be necessary to search many tapes or other media to find the correct secondary copy. The great quantity of storage media, such as tapes, may mean that some secondary storage media has been moved offsite requiring that it first be retrieved before information can be recovered from it. Each of these factors increases the cost of protecting information and the time required to recover information in the event of data loss. Quick recovery of information is often critical to today's businesses, and any additional delay can affect business operations and customers' satisfaction with the business.
Single instancing in a data management system is the process of attempting to store only a single instance of a file or data object. Certain objects are often stored in large, monolithic files that are intended to be read only by the application that created them. For example, a MICROSOFT EXCHANGE email server stores email messages in one or more large data files that typically hold thousands of different users' mailboxes. As another example, a database server often stores tables, forms, reports, and other data objects in one or two large data files that provide persistence for the entire database.
Thus, typical data management systems are only able to perform data management operations on the large data file, rather than the data objects themselves. In the case of the email server, a given electronic mail application may generate multiple email messages that all differ, but which all contain the same attachment. Prior systems may not be able to differentiate these messages, and thus each would be stored with the attachment. Further, if two files had different properties or metadata, such prior systems would store both files, even though the data they contain are identical and differ only by their metadata.
In view of the foregoing, certain embodiments of the invention include a de-duplication system for managing application-generated data objects. The system comprises a first backup agent executing on a first client device and a de-duplication module executing on a computing device. The first backup agent is configured to, in response to a storage operation request: parse first application-specific data of the first client device that is the subject of the storage operation request, the first application-specific data comprising a plurality of first data objects having different sizes; identify, based at least on an application that generated the first application-specific data, one or more of the plurality of first data objects of the first application-specific data to be considered for de-duplication, wherein the one of more first data objects comprises less than the entire first application-specific data; and insert at least one indicator in the first application-specific data that identifies at least one location of the one or more first data objects within the first application-specific data. The de-duplication module is in communication with the first backup agent to receive the first application-specific data and is configured to: process each of the one or more data objects; and, based on said processing, determine if a duplicate copy of any of the one or more first data objects exists in a storage device.
In certain embodiments, a method is disclosed for managing application-generated data objects. The method comprises receiving a first storage operation request for first data generated by a first application executing on a first client device, the first data comprising a plurality of first data objects having non-uniform sizes. The method further comprises parsing the first data to identify one or more of the plurality of first data objects to be considered for de-duplication. The method also includes inserting at least one indicator in the first data that identifies at least one location of the one or more first data objects within the first data and processing each of the one or more first data objects to determine if a duplicate copy of the one or more first data objects exists in at least one storage device. For each of the one or more first data objects, if a duplicate copy does not exist in the storage device, the method stores the first data object in the at least one storage device. Otherwise, a stub file and/or a pointer is stored in place of the first data object in the storage device.
In certain embodiments, a method is disclosed for managing application-generated data objects. The method includes receiving a first storage operation request for first data generated by a first application executing on a first client device, the first data comprising a plurality of first data objects. The method also includes parsing the first data to identify one or more of the first data objects to be considered for de-duplication and receiving a second storage operation request for second data generated by a second application, the second data comprising a plurality of second data objects, and the second data having a different file format than the first data. The method further includes parsing the second data to identify one or more of the plurality of second data objects to be considered for de-duplication. In addition, the method includes inserting in the first data and/or the second data an indicator that denotes, based on the first and second file formats, that the one or more first data objects should not be single instanced with the one or more second data objects.
In certain embodiments, a system is disclosed for managing application-generated data objects. The system includes means for receiving a first storage operation request for first data generated by a first application executing on a first client device, the first data comprising a plurality of first data objects having differing sizes. The system also includes means for parsing the first data to identify one or more of the first data objects to be considered for de-duplication, wherein the one of more first data objects comprises less than the entire first data; means for inserting at least one indicator in the first data that identifies at least one location of the one or more first data objects within the first data; and means for processing each of the one or more first data objects to determine if a duplicate copy of the first data object exists in at least one storage device. In addition, the system includes means for (i) storing each of the one or more first data objects in the storage device that does not have a duplicate copy and (ii) storing a stub and/or a pointer in place of each of the one or more first data objects that does have a duplicate copy in the at least one storage device.
In certain embodiments, a method is disclosed for managing data in a computer system. The method comprises accessing first data, the first data comprising at least one first data object in a first file format, and accessing second data, the second data comprising at least one second data object in a second file format, the second file format being different than the first file format. The method further comprises inserting in at least one of a copy of the first data and a copy of the second data an indicator that denotes that, based on the first and second file formats, the at least one first data object cannot or should not be single instanced with the at least one second data object.
In certain embodiments, the method is performed by one or more content-aware data agents residing on one or more client devices. Such agents can advantageously identify portions of data to be backed up, archived or otherwise stored to be considered for possible de-duplication.
In certain embodiments, a de-duplication system is disclosed for managing application-generated data objects. The system comprises a first backup agent executing on a first client device, the first backup agent being configured to, in response to a storage operation request: (i) parse first application-specific data of the first client device that is the subject of the storage operation request, (ii) identify one or more first data objects of the first application-specific data to be considered for de-duplication, and (iii) insert an indicator in a copy of the application-specific data that identifies the one or more first data objects to be considered for de-duplication.
For purposes of summarizing the disclosure, certain aspects, advantages and novel features of the inventions have been described herein. It is to be understood that not necessarily all such advantages may be achieved in accordance with any particular embodiment of the invention. Thus, the invention may be embodied or carried out in a manner that achieves or optimizes one advantage or group of advantages as taught herein without necessarily achieving other advantages as may be taught or suggested herein.
In the drawings, the same reference numbers and acronyms identify elements or acts with the same or similar functionality for ease of understanding and convenience. To easily identify the discussion of any particular element or act, the most significant digit or digits in a reference number refer to the Figure number in which that element is first introduced (e.g., element 604 is first introduced and discussed with respect to
The headings provided herein are for convenience only and do not necessarily affect the scope or meaning of the claimed invention.
Overview
Described in detail herein are single instancing systems that more intelligently identify multiple copies of the same data object. For example, certain embodiments of the single instancing system recognize documents, files, or data objects produced by multiple different applications, and can parse through and identify those data objects common among data blocks within a file, and thereby only copy a single instance of those common data objects. Further embodiments parse the proprietary data formats of many applications and can identify data objects related to the application, even when those data objects are stored in large, monolithic data files. In addition, if two documents, files, or data objects are substantially similar, but have differing metadata, such as different user permissions, embodiments of the single instancing system can store a single instance of the data object, but retain all differing versions of the metadata (for example, such as by retaining the different user permissions). When a client requests the data object, the single instancing system returns appropriate metadata based on the identity of the client or other information.
According to other embodiments of the single instancing system, if multiple clients and associated media agents are in remote locations, the single instancing system may perform backup of data to a local, single instance database or data store at each remote location. Then, the single instancing system may use continuous data replication (CDR) to copy the data contained in each local data store to a central location.
In further embodiments, content-aware systems and methods are disclosed for improving de-duplication/single instancing in storage operations. In certain examples, backup agents on client devices parse application-specific data to identify data objects that are likely candidates for de-duplication. The backup agents can then insert markers or other indictors in the data that identify the location(s) of the particular data objects. Such markers can, in turn, assist a de-duplication manager to perform object-based de-duplication and increase the likelihood that like blocks are identified within the data. In other examples, the agents can further determine if a data object of one file type can, or should, be single-instanced with a data object of a different file type. Such processing of data on the client side can provide for more efficient storage and back-end processing in a data storage system.
The invention will now be described with respect to various embodiments. The following description provides specific details for a thorough understanding of, and enabling description for, these embodiments of the invention. However, one skilled in the art will understand that the invention may be practiced without these details. In other instances, well-known structures and functions have not been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments of the invention.
The terminology used in the description presented below is intended to be interpreted in its broadest reasonable manner, even though it is being used in conjunction with a detailed description of certain specific embodiments of the invention. Certain terms may even be emphasized below; however, any terminology intended to be interpreted in any restricted manner will be overtly and specifically defined as such in this Detailed Description section.
Unless described otherwise below, embodiments of the invention may be practiced with various data processing systems. Thus, the construction and operation of the various blocks shown in
The file identification component 110 identifies files or data objects, such as in response to a storage operation. As used herein, the terms “file” or “data object” are broad terms and are used in their ordinary sense and include, without limitation, any collection or grouping of bytes of data that can be viewed as one or more logical units. For example, a file could be a computer file on a file system (for example, a word processing file, a spreadsheet file, a database file, etc.) As another example, a data object could be a file or within a file (e.g., an embedded object within a word processing file, a cell or a row in a spreadsheet file, a table or an entry in a table in a database file, a string, an array, or any other collection of electronic information). Moreover, data objects need not be limited to units of the same size but can comprise, in certain embodiments, multiple objects of differing sizes. The file identification component 110 may retrieve additional information related to a file or data object, such as its size, that is used by the single instancing system 100 to uniquely identify the data object.
When the file identification component 110 identifies a file, the application data extraction component 170 determines whether the file contains additional data objects. For example, the file may be an application-specific container (for example, a database file), that stores data objects such as documents, email messages, and other collections of data. The application data extraction component 170 would determine that each of the data objects within the application-specific container should be identified.
The identifier generation component 120 generates a substantially unique identifier of a file or data object that is used to determine if another file or data object already stored by the single instancing system matches the file or data object used to generate the substantially unique identifier. The identifier comparison component 130 performs comparisons of identifiers of various files or data objects to determine if the files or data objects contain similar data (for example, the identifier comparison component 130 can compare substantially unique identifiers of two or more files or data objects to determine if the files or data objects contain similar data).
The single instance database component 140 is a data store that contains entries identifying files or data objects managed by the single instancing system 100, and may also contain supplemental information associated with files or data objects, such as a substantially unique identifier, a path, a location, a reference count, a file size or other information. The restored file cache component 150 provides an intermediate location that may be used by the single instancing system 100 during a restore operation to hold instances of files or data objects for which additional references may need to be restored.
For example, during a restore operation, the single instancing system may restore files or data objects to the cache and then transfer the files or data objects to a target location of the restore operation. When the single instancing system 100 encounters a reference to a single instance copy of a file or data object, the single instancing system 100 may consult the restored file cache component 150 or an index. The single instancing system 100 does so to determine if the file or data object is present in the cache before attempting to restore the file or data object from another location, such as from secondary storage (for example, a tape).
The storage operation manager component 160 coordinates storage operations and invokes the other components of the single instancing system 100 as needed to perform requested storage operations. For example, the storage operation manager component 160 may include an application used by an administrator to manage the single instancing system 100. The storage operation manager component 160 may also maintain indexes of the data objects and each of the references to those data objects through the single instancing system 100, as well as pending operations on the data objects that are part of a data management plan of an organization implementing the single instancing system 100.
The remote synchronization component 180 performs single instancing between a remote location and a central location, such as between an enterprise or organization having a headquarters or central office and one or more satellite offices or remote offices, or vice-versa. The remote synchronization component 180 uses the techniques described in further detail herein to determine whether a file or data object should be copied from the remote location to the central location.
Aspects of the invention can be embodied in a special purpose computer or data processor that is specifically programmed, configured, or constructed to perform one or more of the computer-executable instructions explained in detail herein. Embodiments of the invention can also be practiced in distributed computing environments where tasks or modules are performed by remote processing devices, which are linked through a communications network, such as a Local Area Network (LAN), Wide Area Network (WAN), or the Internet. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.
Embodiments of the invention may be stored or distributed on computer-readable media, including magnetically or optically readable computer discs, hard-wired or preprogrammed chips (e.g., EEPROM semiconductor chips), nanotechnology memory, biological memory, or other data storage media. Indeed, computer implemented instructions, data structures, screen displays, and other data under embodiments of the invention may be distributed over the Internet or over other networks (including wireless networks), on a propagated signal on a propagation medium (e.g., an electromagnetic wave(s), a sound wave, etc.) over a period of time, or they may be provided on any analog or digital network (packet switched, circuit switched, or other scheme). Those skilled in the relevant art will recognize that portions of the invention may reside on a server computer, while corresponding portions reside on a client computer such as a mobile or portable device, and thus, while certain hardware platforms are described herein, embodiments of the invention are equally applicable to nodes on a network.
Single Instancing
Reducing or eliminating redundant instances of data resulting from a storage operation is sometimes referred to here as de-duplication or “single instancing,” because what would traditionally be stored as many instances of the same data is reduced to as few as one. Redundant instances may be detected and reduced at several locations or times throughout the operation of the system that will now be described. These embodiments are provided only as examples, and are not intended to be an exhaustive list of the way in which the system can be implemented.
In some embodiments, the single instancing system performs single instancing of data at a computing system that contains or originally generated the data, such as a client computing system. For example, a client computing system that is providing data that is the target of a storage operation may receive a request from the single instancing system to provide a substantially unique identifier (for example, a hash value, message digest, checksum, digital fingerprint, digital signature or other sequence of bytes that substantially uniquely identifies the file or data object) of each file or data object included in the storage operation.
The word “substantially” is used to modify the term “unique identifier” because algorithms used to produce hash values may result in collisions, where two different files or data objects result in the same hash value. However, depending upon the algorithm or cryptographic hash function used, collisions should be suitably rare and thus the identifier generated for a file or data object should be unique throughout the single instancing system. As an alternative to the single instancing system generating the substantially unique identifier, the client computing system may itself generate substantially unique identifiers for each file or data object that is stored on the client computing system on an ongoing or other basis. When a storage operation is requested, the single instancing system determines if another file or data object exists with a substantially unique identifier matching that of the one provided by the client computing system. If the single instancing system is already aware of a similar file or data object, then the client computing system does not need to send the redundant file or data object to a secondary storage location or destination.
Rather than operating only at the file level, in some embodiments, the single instancing system may also operate at a lower level of granularity by enumerating data objects within files of understood types. For example, the single instancing system may read and parse data files from email servers (for example, MICROSOFT EXCHANGE email servers, Sendmail email servers, etc.), database servers (for example, MICROSOFT SQL Server database servers, ORACLE database servers, etc.), web servers (for example, MICROSOFT IIS web servers, APACHE web servers, etc.), word processing applications (for example, MICROSOFT WORD, COREL WORDPERFECT, etc.), spreadsheet applications (for example, MICROSOFT EXCEL, APPLE NUMBERS, etc.), and many others and enumerate the data objects within these files.
The single instancing system may then generate a substantially unique identifier for each data object within the file, rather than each file, and store this substantially unique identifier in the single instance database component 140 or other index. The single instancing system may also store information about the associations between the data objects and their containing files. Therefore, the single instancing system enables data storage operations such as searching, backing up, restoring, replicating, copying and so forth to be performed at the data object level. Enabling data storage operations at the data object level enables the single instancing system to store data using much less space than traditional single instancing storage systems, because files containing data objects are likely to have redundant data objects, of which only one instance or copy need be stored.
For example, the data of two email servers that each contain similar email messages in a large data file can be stored in much less space than if the entire data files of each email server were to be stored, because the data files are likely to contain redundant email messages, of which only one instance or copy need be stored. This is because traditional single instancing systems would determine that the two data files differed and would store both data files, thereby consuming more storage space.
For example, the single instancing system may identify the type of the data file based on the data file's filename extension. The identified data file type may be used to identify a format or specification that describes the structure (e.g., at which bytes specific data is stored, its particular encoding, etc.) of the data file. This knowledge of the structure enables the application data extraction component 170 to parse the data file 210, identify containers 220, and extract email messages 230 or attachments 240 from each container 220. Alternatively, the data file 210 may contain metadata, a header or other information that describes its data structure.
The application data extraction component 170 can obtain this metadata or other information, which enables it to parse the data file 210, identify containers 220, and extract email messages 230 or attachments 240 from each container 220. The single instancing system can then store only a single instance or copy of the extracted email messages 230 or attachments 240 that are similar. By operating on the data at the data object level, the single instancing system provides substantial benefit by identifying the redundancy of data objects stored within the data files and providing single instancing of the individual data objects.
In some embodiments, the single instancing system or another system performs additional operations on the data after single instancing has occurred. For example, another system may encrypt backup data that is being stored offsite to prevent unauthorized parties from accessing the data. Another system may also compress the data to reduce its size. The single instancing system enables these additional operations to be performed more efficiently, because there is less data on which to perform these additional operations after redundant data has been reduced or eliminated.
Single Instanced Archive and Backup Data Using Single Storage Policy
One example of a single instancing system employs a single storage policy or data store. A storage policy is generally a virtual container with preferences that contains a set of rules for data retention of data objects associated with the storage policy. The single instancing system in this example stores single instance data in a single location and in a manner to ensure that any relevant, unique data is retained, but only a single instance of common data is copied to the data store. Notably, a single instancing agent creates, updates, or maintains a single instance database or index that represents or stores the substantially unique identifiers of each file or data object. The single instance database or index is associated with a single storage policy that is separate from one or more data stores that store the data copies. A single storage policy may represent a storage location that includes existing data, as well as new data that has been compared to the existing data and identified as being unique.
A media agent 340 manages the data copy 330, and creates a single instance copy 350. The single instance copy 350 is a copy in which at least some of the redundant files or data objects have been removed. The media agent 340 uses the methods described herein to eliminate redundant instances of files or data objects contained in the data copy 330, and to produce the single instance copy 350. The single instance copy 350 may then be stored on tape or other media.
The single instancing system may employ one or more specialized single instancing agents. For example, as described below, the single instancing system may employ application agents associated with types of applications, metadata agents for analyzing metadata, encryption/compression agents, and so forth. Alternatively, a single agent employing all of the functions described herein may be employed. These agents operate on all types of data, including documents, files, data objects, or any data blocks.
An application agent associated with each type of application may analyze incoming or previously stored data to identify redundant data. For example, an email agent analyzes incoming or previously stored emails, including attachments to emails, to identify duplicate attachments. The email agent provides a first instance of an email with an attachment to the storage policy, but for subsequent emails having the same attachment, the email agent strips them of their attachments and stores them with a stub or pointer to the previously stored attachment.
A metadata agent analyzes files or data objects for similarities in data, but differences in metadata that is associated with the files or data objects. For example, two files may have the same data, but separate permissions, properties, access control lists (ACLs), operating system metadata (e.g., UNIX and WINDOWS metadata), application-specific metadata, ownership metadata, anti-virus information, security metadata, other metadata, combinations of the same or the like. The metadata agent ascertains that the two files contain the same data but have differing metadata, and thus stores only one instance of the file, but two or more instances of the associated metadata, as well as information about the associations between the metadata, the providers of the files and the files. Such embodiments of the invention can advantageously allow for single instancing, or de-duplication, of files or data objects across different operating systems.
When a client requests the file, the single instancing system provides the file to the client with the appropriate instance of the metadata based on the identity of the requestor. For example, a data file may be stored on a user computer with permissions restricting access to the data file to the user of that computer, and the same data file may be stored on a second computer with permissions restricting access to the data file to an administrator of the single instancing system. The user will still be able to access the data file from the single instance store even though the user is not an administrator of the single instancing system. This is because the single instancing system will use the metadata from the file from the user's computer to ascertain that the user has the permissions needed to access the file, and permit the requested access to the user.
As another example, two files may have the same data, but different properties, such as the creation date or access date. The metadata agent ascertains that the two files contain the same content but have different properties. The metadata agent thus stores only one instance of the file, but two or more instances of the properties, as well as information about the associations between the metadata, the providers of the files and the files. Because an association between the provider of the file and the file is stored, when a client requests the file, the single instancing system can provide the file with the proper metadata to the client.
In yet other embodiments, one or more agents on one or more client devices can perform one or more functions of the above-described metadata agent. Embodiments of agents usable with embodiments of the invention are described in more detail below, with particular reference to
The single instancing system can single instance both non-encrypted and encrypted data received from client computers. Each client computer may generate a substantially unique identifier for the file or data object before the file or data object is encrypted. The client computer does this before encryption because an encrypted file or data object would almost certainly result in a substantially unique identifier that is different from a substantially unique identifier generated from the file or data object before encryption. Alternatively, a client computer could generate a substantially unique identifier for a file or data object after the file or data object is encrypted. An encryption agent may compare the substantially unique identifiers to identify identical files or data objects, and thus only store one instance of them.
In some embodiments, the single instancing system determines whether to store an instance of an encrypted or unencrypted file (or data object, but file is used in this and the following paragraphs discussing encryption for brevity) based on whether the files are encrypted using the same encryption scheme, whether the encrypted files can be decrypted by the single instancing system, and/or other factors.
For example, consider the following three situations involving encrypted files: 1) where the single instancing system has previously stored an instance of a file that is encrypted and a request is made to store another instance of the file that is also encrypted; 2) where the single instancing system has previously stored an instance of a file that is unencrypted and a request is made to store another instance of the file that is encrypted; and 3) where the single instancing system has previously stored an instance of a file that is encrypted and a request is made to store another instance of the file that is unencrypted. Metadata associated with the file is typically not encrypted. Moreover, information about the encryption scheme (for example, what public key is used to encrypt the file) may be provided as metadata to the single instancing system.
For the first situation, if the two file instances are encrypted using the same encryption scheme (for example, using the same public key), the single instancing system can avoid storing the second instance of the file. If the two files are encrypted using different encryption schemes, the single instancing system stores the second instance of the file. This is because the single instancing system has to be able to provide the second instance of the file to the requestor when requested.
Alternatively, if both encryption schemes are known to the single instancing system (for example, using two different public keys of the single instancing system), the single instancing system can avoid storing the file. This is because the single instancing system can decrypt the first instance of the file and re-encrypt it using the encryption scheme used to encrypt the second instance of the file. However, this decryption and re-encryption may be too computationally expensive, depending upon various factors (for example, the time needed to decrypt and re-encrypt the file, the algorithm, etc.), and so the single instancing system may simply store the second instance of the file.
For the second situation, if the second instance of the file is encrypted using an encryption scheme that is known to the single instancing system (for example, using a public key of the single instancing system), the single instancing system can avoid storing the second instance of the file. This is because, when the provider of the second encrypted instance requests the file, the single instancing system can retrieve the first unencrypted instance, encrypt it using the known encryption scheme and provide it to the requestor. However, if such encryption is too computationally expensive, the single instancing system may simply store the second encrypted instance of the file. If the second instance of the file is encrypted using an encryption scheme that is not known to the single instancing system, the single instancing system stores the second instance of the file. This is because the single instancing system has to be able to provide the second encrypted instance of the file to the requestor when requested.
For the third situation, if the first instance of the file is encrypted in such a way that it can be decrypted by the single instancing system, the single instancing system can avoid storing the second unencrypted instance of the file. This is because, when the provider of the second unencrypted instance requests the file, the single instancing system can retrieve the first encrypted instance, decrypt it, and provide it to the requestor. However, if such decryption is too computationally expensive, the single instancing system may simply store the second unencrypted instance of the file. If the first instance of the file is encrypted in such a way that it cannot be decrypted by the single instancing system, the single instancing system stores the second unencrypted instance of the file. This is because the single instancing system has to be able to provide the second unencrypted instance of the file to the requestor.
The single instancing system may also handle compressed files. Each client computer may generate a substantially unique identifier for the file or data object before the file or data object is compressed. The client computer does this before compression because a compressed file or data object would almost certainly result in a substantially unique identifier that is different from a substantially unique identifier generated from the file or data object before compression. Alternatively, a client computer could generate a substantially unique identifier for a file or data object after the file or data object is compressed. A compression agent may compare the substantially unique identifiers to identify identical files or data objects, and thus only store one of them.
In some embodiments, the single instancing system determines whether to store an instance of a compressed or uncompressed file (or data object, but file is used in this and the following paragraphs discussing compression for brevity) based on whether the files are compressed using the same compression scheme, whether the compressed files can be decompressed by the single instancing system, and/or other factors.
For example, consider the following three situations involving compressed files: 1) where the single instancing system has previously stored an instance of a file that is compressed and a request is made to store another instance of the file that is also compressed; 2) where the single instancing system has previously stored an instance of a file that is uncompressed and a request is made to store another instance of the file that is compressed; and 3) where the single instancing system has previously stored an instance of a file that is compressed and a request is made to store another instance of the file that is uncompressed. Metadata associated with the file is typically not compressed. Moreover, information about the compression scheme (for example, what compression algorithm is used to compress the file) may be provided as metadata to the single instancing system.
For the first situation, if the two file instances are compressed using the same compression scheme (for example, using the same compression algorithm), the single instancing system can avoid storing the second instance of the file. Otherwise, the single instancing system stores the second instance of the file. However, if the second instance of the file is compressed using a different compression scheme, the single instancing system may avoid storing the second instance of the file, if the single instancing system is able to decompress the first instance of the file and recompress the first instance of the file using the different compression scheme. If the single instancing system is not able to do so, the single instancing system stores the second instance of the file. However, this decompress and recompression may be too computationally expensive, depending upon various factors (for example, the time needed to decompress and recompress the file, the algorithm, etc.), and so the single instancing system may simply store the second instance of the file.
For the second situation, if the second instance of the file is compressed using a compression scheme that is known to the single instancing system (for example, using a known compression algorithm), the single instancing system can avoid storing the second instance of the file. This is because, when the provider of the second compressed instance requests the file, the single instancing system can retrieve the first uncompressed instance, compress it using the known compression scheme and provide it to the requestor. However, if such compression is too computationally expensive, the single instancing system may simply store the second compressed instance of the file. If the second instance of the file is compressed using a compression scheme that is not known to the single instancing system, the single instancing system stores the second instance of the file. This is because the single instancing system has to be able to provide the second compressed instance of the file to the requestor when requested.
For the third situation, if the first instance of the file is compressed in such a way that it can be decompressed by the single instancing system, the single instancing system can avoid storing the second uncompressed instance of the file. This is because, when the provider of the second uncompressed instance requests the file, the single instancing system can retrieve the first compressed instance, decompress it, and provide it to the requestor. However, if such decompression is too computationally expensive, the single instancing system may simply store the second uncompressed instance of the file. If the first instance of the file is compressed in such that it cannot be decompressed by the single instancing system, the single instancing system stores the second uncompressed instance of the file. This is because the single instancing system has to be able to provide the second uncompressed instance of the file to the requestor.
The single instancing system may be configurable to reduce processing time, transmission bandwidth, and the like with small files. For example, an administrator-configurable value would allow the administrator to configure the single instancing system to ignore files or data objects below a given size. For example, any file or data object below a certain threshold (for example, one kilobyte) may simply be stored, and a substantially unique identifier would not determined for it. Any file or data object greater than the threshold (for example, one kilobyte) would then be single instanced. An administrator may adjust this threshold up or down. As another example, the single instancing system may allow the administrator to configure it to always single instance files or data objects of a certain type or category (for example, executable files or modules may always be single instanced if they rarely change).
Alternatively, embodiments of the single instancing system may allow the administrator to never single instance files or data objects of a certain type or category (for example, log files may never be single instanced, because they typically change quite frequently) or compare data objects of certain different data formats.
In yet other embodiments, the single instancing system can comprise multiple single instancing, or de-duplication, components (e.g., databases) for processing and/or storing single instance copies 350 of data from different applications and/or file formats. For example, in certain embodiments, a file when processed (e.g., checked in) by a SHAREPOINT application can be fundamentally altered such that it differs from the same file when processed in an EXCHANGE server environment.
In view of the foregoing, certain embodiments of the single instancing system can determine that files or data objects from these two applications should not be considered together for single instancing since the copies of the same file will contain different data. Rather, the single instancing system can comprise different components that are each dedicated to single instance processing of the data from the two applications. For example, in certain embodiments, a first single instance database can allocated for SHAREPOINT data, while a second single instance database can be allocated for EXCHANGE data.
Moreover, the same single instancing system components can be dedicated for single instancing data of different applications that process data in a similar manner. For instance, in certain embodiments, the second single instance database described above could be allocated for both EXCHANGE and file system data, while the first single instance database is reserved for SHAREPOINT data.
In such embodiments, storage space can be advantageously allocated on one or more storage devices based on the expected amount of single instanced, or de-duplicated, data from the different applications. For example, a main or primary de-duplication database (e.g., having media with a faster access time) can be set aside for de-duplication of files or data objects from one or more applications or application formats producing a relatively large amount of data within a system, while a secondary de-duplication database can be reserved for data of one or more applications or application formats producing a relatively small amount of data in the system.
In certain embodiments, as discussed below with reference to
The single instancing system may further associate timestamps with the files or data objects or with their generated substantially unique identifiers. A timestamp may indicate the time at which the file or data object was created, last accessed or modified, or the time at which the single instancing system generated the substantially unique identifier for it, or the time at which the file or data object was stored by the single instancing system. The single instancing system may do so to determine whether a file or data object is substantially newer than another file or data object already stored in the single instancing system.
For example, two files or data objects may have the same substantially unique identifier, meaning that they contain the same data. The single instancing system may compare the timestamp of the first, previously stored, file or data object with that of the second file or data object. If the timestamp of the second file or data object is substantially newer than the timestamp of the first file or data object, the single instancing system may nonetheless store the second file or data object, even though it is duplicative of a file or data object that is already stored.
In this context, substantially newer means that the age (as determined by its timestamp) of the second file or data object is less than the age of the first, previously stored, file or data object by a certain amount or more (configurable by an administrator). The single instancing system could then prune off older versions as the data ages and is migrated to other storage, such as longer-term storage. For example, a first file or data object could have a timestamp indicating that its age is ten months. A second file or data object could have a timestamp indicating that its age is three months. If the administrator has configured the amount to be six months, then the second file or data object is substantially newer than the first file or data object, because the age of the second file or data object is less than the age of the first file or data object by more than the configured amount. Therefore, the single instancing system would store the second file or data object, and the first file or data object could be migrated to longer-term storage, such as to tape.
The single instancing system may be employed in any variety of architectures. For example, it may be employed with heterogeneous storage hardware, and thus is not reliant on a specific hardware platform to perform all of the single instancing functions. Instead, multiple, different data storage platforms may be employed for storing data under the storage policies. Further, the architecture may be tiered or federated wherein one server manages multiple cells (and each of those cells in turn may manage lower tier cells). In some embodiments, the cells may be arranged in hierarchies or organized in configurations such as those described in U.S. patent application Ser. No. 12/060,186, filed Mar. 31, 2008, published as U.S. Patent Application Publication No. 2008-0250204 A1, the entirety of which is hereby incorporated herein by reference.
The single instancing system may employ many other functions. For example, it may employ content indexing agents to index the content all data blocks. These content indexes may then be searched to permit users to readily locate desired data objects. Further details regarding content indexing may be found in U.S. patent application Ser. No. 11/694,869, filed Mar. 30, 2007, published as U.S. Patent Application Publication No. 2008-0091655 A1, the entirety of which is hereby incorporated herein by reference.
In certain embodiments, the de-duplication system 360 provides an end-to-end, block-based and/or file-based de-duplication system that can extend across multiple tiers of secondary storage (including disk and tape) and/or encompass reduction of stored data across multiple backups, archives, clients and/or platforms. Embodiments of the de-duplication system 360 further employ application-aware or content-aware de-duplication processes to more accurately find and reduce common patterns in data across disparate applications, operating systems, file systems and/or data types.
As illustrated, the content-aware de-duplication system 360 comprises a de-duplication manager 362 that performs de-duplication of data received from one or more client devices 364. In certain embodiments, the de-duplication manager 362 can comprise the single instancing system 100, or one or more components thereof. For example, the de-duplication manager 362 can receive secondary copies (e.g., backup, migration and/or archive copies) of data from the client devices 364 and store at least portions of the secondary copies on the storage device 366.
A de-duplication database 368 can utilize one or more tables or other data structures to store substantially unique identifiers of data objects (e.g., files, blocks) that have already been stored on the storage device 366. If a copy or instance of a data object received by the de-duplication manager 362 has already been stored on the storage device 366, the de-duplication database 368 can be used to avoid sending an additional copy of the same data to the storage device 366. Examples of similar single instancing configurations and methods usable with embodiments of the invention are further disclosed in U.S. Provisional Patent Application No. 61/100,686, filed Sep. 26, 2008, which is hereby incorporated herein by reference in its entirety.
As further illustrated by
In certain further embodiments, the agents 370 comprise application-specific, or content-aware, modules that are configured to access and process one or more particular data formats and/or data associated with a particular application. In certain embodiments, the agents 370 are advantageously used to process data within a database file, blob data and/or data encapsulated in another data format. For instance, each agent 370 can be aware of the various files, folders, registry files and/or system resources that are impacted by a particular application. In certain embodiments, different individual agents 370 may be designed to handle MICROSOFT EXCHANGE data, SQL SERVER data, LOTUS NOTES data, MICROSOFT WINDOWS 2000 file system data, SHAREPOINT data, and other types of data.
In certain embodiments, multiple agents 370 may be treated as separate agents even though they reside on the same client 364. In other embodiments, the separate agents may be combined to form a virtual agent for performing storage operations related to a specific application. Also, although the agents 370 are illustrated as executing on the client devices 364, it will be understood that embodiments of the agents 370 can also be configured to remotely access data stored on the client devices 364 through one or more application programming interfaces (APIs), remote procedure calls (RPCs) or the like.
In certain embodiments, the agent 370 is configured to perform data management or storage operations in accordance with one or more storage policies or other preferences. The storage polices and/or preferences may include, but are not limited to, information regarding storage locations, relationships between system components, network pathways, retention policies, data characteristics, compression or encryption requirements, preferred system components, combinations of the same or the like.
In certain embodiments, each agent 370 can prepare the secondary copy of the data to facilitate subsequent de-duplication of data objects within the backup data. Such embodiments can advantageously improve the speed of de-duplication since the de-duplication preparation can be integrated with the normal backup process and since the agents 370 can use knowledge of both specific file formats of the client data and the de-duplication process occurring on the back end of the de-duplication system 360.
In certain embodiments, the agent 370 identifies the locations of data objects within a file that should be considered for de-duplication. For instance, the agent 370 can indicate (e.g., via offsets) where the de-duplication manager 362 should start and/or stop considering data for possible de-duplication.
For instance, suppose the agent 370 is instructed to process the data file 210 (see
For example, the agent 370 may insert one or more markers within the data file 210 that identify location(s) where a data object (e.g., the body of an email message, an attachment) begins and/or ends. In certain embodiments, such identification information can be advantageously used by the de-duplication manager 362 to more quickly identify portions of one or more data files that may be more likely to contain duplicate data (e.g., attachments between multiple email messages). In certain embodiments, the marker can further identify the type of data associated with the data object.
In embodiments of the invention that utilize block-level de-duplication, the information inserted by the agent 370 can advantageously assist the de-duplication manager 362 to know where to begin the de-duplication comparison of data such that a higher likelihood of detecting duplicate data is achieved. For instance, the agent 370 can process metadata and/or other file information regarding a particular data file to detect where logical or natural divisions within the file should be identified (e.g., an offset of a mail data file at which an attachment begins). As a result, it is more likely that like blocks are detected since the de-duplication manager 362 can proceed with its comparison and analysis of data blocks in corresponding locations of the data.
For example, suppose two different email messages to be backed up contain the same data object as an attachment, but the bodies of the email messages differ in size. If a block-level de-duplication manager 362 begins its comparison process at the beginning of each email message by comparing set de-duplication block sizes (e.g., 32 KB), the de-duplication manager 362 may not detect a match of the attachments of the two email messages since the attachments may be located at different offsets within the respective email messages (e.g., 40 KB and 46 KB).
However, using the identifying information generated by the agents 370, the de-duplication manager 362 can be instructed to begin the block-level comparison at the respective offsets within the two email messages whereat the particular attachments begin. As a result, such embodiments can advantageously increase the likelihood that duplicate data will be detected between data of like files (e.g., generated from the same application).
Although the agent 370 has been described as processing particular types of data (e.g., mail data), it will be understood that other embodiments of the invention can be used with a variety of different types of data and that different agents 370 can be associated with the different types of data to identify possible common duplicate data therein. For instance, a first agent can be used to identify data objects within an EXCHANGE server data file, while a second agent can be used to identify data objects within file system data such that the de-duplication manager 362 can more easily detect duplicate data objects between the two types of data.
In yet other embodiments, the agent 370 can generate substantially unique identifiers, as discussed above with respect to
The chunk file 380 includes a chunk header 382 comprising metadata regarding the chunk file 380. The chunk file 380 further comprises a plurality of tag headers 384 that are each associated with a data payload 386 containing the actual data to be backed up. In certain embodiments, each tag header 384 comprises metadata (e.g., size, data type, or the like) regarding the data in the corresponding payload section 386.
As shown, each of the tag headers 384 comprises a de-duplication identifier 388. In certain embodiments, the identifier 388 indicates whether or not one or more data objects within the corresponding payload 386 should be considered for de-duplication. For instance, in certain embodiments, the identifier 388 comprises a single bit that indicates whether or not the de-duplication manager 362 should consider the data of the corresponding payload 386 for de-duplication.
In yet other embodiments, the identifier 388 can comprise a file marker or other data structure that demarcates payload data for possible de-duplication. For instance, the identifier 388 may identify particular offsets within the payload 386 whereat the de-duplication manager 362 should start and/or stop considering data objects for de-duplication. In certain embodiments, the identifier 388 can comprise a tag, a pointer, an index or table entry, a flag, a bit, a header, combinations of the same or the like.
In yet other embodiments, the identifier 388 can identify data objects that can or cannot be single instanced with other data objects (e.g., based on file types). In such embodiments, identifiers 388 may indicate that data of a particular payload 386 should be considered for de-duplication with reference to a first de-duplication database, while data of a second payload 386 should be considered for de-duplication with reference to a second de-duplication database.
In certain embodiments, the agent 370 inserts the identifier 388 in the chunk file 380. In certain embodiments, one or more agents 370 can also separate data of different applications and/or formats that should not be considered together for single instancing into separate payloads (e.g., under different tag headers) or separate chunk files.
Although
Determining Data Object Similarity
Various methods of determining if one file or data object is similar (e.g., two instances of the same data) to another file or data object will now be described. However, those of ordinary skill in the art will recognize that many other methods besides those described herein may be used to achieve similar results.
In some embodiments, the single instancing system determines if two files or data objects are similar by performing a binary comparison. For example, a first file or data object can be compared byte by byte with a second file or data object, or portions of the first file or data object can be compared with portions of the second file or data object. If each byte of the compared data in each file or data object matches, then the two files or data objects are identical and therefore similar. Otherwise, the two files or data objects do not match and are not treated as similar.
In some embodiments, the single instancing system determines if two files or data objects are similar by creating a digest or fingerprint of the data contained in each file or data object. For example, as storage operations are performed, the single instancing system may perform a cryptographic hash on each file or data object to create a digest of the file or data object. The single instancing system compares the digest of the file or data object with stored digests created for other files or data objects. If the digests of two files or data objects match, then the single instancing system may consider the files or data objects to be identical. The single instancing system can use any suitable hashing algorithm, such as SHA512. For applications that create data files having embedded data objects, the single instancing system identifies the embedded data objects, and determines the similarity of each data object with other data objects found either within the same data file or in other data files or stores managed by the single instancing system.
For example, the application may provide an object model that the storage operation manager component 160 can invoke to enumerate objects within the file, or the storage operation manager component 160 may understand the format used by the application for storing objects in the file. An application author, manufacturer or third party may also provide a module for parsing the application file that the storage operation manager component 160 can invoke. In this way, the application author, manufacturer or third party can provide access to the data objects within the file without exposing the actual format used to third parties.
In certain embodiments, the storage operation manager component 160 further identifies files or data objects from different applications that should be considered separately for de-duplication. For instance, the storage operation manager component 160 can identify one or more files or data objects from a first application or data format to be associated with a first de-duplication database, while identifying one or more files or data objects from a second application or data format to be associated with a second de-duplication database. In yet other embodiments, one or more agents 370 can perform one or more of the functions described above with respect to blocks 400 and/or 410.
In block 415, the storage operation manager component 160 selects the first identified data object. In block 420, the storage operation manager component 160 determines whether the data object is unique, or if the storage manager component has previously copied the data object. For example, the storage operation manager component 160 may compute a digest in the manner described herein, and compare the digest to the digests of previously copied data objects to determine if the data object is an instance of a previously copied data object. In decision block 430, if the data object is unique, then the storage operation manager component 160 continues at block 450, else the storage operation manager component 160 continues at block 440. In block 440, the storage operation manager component 160 adds a reference (e.g., to an index of data managed by the single instancing system, such as by incrementing a reference count in the index) to the already backed up instance of the data object, and then continues to block 460. In block 450, the component stores the unique data object. In decision block 460, if the storage operation manager component 160 identified more data objects within the file, then the storage operation manager component 160 loops to block 415 to select the next data object, else the storage operation manager component 160 completes.
In decision block 540, if the data objects match, then the single instancing system continues at block 550, otherwise the single instancing system continues at block 570. In block 550, the single instancing system reports to the entity that invoked the storage operation manager component 160 that the file or data object is not unique. In block 560, the single instancing system updates the substantially unique identifier reference count tracked by the single instance database component 140 and then concludes. In block 570, the single instancing system reports to the entity that invoked the storage operation manager component 160 that the file or data object is unique. In block 580, the single instancing system adds the file or data object's substantially unique identifier and other information to the list of files and data objects tracked by the single instance database component 140. The process then concludes.
In decision block 630, if the selected file or data object is a reference to an instance of a file or data object stored somewhere else, then the storage operation manager component 160 continues at block 640, else the storage operation manager component 160 continues at block 650. In block 640, the storage operation manager component 160 locates the referenced instance of the file or data object and continues to block 655. In block 655, the storage operation manager component 160 restores the file or data object from the referenced instance of the file or data object. In block 650, the storage operation manager component 160 restores the file or data object directly from the file or data object. In decision block 660, if there are more files or data objects referred to by the received request, then the storage operation manager component 160 loops to block 620 to select the next file or data object on which to perform these blocks, else the storage operation manager component 160 completes.
Single Instance Continuous Data Replication
Where multiple computing systems containing data that is to be single instanced are located remotely from a single instancing database, various system configurations may be employed to avoid transferring data that is common to the remote multiple computing systems to the single instancing database. Three example configurations are described herein. Under a first configuration, a single instancing database is maintained at a first location and at each remote location (e.g., a second location, a third location, etc.). De-duplicated information in the single instancing database at each remote location is transferred to the single instancing database at the first location.
Under a second configuration, a single instancing database is maintained at a first location, and a computing system at each remote location (e.g., a second location, a third location, etc.) sends queries to the single instancing database at the first location to identify what data to transfer to the single instancing database. Under a third configuration, each computing system at each remote location (e.g., a second location, a third location, etc.) queries a single instancing database at a first location before transferring any data to the single instancing database. Each of these configurations is discussed separately below. In the discussed configurations, the example of a central or home office with one or more remote or satellite offices is used, where each remote office includes one or more computing systems. Although the terminology used to discuss these configurations implies a certain geographical positioning of computing systems, the single instancing system described herein may be used in a wide variety of configurations and is not limited to a specific geographical positioning.
Furthermore, the single instancing system described herein is not limited to the hub-and-spoke model implied by the terminology used to discuss these configurations. For example, a multi-tier hierarchical configuration could be employed, in which computing systems at the leaf nodes transfer data to computing systems at their respective parent nodes, which transfer data to computing systems at their parent nodes, and so on, up to the computing system at the top-most node of the hierarchy. As another example, a mesh configuration could be employed, in which a first computing system in the mesh configuration transfers data to a second computing system, the data to be ultimately transferred to a third computing system.
CDR, also called continuous data protection or continuous backup, refers to copying computer data by automatically saving a copy of every change made to that data, essentially capturing every version of the data that the user saves. It allows an administrator (or other user) to restore data to previous point(s) in time. There are multiple methods known in the art for capturing the continuous changes involving different technologies that serve different needs. CDR-based solutions can provide fine granularities of restorable objects ranging from disk images to logical data objects such as files, email data files, email messages, and database files and logs.
CDR differs from traditional backup or copy operation in that an administrator (or other user) does not have to specify the point in time to which the administrator would like to recover until the administrator is ready to perform a restore. Traditional backups or copy operations can only restore data to the point at which the backup or copy was made. With CDR, there are typically no backup or copy schedules. When data is written to disk, it can also be synchronously or asynchronously written to a second location, usually another computer over the network. In some situations, CDR will require less space on secondary storage media (usually disk) than traditional backup or copy operations. Most CDR solutions save byte- or block-level differences rather than file-level differences. This means that if one byte of a 100 GB file is changed, only the changed byte or block is backed up or copied. In contrast, traditional incremental and differential backups and copy operations generally make copies of entire files when those files change.
Clients 760a are part of remote office 710a, and store data in single instance database 740a. Clients 760b are part of remote office 710b, and store data in single instance database 740b. The clients 760 may be connected to their respective remote office 710 by way of a local area network (wired or wireless), with the remote offices 710 being connected to the home office 730 by a wide area network, or other network such as a public computer network (e.g., the Internet). By employing known CDR techniques, data from each remote single instance database 740 is sent to the home office 730 to be stored in a central data store (not shown in
Each remote office 710 tracks incremental changes for its clients 760, and then employs CDR to transmit those incremental changes to the home office 730. The home office 730 may then in turn employ its own single instance database 790 to avoid any duplication between the remote offices 710. For example, clients 760 that run MICROSOFT WINDOWS typically will each have a similar C:\Windows directory containing operating system files. Multiple client computing systems 760 will have stored a single copy of the C:\Windows directory at the remote single instance database 740, which will then be replicated to the central data store at the home office 730. The home office 730 will in turn store a single copy of the C:\Windows directory in the single instance database 790.
For example, the single instance database 740 may create or have created a substantially unique identifier for each stored file or data object and create or have created a substantially unique identifier for each new file or data object to be stored. The single instance database 740 may then compare the substantially unique identifier of the new file or data object with that of each previously stored file or data object to determine whether the new file or data object is already stored. In some embodiments, the single instance database 740 may store metadata that is specific to some clients while only storing one instance of the file or data object itself.
Accordingly, the client 760 copies data to a computing system that is geographically close, and then later the data from each client 760 can be copied to the home office 730 by copying the single instance or unique data from the single instance database 740 at the remote office 710. In this context, computing systems that are geographically close refers to computing systems that may be in the same room, in the same building or on the same campus. Computing systems that are geographically remote refers to computing systems that are not geographically close. In block 830, the single instancing system replicates the contents of the remote single instance database 740 to the home office 730. This block can be performed using CDR or other traditional methods of copying data. After block 830, these blocks conclude. Using this method, each remote single instance database 740 may send identical files or data objects to the home office single instance database 790 (i.e., data that is unique as to each individual remote office 710, but duplicative as to both remote offices 710). However, only one instance of each file or data object at the remote office 710 is stored by the home office single instance database 790.
For example, the remote office 910 may include a computing system (not shown in
The single instancing system may employ various optimization techniques. For example, the remote office 910 may cache substantially unique identifiers and only transmit one instance of a substantially unique identifier to the home office 930 if it detects more than one instance of the same substantially unique identifier. After the single instancing system has made a request to the single instance database 990 to determine if a particular file or data object is unique, the remote office 910 may cache the response so that subsequent identical files or data objects do not generate a new request to the single instance database 990. Rather, the single instancing system consults the cache first to determine if a previous request determined whether the file or data object is unique. If the cache responds that the file or data object is not unique, then the remote office 910 does not need to query the single instance database 990 to determine if the file or data object is unique.
In block 1030, the single instancing system queries the single instance database 990 at the home office 930 to determine whether the single instance database 990 already has a copy of the file or data object. In decision block 1040, if the file or data object is new to the single instance database 990, then the single instancing system continues at block 1050, else these blocks conclude. In block 1050, the single instancing system copies the new file or data object to the single instance database 990. If there are duplicates of the file or data object, then later when the single instance database 990 is queried for the duplicate files or data objects, the single instance database 990 will indicate that the file or data object is not new and need not be copied. In this way, duplicate data is not sent to the home office 930. After block 1050, these blocks conclude. This method differs from the method of
As shown in
From the foregoing, it will be appreciated that specific embodiments of the storage system have been described herein for purposes of illustration, but that various modifications may be made without deviating from the spirit and scope of the invention. For example, although backup operations have been described, the single instancing system may be used to reduce many types of redundant storage operations. As one example, the storage system may be employed by an Internet proxy server to reduce downloading of redundant files over the Internet by tracking a digest of each downloaded file and the location of a downloaded instance of the file behind the proxy server such that subsequent requests for the file can be serviced from the previously downloaded instance without accessing the file over the Internet. Similarly, the storage system could be used by a file system to reduce storage space by storing a single copy of data placed in multiple locations throughout the file system.
Unless the context clearly requires otherwise, throughout the description and the claims, the words “comprise,” “comprising,” and the like are to be construed in an inclusive sense, as opposed to an exclusive or exhaustive sense; that is to say, in the sense of “including, but not limited to.” The word “coupled”, as generally used herein, refers to two or more elements that may be either directly connected, or connected by way of one or more intermediate elements. Additionally, the words “herein,” “above,” “below,” and words of similar import, when used in this application, shall refer to this application as a whole and not to any particular portions of this application. Where the context permits, words in the above Detailed Description using the singular or plural number may also include the plural or singular number respectively. The word “or” in reference to a list of two or more items, that word covers all of the following interpretations of the word: any of the items in the list, all of the items in the list, and any combination of the items in the list.
The above detailed description of embodiments of the invention is not intended to be exhaustive or to limit the invention to the precise form disclosed above. While specific embodiments of, and examples for, the invention are described above for illustrative purposes, various equivalent modifications are possible within the scope of the invention, as those skilled in the relevant art will recognize. For example, while processes or blocks are presented in a given order, alternative embodiments may perform routines having steps, or employ systems having blocks, in a different order, and some processes or blocks may be deleted, moved, added, subdivided, combined, and/or modified. Each of these processes or blocks may be implemented in a variety of different ways. Also, while processes or blocks are at times shown as being performed in series, these processes or blocks may instead be performed in parallel, or may be performed at different times.
The teachings of the invention provided herein can be applied to other systems, not necessarily the system described above. The elements and acts of the various embodiments described above can be combined to provide further embodiments.
While certain embodiments of the inventions have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the disclosure. Indeed, the novel methods and systems described herein may be embodied in a variety of other forms; furthermore, various omissions, substitutions and changes in the form of the methods and systems described herein may be made without departing from the spirit of the disclosure. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the disclosure.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4084231 | Capozzi et al. | Apr 1978 | A |
| 4267568 | Dechant et al. | May 1981 | A |
| 4283787 | Chambers | Aug 1981 | A |
| 4417321 | Chang et al. | Nov 1983 | A |
| 4641274 | Swank | Feb 1987 | A |
| 4654819 | Stiffler et al. | Mar 1987 | A |
| 4686620 | Ng | Aug 1987 | A |
| 4912637 | Sheedy et al. | Mar 1990 | A |
| 4995035 | Cole et al. | Feb 1991 | A |
| 5005122 | Griffin et al. | Apr 1991 | A |
| 5093912 | Dong et al. | Mar 1992 | A |
| 5133065 | Cheffetz et al. | Jul 1992 | A |
| 5193154 | Kitajima et al. | Mar 1993 | A |
| 5212772 | Masters | May 1993 | A |
| 5226157 | Nakano et al. | Jul 1993 | A |
| 5239647 | Anglin et al. | Aug 1993 | A |
| 5241668 | Eastridge et al. | Aug 1993 | A |
| 5241670 | Eastridge et al. | Aug 1993 | A |
| 5276860 | Fortier et al. | Jan 1994 | A |
| 5276867 | Kenley et al. | Jan 1994 | A |
| 5287500 | Stoppani, Jr. | Feb 1994 | A |
| 5301286 | Rajani | Apr 1994 | A |
| 5321816 | Rogan et al. | Jun 1994 | A |
| 5333315 | Saether et al. | Jul 1994 | A |
| 5347653 | Flynn et al. | Sep 1994 | A |
| 5403639 | Belsan | Apr 1995 | A |
| 5410700 | Fecteau et al. | Apr 1995 | A |
| 5420996 | Aoyagi | May 1995 | A |
| 5448724 | Hayashi et al. | Sep 1995 | A |
| 5454099 | Myers et al. | Sep 1995 | A |
| 5491810 | Allen | Feb 1996 | A |
| 5495607 | Pisello et al. | Feb 1996 | A |
| 5499367 | Bamford et al. | Mar 1996 | A |
| 5504873 | Martin et al. | Apr 1996 | A |
| 5544345 | Carpenter et al. | Aug 1996 | A |
| 5544347 | Yanai et al. | Aug 1996 | A |
| 5559957 | Balk | Sep 1996 | A |
| 5559991 | Kanfi | Sep 1996 | A |
| 5619644 | Crockett et al. | Apr 1997 | A |
| 5625793 | Mirza | Apr 1997 | A |
| 5638509 | Dunphy et al. | Jun 1997 | A |
| 5642496 | Kanfi | Jun 1997 | A |
| 5673381 | Huai et al. | Sep 1997 | A |
| 5699361 | Ding et al. | Dec 1997 | A |
| 5720026 | Uemura | Feb 1998 | A |
| 5729743 | Squibb | Mar 1998 | A |
| 5732240 | Caccavale | Mar 1998 | A |
| 5751997 | Kullick et al. | May 1998 | A |
| 5758359 | Saxon | May 1998 | A |
| 5761677 | Senator et al. | Jun 1998 | A |
| 5764972 | Crouse et al. | Jun 1998 | A |
| 5765173 | Cane et al. | Jun 1998 | A |
| 5778395 | Whiting | Jul 1998 | A |
| 5790828 | Jost | Aug 1998 | A |
| 5812398 | Nielsen | Sep 1998 | A |
| 5813008 | Benson et al. | Sep 1998 | A |
| 5813009 | Johnson et al. | Sep 1998 | A |
| 5813017 | Morris | Sep 1998 | A |
| 5875478 | Blumenau | Feb 1999 | A |
| 5875481 | Ashton | Feb 1999 | A |
| 5878408 | Van Huben | Mar 1999 | A |
| 5887134 | Ebrahim | Mar 1999 | A |
| 5901327 | Ofek | May 1999 | A |
| 5907672 | Matze | May 1999 | A |
| 5924102 | Perks | Jul 1999 | A |
| 5930831 | Marsh et al. | Jul 1999 | A |
| 5940833 | Benson | Aug 1999 | A |
| 5950205 | Aviani, Jr. | Sep 1999 | A |
| 5956519 | Wise | Sep 1999 | A |
| 5974563 | Beeler, Jr. | Oct 1999 | A |
| 5990810 | Williams | Nov 1999 | A |
| 6021415 | Cannon et al. | Feb 2000 | A |
| 6026414 | Anglin | Feb 2000 | A |
| 6038379 | Fletcher et al. | Mar 2000 | A |
| 6044437 | Reinders | Mar 2000 | A |
| 6052735 | Ulrich et al. | Apr 2000 | A |
| 6076148 | Kedem et al. | Jun 2000 | A |
| 6094416 | Ying | Jul 2000 | A |
| 6131095 | Low et al. | Oct 2000 | A |
| 6131190 | Sidwell | Oct 2000 | A |
| 6148412 | Cannon et al. | Nov 2000 | A |
| 6154787 | Urevig et al. | Nov 2000 | A |
| 6161111 | Mutalik et al. | Dec 2000 | A |
| 6163856 | Dion | Dec 2000 | A |
| 6167402 | Yeager | Dec 2000 | A |
| 6212512 | Barney et al. | Apr 2001 | B1 |
| 6260069 | Anglin | Jul 2001 | B1 |
| 6269431 | Dunham | Jul 2001 | B1 |
| 6275953 | Vahalia et al. | Aug 2001 | B1 |
| 6286084 | Wexler et al. | Sep 2001 | B1 |
| 6289432 | Ault et al. | Sep 2001 | B1 |
| 6301592 | Aoyama et al. | Oct 2001 | B1 |
| 6324581 | Xu et al. | Nov 2001 | B1 |
| 6328766 | Long | Dec 2001 | B1 |
| 6330570 | Crighton | Dec 2001 | B1 |
| 6330642 | Carteau | Dec 2001 | B1 |
| 6343324 | Hubis et al. | Jan 2002 | B1 |
| RE37601 | Eastridge et al. | Mar 2002 | E |
| 6353878 | Dunham | Mar 2002 | B1 |
| 6356801 | Goodman et al. | Mar 2002 | B1 |
| 6366986 | St. Pierre | Apr 2002 | B1 |
| 6366988 | Skiba | Apr 2002 | B1 |
| 6374336 | Peters | Apr 2002 | B1 |
| 6389432 | Pothapragada et al. | May 2002 | B1 |
| 6389433 | Bolosky | May 2002 | B1 |
| 6397308 | Ofek | May 2002 | B1 |
| 6418478 | Ignatius et al. | Jul 2002 | B1 |
| 6421711 | Blumenau et al. | Jul 2002 | B1 |
| 6425057 | Cherkasova et al. | Jul 2002 | B1 |
| 6438368 | Phillips | Aug 2002 | B1 |
| 6487561 | Ofek et al. | Nov 2002 | B1 |
| 6496850 | Bowman-Amuah | Dec 2002 | B1 |
| 6519679 | Devireddy et al. | Feb 2003 | B2 |
| 6538669 | Lagueux, Jr. et al. | Mar 2003 | B1 |
| 6542972 | Ignatius et al. | Apr 2003 | B2 |
| 6557030 | Hoang | Apr 2003 | B1 |
| 6557089 | Reed | Apr 2003 | B1 |
| 6564228 | O'Connor | May 2003 | B1 |
| 6625623 | Midgley | Sep 2003 | B1 |
| 6658436 | Oshinsky et al. | Dec 2003 | B2 |
| 6658526 | Nguyen et al. | Dec 2003 | B2 |
| 6662198 | Satyanarayanan | Dec 2003 | B2 |
| 6665815 | Goldstein | Dec 2003 | B1 |
| 6704730 | Moulton et al. | Mar 2004 | B2 |
| 6721767 | De Meno et al. | Apr 2004 | B2 |
| 6732125 | Autry | May 2004 | B1 |
| 6757794 | Cabrera et al. | Jun 2004 | B2 |
| 6760723 | Oshinsky et al. | Jul 2004 | B2 |
| 6760812 | Degenaro et al. | Jul 2004 | B1 |
| 6779093 | Gupta | Aug 2004 | B1 |
| 6789161 | Blendermann | Sep 2004 | B1 |
| 6799258 | Linde | Sep 2004 | B1 |
| 6810398 | Moulton | Oct 2004 | B2 |
| 6823377 | Wu et al. | Nov 2004 | B1 |
| 6865655 | Andersen | Mar 2005 | B1 |
| 6886020 | Zahavi | Apr 2005 | B1 |
| 6912629 | West et al. | Jun 2005 | B1 |
| 6952758 | Chron et al. | Oct 2005 | B2 |
| 6983351 | Gibble | Jan 2006 | B2 |
| 7003641 | Prahlad et al. | Feb 2006 | B2 |
| 7028096 | Lee | Apr 2006 | B1 |
| 7035880 | Crescenti et al. | Apr 2006 | B1 |
| 7065619 | Zhu | Jun 2006 | B1 |
| 7082441 | Zahavi | Jul 2006 | B1 |
| 7085904 | Mizuno et al. | Aug 2006 | B2 |
| 7100089 | Phelps | Aug 2006 | B1 |
| 7103617 | Phatak | Sep 2006 | B2 |
| 7107298 | Prahlad et al. | Sep 2006 | B2 |
| 7107395 | Ofek | Sep 2006 | B1 |
| 7117246 | Christenson et al. | Oct 2006 | B2 |
| 7130860 | Pachet | Oct 2006 | B2 |
| 7130970 | Devassy et al. | Oct 2006 | B2 |
| 7143091 | Charnock | Nov 2006 | B2 |
| 7155465 | Lee | Dec 2006 | B2 |
| 7155633 | Tuma | Dec 2006 | B2 |
| 7162496 | Amarendran et al. | Jan 2007 | B2 |
| 7174433 | Kottomtharayil et al. | Feb 2007 | B2 |
| 7194454 | Hansen | Mar 2007 | B2 |
| 7197665 | Goldstein | Mar 2007 | B2 |
| 7225210 | Guthrie, II | May 2007 | B2 |
| 7243163 | Friend et al. | Jul 2007 | B1 |
| 7246207 | Kottomtharayil et al. | Jul 2007 | B2 |
| 7246272 | Cabezas et al. | Jul 2007 | B2 |
| 7272606 | Borthakur et al. | Sep 2007 | B2 |
| 7284030 | Ackaouy et al. | Oct 2007 | B2 |
| 7287252 | Bussiere et al. | Oct 2007 | B2 |
| 7315923 | Retnamma et al. | Jan 2008 | B2 |
| 7343356 | Prahlad | Mar 2008 | B2 |
| 7343453 | Prahlad et al. | Mar 2008 | B2 |
| 7343459 | Prahlad | Mar 2008 | B2 |
| 7346751 | Prahlad | Mar 2008 | B2 |
| 7383462 | Osaki et al. | Jun 2008 | B2 |
| 7389311 | Crescenti et al. | Jun 2008 | B1 |
| 7395282 | Crescenti et al. | Jul 2008 | B1 |
| 7412583 | Burton | Aug 2008 | B2 |
| 7437388 | DeVos | Oct 2008 | B1 |
| 7440982 | Lu et al. | Oct 2008 | B2 |
| 7454569 | Kavuri et al. | Nov 2008 | B2 |
| 7472238 | Gokhale et al. | Dec 2008 | B1 |
| 7472242 | Deshmukh et al. | Dec 2008 | B1 |
| 7490207 | Amarendran et al. | Feb 2009 | B2 |
| 7500053 | Kavuri et al. | Mar 2009 | B1 |
| 7512595 | McBride et al. | Mar 2009 | B1 |
| 7516186 | Borghetti et al. | Apr 2009 | B1 |
| 7519726 | Palliyll et al. | Apr 2009 | B2 |
| 7529782 | Prahlad et al. | May 2009 | B2 |
| 7536291 | Vijayan et al. | May 2009 | B1 |
| 7539710 | Haustein et al. | May 2009 | B1 |
| 7543125 | Gokhale | Jun 2009 | B2 |
| 7546324 | Prahlad et al. | Jun 2009 | B2 |
| 7552358 | Asgar-Deen et al. | Jun 2009 | B1 |
| 7567188 | Anglin | Jul 2009 | B1 |
| 7568080 | Prahlad et al. | Jul 2009 | B2 |
| 7574692 | Herscu | Aug 2009 | B2 |
| 7577806 | Rowan | Aug 2009 | B2 |
| 7581077 | Ignatius et al. | Aug 2009 | B2 |
| 7584338 | Bricker et al. | Sep 2009 | B1 |
| 7603386 | Amarendran et al. | Oct 2009 | B2 |
| 7606844 | Kottomtharayil | Oct 2009 | B2 |
| 7613748 | Brockway et al. | Nov 2009 | B2 |
| 7613752 | Prahlad et al. | Nov 2009 | B2 |
| 7617253 | Prahlad et al. | Nov 2009 | B2 |
| 7617262 | Prahlad et al. | Nov 2009 | B2 |
| 7620710 | Kottomtharayil et al. | Nov 2009 | B2 |
| 7631194 | Wahlert et al. | Dec 2009 | B2 |
| 7636743 | Erofeev | Dec 2009 | B2 |
| 7651593 | Prahlad et al. | Jan 2010 | B2 |
| 7657550 | Prahlad et al. | Feb 2010 | B2 |
| 7660807 | Prahlad et al. | Feb 2010 | B2 |
| 7661028 | Prahlad et al. | Feb 2010 | B2 |
| 7664771 | Kusters | Feb 2010 | B2 |
| 7685126 | Patel et al. | Mar 2010 | B2 |
| 7702782 | Pai | Apr 2010 | B1 |
| 7720841 | Gu et al. | May 2010 | B2 |
| 7730113 | Payette | Jun 2010 | B1 |
| 7734669 | Kottomtharayil et al. | Jun 2010 | B2 |
| 7734820 | Ranade et al. | Jun 2010 | B1 |
| 7739235 | Rousseau | Jun 2010 | B2 |
| 7743051 | Kashyap | Jun 2010 | B1 |
| 7747577 | Cannon | Jun 2010 | B2 |
| 7747579 | Prahlad et al. | Jun 2010 | B2 |
| 7761425 | Erickson et al. | Jul 2010 | B1 |
| 7779032 | Garfinkel | Aug 2010 | B1 |
| 7797279 | Starling et al. | Sep 2010 | B1 |
| 7801864 | Prahlad et al. | Sep 2010 | B2 |
| 7809914 | Kottomtharayil et al. | Oct 2010 | B2 |
| 7814074 | Anglin et al. | Oct 2010 | B2 |
| 7814149 | Stringham | Oct 2010 | B1 |
| 7822939 | Veprinsky et al. | Oct 2010 | B1 |
| 7827150 | Wu et al. | Nov 2010 | B1 |
| 7831795 | Prahlad et al. | Nov 2010 | B2 |
| 7840533 | Prahlad et al. | Nov 2010 | B2 |
| 7899871 | Kumar | Mar 2011 | B1 |
| 7962452 | Anglin et al. | Jun 2011 | B2 |
| 8041907 | Wu | Oct 2011 | B1 |
| 8074043 | Zeis | Dec 2011 | B1 |
| 8095756 | Somavarapu | Jan 2012 | B1 |
| 8108446 | Christiaens | Jan 2012 | B1 |
| 8108638 | Kishi | Jan 2012 | B2 |
| 8131669 | Cannon | Mar 2012 | B2 |
| 8145614 | Zimran et al. | Mar 2012 | B1 |
| 8156086 | Lu et al. | Apr 2012 | B2 |
| 8170995 | Prahlad et al. | May 2012 | B2 |
| 8199911 | Tsaur et al. | Jun 2012 | B1 |
| 8200638 | Zheng et al. | Jun 2012 | B1 |
| 8200923 | Healey et al. | Jun 2012 | B1 |
| 8204862 | Paulzagade et al. | Jun 2012 | B1 |
| 8209334 | Doerner | Jun 2012 | B1 |
| 8224875 | Christiaens et al. | Jul 2012 | B1 |
| 8229954 | Kottomtharayil et al. | Jul 2012 | B2 |
| 8230195 | Amarendran et al. | Jul 2012 | B2 |
| 8261240 | Hoban | Sep 2012 | B2 |
| 8280854 | Emmert | Oct 2012 | B1 |
| 8285681 | Prahlad et al. | Oct 2012 | B2 |
| 8307177 | Prahlad et al. | Nov 2012 | B2 |
| 8352422 | Prahlad et al. | Jan 2013 | B2 |
| 8364652 | Vijayan et al. | Jan 2013 | B2 |
| 8370315 | Efstathopoulos et al. | Feb 2013 | B1 |
| 8370542 | Lu et al. | Feb 2013 | B2 |
| 8375008 | Gomes | Feb 2013 | B1 |
| 8375181 | Kekre et al. | Feb 2013 | B1 |
| 8407190 | Prahlad et al. | Mar 2013 | B2 |
| 8468320 | Stringham | Jun 2013 | B1 |
| 8479304 | Clifford | Jul 2013 | B1 |
| 8484162 | Prahlad et al. | Jul 2013 | B2 |
| 8510573 | Muller et al. | Aug 2013 | B2 |
| 8527469 | Hwang | Sep 2013 | B2 |
| 8549350 | Dutch et al. | Oct 2013 | B1 |
| 8572055 | Wu | Oct 2013 | B1 |
| 8572340 | Vijayan et al. | Oct 2013 | B2 |
| 8577851 | Vijayan et al. | Nov 2013 | B2 |
| 8578109 | Vijayan et al. | Nov 2013 | B2 |
| 8578120 | Attarde et al. | Nov 2013 | B2 |
| 8595191 | Prahlad et al. | Nov 2013 | B2 |
| 8621240 | Auchmoody et al. | Dec 2013 | B1 |
| 8645320 | Prahlad et al. | Feb 2014 | B2 |
| 8719264 | Varadharajan | May 2014 | B2 |
| 8725688 | Lad | May 2014 | B2 |
| 8726242 | Ngo | May 2014 | B2 |
| 8745105 | Erofeev | Jun 2014 | B2 |
| 8775823 | Gokhale et al. | Jul 2014 | B2 |
| 8825720 | Xie et al. | Sep 2014 | B1 |
| 8849762 | Kumarasarny et al. | Sep 2014 | B2 |
| 8909980 | Lewis et al. | Dec 2014 | B1 |
| 8930306 | Ngo et al. | Jan 2015 | B1 |
| 8938481 | Kumarasamy et al. | Jan 2015 | B2 |
| 8954446 | Vijayan et al. | Feb 2015 | B2 |
| 9015181 | Kottomtharayil et al. | Apr 2015 | B2 |
| 9020900 | Vijayan et al. | Apr 2015 | B2 |
| 9092441 | Patiejunas et al. | Jul 2015 | B1 |
| 9098495 | Gokhale | Aug 2015 | B2 |
| 9104623 | Vijayan et al. | Aug 2015 | B2 |
| 9110602 | Vijayan et al. | Aug 2015 | B2 |
| 9116850 | Vijayan et al. | Aug 2015 | B2 |
| 9128901 | Nickurak | Sep 2015 | B1 |
| 9171008 | Prahlad et al. | Oct 2015 | B2 |
| 9208160 | Prahlad et al. | Dec 2015 | B2 |
| 9218374 | Muller et al. | Dec 2015 | B2 |
| 9218375 | Muller et al. | Dec 2015 | B2 |
| 9218376 | Muller et al. | Dec 2015 | B2 |
| 9239687 | Vijayan et al. | Jan 2016 | B2 |
| 9244779 | Littlefield et al. | Jan 2016 | B2 |
| 9251186 | Muller et al. | Feb 2016 | B2 |
| 9298386 | Baldwin et al. | Mar 2016 | B2 |
| 9298715 | Kumarasamy et al. | Mar 2016 | B2 |
| 9298724 | Patil et al. | Mar 2016 | B1 |
| 9323820 | Lauinger et al. | Apr 2016 | B1 |
| 9336076 | Baldwin et al. | May 2016 | B2 |
| 9342537 | Kumarasamy et al. | May 2016 | B2 |
| 9405631 | Prahlad et al. | Aug 2016 | B2 |
| 9405763 | Prahlad et al. | Aug 2016 | B2 |
| 9442806 | Bardale | Sep 2016 | B1 |
| 9483486 | Christiaens | Nov 2016 | B1 |
| 9575673 | Mitkar et al. | Feb 2017 | B2 |
| 9619480 | Vijayan et al. | Apr 2017 | B2 |
| 9633033 | Vijayan et al. | Apr 2017 | B2 |
| 9633056 | Attarde et al. | Apr 2017 | B2 |
| 9639289 | Vijayan et al. | May 2017 | B2 |
| 9665591 | Vijayan et al. | May 2017 | B2 |
| 9678968 | Taylor et al. | Jun 2017 | B1 |
| 9858156 | Muller et al. | Jan 2018 | B2 |
| 9898225 | Vijayan et al. | Feb 2018 | B2 |
| 9898478 | Vijayan et al. | Feb 2018 | B2 |
| 9934238 | Mitkar et al. | Apr 2018 | B2 |
| 9990253 | Rajimwale et al. | Jun 2018 | B1 |
| 10061663 | Vijayan et al. | Aug 2018 | B2 |
| 10126973 | Vijayan et al. | Nov 2018 | B2 |
| 10176053 | Muller et al. | Jan 2019 | B2 |
| 10191816 | Vijayan et al. | Jan 2019 | B2 |
| 10229133 | Vijayan et al. | Mar 2019 | B2 |
| 10255143 | Vijayan et al. | Apr 2019 | B2 |
| 10310953 | Vijayan et al. | Jun 2019 | B2 |
| 10339106 | Vijayan et al. | Jul 2019 | B2 |
| 10380072 | Attarde et al. | Aug 2019 | B2 |
| 10387269 | Muller et al. | Aug 2019 | B2 |
| 10445293 | Attarde et al. | Oct 2019 | B2 |
| 10474638 | Mitkar et al. | Nov 2019 | B2 |
| 10481824 | Vijayan et al. | Nov 2019 | B2 |
| 10481825 | Vijayan et al. | Nov 2019 | B2 |
| 10481826 | Vijayan et al. | Nov 2019 | B2 |
| 10540327 | Ngo et al. | Jan 2020 | B2 |
| 10592357 | Vijayan et al. | Mar 2020 | B2 |
| 10740295 | Vijayan et al. | Aug 2020 | B2 |
| 20010052015 | Lin et al. | Dec 2001 | A1 |
| 20020062439 | Cotugno et al. | May 2002 | A1 |
| 20020065892 | Malik | May 2002 | A1 |
| 20020083055 | Pachet | Jun 2002 | A1 |
| 20020107877 | Whiting et al. | Aug 2002 | A1 |
| 20020133601 | Kennamer et al. | Sep 2002 | A1 |
| 20020143892 | Mogul | Oct 2002 | A1 |
| 20020144250 | Yen | Oct 2002 | A1 |
| 20020169934 | Krapp et al. | Nov 2002 | A1 |
| 20030033308 | Patel et al. | Feb 2003 | A1 |
| 20030084076 | Sekiguchi et al. | May 2003 | A1 |
| 20030105716 | Sutton, Jr. | Jun 2003 | A1 |
| 20030115346 | McHenry et al. | Jun 2003 | A1 |
| 20030149750 | Franzenburg | Aug 2003 | A1 |
| 20030172130 | Fruchtman et al. | Sep 2003 | A1 |
| 20030174648 | Wang et al. | Sep 2003 | A1 |
| 20030182310 | Charnock | Sep 2003 | A1 |
| 20030187917 | Cohen | Oct 2003 | A1 |
| 20030188106 | Cohen | Oct 2003 | A1 |
| 20040010562 | Itonaga | Jan 2004 | A1 |
| 20040128442 | Hinshaw et al. | Jul 2004 | A1 |
| 20040148306 | Moulton et al. | Jul 2004 | A1 |
| 20040181519 | Anwar | Sep 2004 | A1 |
| 20040215746 | McCanne et al. | Oct 2004 | A1 |
| 20040230753 | Amiri et al. | Nov 2004 | A1 |
| 20050033756 | Kottomtharayil et al. | Feb 2005 | A1 |
| 20050060643 | Glass | Mar 2005 | A1 |
| 20050066118 | Perry | Mar 2005 | A1 |
| 20050066225 | Rowan | Mar 2005 | A1 |
| 20050108292 | Burton | May 2005 | A1 |
| 20050114450 | DeVos | May 2005 | A1 |
| 20050117558 | Angermann et al. | Jun 2005 | A1 |
| 20050144202 | Chen | Jun 2005 | A1 |
| 20050204108 | Ofek et al. | Sep 2005 | A1 |
| 20050216659 | Ogawa et al. | Sep 2005 | A1 |
| 20050243609 | Yang et al. | Nov 2005 | A1 |
| 20050246393 | Coates et al. | Nov 2005 | A1 |
| 20050268068 | Ignatius et al. | Dec 2005 | A1 |
| 20050273654 | Chen et al. | Dec 2005 | A1 |
| 20060004808 | Hsu et al. | Jan 2006 | A1 |
| 20060005048 | Osaki | Jan 2006 | A1 |
| 20060010227 | Atluri | Jan 2006 | A1 |
| 20060020660 | Prasad et al. | Jan 2006 | A1 |
| 20060064456 | Kalthoff et al. | Mar 2006 | A1 |
| 20060074957 | Yamamoto et al. | Apr 2006 | A1 |
| 20060089954 | Anschutz | Apr 2006 | A1 |
| 20060095527 | Malik | May 2006 | A1 |
| 20060101096 | Fuerst | May 2006 | A1 |
| 20060129537 | Torii | Jun 2006 | A1 |
| 20060136685 | Griv | Jun 2006 | A1 |
| 20060167900 | Pingte et al. | Jul 2006 | A1 |
| 20060168318 | Twiss | Jul 2006 | A1 |
| 20060179261 | Twiss | Aug 2006 | A1 |
| 20060179405 | Chao et al. | Aug 2006 | A1 |
| 20060224846 | Amarendran et al. | Oct 2006 | A1 |
| 20060277154 | Lunt | Dec 2006 | A1 |
| 20070006018 | Thompson | Jan 2007 | A1 |
| 20070038714 | Sell | Feb 2007 | A1 |
| 20070043757 | Benton | Feb 2007 | A1 |
| 20070050526 | Abe et al. | Mar 2007 | A1 |
| 20070067263 | Syed | Mar 2007 | A1 |
| 20070073814 | Kamat et al. | Mar 2007 | A1 |
| 20070156966 | Sundarrajan et al. | Jul 2007 | A1 |
| 20070162462 | Zhang et al. | Jul 2007 | A1 |
| 20070179990 | Zimran et al. | Aug 2007 | A1 |
| 20070179995 | Prahlad et al. | Aug 2007 | A1 |
| 20070192444 | Ackaouy et al. | Aug 2007 | A1 |
| 20070192542 | Frolund et al. | Aug 2007 | A1 |
| 20070192544 | Frolund et al. | Aug 2007 | A1 |
| 20070203937 | Prahlad et al. | Aug 2007 | A1 |
| 20070250670 | Fineberg et al. | Oct 2007 | A1 |
| 20070255758 | Zheng et al. | Nov 2007 | A1 |
| 20080005141 | Zheng et al. | Jan 2008 | A1 |
| 20080005509 | Smith et al. | Jan 2008 | A1 |
| 20080016131 | Sandorfi | Jan 2008 | A1 |
| 20080028149 | Pardikar et al. | Jan 2008 | A1 |
| 20080089342 | Lansing et al. | Apr 2008 | A1 |
| 20080091655 | Gokhale et al. | Apr 2008 | A1 |
| 20080091725 | Hwang et al. | Apr 2008 | A1 |
| 20080098041 | Chidambaran et al. | Apr 2008 | A1 |
| 20080098083 | Shergill | Apr 2008 | A1 |
| 20080133561 | Dubnicki et al. | Jun 2008 | A1 |
| 20080140630 | Sato et al. | Jun 2008 | A1 |
| 20080159331 | Mace et al. | Jul 2008 | A1 |
| 20080229037 | Bunte | Sep 2008 | A1 |
| 20080243769 | Arbour | Oct 2008 | A1 |
| 20080243879 | Gokhale et al. | Oct 2008 | A1 |
| 20080243914 | Prahlad et al. | Oct 2008 | A1 |
| 20080243953 | Wu et al. | Oct 2008 | A1 |
| 20080243957 | Prahlad et al. | Oct 2008 | A1 |
| 20080243958 | Prahlad et al. | Oct 2008 | A1 |
| 20080244172 | Kano | Oct 2008 | A1 |
| 20080244199 | Nakamura et al. | Oct 2008 | A1 |
| 20080244204 | Cremelie | Oct 2008 | A1 |
| 20080244205 | Amano | Oct 2008 | A1 |
| 20080250204 | Kavuri et al. | Oct 2008 | A1 |
| 20080256326 | Patterson | Oct 2008 | A1 |
| 20080256431 | Hornberger | Oct 2008 | A1 |
| 20080281908 | McCanne et al. | Nov 2008 | A1 |
| 20080294660 | Patterson et al. | Nov 2008 | A1 |
| 20080294696 | Frandzel | Nov 2008 | A1 |
| 20080313236 | Vijayakumar et al. | Dec 2008 | A1 |
| 20080320151 | McCanne et al. | Dec 2008 | A1 |
| 20090013129 | Bondurant | Jan 2009 | A1 |
| 20090013258 | Hintermeister et al. | Jan 2009 | A1 |
| 20090043767 | Joshi et al. | Feb 2009 | A1 |
| 20090055425 | Evans et al. | Feb 2009 | A1 |
| 20090055471 | Kozat et al. | Feb 2009 | A1 |
| 20090077140 | Anglin et al. | Mar 2009 | A1 |
| 20090138481 | Chatley et al. | May 2009 | A1 |
| 20090144416 | Chatley et al. | Jun 2009 | A1 |
| 20090144422 | Chatley et al. | Jun 2009 | A1 |
| 20090171888 | Anglin | Jul 2009 | A1 |
| 20090172139 | Wong et al. | Jul 2009 | A1 |
| 20090182789 | Sandorfi | Jul 2009 | A1 |
| 20090183162 | Kindel et al. | Jul 2009 | A1 |
| 20090204636 | Li et al. | Aug 2009 | A1 |
| 20090204649 | Wong et al. | Aug 2009 | A1 |
| 20090210431 | Marinkovic et al. | Aug 2009 | A1 |
| 20090228599 | Anglin | Sep 2009 | A1 |
| 20090243846 | Yuuki | Oct 2009 | A1 |
| 20090254507 | Hosoya et al. | Oct 2009 | A1 |
| 20090268903 | Bojinov et al. | Oct 2009 | A1 |
| 20090271454 | Anglin et al. | Oct 2009 | A1 |
| 20090276454 | Smith | Nov 2009 | A1 |
| 20090307251 | Heller et al. | Dec 2009 | A1 |
| 20090319534 | Gokhale | Dec 2009 | A1 |
| 20090319585 | Gokhale | Dec 2009 | A1 |
| 20090327625 | Jaquette et al. | Dec 2009 | A1 |
| 20100005259 | Prahlad | Jan 2010 | A1 |
| 20100011178 | Feathergill | Jan 2010 | A1 |
| 20100031086 | Leppard | Feb 2010 | A1 |
| 20100036887 | Anglin et al. | Feb 2010 | A1 |
| 20100042790 | Mondal et al. | Feb 2010 | A1 |
| 20100049926 | Fuente et al. | Feb 2010 | A1 |
| 20100049927 | Fuente et al. | Feb 2010 | A1 |
| 20100070478 | Anglin | Mar 2010 | A1 |
| 20100077161 | Stoakes | Mar 2010 | A1 |
| 20100082558 | Anglin et al. | Apr 2010 | A1 |
| 20100082672 | Kottomtharayil et al. | Apr 2010 | A1 |
| 20100088296 | Periyagaram et al. | Apr 2010 | A1 |
| 20100094817 | Ben-Shaul et al. | Apr 2010 | A1 |
| 20100100529 | Erofeev | Apr 2010 | A1 |
| 20100114833 | Mu | May 2010 | A1 |
| 20100153511 | Lin | Jun 2010 | A1 |
| 20100169287 | Klose | Jul 2010 | A1 |
| 20100180075 | McCloskey et al. | Jul 2010 | A1 |
| 20100198864 | Ravid | Aug 2010 | A1 |
| 20100223495 | Leppard | Sep 2010 | A1 |
| 20100250501 | Mandagere et al. | Sep 2010 | A1 |
| 20100250549 | Muller et al. | Sep 2010 | A1 |
| 20100250896 | Matze | Sep 2010 | A1 |
| 20100257142 | Murphy et al. | Oct 2010 | A1 |
| 20100257346 | Sosnosky et al. | Oct 2010 | A1 |
| 20100257403 | Virk et al. | Oct 2010 | A1 |
| 20100306283 | Johnson et al. | Dec 2010 | A1 |
| 20100312752 | Zeis et al. | Dec 2010 | A1 |
| 20100318759 | Hamilton et al. | Dec 2010 | A1 |
| 20100332401 | Prahlad et al. | Dec 2010 | A1 |
| 20100332454 | Prahlad et al. | Dec 2010 | A1 |
| 20110010498 | Lay et al. | Jan 2011 | A1 |
| 20110060940 | Taylor et al. | Mar 2011 | A1 |
| 20110072291 | Murase | Mar 2011 | A1 |
| 20110113012 | Gruhl et al. | May 2011 | A1 |
| 20110113013 | Reddy et al. | May 2011 | A1 |
| 20110113016 | Gruhl et al. | May 2011 | A1 |
| 20110119741 | Kelly et al. | May 2011 | A1 |
| 20110153570 | Kim et al. | Jun 2011 | A1 |
| 20110161723 | Taleck et al. | Jun 2011 | A1 |
| 20110167221 | Pangal et al. | Jul 2011 | A1 |
| 20110258161 | Constantinescu et al. | Oct 2011 | A1 |
| 20110276543 | Matze | Nov 2011 | A1 |
| 20110289281 | Spackman | Nov 2011 | A1 |
| 20110302140 | Gokhale et al. | Dec 2011 | A1 |
| 20110314070 | Brown et al. | Dec 2011 | A1 |
| 20110314400 | Mital et al. | Dec 2011 | A1 |
| 20120011101 | Fang et al. | Jan 2012 | A1 |
| 20120016839 | Yueh | Jan 2012 | A1 |
| 20120016845 | Bates | Jan 2012 | A1 |
| 20120078881 | Crump et al. | Mar 2012 | A1 |
| 20120084272 | Garces-Erice et al. | Apr 2012 | A1 |
| 20120089574 | Doerner | Apr 2012 | A1 |
| 20120150818 | Vijayan et al. | Jun 2012 | A1 |
| 20120166403 | Kim et al. | Jun 2012 | A1 |
| 20120185437 | Pavlov et al. | Jul 2012 | A1 |
| 20120221817 | Yueh | Aug 2012 | A1 |
| 20120233417 | Kalach | Sep 2012 | A1 |
| 20120303622 | Dean et al. | Nov 2012 | A1 |
| 20130006943 | Chavda et al. | Jan 2013 | A1 |
| 20130219470 | Chintala et al. | Aug 2013 | A1 |
| 20130238562 | Kumarasamy et al. | Sep 2013 | A1 |
| 20130238572 | Prahlad et al. | Sep 2013 | A1 |
| 20130262396 | Kripalani et al. | Oct 2013 | A1 |
| 20130339298 | Muller et al. | Dec 2013 | A1 |
| 20130339310 | Muller et al. | Dec 2013 | A1 |
| 20140032940 | Sartirana et al. | Jan 2014 | A1 |
| 20140115287 | Schnapp et al. | Apr 2014 | A1 |
| 20140181028 | Prahlad et al. | Jun 2014 | A1 |
| 20140195749 | Colgrove et al. | Jul 2014 | A1 |
| 20140196037 | Gopalan et al. | Jul 2014 | A1 |
| 20140201142 | Varadharajan et al. | Jul 2014 | A1 |
| 20140201150 | Kumarasamy et al. | Jul 2014 | A1 |
| 20140201153 | Vijayan et al. | Jul 2014 | A1 |
| 20140229451 | Venkatesh et al. | Aug 2014 | A1 |
| 20140250076 | Lad | Sep 2014 | A1 |
| 20140258245 | Estes | Sep 2014 | A1 |
| 20140281758 | Klein et al. | Sep 2014 | A1 |
| 20140289225 | Chan et al. | Sep 2014 | A1 |
| 20140337285 | Gokhale et al. | Nov 2014 | A1 |
| 20140337664 | Gokhale et al. | Nov 2014 | A1 |
| 20150012698 | Bolla et al. | Jan 2015 | A1 |
| 20150088821 | Blea et al. | Mar 2015 | A1 |
| 20150089185 | Brandyberry et al. | Mar 2015 | A1 |
| 20150134611 | Avati et al. | May 2015 | A1 |
| 20150154220 | Ngo et al. | Jun 2015 | A1 |
| 20150161015 | Kumarasamy et al. | Jun 2015 | A1 |
| 20150212893 | Pawar et al. | Jul 2015 | A1 |
| 20150212894 | Pawar et al. | Jul 2015 | A1 |
| 20150212895 | Pawar et al. | Jul 2015 | A1 |
| 20150212896 | Pawar et al. | Jul 2015 | A1 |
| 20150212897 | Pawar et al. | Jul 2015 | A1 |
| 20150248466 | Jernigan, IV et al. | Sep 2015 | A1 |
| 20150261776 | Attarde et al. | Sep 2015 | A1 |
| 20150269032 | Muthyala et al. | Sep 2015 | A1 |
| 20150269212 | Kramer et al. | Sep 2015 | A1 |
| 20150278104 | Moon et al. | Oct 2015 | A1 |
| 20150347306 | Gschwind | Dec 2015 | A1 |
| 20150378839 | Langouev et al. | Dec 2015 | A1 |
| 20160026405 | Dhuse | Jan 2016 | A1 |
| 20160041880 | Mitkar et al. | Feb 2016 | A1 |
| 20160042090 | Mitkar et al. | Feb 2016 | A1 |
| 20160062846 | Nallathambi et al. | Mar 2016 | A1 |
| 20160065671 | Nallathambi et al. | Mar 2016 | A1 |
| 20160139836 | Nallathambi et al. | May 2016 | A1 |
| 20160142483 | Nallathambi et al. | May 2016 | A1 |
| 20160154709 | Mitkar et al. | Jun 2016 | A1 |
| 20160170657 | Suehr et al. | Jun 2016 | A1 |
| 20160188416 | Muller et al. | Jun 2016 | A1 |
| 20160196070 | Vijayan et al. | Jul 2016 | A1 |
| 20160266980 | Muller et al. | Sep 2016 | A1 |
| 20160299818 | Vijayan et al. | Oct 2016 | A1 |
| 20160306707 | Vijayan et al. | Oct 2016 | A1 |
| 20160306708 | Prahlad et al. | Oct 2016 | A1 |
| 20160306818 | Vijayan et al. | Oct 2016 | A1 |
| 20160350391 | Vijayan et al. | Dec 2016 | A1 |
| 20170031768 | Sarab | Feb 2017 | A1 |
| 20170083558 | Vijayan et al. | Mar 2017 | A1 |
| 20170083563 | Vijayan et al. | Mar 2017 | A1 |
| 20170090773 | Vijayan et al. | Mar 2017 | A1 |
| 20170090786 | Parab et al. | Mar 2017 | A1 |
| 20170168903 | Dornemann et al. | May 2017 | A1 |
| 20170192860 | Vijayan et al. | Jul 2017 | A1 |
| 20170192861 | Vijayan et al. | Jul 2017 | A1 |
| 20170192866 | Vijayan et al. | Jul 2017 | A1 |
| 20170192868 | Vijayan et al. | Jul 2017 | A1 |
| 20170193003 | Vijayan et al. | Jul 2017 | A1 |
| 20170199699 | Vijayan et al. | Jul 2017 | A1 |
| 20170206219 | Vijayan et al. | Jul 2017 | A1 |
| 20170235647 | Kilaru et al. | Aug 2017 | A1 |
| 20170242871 | Kilaru et al. | Aug 2017 | A1 |
| 20170262217 | Pradhan et al. | Sep 2017 | A1 |
| 20170315876 | Dornquast et al. | Nov 2017 | A1 |
| 20180075055 | Ngo et al. | Mar 2018 | A1 |
| 20180189314 | Mitkar et al. | Jul 2018 | A1 |
| 20180196720 | Muller et al. | Jul 2018 | A1 |
| 20190179805 | Prahlad et al. | Jun 2019 | A1 |
| 20190188088 | Muller et al. | Jun 2019 | A1 |
| 20190205290 | Vijayan et al. | Jul 2019 | A1 |
| 20190227879 | Vijayan et al. | Jul 2019 | A1 |
| 20190272220 | Vijayan et al. | Sep 2019 | A1 |
| 20190272221 | Vijayan et al. | Sep 2019 | A1 |
| 20190310968 | Attarde et al. | Oct 2019 | A1 |
| 20200104052 | Vijayan et al. | Apr 2020 | A1 |
| 20200104213 | Muller et al. | Apr 2020 | A1 |
| 20200117641 | Mitkar et al. | Apr 2020 | A1 |
| 20200167091 | Haridas et al. | May 2020 | A1 |
| 20200167240 | Haridas et al. | May 2020 | A1 |
| 20200250145 | Ngo et al. | Aug 2020 | A1 |
| Number | Date | Country |
|---|---|---|
| 0259912 | Mar 1988 | EP |
| 0405926 | Jan 1991 | EP |
| 0467546 | Jan 1992 | EP |
| 0541281 | May 1993 | EP |
| 0774715 | May 1997 | EP |
| 0809184 | Nov 1997 | EP |
| 0899662 | Mar 1999 | EP |
| 0981090 | Feb 2000 | EP |
| WO 1995013580 | May 1995 | WO |
| WO 99009480 | Feb 1999 | WO |
| WO 1999012098 | Mar 1999 | WO |
| WO 2002005466 | Jan 2002 | WO |
| WO 2006052872 | May 2006 | WO |
| WO 2010013292 | Feb 2010 | WO |
| WO 2010140264 | Dec 2010 | WO |
| WO 2012044366 | Apr 2012 | WO |
| WO 2012044367 | Apr 2012 | WO |
| WO 2013188550 | Dec 2013 | WO |
| Entry |
|---|
| Armstead et al., “Implementation of a Campus-Wide Distributed Mass Storage Service: The Dream vs. Reality,” IEEE, 1995, pp. 190-199. |
| Arneson, “Mass Storage Archiving in Network Environments,” Digest of Papers, Ninth IEEE Symposium on Mass Storage Systems, Oct. 31, 1988-Nov. 3, 1988, pp. 45-50, Monterey, CA. |
| Ashton, et al., “Two Decades of policy-based storage management for the IBM mainframe computer”, www.research.ibm.com, 19 pages, published Apr. 10, 2003, printed Jan. 3, 2009, www.research.ibm.com, Apr. 10, 2003, pp. 19. |
| Bhagwat, Extreme Binning: Scalable, Parallel Deduplication for Chunk-based File Backup. IEEE 2009, 9 pages. |
| Cabrera, et al. “ADSM: A Multi-Platform, Scalable, Back-up and Archive Mass Storage System,” Digest of Papers, Compcon '95, Proceedings of the 40th IEEE Computer Society International Conference, Mar. 5, 1995-Mar. 9, 1995, pp. 420-427, San Francisco, CA. |
| Cohen, Edith, et al, “The Age Penalty and Its Effect on Cache Performance.” In USITS, pp. 73-84. 2001. |
| Cohen, Edith, et al, “Aging through cascaded caches: Performance issues in the distribution of web content.” In ACM Sigcomm Computer Communication Review, vol. 31, No. 4, pp. 41-53. ACM, 2001. |
| Cohen, Edith, et al,. “Refreshment policies for web content caches.” Computer Networks 38.6 (2002): 795-808. |
| CommVault Systems, Inc. “Continuous Data Replicator 7.0,” Product Data Sheet, 2007. |
| CommVault Systems, Inc., “Deduplication—How To,” http://documentation.commvault.com/commvault/release_8_0_0/books_online_1/english_US/features/single_instance/single_instance_how_to.htm, internet accessed on Jan. 26, 2009, 7 pages. |
| CommVault Systems, Inc., “Deduplication,” http://documentation.commvault.com/commvault/release_8_0_0/books_online_1/english_US/features/single_instance/single_instance.htm, internet accessed on Jan. 26, 2009, 9 pages. |
| Diligent Technologies HyperFactor, http://www.dilligent.com/products:protecTIER-1:HyperFactor-1, Internet accessed on Dec. 5, 2008, 2 pages. |
| Dubnicki, et al. “HYDRAstor: A Scalable Secondary Storage.” FAST. vol. 9.2009, 74 pages. |
| Eitel, “Backup and Storage Management in Distributed Heterogeneous Environments,” IEEE, 1994, pp. 124-126. |
| Gait, “The Optical File Cabinet: A Random-Access File system for Write-Once Optical Disks,” IEEE Computer, vol. 21, No. 6, pp. 11-22 (1988). |
| Gray (#2 of 2, pp. 604-609), Jim; Reuter Andreas, Transaction Processing Concepts and Techniques, Morgan Kaufmann Publisher, USA 1994, pp. 604-609. |
| Guo et al., Building a High-performance Deduplication System, Jun. 15, 2011, retrieved from the Internet at <U RL: http://dl.acm.org/citation.cfm?id=2002206>, pp. 1-14. |
| Huff, KL, “Data Set Usage Sequence Number,” IBM Technical Disclosure Bulletin, vol. 24, No. 5, Oct. 1981 New York, US, pp. 2404-2406. |
| Jander, “Launching Storage-Area Net,” Data Communications, US, McGraw Hill, NY, vol. 27, No. 4(Mar. 21, 1998), pp. 64-72. |
| Kashyap, et al., “Professional Services Automation: A knowledge Management approach using LSI and Domain specific Ontologies”, FLAIRS-01 Proceedings, 2001, pp. 300-302. |
| Kornblum, Jesse, “Identifying Almost Identical Flies Using Context Triggered Piecewise Hashing,” www.sciencedirect.com, Digital Investigation 3S (2006), pp. S91-S97. |
| Lortu Software Development, “Kondar Technology-Deduplication,” http://www.lortu.com/en/deduplication.asp, Internet accessed on Dec. 5, 2008, 3 pages. |
| Overland Storage, “Data Deduplication,” http://www.overlandstorage.com/topics/data_deduplication.html, Internet accessed on Dec. 5, 2008, 2 pages. |
| Quantum Corporation, “Data De-Duplication Background: A Technical White Paper,” May 2008, 13 pages. |
| Rosenblum et al., “The Design and Implementation of a Log-Structure File System,” Operating Systems Review SIGOPS, vol. 25, No. 5, New York, US, pp. 1-15 (May 1991). |
| Wei, et al. “MAD2: A scalable high-throughput exact deduplication approach for network backup services.” Mass Storage Systems and Technologies (MSST), 2010 IEEE 26th Symposium on. IEEE, 2010, 14 pages. |
| Wolman et al., On the scale and performance of cooperative Web proxy caching, 1999. |
| Wu, et al., Load Balancing and Hot Spot Relief for Hash Routing among a Collection of Proxy Caches, 1999. |
| Final Office Action for Japanese Application No. 2003531581, Examiner Kudoh, dated Mar. 24, 2009, 6 pages. |
| International Search Report and Written Opinion, International Application No. PCT/US2009/58137, dated Dec. 23, 2009, 14 pages. |
| International Search Report and Written Opinion, International Application No. PCT/US2011/030804, dated Jun. 9, 2011. |
| International Search Report and Written Opinion, International Application No. PCT/US2011/030814, dated Jun. 9, 2011. |
| International Search Report and Written Opinion, International Application No, PCT/US2013/045443 dated Nov. 14, 2013, 16 pages. |
| International Preliminary Report on Patentability, International Application No. PCT/US2013/045443 dated Dec. 16, 2014 11 pages. |
| Number | Date | Country | |
|---|---|---|---|
| 20190012237 A1 | Jan 2019 | US |
| Number | Date | Country | |
|---|---|---|---|
| 61165071 | Mar 2009 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 15198269 | Jun 2016 | US |
| Child | 15991849 | US | |
| Parent | 13931654 | Jun 2013 | US |
| Child | 15198269 | US | |
| Parent | 12724292 | Mar 2010 | US |
| Child | 13931654 | US |
