Computer systems contain large amounts of data. This data includes personal data, such as financial data, customer/client/patient contact data, audio/visual data, and much more. This data also includes information related to the correct operation of the computer system, such as operating system files, application files, user settings, and so on. With the increased reliance on computer systems to store critical information, the importance of protecting data has grown. Traditional data protection systems, such as backup systems, receive an identification of a file location to protect, then create one or more secondary copies containing the contents of the protected up location. These secondary copies can then later be used to restore the original data should anything happen to the original data.
In corporate environments, protecting data is generally part of a routine process that is performed for many computer systems within an organization. For example, a company might back up critical computing systems related to e-commerce such as databases, file servers, web servers, and so on. The company may also back up computing systems used by each of its employees, such as those used by an accounting department, marketing department, engineering, and so forth.
Because of the amount of data in an organization, secondary copies of data for an organization's computing systems are often very large and can require the purchase of expensive storage devices and storage media. The restoration of data in the event of data loss is also slowed by the large size of the secondary copy. As the amount of protected data increases, locating and restoring data requires more actions to be taken. For example, it may be necessary to search many backup tapes to find the correct data. The quantity of secondary copy media, such as tapes, may mean that some secondary copy media has been moved offsite such that it must first be retrieved before data can be recovered from it. Each of these factors increases the cost of protecting data and the time required to recover data in the event of data loss. Quick recovery of data is often critical to today's businesses, and any additional delay could affect business operations and customers' satisfaction with the business.
Management of data in this way also consumes resources, and it is often desirable to minimize impact to computing systems to provide maximum availability for handling customer or employee requests. Some organizations defer activities such as performing backups until off hours, such as early in the morning to reduce the impact to the availability of systems. However, recovery of a file may be needed during business hours or at other inconvenient times, and choosing the best way to retrieve the data is important for maintaining the availability of the system.
There is a need for a system that overcomes the above problems, as well as one that provides additional benefits.
In the drawings, the same reference numbers and acronyms identify elements or acts with the same or similar functionality for ease of understanding and convenience. To easily identify the discussion of any particular element or act, the most significant digit or digits in a reference number refer to the Figure number in which that element is first introduced (e.g., element 110 is first introduced and discussed with respect to
The headings provided herein are for convenience only and do not necessarily affect the scope or meaning of the claimed invention.
A data management system often contains a primary or production copy of data, and one or more secondary copies created from the primary copy of the data. For example, a web server may contain the primary copy of the data. A snapshot, change journal, replication, migration, backup, or other storage operation may be performed to create the one or more secondary copies of the data. For example, a snapshot operation may create an image of the current state of the primary copy, and the data management system may transfer the snapshot to another system for storage as a secondary copy. The secondary copy may later be encrypted or otherwise processed to create additional secondary copies. For example, an organization may initially backup data to a first secondary copy and later encrypt the data to a second secondary copy before the data is stored in a third-party offsite storage facility.
A method and system for managing copies of data is provided, referred to as the data management system. The data management system creates and manages copies of data. For example, the data management system may create a secondary copy such as a backup of the data stored on each of the computer systems within an organization. The secondary copy represents a second version of the original source version of the data. The data management system may make additional secondary copies of the data, called auxiliary copies. For example, the data management system may create secondary copies to perform additional operations on the data (e.g., encryption and protection) and to store the data in a new location (e.g., in a tape library or at an off-site data vault). The data management system may also store information about the data managed by the system within a data structure such as a database, also called a metabase. The metabase may contain information such as when the data management system performed the last data copy, where each of the data copies is stored, what files are contained within each data copy, and so on. When the data management system receives a request to access data, there may be many copies of the data available, and the data management system selects a desirable copy of the data to satisfy the access request as described in further detail below. The data management system may first consult the metabase to determine which copies contain the requested data without accessing the computer systems where the data is actually stored. In this way, the data management system provides high availability of data while reducing the impact of access requests on critical computer systems.
The invention will now be described with respect to various embodiments. The following description provides specific details for a thorough understanding of, and enabling description for, these embodiments of the invention. However, one skilled in the art will understand that the invention may be practiced without these details. In other instances, well-known structures and functions have not been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments of the invention.
The terminology used in the description presented below is intended to be interpreted in its broadest reasonable manner, even though it is being used in conjunction with a detailed description of certain specific embodiments of the invention. Certain terms may even be emphasized below; however, any terminology intended to be interpreted in any restricted manner will be overtly and specifically defined as such in this Detailed Description section.
Unless described otherwise below, aspects of the invention may be practiced with conventional systems. Thus, the construction and operation of the various blocks shown in
Aspects of the invention can be embodied in a special purpose computer or data processor that is specifically programmed, configured, or constructed to perform one or more of the computer-executable instructions explained in detail herein. Aspects of the invention can also be practiced in distributed computing environments where tasks or modules are performed by remote processing devices, which are linked through a communications network, such as a Local Area Network (LAN), Wide Area Network (WAN), Storage Area Network (SAN), Fibre Channel, or the Internet. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.
Aspects of the invention may be stored or distributed on computer-readable media, including magnetically or optically readable computer discs, hard-wired or preprogrammed chips (e.g., EEPROM semiconductor chips), nanotechnology memory, biological memory, or other data storage media. Indeed, computer implemented instructions, data structures, screen displays, and other data under aspects of the invention may be distributed over the Internet or over other networks (including wireless networks), on a propagated signal on a propagation medium (e.g., an electromagnetic wave(s), a sound wave, etc.) over a period of time, or they may be provided on any analog or digital network (packet switched, circuit switched, or other scheme). Those skilled in the relevant art will recognize that portions of the invention reside on a server computer, while corresponding portions reside on a client computer such as a mobile or portable device, and thus, while certain hardware platforms are described herein, aspects of the invention are equally applicable to nodes on a network.
The data management system may create the primary copy of the data stored on a computer system in a variety of ways. In some embodiments, snapshot software installed on each computer system is used to capture a point-in-time view of the data on a computer system. The data management system may copy the snapshot data to another computer system. Snapshots may contain a snapshot of all of the data on a system, or they may be incremental and provide information about the data that has changed since a previous snapshot was taken. In some embodiments, change journaling software (often built into the operating system) is used to detect changes to data and to update the primary copy. In some embodiments, an agent is installed for each type of data stored on a computer system. The agent may be designed to copy a specific type of data (e.g., database data or data produced by a specific application) more efficiently than a simple copy of the underlying data files. Once a primary copy has been created, additional operations can be performed on the data without accessing the original computer system from which the data was copied.
The data management system may copy data from a computer system as files or the data management system may process the contents of the files as application data. As described above, an agent may be installed that interprets data produced by a particular application. For example, data stored by an email server (e.g., Microsoft Exchange Server), may be stored in a series of files. Creating a copy of the files is often not as useful as interpreting the data itself and storing the data in its interpreted form. For example, the data from an email server may be stored in one large file with a non-descriptive name, such as “mailbox.dat,” but it is more useful when searching for data to understand that the file contains mailboxes belonging to specific users, and that each of those mailboxes contains email messages having descriptive information such as a subject, TO header, and so on. Therefore, rather than simply storing the file, the data management system may parse, recognize, and store information about each user's mailbox and the email messages contained within it. Similar processes may be used to store data created by database applications, word processors, accounting software, and so on.
In some embodiments, the data management system performs additional operations on secondary copies of data. For example, an organization may create an unencrypted first secondary copy, but perform encryption on an auxiliary secondary copy to allow the organization to safely store the auxiliary secondary copy with a third-party data storage provider without worrying that the third party may view confidential information contained within the data. The data management system may also examine the first secondary copy and eliminate duplicate data objects within the first secondary copy to create a single instance of each data object. This process is generally referred to as single instancing. For example, when copying data from multiple computer systems within an organization, each computer system will often contain similar files for the operating system and configuration data. Storing multiple copies of the same data wastes storage space, and increases the time and cost associated with data management, so single instancing can provide substantial cost savings to an organization.
In some embodiments, the data management system creates an index of the content contained within each copy. The index information may be stored within the metabase. The index may contain information such as each of the locations where the data is located, keywords contained within the data, classifications assigned to the data, and user access information describing which users are permitted to view the contents of the data. The content index may be used to facilitate search and retrieval of the data, such as in response to a user request to restore a particular file.
In some embodiments, the data management system performs the operations described above based on a storage policy. For example, operations such as the creation of a primary copy of data may be performed automatically on a schedule. The storage policy defines the types of operations to be performed, and the conditions that trigger their performance. For example, a storage policy may specify the creation of a secondary copy of data onto tape when the primary copy of the data reaches a certain size or is a certain number of days old. The storage policy may also define which computer systems are involved in a particular data management operation, such as a particular pool of servers to use for performing encryption of data.
The data management system periodically receives requests to retrieve data from users and from the system itself. For example, a user may search for a specific file or document, and the data management system finds matching data. The system may request a copy of the data on which to perform a data management operation. For example, if a storage policy specifies that an encrypted copy of data should be created, the data management system may query the metabase to locate an available copy of the data to encrypt, so that the original source of the data is not impacted by the encryption operation.
In some embodiments, the data management system determines availability based on the environment of the entity requesting the data. For example, if a user of a computer system within an organization is requesting data from another computer system within the organization, then the data management system may factor in the subnet of the two computer systems relative to each other. If a copy is available on the same subnet as the requesting user, then that copy may be preferred over a copy on a different subnet. As another example, the data management system may hierarchically group computer systems within an organization into storage cells that share certain storage characteristics (e.g., network location or storage policies), and a copy may be preferred based on the storage cell that the computer system storing it is associated with. Alternatively or additionally, the purpose of the retrieval request may influence the copy used to retrieve the data. For example, a legal discovery request may specify certain documents that must be retrieved within a long timeframe (e.g., a year), such that the most available copy is not required and a slower copy can be used so that other operations can use the most available copy.
In some embodiments, the data management system determines the age of the data available in each copy. For example, the time a copy was created may be stored with the copy, or the copy may contain other information indicating the time that the data was last modified. For some search requests, an older copy of the data available on tape media may be acceptable, whereas for other search requests only a very recent copy will satisfy the request. If the user is searching for a particular file, the data management system may determine that although a particular copy is older and out of date with respect to some files, the file requested by the user has not changed since the copy was made and the copy can adequately satisfy the user's request. Using this type of analysis, the data management system can select the most appropriate copy to use for many different circumstances.
In some embodiments, the data management system restricts the data accessible to a user based on the user's identity and authentication information stored within the system. Each data object may contain authentication information stored within the metabase that specifies the accessibility of the data object for each user, or the data management system may leverage an existing authentication system, such as Microsoft Windows Active Directory. For example, a person on the engineering team of an organization may be unable to view data objects copied from the CEO's computer system, even though the data objects match the engineer's search criteria. Some copies may also be inaccessible to certain users. For example, some employees within an organization may not be permitted to decrypt encrypted data copies, such that those copies are unavailable for satisfying a data retrieval request from the employee.
In some embodiments, the data management system provides a user interface for identifying and retrieving data. For example, the data management system may host a web server that provides access to the metabase, such that a user can submit a search request and find matching data categorized by the metabase. Then, the user can access the web server using a web browser. This allows the user to access data regardless of the operating system of the computer system on which the data was originally stored and regardless of the operating system of the computer system that the user is submitting the request from. In some embodiments, the data management system converts the data within search results returned in response to a user search into a format that is viewable within the web browser. For example, if a search returns a document that is normally opened by a particular word processing program, the data management system may provide the ability to convert and view the document as Hypertext Markup Language (HTML) within the requesting user's web browser. In this way, a user can search for documents created by many types of applications and on many platforms without installing additional software.
Unless the context clearly requires otherwise, throughout the description and the claims, the words “comprise,” “comprising,” and the like are to be construed in an inclusive sense, as opposed to an exclusive or exhaustive sense; that is to say, in the sense of “including, but not limited to.” The word “coupled,” as generally used herein, refers to two or more elements that may be either directly connected, or connected by way of one or more intermediate elements. Additionally, the words “herein,” “above,” “below,” and words of similar import, when used in this application, shall refer to this application as a whole and not to any particular portions of this application. Where the context permits, words in the above Detailed Description using the singular or plural number may also include the plural or singular number respectively. The word “or” in reference to a list of two or more items, that word covers all of the following interpretations of the word: any of the items in the list, all of the items in the list, and any combination of the items in the list.
The above detailed description of embodiments of the invention is not intended to be exhaustive or to limit the invention to the precise form disclosed above. While specific embodiments of, and examples for, the invention are described above for illustrative purposes, various equivalent modifications are possible within the scope of the invention, as those skilled in the relevant art will recognize. For example, while processes or blocks are presented in a given order, alternative embodiments may perform routines having steps, or employ systems having blocks, in a different order, and some processes or blocks may be deleted, moved, added, subdivided, combined, and/or modified. Each of these processes or blocks may be implemented in a variety of different ways. In addition, while processes or blocks are at times shown as being performed in series, these processes or blocks may instead be performed in parallel, or may be performed at different times.
The teachings of the invention provided herein can be applied to other systems, not necessarily the system described above. The elements and acts of the various embodiments described above can be combined to provide further embodiments.
These and other changes can be made to the invention in light of the above Detailed Description. While the above description details certain embodiments of the invention and describes the best mode contemplated, no matter how detailed the above appears in text, the invention can be practiced in many ways. Details of the system may vary considerably in implementation details, while still being encompassed by the invention disclosed herein. As noted above, particular terminology used when describing certain features or aspects of the invention should not be taken to imply that the terminology is being redefined herein to be restricted to any specific characteristics, features, or aspects of the invention with which that terminology is associated. In general, the terms used in the following claims should not be construed to limit the invention to the specific embodiments disclosed in the specification, unless the above Detailed Description section explicitly defines such terms. Accordingly, the actual scope of the invention encompasses not only the disclosed embodiments, but also all equivalent ways of practicing or implementing the invention under the claims.
While certain aspects of the invention are presented below in certain claim forms, the inventors contemplate the various aspects of the invention in any number of claim forms. For example, while only one aspect of the invention is recited as embodied in a computer-readable medium, other aspects may likewise be embodied in a computer-readable medium. Accordingly, the inventors reserve the right to add additional claims after filing the application to pursue such additional claim forms for other aspects of the invention.
The present application is a continuation of U.S. application Ser. No. 11/694,890 entitled “MANAGING COPIES OF DATA” and filed on Mar. 30, 2007, now U.S. Pat. No. 7,734,669, which claims priority to U.S. Provisional Application No. 60/871,735 entitled “METHOD AND SYSTEM FOR SEARCHING STORED DATA” and filed on Dec. 22, 2006, each of which are hereby incorporated by reference.
Number | Name | Date | Kind |
---|---|---|---|
4686620 | Ng | Aug 1987 | A |
4995035 | Cole et al. | Feb 1991 | A |
5005122 | Griffin et al. | Apr 1991 | A |
5093912 | Dong et al. | Mar 1992 | A |
5133065 | Cheffetz et al. | Jul 1992 | A |
5193154 | Kitajima et al. | Mar 1993 | A |
5212772 | Masters | May 1993 | A |
5226157 | Nakano et al. | Jul 1993 | A |
5239647 | Anglin et al. | Aug 1993 | A |
5241668 | Eastridge et al. | Aug 1993 | A |
5241670 | Eastridge et al. | Aug 1993 | A |
5276860 | Fortier et al. | Jan 1994 | A |
5276867 | Kenley et al. | Jan 1994 | A |
5287500 | Stoppani, Jr. | Feb 1994 | A |
5321816 | Rogan et al. | Jun 1994 | A |
5333315 | Saether et al. | Jul 1994 | A |
5347653 | Flynn et al. | Sep 1994 | A |
5410700 | Fecteau et al. | Apr 1995 | A |
5448724 | Hayashi et al. | Sep 1995 | A |
5491810 | Allen | Feb 1996 | A |
5495607 | Pisello et al. | Feb 1996 | A |
5504873 | Martin et al. | Apr 1996 | A |
5537568 | Yanai et al. | Jul 1996 | A |
5544345 | Carpenter et al. | Aug 1996 | A |
5544347 | Yanai et al. | Aug 1996 | A |
5559957 | Balk | Sep 1996 | A |
5619644 | Crockett et al. | Apr 1997 | A |
5638509 | Dunphy et al. | Jun 1997 | A |
5673381 | Huai et al. | Sep 1997 | A |
5699361 | Ding et al. | Dec 1997 | A |
5729743 | Squibb | Mar 1998 | A |
5751997 | Kullick et al. | May 1998 | A |
5758359 | Saxon | May 1998 | A |
5761677 | Senator et al. | Jun 1998 | A |
5764972 | Crouse et al. | Jun 1998 | A |
5778395 | Whiting et al. | Jul 1998 | A |
5812398 | Nielsen | Sep 1998 | A |
5813009 | Johnson et al. | Sep 1998 | A |
5813017 | Morris | Sep 1998 | A |
5875478 | Blumenau | Feb 1999 | A |
5887134 | Ebrahim | Mar 1999 | A |
5901327 | Ofek | May 1999 | A |
5924102 | Perks | Jul 1999 | A |
5950205 | Aviani, Jr. | Sep 1999 | A |
5974563 | Beeler, Jr. | Oct 1999 | A |
6021415 | Cannon et al. | Feb 2000 | A |
6026414 | Anglin | Feb 2000 | A |
6052735 | Ulrich et al. | Apr 2000 | A |
6076148 | Kedem et al. | Jun 2000 | A |
6094416 | Ying | Jul 2000 | A |
6131095 | Low et al. | Oct 2000 | A |
6131190 | Sidwell | Oct 2000 | A |
6148412 | Cannon et al. | Nov 2000 | A |
6154787 | Urevig et al. | Nov 2000 | A |
6161111 | Mutalik et al. | Dec 2000 | A |
6167402 | Yeager | Dec 2000 | A |
6212512 | Barney et al. | Apr 2001 | B1 |
6260069 | Anglin | Jul 2001 | B1 |
6269431 | Dunham | Jul 2001 | B1 |
6275953 | Vahalia et al. | Aug 2001 | B1 |
6301592 | Aoyama et al. | Oct 2001 | B1 |
6324581 | Xu et al. | Nov 2001 | B1 |
6328766 | Long | Dec 2001 | B1 |
6330570 | Crighton et al. | Dec 2001 | B1 |
6330642 | Carteau | Dec 2001 | B1 |
6343324 | Hubis et al. | Jan 2002 | B1 |
RE37601 | Eastridge et al. | Mar 2002 | E |
6356801 | Goodman et al. | Mar 2002 | B1 |
6389432 | Pothapragada et al. | May 2002 | B1 |
6421711 | Blumenau et al. | Jul 2002 | B1 |
6487561 | Ofek et al. | Nov 2002 | B1 |
6519679 | Devireddy et al. | Feb 2003 | B2 |
6538669 | Lagueux, Jr. et al. | Mar 2003 | B1 |
6564228 | O'Connor | May 2003 | B1 |
6658526 | Nguyen et al. | Dec 2003 | B2 |
6996616 | Leighton et al. | Feb 2006 | B1 |
7240100 | Wein et al. | Jul 2007 | B1 |
7346676 | Swildens et al. | Mar 2008 | B1 |
7606844 | Kottomtharayil | Oct 2009 | B2 |
20070043715 | Kaushik et al. | Feb 2007 | A1 |
Number | Date | Country |
---|---|---|
0259912 | Mar 1988 | EP |
0405926 | Jan 1991 | EP |
0467546 | Jan 1992 | EP |
405926 | Apr 1996 | EP |
0774715 | May 1997 | EP |
0809184 | Nov 1997 | EP |
0899662 | Mar 1999 | EP |
0981090 | Feb 2000 | EP |
WO-9513580 | May 1995 | WO |
WO-9912098 | Mar 1999 | WO |
Number | Date | Country | |
---|---|---|---|
20080243939 A1 | Oct 2008 | US |
Number | Date | Country | |
---|---|---|---|
60871735 | Dec 2006 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 11694890 | Mar 2007 | US |
Child | 12060055 | US |