Accelerated data transfer using common prior data segments

Information

  • Patent Grant
  • 9317506
  • Patent Number
    9,317,506
  • Date Filed
    Friday, September 22, 2006
    20 years ago
  • Date Issued
    Tuesday, April 19, 2016
    10 years ago
Abstract
Accelerating data transfers is described herein. When a second computing system is requested to transfer a file to a first computing system, a data segment is sent to the first computing system instead of the entire file. The data segment is then compared to data stored within a data store on the first computing system. If the data segment and data within the data store match, then the file does not need to be transferred, and a pointer points to the file already located on the first computing system. If the data segment does not match any data stored in the data store, then the file is transferred from the second computing system to the first computing system. By comparing only the data segment instead of sending an entire file, data transfer is able to be greatly expedited in situations where the data is common between systems.
Description
FIELD OF THE INVENTION

The present invention relates to the field of data transfers. More specifically, the present invention relates to the field of transferring data using common prior data segments.


BACKGROUND OF THE INVENTION

As networking technologies grow, including the Internet, so do their capabilities and requirements. For many years, users dialed up to access the Internet at speeds of 14.4 kilobits per second (kps), then 28.8 kps and 56.6 kps. Then ISDN lines made 128 kps a possibility. Currently cable modems and DSL provide extremely fast connections with high bandwidth to home users. Other technologies such as T1 and T3 lines provide possibly even faster connections and are usually implemented by businesses and universities. As these technologies are increasing in capabilities, so too is the size and amount of the data traveling from one networked device to another. For example, when 14.4 kps connections were prominent, a file of a few hundred kilobytes was considered quite large and took a while to download. With current broadband technologies utilizing cable modems and DSL, a file of multiple megabytes is able to be downloaded in a few minutes. Hence, technology has improved substantially, enabling larger files to be downloaded in a short amount of time. However, many data files are currently in the range of gigabytes such as movie files which could take hours to download even on fast connections and would take days with older dial-up connections. Since the Internet and other networks are being used to couple everything together lately, even toasters and refrigerators, many attempts have been made to make network connections more efficient utilizing data processing techniques.


One technique is to compress the data before sending it over the network. However, that has its drawbacks of adding steps of compressing the data and before it is sent and uncompressing the data after it is received, simply adding time to the process in a different way. Furthermore, since many files like .mp3s are already compressed yet still quite large, compressing them again will do little if anything to improve network speed.


Another technique is described in U.S. Patent App. No. 2004/0148306 to Moulton, et al. Moulton describes a hash file system that is based and organized upon hashes and which is able to eliminate redundant copies of aggregate blocks of data or parts of data blocks from the system. The hash file system taught by Moulton utilizes hash values for computer files or file pieces which are produced by a checksum generating program, engine or algorithm. The hash file system as taught by Moulton is able to be used as a network accelerator by sending hashes for the data instead of the data itself.


SUMMARY OF THE INVENTION

Accelerating data transfers is described herein. When a second computing system is requested to transfer a file to a first computing system, a data segment is sent to the first computing system instead of the entire file. The data segment is then compared to data stored within a data store on the first computing system. If the data segment and data within the data store match, then the file does not need to be transferred, and a pointer points to the file already located on the first computing system. If the data segment does not match any data stored in the data store, then the file is transferred from the second computing system to the first computing system. By comparing only the data segment instead of sending an entire file, data transfer is able to be greatly expedited in situations where the data is common between systems.


In one aspect, a method of accelerating data transfer comprises storing data in a data store on a first computing system wherein the data corresponds to one or more files stored on the first computing system, transferring a data segment from a source file from a second computing system to the first computing system over a network, scanning the data store for the data segment, generating one or more pointers to the one or more corresponding files of one or more matching data segments, if the one or more matching data segments are identified in the data store and transferring a copy of the source file, if the one or more matching data segments are not identified in the data store. The data store is a database. The first computing system is a target system and the second computing system is a source system. The first computing system is a server and the second computing system is a client system. The client system is selected from the group consisting of a personal computer, a PDA, a cell phone, a laptop, a thin client, a Mac computer, an mp3 player and a gaming console. Alternatively, the first computing system is a first client system and the second computing system is a second client system. The data segment is one or more cyclic redundancy checks and the data in the data store includes cyclic redundancy checks and the data segment and the data are compared. Alternatively, the data segment is a unique database key and the data in the data store includes database keys and the data segment and the data are compared. Alternatively, the data segment is a hash and the data in the data store includes hashes and the data segment and the data are compared. The data store grows as more files are stored on the first computing system. The files stored on the first computing system are minimized by implementing the data store. One or more additional computing systems are coupled to the first computing system. The method further comprises transferring only a first section of the source file when only a second section of the source file is found within the data store. A standard operating system and file system are utilized on the first computing system and the second computing system.


In another aspect, a system for accelerating data transfer comprises a first computing system for storing one or more files and a data store for storing data corresponding to the one or more files and a second computing system coupled to the first computing system, wherein a data segment is compared to the data within the data store on the first computing system after being received from the second computing system, further wherein a pointer to the one or more files is added on the first computing system if the data segment is found within the data store, but a copy of a source file corresponding to the data segment is transferred from the second computing system to the first computing system if the data segment is not found in the data store. The data store is a database. The first computing system is a target system and the second computing system is a source system. The first computing system is a server and the second computing system is a client system. The client system is selected from the group consisting of a personal computer, a PDA, a cell phone, a laptop, a thin client, a Mac computer, an mp3 player and a gaming console. Alternatively, the first computing system is a first client system and the second computing system is a second client system. The data segment is one or more cyclic redundancy checks and the data in the data store includes cyclic redundancy checks and the data segment and the data are compared. Alternatively, the data segment is a unique database key and the data in the data store includes database keys and the data segment and the data are compared. Alternatively, the data segment is a hash and the data in the data store includes hashes and the data segment and the data are compared. The data store grows as more files are stored on the first computing system. The files stored on the first computing system are minimized by implementing the data store. The system further comprises one or more additional computing systems coupled to the first computing system. Only a first section of the source file is transferred when only a second section of the source file is found within the data store. A standard operating system and file system are utilized on the first computing system and the second computing system. The system further comprises a network coupling the first computing system and the second computing system.


In another aspect, a network of systems for accelerating data transfers comprises one or more source systems for transferring a data segment corresponding to a source file stored on the one or more source systems, one or more target systems coupled to the one or more source systems for storing data in a data store corresponding to one or more files and for comparing the data segment received from the one or more source systems with the data in the data store where if the data segment is found, a pointer is generated to point to a corresponding file in the one or more files on the target system instead of transferring the source file over a network. The data store is a database. The one or more target systems are one or more servers and the one or more source systems are one or more client systems. The one or more client systems are selected from the group consisting of personal computers, PDAs, cell phones, laptops, thin clients, Mac computers, mp3 players and gaining consoles. The data segment is one or more cyclic redundancy checks and the data in the data store includes cyclic redundancy checks and the data segment and the data are compared. Alternatively, the data segment is a unique database key and the data in the data store includes database keys and the data segment and the data are compared. Alternatively, the data segment is a hash and the data in the data store includes hashes and the data segment and the data are compared. The data store grows as more files are stored on the one or more target systems. The files stored on the one or more target systems are minimized by implementing the data store. Only a first section of the source file is transferred when only a second section of the source file is found within the data store. A standard operating system and file system are utilized on the one or more target systems and the one or more source systems.


In yet another aspect, a storage system configured to receive data from a plurality of computing systems comprises one or more files, a set of information corresponding to the one or more files and a data store for storing the set of information, wherein a data segment received from a source system is compared with the set of information stored within the data store and a pointer is generated to point to a corresponding file in the one or more files if the data segment is found but if the data segment is not found within the data store, a copy of a source file corresponding to the data segment is transferred. The data store is a database. The data segment is one or more cyclic redundancy checks and the set of information in the data store includes cyclic redundancy checks and the data segment and the set of information are compared. Alternatively, the data segment is a unique database key and the set of information in the data store includes database keys and the data segment and the set of information are compared. Alternatively, the data segment is a hash and the set of information in the data store includes hashes and the data segment and the set of information are compared. The data store grows as more files are stored on the storage system. The files stored on the storage system are minimized by implementing the data store. Only a first section of the source file is transferred when only a second section of the source file is found within the data store. A standard operating system and file system are utilized on the storage system.





BRIEF DESCRIPTION OF THE DRAWINGS


FIG. 1 illustrates a graphical representation of a configuration of an embodiment of the present invention.



FIG. 2 illustrates an exemplary graphical representation of files and directories stored in a target system in an embodiment of the present invention.



FIG. 3 illustrates a graphical representation of a network of systems configured in the present invention.



FIG. 4 illustrates a flowchart of an embodiment of the present invention.





DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

A system for and method of accelerating data transfers over a network is described herein. In the past, data was transferred with a minimal check to determine if the data is already located on the destination system. Essentially, a check was made whether a file with the same file name is located in the location of the desired destination. For example, if a user was copying movie.avi from his personal computer to a folder entitled “comedy” on a backup server used for people to store their movies, the server only checks if movie.avi exists in the “comedy” folder. However, there are a number of problems with this. The main one is that the file movie.avi could already be located on the server just in a different folder. It would be a waste of network resources to copy the entire movie.avi file, considering a typical movie file is a few hundred megabytes or possibly gigabytes. Using the present invention, only a data segment is sent from the user's computer to the server, and then the server searches its system and locates the preexisting movie.avi and simply generates a pointer to it. Thus, only a very small amount of data is sent over the network instead of a huge movie file, and each file is only stored a single time on the storage system.



FIG. 1 illustrates a graphical representation of a configuration of an embodiment of the present invention. A source computing system 100 is coupled to a target computing system 120 through a network 110. Both the source computing system 100 and the target computing system 120 are able to be any computing system with the ability to transfer data to another system. Such computing systems include but are not limited to, personal computers, laptops, servers, thin clients, cell phones, PDAs, Mac computers, mp3 players and gaming consoles. The network 110 is able to be a Local Area Network (LAN), Wide Area Network (WAN), Metropolitan Area Network (MAN), the Internet, or any other type of network. Although the configuration in FIG. 1 shows the two systems coupled through the network 110, it is possible for the source computing system 100 and the target computing system 120 to be directly coupled to each other. Within the source computing system 100 are standard computing elements including a hard drive 102 where files 104 are stored on a source file system 106. In some embodiments, the hard drive 102 is not a standard hard disk drive but another type of storage system including, but not limited to, a compact disc, a DVD, an optical drive, a network drive or a Redundant Array of Inexpensive Disks (RAID). When a user desires to transfer a file, a source file 104′ is selected by the user. For example, a file named, movie.avi, is selected to be transferred. However, unlike past implementations of transferring data, the process does not begin by transferring the entire source file 104′.


After the source file 104′ is selected to be transferred, a data segment 112 of the source file 104′ is sent across the network 110. In an embodiment, the data segment 112 is a section of the source file 104′. Using the movie.avi example, only a section of the file is sent over the network. In other embodiments, the data segment 112 is a different representation of the data such as a hash or a sliding Cyclic Redundancy Check (CRC) of the source file 104′. In other embodiments, other similar implementations are used where a representation of the source file 104′ is sent over the network 110 instead of the entire file. Additionally, representations of parts of the source file 104′ are able to be sent.


The target computing system 120 similarly has standard computing components including a hard drive 122. In some embodiments, the hard drive 122 is not a standard hard disk drive but another type of storage system as described above. Within the hard drive 122 is a standard operating system such as Microsoft® Windows XP and a standard file system 128 such as New Technology File System (NTFS) where one or more files 124 are stored. In alternate embodiments, the file system is a non-standard file system. The file system 128 also contains a data store 126. The file system 128 utilizes typical structures such as directories or folders to store the files 124. The data store 126 is an implementation that is able to store data 126′ in an organized manner so that it is searchable. In some embodiments, the data store 126 is a database. The data 126′ stored within the data store 126 corresponds to the files 124 stored in the file system 128. For example, since a movie.avi file 124′ is stored within the hard drive 122, the data store 126 contains data 126′ corresponding to movie.avi. The data 126′ within the data store 126 depends on the embodiment implemented wherein some embodiments store segments of files, hashes, CRCs, unique database keys and/or other similar implementations of data representation.


The data segment 112 sent from the source computing system 100 is received by the target computing system 120. The target computing system 120 then searches within the data store 126 for a matching data segment. Continuing with the movie.avi example, a matching section of the movie.avi file is searched for within the data store 126. Since the data store 126 contains the movie.avi data 126′, a match is found. Hence, the system knows that the movie.avi file already exists on the target computing system 120. The target computing system 120, then sends a status 114 or some form of response to the source computing system 100 indicating that the file is already located at the target computing system 120. In the situation where the source file 104′ is already located at the target computing system 120, the source computing system 100 does not need to send any more data, and the target computing system 120 adds a pointer or indicates in some way where the data is located, so that the user copying the data is able to retrieve it later on. If the source file 104′ is not located on the target computing system 120, then the status 114 sent back indicates as such. At that point, a copy 116 of the source file 104′ is sent from the source computing system 100 to the target computing system 120. Once the new file is received on the target computing system 120, it is stored with the rest of the files 124 and a representation is stored within the data store 126, so that in the future when a user wants to copy that same file, the target computing system 120 will know that it is there and is able to expedite the data transfer by not having to actually transfer the entire file.



FIG. 2 illustrates an exemplary graphical representation of files and directories stored in a target system in an embodiment of the present invention. Within the example, two users' directories are shown, Brian and Paul. Within each user's storage area, there are four directories: documents, pictures, music and movies, each for storing files related to their respective category. Files that include just the file name within FIG. 2 signify that they are the only file containing that data on the system. For example, Brian's Documents directory contains resume.doc and report.doc which, as expected, are specific to his personal information, so there are no copies of that information found elsewhere on the system. This also means that when Brian transferred these files to the target computing system from his source computing system, the entire files were copied. However, there are types of data where duplicates are commonly found such as music and movies. These common files are the ones that are able to improve network data transfers by not actually copying the entire file and instead linking to the appropriate file already located on the target computing system. Files with a box around them with an arrow pointing outward such as DMB1.mp3, DMB2.mp3 and DMB3.mp3 within Brian's Music directory indicate that those files are actually pointers or links to another file on the system. Here, in Paul's Music directory, he also has DMB1.mp3, DMB2.mp3 and DMB3.mp3 amongst other music files. Paul copied his files before Brian, so his copying included transferring all of the file contents over the network. However, when Brian initiated his transfer, the system found Paul's copies using the methods described herein and instead of transferring the entire files, generated a pointer to Paul's files which are denoted by a box with an arrow pointing inward since they are being linked to. Furthermore, from the user's perspective, the files appear the same, even though there are no actual files with music data stored within Brian's Music directory on the system. The process continues as the users transfer files to the system, and when a file is copied determines whose directory includes the actual data and whose directory includes a pointer to data elsewhere on the system. As shown in FIG. 2, Brian copied Spider-Man.avi and then some time later, Paul did as well. Since Brian made his transfer first, the actual data is stored in his directory.


In some embodiments, the data is not stored in a user's directory, but is stored centrally so that everyone has pointers to the data. This alleviates the issue of one user deleting the file while the other user still wants it to remain. For example if Paul deletes Crash.avi, since the actual movie content is stored in his directory, Brian's pointer would point to nothing if the file is removed from Paul's directory. Using a central storage system where each user points to the central storage, the actual data would not be deleted, just Paul's link to the data, and Brian's link would remain intact. Another embodiment still stores the files in the individual locations, but also keeps track of whom is pointing to the files as well. Therefore, if the user with the actual content deletes it, the file is transferred to another user whose link is pointing to the data. The pointers pointing to the file are reconfigured to point to the data's new location. By transferring the data to another user before the actual data is deleted, this safeguards that the actual data is not lost when other users still want the file.


The above example is not meant to limit the present invention in any way. Although only two users are described, any number of users are able to store data on a system. Furthermore, the number of directories and the directory names are variable as well. The file types are not restricted to those described in the example either; any file types are able to be used. Also, when the files are linked, the filenames do not have to be the same. Comparisons performed by the methods described herein focus on the content of the data not the filenames. Hence, if a filename is Spider-Man.avi on a target and the source filename is Spiderman.avi, but they have the same content, the system is able to recognize they are the same file. The converse is true as well, that just because two files have the same filename, does not mean they have the same content, so links will not incorrectly point to the wrong data as they will not have the same content.


By implementing the present invention, not only are data transfers accelerated, but storage requirements are reduced as well. Using the example in FIG. 2, there are three music files and two movie files that would have been contained as two separate copies in conventional systems. Assuming the music files are 5 MB and the movie files are 1 GB, that is over 2 GB of data being stored in duplicate. Furthermore, since data on network systems are typically backed up, 4 GB of space is being wasted. Using the present invention, where a few bytes are used to point to the data, over 4 GB of space is saved. Furthermore, since this example only shows two users with a small number of files, the space savings on a large system with thousands of users could be extremely large.



FIG. 3 illustrates a graphical representation of a network of systems configured in the present invention. As described above, the present invention includes one or more source computing systems and one or more target computing systems. The source computing system is where the data to be transferred is located, and the target computing system is where the data will be stored after the transfer. Although FIG. 1 illustrates one source computing system and one target computing system, a network of systems 300 is able to include any number of source and target computing systems. FIG. 3 illustrates the computing systems coupled by a network 302. The computing systems include, but are not limited to, a server 304, a personal computer 306, a PDA 308, a cell phone 310, a laptop 312, a thin client 314, a Mac computer 316, an mp3 player 318 and a gaming console 320. Generally, the target computing systems are servers and the source computing systems are personal computers, PDAs, cell phones, laptops, thin clients, Mac computers, mp3 players and gaming consoles. However, any of the systems are able to be either the source or the target.


As an example, a typical configuration for use at a business includes one or more servers 304 as the target systems where users are able to back up their data. The employees then utilize one or more personal computers 306, PDAs 308, cell phones 310 and laptops 312 as the sources for the data. As data is backed up onto the server 304, the accelerated data transfer described herein is utilized. Fewer servers are required because the inefficiencies of duplicated data are resolved. Furthermore, there is less traffic on the network because transfers are much more efficient. Hence, in this setting it is reasonable to have the server be the target computing system and the other systems be the source computing systems.


It is possible though to have the roles of the systems switched or modified. For example, in a home network, a user is able to couple his cell phone, PDA, gaming system and personal computer together where the personal computer is the target system and his cell phone, PDA and gaming system are the source systems.



FIG. 4 illustrates a flowchart of an embodiment of the present invention. In the step 400, data is stored in a data store on a first computing system also referred to as a target computing system. Additionally, files corresponding to the data stored in the data store are also stored on the first computing system. In the step 402, a data segment is transferred from a second computing system or a source computing system to the first computing system. Generally a user selects a file to be transferred from the second computing system to the first computing system, and the data segment is a part of the file, a hash of the file and/or a CRC of the file. In the step 404, the data segment transferred is compared with the data in the data store. Comparing includes scanning the data store for the data segment and then identifying matching data in the data store. In the step 406, if a match is found then a pointer is generated to point to the corresponding file or files in the step 408. However, if a match is not found in the step 406, then a copy of the source file is transferred from the second computing system to the first computing system.


Although the present invention has been described where a data segment is compared to data, and then a link is generated to point to the entire file corresponding with the data, sections of files are able to be matched as well where the entire file is not the same. For example, sometimes additional data is included at the beginning or end of a music or movie file making the file slightly different from one that has very similar contents. Or, for example, one person has a fifteen second clip of a five minute long video, so the fifteen second clip is contained within the file of the long video. Such sections of data are able to be compared and matched by the present invention using a section of the file or a CRC or hash of a section of the file. In those instances, instead of transferring the entire file across the network because there is some offset or slight difference between the data, the present invention copies the data from the file residing on the target system. The sections of the file that are not already existing on the target system are transferred over the network, and the file is combined to generate the file initially intended to be transferred. In another embodiment, a master file is stored on the target system where the master file contains more data than a smaller file which only contains a portion of the master file. A pointer then points to the correct sections of the master file to represent the smaller file.


To utilize the present invention a user selects a file or files on a source computing system to be transferred over a network to a target computing system. In some embodiments, a user is not required to initiate the data transfer and the transfer is automated. The target computing system performs the necessary search to determine if any common data is already located on the target computing system. If there is common data, then the file is not transferred or only a portion of the file that is not common is transferred, and a pointer points to the common data. When a user views the data on the target computing system, the appearance is no different whether the file was transferred or is pointed to by a pointer. Furthermore, the present invention is able to be utilized without a specially modified file system.


In operation, users experience accelerated data transfers, but otherwise do not have to modify their ways of transferring data. After a user initiates the data transfer, the target computing system receives a data segment representing the file on the source computing system. The target computing system then compares the data segment with data stored within a data store by scanning the data store for a match. If a match is found, then the source file is not actually transferred over the network, and a pointer is generated on the target computing system. If the target computing system does not locate matching data, then the source file is transferred over the network. By expediting transfers of common data, network efficiency increases greatly in addition to storage requirements being reduced.


The present invention has been described in terms of specific embodiments incorporating details to facilitate the understanding of principles of construction and operation of the invention. Such reference herein to specific embodiments and details thereof is not intended to limit the scope of the claims appended hereto. It will be readily apparent to one skilled in the art that other various modifications may be made in the embodiment chosen for illustration without departing from the spirit and scope of the invention as defined by the claims.

Claims
  • 1. A method of accelerating data transfer comprising: a. storing data in a data store on a first computing system wherein the data corresponds to one or more files stored on the first computing system;b. transferring a data segment from a source file from a second computing system to the first computing system over a network;c. scanning the data store on the first computing system for the data segment received from the second computing system;d. generating one or more pointers on the first computing system to the one or more corresponding files of one or more matching data segments, if the one or more matching data segments are identified in the data store on the first computing system; ande. transferring a copy of the source file from the second computing system to the first computing system, if the one or more matching data segments are not identified in the data store on the first computing system.
  • 2. The method as claimed in claim 1 wherein the data store is a database.
  • 3. The method as claimed in claim 1 wherein the first computing system is a target system and the second computing system is a source system.
  • 4. The method as claimed in claim 1 wherein the first computing system is a server and the second computing system is a client system.
  • 5. The method as claimed in claim 4 wherein the client system is selected from the group consisting of a personal computer, a PDA, a cell phone, a laptop, a thin client, a Mac computer, an mp3 player and a gaming console.
  • 6. The method as claimed in claim 1 wherein the first computing system is a first client system and the second computing system is a second client system.
  • 7. The method as claimed in claim 1 wherein the data segment is one or more cyclic redundancy checks and the data in the data store includes cyclic redundancy checks and the data segment and the data are compared.
  • 8. The method as claimed in claim 1 wherein the data segment is a unique database key and the data in the data store includes database keys and the data segment and the data are compared.
  • 9. The method as claimed in claim 1 wherein the data segment is a hash and the data in the data store includes hashes and the data segment and the data are compared.
  • 10. The method as claimed in claim 1 wherein the data store grows as more files are stored on the first computing system.
  • 11. The method as claimed in claim 1 wherein the files stored on the first computing system are minimized by implementing the data store.
  • 12. The method as claimed in claim 1 wherein one or more additional computing systems are coupled to the first computing system.
  • 13. The method as claimed in claim 1 further comprising transferring only a first section of the source file when only a second section of the source file is found within the data store.
  • 14. The method as claimed in claim 1 wherein a standard operating system and file system are utilized on the first computing system and the second computing system.
  • 15. The method of claim 1 wherein if at least one of the corresponding files of the matching data segments is deleted from a first location on the first computer system where the file was stored, the file is automatically transferred to and stored at a second location on the first computer system where at least one of the one or more pointers were stored as a replacement for the at least one of the one or more pointers.
  • 16. The method of claim 15 wherein the remaining one or more pointers that were not replaced by the file are automatically reconfigured such that the remaining one or more pointers point to the file stored at the second location instead of the first location.
  • 17. A system for accelerating data transfer comprising: a. a first computing system having a first memory device for storing one or more files and a data store for storing data corresponding to the one or more files; andb. a second computing system having a second memory device, wherein the second computing system is coupled to the first computing system,wherein a data segment stored in the second memory device is transferred to the first computing system, and is compared to the data within the data store on the first memory device after being received from the second memory device, further wherein a pointer to the one or more files on the first memory device is added on the first memory device if the data segment is found within the data store on the first memory device, but a copy of a source file corresponding to the data segment is transferred from the second memory device to the first memory device if the data segment is not found in the data store.
  • 18. The system as claimed in claim 17 wherein the data store is a database.
  • 19. The system as claimed in claim 17 wherein the first computing system is a target system and the second computing system is a source system.
  • 20. The system as claimed in claim 17 wherein the first computing system is a server and the second computing system is a client system.
  • 21. The system as claimed in claim 20 wherein the client system is selected from the group consisting of a personal computer, a PDA, a cell phone, a laptop, a thin client, a Mac computer, an mp3 player and a gaming console.
  • 22. The system as claimed in claim 17 wherein the first computing system is a first client system and the second computing system is a second client system.
  • 23. The system as claimed in claim 17 wherein the data segment is one or more cyclic redundancy checks and the data in the data store includes cyclic redundancy checks and the data segment and the data are compared.
  • 24. The system as claimed in claim 17 wherein the data segment is a unique database key and the data in the data store includes database keys and the data segment and the data are compared.
  • 25. The system as claimed in claim 17 wherein the data segment is a hash and the data in the data store includes hashes and the data segment and the data are compared.
  • 26. The system as claimed in claim 17 wherein the data store grows as more files are stored on the first memory device.
  • 27. The system as claimed in claim 17 wherein the files stored on the first memory device are minimized by implementing the data store.
  • 28. The system as claimed in claim 17 further comprising one or more additional computing systems coupled to the first computing system.
  • 29. The system as claimed in claim 17 wherein only a first section of the source file is transferred when only a second section of the source file is found within the data store.
  • 30. The system as claimed in claim 17 wherein a standard operating system and file system are utilized on the first computing system and the second computing system.
  • 31. The system as claimed in claim 17 further comprising a network coupling the first computing system and the second computing system.
  • 32. A network of systems for accelerating data transfers comprising: a. one or more source systems having one or more source memory devices for transferring a data segment corresponding to a source file stored on the one or more source memory devices; andb. one or more target systems having one or more target memory devices for storing data in a data store corresponding to one or more files on the one or more target memory devices and for comparing the data segment received from the one or more source memory devices with the data in the data store where if the data segment is found, a pointer is generated on the one or more target memory devices to point to a corresponding file in the one or more files on the target memory device instead of transferring the source file over a network, wherein the one or more target systems are coupled to the one or more source systems.
  • 33. The network of systems as claimed in claim 32 wherein the data store is a database.
  • 34. The network of systems as claimed in claim 32 wherein the one or more target systems are one or more servers and the one or more source systems are one or more client systems.
  • 35. The network of systems as claimed in claim 32 wherein the one or more client systems are selected from the group consisting of personal computers, PDAs, cell phones, laptops, thin clients, Mac computers, mp3 players and gaming consoles.
  • 36. The network of systems as claimed in claim 32 wherein the data segment is one or more cyclic redundancy checks and the data in the data store includes cyclic redundancy checks and the data segment and the data are compared.
  • 37. The network of systems as claimed in claim 32 wherein the data segment is a unique database key and the data in the data store includes database keys and the data segment and the data are compared.
  • 38. The network of systems as claimed in claim 32 wherein the data segment is a hash and the data in the data store includes hashes and the data segment and the data are compared.
  • 39. The network of systems as claimed in claim 32 wherein the data store grows as more files are stored on the one or more target memory devices.
  • 40. The network of systems as claimed in claim 32 wherein the files stored on the one or more target memory devices are minimized by implementing the data store.
  • 41. The network of systems as claimed in claim 32 wherein only a first section of the source file is transferred when only a second section of the source file is found within the data store.
  • 42. The network of systems as claimed in claim 32 wherein a standard operating system and file system are utilized on the one or more target systems and the one or more source systems.
  • 43. A storage system configured to receive data from a plurality of source computing systems comprising: a memory device comprising:a. one or more files;b. a set of information corresponding to the one or more files; andc. a data store for storing the set of information,wherein a data segment received from a source computing system in the plurality of source computing systems is compared with the set of information stored within the data store and a pointer is generated on the storage system to point to a corresponding file in the one or more files if the data segment is found but if the data segment is not found within the data store, a copy of a source file on the source computing system corresponding to the data segment is transferred to the storage system.
  • 44. The storage system as claimed in claim 43 wherein the data store is a database.
  • 45. The storage system as claimed in claim 43 wherein the data segment is one or more cyclic redundancy checks and the set of information in the data store includes cyclic redundancy checks and the data segment and the set of information are compared.
  • 46. The storage system as claimed in claim 43 wherein the data segment is a unique database key and the set of information in the data store includes database keys and the data segment and the set of information are compared.
  • 47. The storage system as claimed in claim 43 wherein the data segment is a hash and the set of information in the data store includes hashes and the data segment and the set of information are compared.
  • 48. The storage system as claimed in claim 43 wherein the data store grows as more files are stored on the storage system.
  • 49. The storage system as claimed in claim 43 wherein the files stored on the storage system are minimized by implementing the data store.
  • 50. The storage system as claimed in claim 43 wherein only a first section of the source file is transferred when only a second section of the source file is found within the data store.
  • 51. The storage system as claimed in claim 43 wherein a standard operating system and file system are utilized on the storage system.
  • 52. A method of accelerating data transfer comprising: a. storing data in a data store on a target computing system wherein the data corresponds to one or more files stored on the target computing system;b. transferring a data segment of a source file from a source computing system to the target computing system over a network in response to a request for the source file by the first computing system;c. scanning the data store on the target computing system for the data segment received from the source computing system;d. generating a pointer on the target computing system that points to at least one of the one or more files, if the data that corresponds to the at least one of the one or more files matches the data segment; ande. transferring a copy of the source file from the source computing system to the target computing system, if the data that corresponds to the one or more files does not match the data segment.
US Referenced Citations (270)
Number Name Date Kind
4866635 Kahn et al. Sep 1989 A
5204897 Wyman Apr 1993 A
5479654 Squibb Dec 1995 A
5602990 Leete Feb 1997 A
5649196 Woodhill Jul 1997 A
5659743 Adams et al. Aug 1997 A
5787409 Seiffert et al. Jul 1998 A
5802364 Senator et al. Sep 1998 A
5812751 Ekrot et al. Sep 1998 A
5835911 Nakagawa et al. Nov 1998 A
5897635 Torres et al. Apr 1999 A
5930357 Fukui Jul 1999 A
5933647 Aronberg et al. Aug 1999 A
5950010 Hesse et al. Sep 1999 A
5974547 Klimenko Oct 1999 A
6012152 Douik et al. Jan 2000 A
6029196 Lenz Feb 2000 A
6067582 Smith et al. May 2000 A
6144959 Anderson et al. Nov 2000 A
6170065 Kobata et al. Jan 2001 B1
6189101 Dusenbury, Jr. Feb 2001 B1
6209089 Selitrennikoff et al. Mar 2001 B1
6212525 Guha Apr 2001 B1
6212660 Joeressen et al. Apr 2001 B1
6282711 Halpern et al. Aug 2001 B1
6301612 Selitrennikoff et al. Oct 2001 B1
6314428 Brew et al. Nov 2001 B1
6317761 Landsman et al. Nov 2001 B1
6339826 Hayes, Jr. et al. Jan 2002 B2
6349137 Hunt et al. Feb 2002 B1
6356915 Chtchetkine et al. Mar 2002 B1
6363400 Chtchetkine et al. Mar 2002 B1
6366296 Boreczky et al. Apr 2002 B1
6378035 Parry et al. Apr 2002 B1
6421777 Pierre-Louis et al. Jul 2002 B1
6449658 Lafe et al. Sep 2002 B1
6459499 Tomat Oct 2002 B1
6463530 Sposato Oct 2002 B1
6473794 Guheen et al. Oct 2002 B1
6477531 Sullivan et al. Nov 2002 B1
6490677 Aquilar et al. Dec 2002 B1
6536037 Guheen et al. Mar 2003 B1
6553375 Huang et al. Apr 2003 B1
6556950 Schwenke et al. Apr 2003 B1
6578142 Anderson et al. Jun 2003 B1
6606744 Mikurak Aug 2003 B1
6625651 Swartz et al. Sep 2003 B1
6625754 Aguilar et al. Sep 2003 B1
6633782 Schleiss et al. Oct 2003 B1
6636857 Thomas et al. Oct 2003 B2
6654797 Kamper Nov 2003 B1
6654801 Mann et al. Nov 2003 B2
6694375 Beddus et al. Feb 2004 B1
6697852 Ryu Feb 2004 B1
6704886 Gill et al. Mar 2004 B1
6718464 Cromer et al. Apr 2004 B2
6728530 Heinonen et al. Apr 2004 B1
6735625 Ponna May 2004 B1
6751658 Haun et al. Jun 2004 B1
6757729 Devarakonda et al. Jun 2004 B1
6795912 Itoh et al. Sep 2004 B1
6816462 Booth, III et al. Nov 2004 B1
6816882 Conner et al. Nov 2004 B1
6820180 McBrearty et al. Nov 2004 B2
6854009 Hughes Feb 2005 B1
6871210 Subramanian Mar 2005 B1
6880108 Gusler et al. Apr 2005 B1
6885481 Dawe Apr 2005 B1
6886020 Zahavi et al. Apr 2005 B1
6898286 Murray May 2005 B2
6915343 Brewer et al. Jul 2005 B1
6954853 Wang et al. Oct 2005 B2
6954930 Drake et al. Oct 2005 B2
6959235 Abdel-Malek et al. Oct 2005 B1
6985967 Hipp Jan 2006 B1
7003560 Mullen et al. Feb 2006 B1
7003663 Lagosanto et al. Feb 2006 B2
7024471 George et al. Apr 2006 B2
7058698 Chatterjee et al. Jun 2006 B2
7080118 Hildebrand Jul 2006 B2
7143307 Witte et al. Nov 2006 B1
7149698 Guheen et al. Dec 2006 B2
7165087 Graupner et al. Jan 2007 B1
7175078 Ban et al. Feb 2007 B2
7178166 Taylor et al. Feb 2007 B1
7194445 Chan et al. Mar 2007 B2
7200779 Coss, Jr. et al. Apr 2007 B1
7210143 Or et al. Apr 2007 B2
7237122 Kadam et al. Jun 2007 B2
7260597 Hofrichter et al. Aug 2007 B1
7287053 Bodin Oct 2007 B2
7305671 Davidov et al. Dec 2007 B2
7328367 Ukai et al. Feb 2008 B2
7330997 Odom Feb 2008 B1
7337311 Chen et al. Feb 2008 B2
7392046 Leib et al. Jun 2008 B2
7401125 Uchida et al. Jul 2008 B1
7480822 Arbon et al. Jan 2009 B1
7487383 Bensinger Feb 2009 B2
7512584 Keith, Jr. Mar 2009 B2
7571467 Priestley et al. Aug 2009 B1
7577431 Jiang Aug 2009 B2
7624086 Keith, Jr. Nov 2009 B2
7627694 Sreenivasan et al. Dec 2009 B2
7664834 Keith, Jr. Feb 2010 B2
7698487 Rothman et al. Apr 2010 B2
7703142 Wilson et al. Apr 2010 B1
7752486 Satran et al. Jul 2010 B2
7788524 Wing et al. Aug 2010 B2
7840514 Keith, Jr. Nov 2010 B2
7844686 Keith, Jr. Nov 2010 B1
7886031 Taylor et al. Feb 2011 B1
7908339 Keith, Jr. Mar 2011 B2
7912902 Cheng et al. Mar 2011 B2
8099378 Keith, Jr. Jan 2012 B2
8126722 Robb et al. Feb 2012 B2
8175418 Keith, Jr. May 2012 B1
8234238 Keith, Jr. Jul 2012 B2
8244864 Bahl et al. Aug 2012 B1
8307239 Keith, Jr. Nov 2012 B1
8423821 Keith, Jr. Apr 2013 B1
8612741 Crisan et al. Dec 2013 B2
20010037323 Moulton et al. Nov 2001 A1
20010049793 Sugimoto Dec 2001 A1
20020013827 Edstrom et al. Jan 2002 A1
20020035674 Vetrivelkumaran et al. Mar 2002 A1
20020049764 Boothby Apr 2002 A1
20020059505 St. Pierre et al. May 2002 A1
20020083183 Pujare et al. Jun 2002 A1
20020087625 Toll et al. Jul 2002 A1
20020087963 Eylon et al. Jul 2002 A1
20020091763 Shah et al. Jul 2002 A1
20020094868 Tuck et al. Jul 2002 A1
20020099934 Cromer et al. Jul 2002 A1
20020104080 Woodard et al. Aug 2002 A1
20020107920 Hotti Aug 2002 A1
20020107945 George et al. Aug 2002 A1
20020116585 Scherr Aug 2002 A1
20020124092 Urien Sep 2002 A1
20020131404 Mehta et al. Sep 2002 A1
20020138640 Raz et al. Sep 2002 A1
20020157089 Patel et al. Oct 2002 A1
20020161868 Paul et al. Oct 2002 A1
20020188941 Cicciarelli et al. Dec 2002 A1
20030005096 Paul et al. Jan 2003 A1
20030014470 Iijima Jan 2003 A1
20030031164 Nabkel et al. Feb 2003 A1
20030033379 Civanlar et al. Feb 2003 A1
20030036882 Harper et al. Feb 2003 A1
20030037328 Cicciarelli et al. Feb 2003 A1
20030041136 Cheline et al. Feb 2003 A1
20030046371 Falkner Mar 2003 A1
20030051128 Rodriguez et al. Mar 2003 A1
20030055878 Fletcher et al. Mar 2003 A1
20030061067 Atwal et al. Mar 2003 A1
20030078960 Murren et al. Apr 2003 A1
20030110188 Howard et al. Jun 2003 A1
20030120502 Robb et al. Jun 2003 A1
20030126242 Chang Jul 2003 A1
20030177381 Ofek et al. Sep 2003 A1
20030191730 Adkins et al. Oct 2003 A1
20030204562 Hwang Oct 2003 A1
20030233383 Koskimies Dec 2003 A1
20030233493 Boldon et al. Dec 2003 A1
20040010716 Childress et al. Jan 2004 A1
20040025018 Haas et al. Feb 2004 A1
20040068554 Bales et al. Apr 2004 A1
20040073787 Ban et al. Apr 2004 A1
20040093492 Daude et al. May 2004 A1
20040098426 Ishii et al. May 2004 A1
20040104927 Husain et al. Jun 2004 A1
20040107273 Biran et al. Jun 2004 A1
20040123153 Wright et al. Jun 2004 A1
20040133634 Luke Jul 2004 A1
20040148306 Moulton et al. Jul 2004 A1
20040180721 Rowe Sep 2004 A1
20040193876 Donley et al. Sep 2004 A1
20040201604 Kraenzel et al. Oct 2004 A1
20040236843 Wing et al. Nov 2004 A1
20040243650 McCrory et al. Dec 2004 A1
20040243928 Hesmer et al. Dec 2004 A1
20040267944 Britt, Jr. Dec 2004 A1
20050027846 Wolfe et al. Feb 2005 A1
20050033757 Greenblatt et al. Feb 2005 A1
20050033808 Cheng et al. Feb 2005 A1
20050044197 Lai Feb 2005 A1
20050044332 de Brebisson Feb 2005 A1
20050044544 Slivka et al. Feb 2005 A1
20050060281 Bucher et al. Mar 2005 A1
20050071267 Takahashi et al. Mar 2005 A1
20050096920 Matz et al. May 2005 A1
20050108297 Rollin et al. May 2005 A1
20050108538 Howard May 2005 A1
20050108546 Lehew et al. May 2005 A1
20050108593 Purushothaman et al. May 2005 A1
20050144218 Heintz Jun 2005 A1
20050149729 Zimmer et al. Jul 2005 A1
20050160289 Shay Jul 2005 A1
20050188315 Campbell et al. Aug 2005 A1
20050192035 Jiang Sep 2005 A1
20050193245 Hayden et al. Sep 2005 A1
20050198196 Bohn et al. Sep 2005 A1
20050198239 Hughes Sep 2005 A1
20050216524 Gomes et al. Sep 2005 A1
20050216902 Schaefer Sep 2005 A1
20050226059 Kavuri et al. Oct 2005 A1
20050256952 Mouhanna et al. Nov 2005 A1
20050262503 Kane Nov 2005 A1
20050268145 Hufferd et al. Dec 2005 A1
20050273486 Keith, Jr. Dec 2005 A1
20050283606 Williams Dec 2005 A1
20050286435 Ogawa et al. Dec 2005 A1
20060021040 Boulanger et al. Jan 2006 A1
20060031377 Ng et al. Feb 2006 A1
20060031407 Dispensa et al. Feb 2006 A1
20060031529 Keith, Jr. Feb 2006 A1
20060035713 Cockerille et al. Feb 2006 A1
20060041641 Breiter et al. Feb 2006 A1
20060041759 Kaliski, Jr. et al. Feb 2006 A1
20060047716 Keith, Jr. Mar 2006 A1
20060047946 Keith, Jr. Mar 2006 A1
20060059384 Helliker Mar 2006 A1
20060074943 Nakano et al. Apr 2006 A1
20060095705 Wichelman et al. May 2006 A1
20060106842 Diament et al. May 2006 A1
20060106896 Carlson et al. May 2006 A1
20060129459 Mendelsohn Jun 2006 A1
20060143709 Brooks et al. Jun 2006 A1
20060149955 Velhal et al. Jul 2006 A1
20060179061 D'Souza et al. Aug 2006 A1
20060200736 Smit Sep 2006 A1
20060224544 Keith, Jr. Oct 2006 A1
20060224545 Keith, Jr. Oct 2006 A1
20060233310 Adams, Jr. et al. Oct 2006 A1
20070011292 Fritsch et al. Jan 2007 A1
20070078982 Aidun et al. Apr 2007 A1
20070094269 Mikesell et al. Apr 2007 A1
20070143374 D'Souza et al. Jun 2007 A1
20070174658 Takamoto et al. Jul 2007 A1
20070174690 Kambara et al. Jul 2007 A1
20070185936 Derk et al. Aug 2007 A1
20070233633 Keith, Jr. Oct 2007 A1
20070239905 Banerjee Oct 2007 A1
20070255604 Seelig Nov 2007 A1
20070271290 Keith, Jr. Nov 2007 A1
20070271428 Atluri Nov 2007 A1
20070274315 Keith, Jr. Nov 2007 A1
20070276836 Chatterjee et al. Nov 2007 A1
20080016387 Bensinger Jan 2008 A1
20080034019 Cisler et al. Feb 2008 A1
20080034071 Wilkinson et al. Feb 2008 A1
20080072002 Kuwahara et al. Mar 2008 A1
20080077622 Keith, Jr. Mar 2008 A1
20080077630 Keith Mar 2008 A1
20080090560 Bouzid et al. Apr 2008 A1
20080127294 Keith, Jr. May 2008 A1
20080209142 Obernuefemann Aug 2008 A1
20080216168 Larson et al. Sep 2008 A1
20080294860 Stakutis et al. Nov 2008 A1
20080313632 Kumar et al. Dec 2008 A1
20090075593 Demirbasa et al. Mar 2009 A1
20090094362 Huff Apr 2009 A1
20100050011 Takamoto et al. Feb 2010 A1
20100125770 Keith, Jr. May 2010 A1
20110047118 Keith, Jr. Feb 2011 A1
20110161729 Ritz et al. Jun 2011 A1
20120084364 Sivavakeesar Apr 2012 A1
20120198154 Keith, Jr. Aug 2012 A1
20120272099 Keith, Jr. Oct 2012 A1
20130031405 Keith, Jr. Jan 2013 A1
Foreign Referenced Citations (11)
Number Date Country
WO2005119493 Dec 2005 WO
WO2005119494 Dec 2005 WO
WO2005119495 Dec 2005 WO
WO2006010131 Jan 2006 WO
WO2006094282 Sep 2006 WO
WO2006094301 Sep 2006 WO
WO2007139962 Dec 2007 WO
WO2007139963 Dec 2007 WO
WO2008039386 Apr 2008 WO
WO2008039394 Apr 2008 WO
WO2008039395 Apr 2008 WO
Non-Patent Literature Citations (24)
Entry
http://www.macrovision.com/products/flexnet—installshield/installshield/overview/index.shtml, pp. 1-3.
http://macrovision.com/products/flexnet—installshield/installshield/index.shtml, pp. 1-2.
http:/www.stealthbits.com/, pp. 1-2.
http://www.microsoft.com/technet/prodtechnol/windows2000serv/maintain/featusability/inmnwp.mspx, pp. 1-36.
http://en.wikipedia.org/wik/Expert—system, pp. 1-9.
http://www.pctools.com/registry-mechanic/, pp. 1-2.
http://www.backup.com.
http://www.swapdrive.com/sdbackup.asp.
Bandwidth Capping at Source http://www.24onlinebilling.com/download/whitepapers/Bandwidth%20Capping%20At%20Source.pdf pp. 1-9.
IBM Corporation, AFS: “User Guide”, First Edition, Version 3.6. Apr. 2000, pp. 112.
Shepler et al. “RFC 3530—Network File System (NFS) version 4 Protocol”, Standards Track, Apr. 2003, pp. 278, http://tools.ietf.org/html/rfc3530#page-119.
IEEE100 the Authoritative Dictionary of IEEE Standards Terms, Seventh Edition, Jan. 2003, 3 pages.
VMware, VMware Infrastructure Architecture Overview, Jun. 14, 2006, http:/www.vmware.com/resourse/techresources/, pp. 1-14.
VMware, Using VMware Infrastructure for Backup and Restore, Oct. 26, 2006, http:/www.vmware.com/resourse/techresources/, pp. 1-20.
Microsoft@ Computer Dictionary, Fifth Edition, 2002, 3 pages.
Tridgell, A., “Efficient Algorithms for Sorting and Synchronization,” Thesis, The Australian National University, Feb. 1999, 106 pages.
MacDonald, J.P., “File System Support for Delta Compression,” University of California at Berkeley, Dept. of Electrical Engineering and Computer Sciences, May 19, 2000, 32 pages.
Muthitacharoen et al., “A Low-bandwidth Network File System,” MIT Laboratory for Computer Science and NYU Department of Computer Science, 2001, 14 pages.
Yu et al., “Scalable network resource management for large scale Virtual Private Networks,” Simulation Modeling Practice and Theory, 12 (2004) pp. 263-285.
Cisco PIX “Configuring an IPSec Tunnel Between a Cisco Secure PIX Firewall and a Checkpoint NG Firewall,” May 12, 2006, document ID 23785.
Random House Unabridged Dictionary, “device: meaning and definitions,” 1997, Random House Inc., retrieved via “http://dictionary.infoplease.com/device”.
Microsoft Press, Microsoft® Computer Dictionary, Fifth Edition, May 1, 2002, p. 39.
Definition of Plug-in (computing) from http://en.wikipedia.org/wiki/Plug-in—(computing), printed on Jun. 18, 2011, 1 page.
Plug-in definition from Wikipedia, http://en.wikipedia.org/wiki/Plug-in—(computing) printed Nov. 6, 2011, 5 pages.
Related Publications (1)
Number Date Country
20080077630 A1 Mar 2008 US