Claims
- 1. A method for creating a virtual hot spare in an intelligent distributed storage system, the method comprising:
reserving at least one virtual hot spare block in a stripe of blocks in the file system; and, repeating the reservation for each stripe in the file.
- 2. The method of claim 1, wherein the reserving further comprises allocation of at least one virtual hot spare block.
- 3. The method of claim 1, further comprising creating a map associated with the virtual hot spare block.
- 4. The method of claim 1, further comprising creating a map associated with the stripe, including the virtual hot spare block.
- 5. The method of claim 1, further comprising creating a map associated with the stripe.
- 6. A method for creating a virtual hot spare for a file stored in an intelligent distributed storage system, the method comprising:
selecting at least one smart storage unit from the set of smart storage units in the intelligent distributed file system; reserving at least one virtual hot spare block on the selected smart storage unit for a stripe of the file; and, repeating the reservation for each stripe in the file.
- 7. The method of claim 6, wherein reserving further comprises allocating at least one virtual hot spare block.
- 8. The method of claim 6, further comprising creating a map associated with the virtual hot spare block.
- 9. The method of claim 6, further comprising creating a map associated with the stripe.
- 10. The method of claim 6, further comprising creating a map associated with the stripe, including the virtual hot spare block.
- 11. A method for creating a virtual hot spare in an intelligent distributed storage system, the method comprising:
reserving at least one virtual hot spare block associated with a stripe; and, repeating the reservation for each stripe created in the file system.
- 12. The method of claim 11, wherein reserving further comprises allocating at least one virtual hot spare block.
- 13. The method of claim 11, further comprising creating a map associated with the virtual hot spare block, and returning the map to the intelligent distributed storage system.
- 14. The method of claim 11, further comprising creating a map associated with the stripe, and returning the map to the intelligent distributed storage system.
- 15. The method of claim 11, further comprising creating a map associated with the stripe, including the virtual hot spare block, and returning the map to the intelligent distributed storage system.
- 16. A method for using a virtual hot spare in an intelligent distributed file system, the method comprising:
detecting the failure of at least one smart storage unit in the intelligent distributed file system; recovering data blocks stored on the at least one failed smart storage unit in each stripe in the file system; determining the location of virtual hot spare blocks on at least one functional smart storage unit in the intelligent distributed file system; and, storing the recovered data blocks in stripes on the virtual hot spare blocks on the at least one functional smart storage unit.
- 17. The method of claim 16, further comprising determining the location of virtual hot spare blocks using a virtual hot spare block map.
- 18. The method of claim 16, further comprising determining the location of virtual hot spare blocks based on a predetermined virtual hot spare location relative to the stripe.
- 19. A method for using a virtual hot spare in an intelligent distributed file system 110, the method comprising:
detecting the failure of a smart storage unit; for a stripe, recovering the contents of at least one of a lost data block and lost parity block from the stripe stored on the failed smart storage unit, through at least one of the data blocks and parity blocks stored in the stripe; and, storing the recovered block in an at least one virtual hot spare block associated with the stripe.
- 20. The method of claim 19, further comprising determining the location of the virtual hot spare block using a virtual hot spare block map.
- 21. The method of claim 19, further comprising determining the location of virtual hot spare block based on a predetermined virtual hot spare location relative to the stripe.
- 22. A method for using a virtual hot spare in an intelligent distributed file system 110, the method comprising:
detecting the failure of a smart storage unit; for each stripe, recovering the contents of at least one lost block stored on the failed smart storage unit; and, storing the recovered block in an at least one virtual hot spare block associated with the stripe.
- 23. A system for providing a virtual hot spare in an intelligent distributed file system 110, comprising:
a plurality of smart units, the smart units configured to store data in stripes distributed among the smart units; a device width, the device width representing the number of smart units configured in the system; a parity width, the parity width representing the number of data blocks associated with a set of one or more parity blocks; a stripe width, the stripe width representing a set of blocks used for operations by the file system; a virtual hot spare width, the virtual hot spare width representing the number of hot spare blocks to be associated with a stripe of data blocks; an allocator module, the allocator module configured to allocate at least one virtual hot spare block for each stripe; and, a recovery module, the recovery module restoring data blocks to the at least one virtual hot spare block in the event of failure of one or more of the plurality of smart units.
- 24. A system for providing a virtual hot spare in an intelligent distributed file system, comprising:
a first module configured to store a stripe across a plurality of smart storage units; a second module configured to allocate an at least one virtual hot spare block for the stripe; a third module configured to recover at least one lost block from the stripe in the event of failure of one or more smart storage units; a fourth module configured to write the recovered at least one lost block to the at least one virtual hot spare block for each stripe.
- 25. A virtual hot spare data structure for storing information about at least one virtual hot spare block location for at least one of stripes, files, and file systems, the virtual hot spare data structure comprising the at least one virtual hot spare block location and smart unit location in the file system.
- 26. The data structure of claim 25, further comprising a metadata portion associated with at least one of the stripe, file, data directory, and file system corresponding to the virtual hot spare, wherein the metadata includes information about the location of the virtual hot spare block and smart unit location in the file system.
- 27. A method for implementing a virtual hot spare in a file system, comprising:
allocating at least one virtual hot spare block for a stripe in the file system; detecting failure of at least one storage device in the file system; recovering a block of the stripe previously stored on the failed storage device; and, writing the portion of the stripe previously stored on the failed storage device to the at least one virtual hot spare block.
- 28. The method of claim 27, wherein the allocation of the at least one virtual hot spare block for a stripe is determined by the approximate even distribution of data blocks, parity blocks, and hot spare blocks amongst the storage devices.
- 29. The method of claim 27, wherein the allocation of the at least one virtual hot spare block for a stripe is determined by the selection of one storage device for all virtual hot spare blocks for all stripes for a file on the file system.
- 30. The method of claim 27, wherein the allocation of the at least one virtual hot spare block for a stripe is determined by the preselection of a location relative to each stripe in the file system.
- 31. The method of claim 27, wherein the allocation of the at least one virtual hot spare block for a stripe is determined by the algorithmic distribution of data blocks, parity blocks, and hot spare blocks amongst the storage devices.
- 32. The method of claim 27, wherein the allocation of the at least one virtual hot spare block for a stripe is determined by the combination of at least two of: the algorithmic distribution of data blocks, parity blocks, and hot spare blocks amongst the storage devices, the preselection of a location relative to each stripe in the file system, and the selection of one storage device for virtual hot spare blocks for stripes for a file on the file system.
- 33. A system for providing a virtual hot spare in an intelligent distributed file system, comprising:
a plurality of smart units, the smart units configured to store data in stripes distributed among the plurality of smart units; a stripe width, the stripe width representing a set of blocks used for operations by the file system; a virtual hot spare width, the virtual hot spare width representing the number of hot spare blocks to be associated with a stripe of data blocks; an allocator module, the allocator module configured to allocate at least one virtual hot spare block for each stripe; a recovery module, the recovery module configured to restore data blocks in the event of failure of one or more smart units; and, a restored data module, the restored data module configured to determine the location of virtual hot spare blocks on the smart storage system, to write restored data blocks to the virtual hot spare blocks, and to create a metadata structure representing the location of restored data blocks in the virtual hot spare blocks.
- 34. A method for retrieving data in an intelligent distributed file system comprised of a plurality of smart storage units, wherein the data is distributed among a subset of the plurality of smart storage units, the method comprising:
receiving a location corresponding to the data; retrieving locally stored portions of the data; sending requests for remotely stored portions of the data; receiving the remotely stored portions of the data; and, returning the data.
- 35. The method of claim 34 further comprising:
forming the data from the retrieved locally stored portions of the data and the received remotely stored portions of the data.
- 36. A method for performing name resolution for files stored in an intelligent distributed file system, the method comprising:
receiving a file name comprised of a set of tokens, wherein the file name corresponds to a root directory metadata address; setting a current variable to the root directory metadata address; and traversing through the set of tokens comprising:
requesting an address of the token's metadata determined from the current variable; receiving the requested address; setting the current variable to the received address; and obtaining the next token.
- 37. A method of retrieving a file stored among a set of smart storage units in an intelligent distributed file system, the method comprising:
receiving a file designator corresponding to a file; determining the location of metadata that corresponds to the file; retrieving the metadata; for each block of data in the file, obtaining the location of the block of data and retrieving the block of data; and, returning the file.
- 38. The method of claim 37, further comprising:
forming the file from the retrieved blocks of data.
- 39. A method for creating parity data for a file stored among a set of smart storage units in an intelligent distributed file system, the method comprising:
receiving parity information designating a set of parity groups; receiving data location information relating to the file; for each parity group within the set of parity groups,
retrieving a data set related the to parity group, generating parity data, and storing the parity data; and storing the location of the parity data.
- 40. The method of claim 39, further comprising:
allocating storage for the parity data within the set of smart storage units.
- 41. The method of claim 39, wherein the parity information designates the parity scheme.
- 42. A method of recovering file data of a file stored among a set of smart storage units of an intelligent distributed file system, wherein the set of smart storage units includes at least one inactive device and a set of active devices, the method comprising:
receiving information about the inactive device; receiving address information for a parity group that corresponds to data on the inactive device; retrieving the parity group; retrieving data from the active devices; performing error correction on the retrieved data using the retrieved data and the parity group to create the data stored on the inactive device; and, storing the created data.
- 43. The method of claim 42, wherein performing error correction includes performing an XOR operation on the data blocks.
- 44. A data structure for storing information about a file stored among a set of smart storage units of an intelligent distributed file system, the data structure comprising:
data address information corresponding to locations in which portions of the file are stored among the smart storage units.
- 45. The data structure of claim 44, further comprising:
parity address information corresponding to locations in which parity data relating to the portions of the file are stored among the smart storage units.
PRIORITY INFORMATION
[0001] This application is a continuation-in-part of U.S. patent application Ser. No. 10/007,003, filed Nov. 9, 2001, entitled “SYSTEMS AND METHODS FOR PROVIDING A DISTRIBUTED FILE SYSTEM UTILIZING METADATA TO TRACK INFORMATION ABOUT DATA STORED THROUGHOUT THE SYSTEM,” which is herein incorporated by reference in its entirety, and which claims the benefit of U.S. Provisional Application No. 60/309,803, filed Aug. 3, 2001 which is herein incorporated by reference in its entirety.
Provisional Applications (1)
|
Number |
Date |
Country |
|
60309803 |
Aug 2001 |
US |
Continuation in Parts (1)
|
Number |
Date |
Country |
Parent |
10007003 |
Nov 2001 |
US |
Child |
10281467 |
Oct 2002 |
US |