Data storage system including unique block pool manager and applications in tiered storage

Abstract

A data storage system (10) that receives a data set from a software module (18A-D) includes a first tier storage device (240), a second tier storage device (242), a redundancy reducer (21) and a migration engine (28). The first tier storage device (240) has a first effective storage capacity and the second tier storage device (242) can have a second effective storage capacity that is greater than the first effective storage capacity. The redundancy reducer (21) subdivides the data set into a plurality of data blocks (20) and reduces the redundancy of the data blocks (20). The migration engine (28) moves one or more of the data blocks (20) between the first tier storage device (240) and the second tier storage device (242) based on a migration parameter of the data block (20). The first tier storage device (240) can store data in a random access manner and the second tier storage device (242) can store data in a random or sequential access manner. The first tier storage device (240) has a first I/O bandwidth, and the second tier storage device (242) can have a second I/O bandwidth that is less than the first I/O bandwidth. The first tier storage device (240) has a first access time to data, and the second tier storage device (242) can have a second access time to data that is lengthier than the first access time to data. The data storage system (10) can also include a third tier storage device (238) used for retired data blocks (20).

Description

BRIEF DESCRIPTION OF THE DRAWINGS

The invention, together with further advantages thereof, may best be understood by reference to the following description taken in conjunction with the accompanying drawings, taken in conjunction with the accompanying description, in which similar reference characters refer to similar parts, and in which:

FIG. 1 is a simplified block diagram of one embodiment of a data storage system having features of the present invention;

FIG. 2 is a simplified block diagram of another embodiment of the data storage system;

FIG. 3 is a simplified block diagram of yet another embodiment of the data storage system;

FIG. 4 is a schematic diagram illustrating one embodiment of the data storage system including a plurality of different tiers of storage devices;

FIG. 5A is a simplified top view of one embodiment of a form factor for a portion of the data storage system;

FIG. 5B is a simplified side view of the portion of the data storage system illustrated in FIG. 5A;

FIG. 5C is a simplified front view of the portion of the data storage system illustrated in FIG. 5A;

FIG. 6A is a simplified top view of another embodiment of a form factor for a portion of the data storage system;

FIG. 6B is a simplified side view of the portion of the data storage system illustrated in FIG. 6A;

FIG. 6C is a simplified front view of the portion of the data storage system illustrated in FIG. 6A; and

FIG. 7 is a simplified front view of another embodiment of a form factor for a portion of the data storage system.

Claims

1. A data storage system that receives a data set from a software module, the data storage system comprising: a first tier storage device having a first effective storage capacity;a second tier storage device having a second effective storage capacity that is greater than the first effective storage capacity;a redundancy reducer that subdivides the data set into a plurality of data blocks and reduces the redundancy of the data blocks; anda migration engine that moves one of the data blocks between the first tier storage device and the second tier storage device based on a migration parameter of the data block.

2. The data storage system of claim 1 wherein the redundancy reducer includes a hash engine.

3. The data storage system of claim 1 wherein the redundancy reducer includes a lookup engine.

4. The data storage system of claim 1 wherein the migration parameter includes a reference count.

5. The data storage system of claim 1 wherein the migration parameter includes a last access time.

6. The data storage system of claim 1 wherein the migration parameter includes an access rate.

7. The data storage system of claim 1 wherein the migration parameter includes a user-defined access policy.

8. The data storage system of claim 1 further comprising a metadata index that is stored one of the storage devices, wherein the migration engine updates the metadata index in conjunction with movement of one of the data blocks by the migration engine.

9. The data storage system of claim 1 wherein the first tier storage device includes a disk drive that stores at least a portion of the data set prior to reduction of the redundancy of the data blocks by the redundancy reducer.

10. The data storage system of claim 9 wherein the second tier storage device includes a disk drive that stores at least a portion of the data set following reduction of the redundancy of the data blocks by the redundancy reducer.

11. The data storage system of claim 10 further comprising a third tier storage device having a third effective storage capacity that is greater than the second effective storage capacity.

12. The data storage system of claim 11 wherein the migration engine moves one of the data blocks between the second tier storage device and the third tier storage device based at least partially on a migration parameter of the data block.

13. The data storage system of claim 11 wherein the third tier storage device includes a tape library.

14. The data storage system of claim 11 wherein the third tier storage device includes a MAID.

15. The data storage system of claim 9 wherein the second tier storage device includes a removable media that stores at least a portion of the data set following reduction of the redundancy of the data blocks by the redundancy reducer.

16. The data storage system of claim 1 wherein the first tier storage device includes a disk drive that stores at least a portion of the data set following reduction of the redundancy of the data blocks by the redundancy reducer.

17. The data storage system of claim 16 wherein the second tier storage device includes a removable media that stores at least a portion of the data set following reduction of the redundancy of the data blocks by the redundancy reducer.

18. The data storage system of claim 1 wherein the software module is selected from the group consisting of virtual tape, network attached storage, content access storage and a cluster file system.

19. The data storage system of claim 1 wherein the first tier storage device stores data in a random access manner, and the second tier storage device stores data in a sequential access manner.

20. The data storage system of claim 1 wherein the first tier storage device stores data in a random access manner, and the second tier storage device stores data in a random access manner.

21. The data storage system of claim 1 wherein the first tier storage device has a first I/O bandwidth, and the second tier storage device has a second I/O bandwidth that is less than the first I/O bandwidth.

22. The data storage system of claim 1 wherein the first tier storage device has a first access time to data, and the second tier storage device has a second access time to data that is lengthier than the first access time to data.

23. The data storage system of claim 1 wherein the data set includes (i) a first data pool having a plurality of first data blocks, and (ii) a separately-stored second data pool having a plurality of second data blocks, and wherein the migration engine is adapted to merge the first data pool with the second data pool.

24. The data storage system of claim 1 wherein the data set includes a data pool having a plurality of data blocks, and wherein the migration engine is adapted to split the data pool into two separately-stored data pools.

25. A data storage system that receives a data set from a software module, the data storage system comprising: a first tier storage device that stores data in a random access manner;a second tier storage device that stores data in a sequential access manner;a redundancy reducer that subdivides the data set into a plurality of data blocks and reduces the redundancy of the data blocks; anda migration engine that moves one of the data blocks between the first tier storage device and the second tier storage device based on a migration parameter of the data block.

26. The data storage system of claim 25 wherein the redundancy reducer includes a hash engine and a lookup engine.

27. The data storage system of claim 25 wherein the migration parameter includes at least one of a reference count, a last access time, an access rate and a user-defined access policy.

28. The data storage system of claim 25 further comprising a metadata index that is stored one of the storage devices, wherein the migration engine updates the metadata index in conjunction with movement of one of the data blocks by the migration engine.

29. The data storage system of claim 25 wherein the first tier storage device includes a disk drive that stores at least a portion of the data set following reduction of the redundancy of the data blocks by the redundancy reducer, and the second tier storage device includes a removable media that stores at least a portion of the data set following reduction of the redundancy of the data blocks by the redundancy reducer.

30. The data storage system of claim 25 wherein the second tier storage device includes one of a tape library and a MAID.

31. The data storage system of claim 25 wherein the software module is selected from the group consisting of virtual tape, network attached storage, content access storage and a cluster file system.

32. The data storage system of claim.25 wherein the first tier storage device has a first I/O bandwidth, and the second tier storage device has a second I/O bandwidth that is less than the first I/O bandwidth.

33. The data storage system of claim 25 wherein the first tier storage device has a first access time to data, and the second tier storage device has a second access time to data that is lengthier than the first access time to data.

34. A data storage system that receives a data set from a software module, the data storage system comprising: a first tier storage device having a first I/O bandwidth;a second tier storage device having a second I/O bandwidth that is less than the first I/O bandwidth;a redundancy reducer that subdivides the data set into a plurality of data blocks and reduces the redundancy of the data blocks; anda migration engine that moves one of the data blocks between the first tier storage device and the second tier storage device based on a migration parameter of the data block.

35. The data storage system of claim 34 wherein the redundancy reducer includes at least one of a hash engine and a lookup engine.

36. The data storage system of claim 34 wherein the migration parameter includes at least one of a reference count, a last access time, an access rate and a user-defined access policy.

37. The data storage system of claim 34 further comprising a metadata index that is stored one of the storage devices, wherein the migration engine updates the metadata index in conjunction with movement of one of the data blocks by the migration engine.

38. The data storage system of claim 34 wherein the first tier storage device includes a disk drive that stores at least a portion of the data set prior to reduction of the redundancy of the data blocks by the redundancy reducer, and the second tier storage device includes a disk drive that stores at least a portion of the data set following reduction of the redundancy of the data blocks by the redundancy reducer.

39. The data storage system of claim 38 further comprising a third tier storage device having a third I/O bandwidth that is less than the second I/O bandwidth.

40. The data storage system of claim 39 wherein the migration engine moves one of the data blocks between the second tier storage device and the third tier storage device based at least partially on a migration parameter of the data block.

41. The data storage system of claim 40 wherein the third tier storage device includes one of a tape library and a MAID.

42. The data storage system of claim 34 wherein the first tier storage device includes a disk drive that stores at least a portion of the data set following reduction of the redundancy of the data blocks by the redundancy reducer, and the second tier storage device includes a removable media that stores at least a portion of the data set following reduction of the redundancy of the data blocks by the redundancy reducer.

43. The data storage system of claim 34 wherein the software module is selected from the group consisting of virtual tape, network attached storage, content access storage and a cluster file system.

44. The data storage system of claim 34 wherein the first tier storage device has a first access time to data, and the second tier storage device has a second access time to data that is lengthier than the first access time to data.

45. A data storage system that receives a data set from a software module, the data storage system comprising: a first tier storage device having a first access time to data;a second tier storage device having a second access time to data that is lengthier than the first access time to data;a redundancy reducer that subdivides the data set into a plurality of data blocks and reduces the redundancy of the data blocks; anda migration engine that moves one of the data blocks between the first tier storage device and the second tier storage device based on a migration parameter of the data block.

46. The data storage system of claim 45 wherein the redundancy reducer includes at least one of a hash engine and a lookup engine.

47. The data storage system of claim 45 wherein the migration parameter includes at least one of a reference count, a last access time, an access rate and a user-defined access policy.

48. The data storage system of claim 45 further comprising a metadata index that is stored one of the storage devices, wherein the migration engine updates the metadata index in conjunction with movement of one of the data blocks between the storage devices by the migration engine.

49. The data storage system of claim 45 wherein the first tier storage device includes a disk drive that stores at least a portion of the data set prior to reduction of the redundancy of the data blocks by the redundancy reducer, and the second tier storage device includes a disk drive that stores at least a portion of the data set following reduction of the redundancy of the data blocks by the redundancy reducer.

50. The data storage system of claim 49 further comprising a third tier storage device having a third access time to data that is lengthier than the second access time to data.

51. The data storage system of claim 50 wherein the migration engine moves one of the data blocks between the second tier storage device and the third tier storage device based at least partially on a migration parameter of the data block.

52. The data storage system of claim 51 wherein the third tier storage device includes one of a tape library and a MAID.

53. The data storage system of claim 45 wherein the first tier storage device includes a disk drive that stores at least a portion of the data set following reduction of the redundancy of the data blocks by the redundancy reducer, and the second tier storage device includes a removable media that stores at least a portion of the data set following reduction of the redundancy of the data blocks by the redundancy reducer.

54. The data storage system of claim 45 wherein the software module is selected from the group consisting of virtual tape, network attached storage, content access storage and a cluster file system.

55. A data storage system that receives a data set from a software module, the data storage system comprising: a first tier storage device having (i) a first effective storage capacity, (ii) a first I/O bandwidth, and (iii) a first access time to data, the first tier storage device storing data in a random access manner;a second tier storage device having (i) a second effective storage capacity that is greater than the first effective storage capacity, (ii) a second I/O bandwidth that is less than the first I/O bandwidth, and (iii) a second access time to data that is greater than the first access time to data;a third tier storage device having a (i) a third effective storage capacity that is greater than the second effective storage capacity, (ii) a third I/O bandwidth that is less than the second I/O bandwidth, and (iii) a third access time to data that is greater than the second access time to data, the third tier storage device storing data in a sequential access manner;a redundancy reducer that subdivides the data set into a plurality of data blocks and reduces the redundancy of the data blocks;a migration engine that moves one of the data blocks between the first tier storage device, the second tier storage device and the third tier storage device based at least partially on a migration parameter of the data block, the migration parameter including one of a reference count, a last access time and an access rate; anda metadata index that is stored on one of the storage devices, wherein the migration engine updates the metadata index in conjunction with movement of one of the data blocks by the migration engine.