Patent Grant
7400510
1. Field of the Invention
The present invention relates to storage systems capable of holding large quantities of information.
2. Background Art
Data storage systems provide long term, low cost storage for data. Many of these systems utilize hard disks for short term storage and removable tapes for long term storage. “Hard disks” refer to disks, magnetic, optical or otherwise, that combine data storage media with an access head and at least some support electronics. For simplicity, these will be referred to as simply “disks.” These disks offer shorter latency and higher throughput rates than tapes or removable disks. However, these disks have traditionally been higher in cost per bit.
Recent advances in disk technology has decreased the cost per bit of hard disks. What is needed is a long term storage system that takes advantage of these improvements to create greater efficiencies in storage systems.
The present invention provides a flexible storage system through the use of portable, removable canisters holding multiple storage subsystems.
Referring to
Cover 108 includes a portion of a retention system, shown generally by 112. Shaft 114 is affixed in cover 108 to rotate but not translate. Paddle 116 is affixed to the front portion of shaft 114. The back portion of shaft 114 includes threaded portion 118. Retention system 112 is accessed from the front of module and provides a first mechanical stop at the rear of module 100 when module 100 is inserted into a secondary packaging such as a dock, rack or the like. Thus, retention system 112 avoids damage to rear electrical connectors. Retention system 112 also provides mechanical advantage by threading threaded portion 118 into the secondary package, seating module 100 into the secondary package. Retention system 112 may be accessed by manually turning paddle 116 or through the use of automated equipment rotating paddle 116.
Module 100 includes front face 120. Front face 120 includes module lock 122. Rotating lock 122 moves tab 124, providing retention and/or lock functionality. Lock functionality may be used to affix module 100 within operational packages, transport packages, storage packages, and the like. Module front face 120 also includes vent openings, shown generally by 126, which permit cooling air to escape from module 100. Vent openings 126 seal when module 100 is not in use. Front face 120 also includes user interface 128 which may include a graphical display, identification label, keypad, and the like. Status indicators 130 provide and indication of the operational status and error conditions of module 100. Handle 132 attached to front face 120 permits transport of module 100. Front face 120 may also support identifying markings readable by an operator or by automated equipment.
Module 100 can be labeled independent of each sub-module 102 housed within the canister. The label can be in a multiplicity of synchronized forms, with some formats presenting a subset of the metadata found in other formats. An example would be a physical paper label with volume information being a subset of a radio frequency label having that information plus additional environmental information, which in turn is a subset of the contents information, such as a virtual table of content that is contained in internal solid state memory.
Information such as serial number, manufacturer information, and the like, is part of the information contained at some level of the labeling hierarchy. This information is checked on power up and on insertion of module 100 into any inventory capable system. The label information is a primary means of maintaining data in a physical warehouse of a large number of modules 100.
In an embodiment of the present invention, module 100 will include sixteen 3.5″ disk drives as sub-modules 102. Such a module 100 would be approximately ⅓ rack wide, 4 U high, and 21 in. (53 cm.) long. Disk drives fitting the 2½″ standard may also be used. In addition, sub-modules may have different form factors. Module 100 may be sealed to prevent user access.
Referring now to
Referring now to
Referring now to
Referring now to
Referring now to
The manifold sub-module assembly, including manifolds 106, sub-modules 102 and printed wiring board 180, slides into the back of housing 107. Attaching screws, one of which is indicated by 110, pass through cover 108, manifolds 106 and into housing 107. Handle 132, module lock 122, indicators 130 and user interface 128 attach to front face 120 of housing 107. Protective frame 160 slides over housing 107 and cover 108.
During operation, cooling air enters module 100 through rear vent openings 172 on rear face 170. Air flows along right manifold 188, entering slots in right manifold 188. The air passes between sub-modules 102 before passing through left manifold 190. The air travels forward along left manifold 190, exiting through front vent openings 126 in front face 120. Slots in right manifold 188 and left manifold 190 may be partially or completely blocked with plastic inserts based on the cooling needs of sub-modules 102 near these slots.
Referring now to
Referring now to
Referring now to
Referring now to
Referring now to
Referring now to
Referring now to
Referring now to
In one embodiment, one or more slots of rack 250 is used for power, cooling air, storage controller, network functions, and the like. These functions may also be built into the back, top, sides and/or bottom of rack 250.
Rack 250 may implement a wide variety of system capability. At a high level of capability, rack 250 includes host connections and storage controllers for handling multiple simultaneous high-bandwidth accesses to storage modules 100. A lower level of capability is provided through only one controller and one external connection for rack 250. A still lower level of capability is to provide only enough power, bandwidth and other support to operate one sub-module 102 within any module 100 in rack 250. Another step down in capability is to provide only enough power to maintain internal memory for all modules 100. Further down is to supply only enough power to read information in identification tags for one or a few of modules 100. At the lowest level of capability, rack 250 is nothing more than shelf space for holding modules 100.
Referring now to
Referring now to
Extender 270 may or may not contain modules 100. If extender 270 is designed to accept sub-modules 102, user interface 128 and displays 130 provide input and reflect the status for extender 270 and sub-modules 102 within extender 270 as well as for module 100.
Referring now to
To engage module 100, solenoid 286 extends shaft 288 to hold slotted wheel 284 stationary. Paddle 116 is then rotated, either manually or by automated equipment, to thread portion 118 of shaft 114 into threaded hole 290 of slotted wheel 284. This action seats module 100. During access of module 100, solenoid 286 retracts shaft 288, permitting wheel 284 to rotate. Turning paddle 116 at this point will not unthread shaft 114 from wheel 284, preventing module 100 from being pulled away from back wall 280. Solenoid 286 reengages wheel 284 with shaft 288 for removal of shaft 114 from opening 282. Solenoid 286 may be controlled to prevent removal of module 100 during data transfer between module 100 and secondary packaging.
As will be recognized by one of ordinary skill in the art, other retention systems are possible. For example, a lever arm may be used to provide a latched insertion of canister connectors 174 via a cam mechanism that properly aligns connectors and provides even insertion pressure. The lever arm protrudes from module 100 to first provide a stop to prevent excessive force on connectors 174.
Referring now to
Referring now to
If an active IEEE 1394 connector, in secondary packaging to which module 100 is mounted, is plugged into connector 174, control input 320 routes communication signals 314, 316. If an active IEEE 1394 connector is not plugged into connector 174, control input 320 routes communication signals 314, 316 to adjacent communication signals 314, 316. For the example shown in
Referring now to
The present invention allows a function sophistication nesting to be supported so that costs can be kept to a minimum in the units that are replicated the most with the more expensive technologies being relegated to the sections of the subsystem with the fewest instances of existence. One example is the use of Fire Wire (IEEE 1394) or USB connector for each disk, concentrated into Ethernet for each module 100, with these being concentrated into fiber channel for each controller. Another example is the use of inexpensive processors in module 100 to manage disks that are individually powered up, thus eliminating the need for a processor in each disk drive. The inexpensive processor, in turn, relies on the help of a more powerful processor in rack 250 associated with the slot holding module 100. The powerful processor can be amortized across all modules 100 that can be plugged into that slot.
Module 100 can include interfaces that allow attachment of additional functionality such as, for example, a PMC connector that accepts a processor or memory containing an application to be mounted directly on module 100. This allows for functions like parallel processing of data stores to be executed. An application, such as a parameter driven query, could be multicast and directed to the processing function in each module 100 of a storage system. Then the data could be evaluated in each module 100 in parallel with all the other modules 100 with the results being cast back to the query origination point for compilation.
Each module 100 can include a multilevel hierarchy of storage such as, for example, having a cache for disks of solid state memory or nano devices. Accessing different levels can be made available in different stages. For example, the power to drive a solid state memory, which may contain metadata and index information, may be significantly less than that necessary to drive a disk drive. When module 100 is first inserted into secondary packaging, the solid state memory can be activated long before any disk storage could come ready. This is especially true of technologies that require even less power to access, such as RF chips.
A variety of techniques may be employed, such as RAID 0, 1, or 5 along with single or multiple parity, within a single module 100 for establishing specific performance and reliability levels that may be dynamically altered by demand. The stripe width and the number and style of redundancy components used can be changed dynamically depending on changing circumstances in the operational requirements such as specific demand explicitly requested externally (e.g., for higher reliability), implied demand due to reaching limitations set by a policy engine interface (e.g., wider striping when demand for data reaches a threshold), and the like. In addition, techniques employed within a single module 100 for establishing specific performance and reliability levels may be dynamically altered depending on changing circumstances in the operational environment such as, for example, if a disk drive in module 100 has failed and must be mapped out of use and the data reconstructed. Apparent conflicting requirements such as, for example, performance specification requiring more drives than are available in a single canister, may be resolved by combining the resources of more than one canister. In addition, capacity that has been delegated (and maybe already used) for storing redundancy information may be reallocated and used for data storage, thus extending the ability to respond to immediate data storage requirements. Conversely, a demand for increased data integrity can be met by reallocating storage capacity to redundant information storage.
The specified or implied capacity of a single virtual entity can be altered dynamically (made larger or made smaller) without requiring reconfiguration. If the capacity of a virtual data grouping, such as a virtual volume, is defined beyond the limits of a single module 100, the capacity of additional modules 100 can be concatenated unless specifically prohibited by command or policy. If specifically requested capacity is not used, it can be recovered and allocated to a different virtual entity either specifically or indirectly via policy or by use. Not all modules 100 of an extremely large virtual entity need be active or even present at all times. Some of a virtual volume may be on modules 100 that have been dismounted and physically moved from the system to an archive facility.
Typically, virtual volumes are explicitly created. However virtual volumes may be created by virtue of a mount command. When created in this implicit manner, a default performance, redundancy, and capacity definition can be used that is specified for the enterprise via specific instruction or via policy interfaces. Once created, the above mechanisms for modifying the definition then apply.
The creation of physical groupings of n+p units to meet performance and redundancy quality of service (QoS) demands is supported where the data is written to only one of the n physical devices at a time. The other n−1 devices need not even be present in the system.
Over time, modules 100 can be built with newer-technologies. The newer technologies will significantly improve functionality and cost for capacities. It is expected that after a very few iterations of technology, an existing module 100 will be viewed as outdated and the customer will prefer to retire module 100 in favor of using the newer ones. This will be especially true when the capacity has increased significantly. For example, the initial module 100 might hold 2 TB and a newer generation might hold 20 TB, making use of the old canister inefficient from a capacity stand point. Slots for modules 100 become a focus of resource management and cost management. New module 100 may be temporarily paired with another slot in rack 250 holding new module 100 or another rack 250. Once paired, the data in any module 100 in companion slot is transferred to new module 100 transparently. All writes to virtual volumes that are in the companion module 100 are directed to a space in new module 100, minimizing the need to read all data from the companion module 100. Once the data from old module 100 is fully copied to new module 100, the pairing is broken and old module 100 can be pulled and retired. The copy operation can also delete data from old module 100 and render the space as empty so that old module 100 can be discarded without any concern for compromising data security. This is especially possible if the copy operation uses a new key structure for encryption. This concept also supports the idea of a perpetual store that has the entire history of all data ever written to the subsystem.
Referring now to
Referring now to
Referring now to
Referring now to
While embodiments of the invention have been illustrated and described, it is not intended that these embodiments illustrate and describe all possible forms of the invention. Rather, the words used in the specification are words of description rather than limitation, and it is understood that various changes may be made without departing from the spirit and scope of the invention.
This application claims the benefit of U.S. provisional application Ser. No. 60/451,460 filed Mar. 3, 2003, which is hereby incorporated by reference in its entirety. This application is a continuation of U.S. application Ser. No. 10/791,205 filed Mar. 2, 2004, which is hereby incorporated by reference in its entirety.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4504936 | Faber et al. | Mar 1985 | A |
| 4674004 | Smith et al. | Jun 1987 | A |
| 4754397 | Varaiya et al. | Jun 1988 | A |
| 5063476 | Hamadah et al. | Nov 1991 | A |
| 5077722 | Geist et al. | Dec 1991 | A |
| 5206845 | Baxter et al. | Apr 1993 | A |
| 5237484 | Ferchau et al. | Aug 1993 | A |
| 5274530 | Anderson | Dec 1993 | A |
| 5412661 | Hao et al. | May 1995 | A |
| 5446855 | Dang et al. | Aug 1995 | A |
| 5506750 | Carteau et al. | Apr 1996 | A |
| 5515515 | Kennedy et al. | May 1996 | A |
| 5566316 | Fechner et al. | Oct 1996 | A |
| 5604662 | Anderson et al. | Feb 1997 | A |
| 5615352 | Jacobson et al. | Mar 1997 | A |
| 5634810 | Niitsu et al. | Jun 1997 | A |
| 5680294 | Stora et al. | Oct 1997 | A |
| 5737189 | Kammersgard et al. | Apr 1998 | A |
| 5751549 | Eberhardt et al. | May 1998 | A |
| 5757617 | Sherry | May 1998 | A |
| 5822184 | Rabinovitz | Oct 1998 | A |
| 5832198 | Lucht | Nov 1998 | A |
| 5914858 | McKeen et al. | Jun 1999 | A |
| 5941994 | DeKoning et al. | Aug 1999 | A |
| 6052278 | Tanzer et al. | Apr 2000 | A |
| 6098114 | McDonald et al. | Aug 2000 | A |
| 6138176 | McDonald et al. | Oct 2000 | A |
| 6145028 | Shank et al. | Nov 2000 | A |
| 6166900 | Flynn et al. | Dec 2000 | A |
| 6222699 | Luffel et al. | Apr 2001 | B1 |
| 6272573 | Coale et al. | Aug 2001 | B1 |
| 6288902 | Kim et al. | Sep 2001 | B1 |
| 6301625 | McDonald et al. | Oct 2001 | B1 |
| 6317334 | Abruzzini et al. | Nov 2001 | B1 |
| 6356441 | Claprood | Mar 2002 | B1 |
| 6381139 | Sun | Apr 2002 | B1 |
| 6473297 | Behl et al. | Oct 2002 | B1 |
| 6480391 | Monson et al. | Nov 2002 | B1 |
| 6501660 | Ho et al. | Dec 2002 | B1 |
| 6560098 | Beinor, Jr. et al. | May 2003 | B1 |
| 6775720 | Glynn | Aug 2004 | B1 |
| 20010021099 | Shedow et al. | Sep 2001 | A1 |
| 20040010660 | Konshak et al. | Jan 2004 | A1 |
| Number | Date | Country | |
|---|---|---|---|
| 60451460 | Mar 2003 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 10791205 | Mar 2004 | US |
| Child | 11022613 | US |
