Patent Application
20120192037
The present invention is directed to data storage systems and methods having block group error correction for facilitating file reconstruction and restoration.
With increasing reliance on electronic means of data communication, different models to efficiently and economically store a large amount of data have been proposed. In a traditional networked storage system, a data storage device, such as a hard disk, is associated with a server or a server having a backup server. Access to the data storage device is available only through the server associated with that data storage device. A client processor desiring access to the data storage device would, therefore, access the associated server through the network and the server would access the data storage device as requested by the client. By contrast, in an object-based data storage system, each object-based storage device communicates directly with clients over a network. An example of an object-based storage system is shown in commonly-owned, U.S. Pat. No. 6,985,995, titled “Data File Migration from a Mirrored RAID to a Non-Mirrored XOR-Based RAID Without Rewriting the Data,” incorporated by reference herein in its entirety.
The data on each hard disk is typically stored in “blocks”, each of which contains a number of disk sectors to store the incoming data. In other words, the total physical disk space is divided into “blocks” and “sectors” to store data. However, data stored on disks are subject to various types of storage errors. For example, a catastrophic disk failure may be result in the loss of all, or substantially all, data stored on the disk. Disk errors may also be localized, resulting in the loss of data from isolated areas of the disk, perhaps as small as a single sector. Other read errors may be detected and corrected by the disk reading mechanism, for example, by retrying the operation, and result only in performance degradation.
Data storage systems may have a level of fault tolerance or redundancy to preserve data integrity in the event of one or more disk failures. One group of schemes for fault tolerant data storage is the RAID (Redundant Array of Independent Disks) levels or configurations. A number of RAID levels (e.g., RAID-0, RAID-1, RAID-3, RAID-4, RAID-5, etc.) are designed to provide fault tolerance and redundancy for different data storage applications. RAID-1 employs “mirroring” of data to provide fault tolerance and redundancy. In other words, the contents of each primary disk are mirrored onto a corresponding secondary or mirror disk. The storage mechanism provided by RAID-1 is not the most economical or most efficient fault tolerance scheme. Although RAID-1 storage systems are simple to design and provide 100% redundancy (and, hence, increased reliability) during disk failures, RAID-1 systems substantially increase the storage overhead because of the necessity to mirror everything. Redundancy under RAID-1 may exist at every level of the system—from power supplies to disk drives to cables and storage controllers—to achieve full mirroring and steady availability of data during disk failures.
On the other hand, RAID-5 allows for reduced overhead and higher efficiency, albeit at the expense of increased complexity in the storage controller design and time-consuming data rebuilds when a disk failure occurs. RAID-5 uses the concepts of “parity” and “striping” to provide redundancy and fault tolerance. Simply speaking, “parity” can be thought of as a binary checksum or a single bit of information that the operator can use to tell if all the other corresponding data bits are likely correct. RAID-5 creates blocks of parity, where each bit in a parity block corresponds to the parity of the corresponding data bits in other associated blocks. The parity data is used to reconstruct blocks of data read from a failed disk drive. Furthermore, RAID-5 uses the concept of “striping”, which means that two or more disks store and retrieve data in parallel, thereby accelerating performance of data read and write operations. To achieve striping, the data is stored in different blocks on different drives. A single group of blocks and their corresponding parity block may constitute a single “stripe” within the RAID set. In RAID-5 configuration, the parity blocks are distributed throughout all the disk drives, instead of storing all the parity blocks on a single disk, which is RAID-4.
RAID was originally introduced to handle catastrophic drive failure. After a failure, the complete contents of the failed drive can be rebuilt from the redundant information on the other drives. As drive capacities have increased, nearly doubling in capacity every year, another common error source is the failure to read individual sectors from an otherwise healthy disk. These errors are caused by defects in the recording media or recording faults, and are called “unrecoverable read errors” or “uncorrectable read errors” because the error correction codes on the drive are unable to correct the problem and the read operation fails. Moreover, while disk drive capacities have increased rapidly, the rate of uncorrectable read errors (UREs) has remained constant, at approximately 1 error per 1014 to 1015 bits read. When used in a RAID configuration, the amount of data read from the surviving drives by the rebuild process following a catastrophic drive failure is proportional to the capacity of the lost device. As disk drive capacities increase, the amount of data read from surviving drives increases at roughly the same rate. The implication of these trends is that the chances of encountering a URE during a RAID rebuild is also increasing, at approximately the same rate as drive capacity is increasing. When this occurs, some amount of data in a single failure correcting RAID array (ranging in size from a stripe to the entire array, depending on the implementation of the RAID controller) is irretrievably lost, leading to an indication being returned to the original requester (a user or application) that the data requested is unrecoverable. Such an application/user-visible failure may involve, for example, the interruption of computing service, the need to restore data from back-up copies, and/or the loss of some previously written data.
Various mitigating schemes have been devised for detecting latent errors before they cause an error during a rebuild. The latent errors may include UREs. These mostly revolve around periodically “scrubbing” the disk by attempting to read every sector and correcting any errors that are found, using RAID parity bits. However, these methods are expensive in terms of disk utilization and at best achieve a reduction in the frequency of user/application-visible errors. The lack of an effective technique for correcting the combination of a failed disk drive and a URE during rebuild has led to the industry adoption of two-fault-tolerant RAID schemes. These schemes are known collectively as RAID-6. However, these schemes suffer from common problems. For example, RAID-6 doubles the parity overhead for the array, reducing usable capacity. Moreover, every update to the array requires updating two parity blocks on two different disks, reducing throughput. In addition, the amount of data that must be written to gain the performance advantages of a full stripe write is usually significantly larger to amortize the capacity overhead, which reduces throughput further for workloads that are not purely sequential. Further, as noted above, the reading mechanism may be able to detect and correct some read errors. However, there are a host of read errors that are not detectable or correctable by the reading mechanism.
Hence, it is desirable to construct a mechanism for correcting unrecoverable read errors that does not suffer from the drawbacks characteristic of RAID-6.
In one embodiment, a method for performing error correction on a single physical storage disk is provided. The method includes arranging a plurality of addressable blocks on the single physical storage disk into error correction groups, wherein each error correction group comprises N data blocks and M coding blocks, and for each error correction group; computing, in accordance with the error-correcting code, error-correcting coding data across the N data blocks in the error correction group; and storing the computed error-correcting coding data in the M coding blocks in the error correcting group. The arranging, computing and storing steps are performed by a hardware or software component external to the single physical storage disk. The error-correcting coding data may correspond to XOR-based parity data.
The method may further include receiving an error message if the single physical storage disk is unable to read one or more failed data or coding blocks associated with a given error correction group; in response to the error message, attempting to read a remainder of the data and coding blocks in the given error correction group; and if a sufficient number of the remainder of the data and coding blocks and coding blocks are successfully read, computing a corrected version of the one or more failed data or coding blocks from at least part of the remainder of the data and the coding blocks.
Moreover, the method may further comprise using the corrected version of the one or more failed data or coding blocks to rewrite an unreadable addressable block, optionally to a spare addressable block on the single physical storage disk, thereby repairing a fault associated with the error message.
By way of example, M may equal to one and N may be selected from the group consisting of: 8, 16 and 256. Moreover, the error-correcting coding data may correspond to Reed-Solomon data, and N and M are selected from the group consisting of: N=8 and M=2; N=16 and M=2; N=64 and M=2; and N=256 and M=4.
The method may further include detecting a silent read error by reading, from the disk, data and coding blocks associated with a given error correction group; computing an expected value of the one or more coding blocks from the data blocks read from the disk; and comparing the expected value to the one or more coding blocks read from the disk, wherein a silent read error is identified if the computed value does not match the one or more coding blocks read from the disk. If a silent error is detected, the correct data may be reconstructed from redundant data on other storage disks.
The method may also include storing the K*N data blocks of K error correction groups contiguously, followed by K*M coding blocks associated with said K*N data blocks. For example, K may equal 4, N may equal 8, and exclusive OR (XOR) parity may be used as the error-correcting code.
The method may include logically arranging the N data blocks in each error correction group into a rectangular array having rows and columns, and computing the error correcting code across both the rows and columns of the array. In addition, the method may include interleaving the data blocks and coding blocks from K error correction groups, such that consecutive addressable blocks on the physical disk contain data or coding blocks from different error correction groups. Both the data blocks and coding blocks from each error correction group may be transmitted to a host or client machine which is an end-user of the data represented by the error correction groups. Moreover, M may be determined in accordance with a desired failure tolerance of the error correction groups and an error-correcting code.
In another instance, a method for recovering data from a physical storage device in the event of a read error is provided. The storage device stores data organized in a plurality of correction groups, each correction group comprising a plurality of addressable blocks for storing data and an addressable block for storing error-correcting code coding information corresponding to the data of the plurality of blocks of the correction group. The method comprising includes attempting to read data contents of a selected addressable block of the storage device; if a read error of the physical storage device occurs preventing the selected addressable block from being properly read, then reading the contents of the correction group to which the selected addressable block belongs; and computing correct data of the selected addressable block using the data contents of the remainder of the addressable blocks of the correction group and error-correcting code information of the correction group.
The method may include storing the computed correct data in another addressable block. The method may also include attempting to read the data contents of multiple addressable blocks of the storage device, including the selected addressable block.
In another instance, a method for detecting silent read errors of data stored in a selected addressable block of a physical storage device is provided. The storage device stores data organized in a plurality of correction groups, each correction group comprising a plurality of addressable blocks for storing data and an addressable block for storing error-correcting code coding information corresponding to the stored data of the plurality of blocks of the correction group. The method includes reading data contents of a correction group corresponding to the selected addressable block from the storage device, the data contents including stored data of addressable blocks of the correction group and error-correcting code information of the correction group; computing error-correcting code information using the data of the plurality of addressable blocks of the correction group; comparing the computed error-correcting code information to the error-correcting code information read from storage device; and indicating a silent read error if the computed error-correcting code information does not match the error-correcting code information read from storage device.
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention that together with the description serve to explain the principles of the invention. In the drawings:
Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings. It is to be understood that the figures and descriptions of the present invention included herein illustrate and describe elements that are of particular relevance to the present invention, while eliminating, for purposes of clarity, other elements found in typical data storage systems or networks.
It is worthy to note that any reference in the specification to “one embodiment” or “an embodiment” means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. The appearances of the phrase “in one embodiment” at various places in the specification do not necessarily all refer to the same embodiment.
Embodiments set forth below correspond to examples of object-based data storage implementations of the present invention. However, the various teachings of the present invention can be applied in object-based data storage systems as well as other data storage systems.
The data storage network 10 is implemented via a combination of hardware and software units and generally consists of managers 14, 16, 18, and 22, data storage systems 12, and clients 24, 26. It is noted that
The network 28 may be a LAN (Local Area Network), WAN (Wide Area Network), MAN (Metropolitan Area Network), SAN (Storage Area Network), wireless LAN, or any other suitable data communication network, or combination of networks. The network may be implemented, in whole or in part, using a TCP/IP (Transmission Control Protocol/Internet Protocol) based network (e.g., the Internet). A client 24, 26 may be any computer (e.g., a personal computer or a workstation) coupled to the network 28 and running appropriate operating system software as well as client application software designed for the network 10.
The manager (or server) and client portions of the program code may be written in C, C++, or in any other compiled or interpreted language suitably selected. The client and manager software modules may be designed using standard software tools including, for example, compilers, linkers, assemblers, loaders, bug tracking systems, memory debugging systems, etc.
In one embodiment, the manager software and program codes running on the clients may be designed without knowledge of a specific network topology. In such case, the software routines may be executed in any given network environment, imparting software portability and flexibility in storage system designs. However, it is noted that a given network topology may be considered to optimize the performance of the software applications running on it. This may be achieved without necessarily designing the software exclusively tailored to a particular network configuration.
In some storage networks, a data storage device, such as a hard disk, is associated with a particular server or a particular server having a particular backup server. Thus, access to the data storage device is available only through the server associated with that data storage device. A client processor desiring access to the data storage device would, therefore, access the associated server through the network and the server would access the data storage device as requested by the client.
Alternatively, each data storage system 12 may communicate directly with clients 24, 26 on the network 28, possibly through routers and/or bridges. The data storage systems, clients, managers, etc., may be considered as “nodes” on the network 28. In storage system network 10, no assumption needs to be made about the network topology (as noted hereinbefore) except that each node should be able to contact every other node in the system. The servers (e.g., servers 14, 16, 18, etc.) in the network 28 merely enable and facilitate data transfers between clients and data storage systems, but the servers do not normally implement such transfers.
In one embodiment, the data storage systems 12 themselves support a security model that allows for privacy (i.e., assurance that data cannot be eavesdropped while in flight between a client and a data storage system), authenticity (i.e., assurance of the identity of the sender of a command), and integrity (i.e., assurance that in-flight data cannot be tampered with). This security model may be capability-based. A manager grants a client the right to access the data stored in one or more of the data storage systems by issuing to it a “capability.” Thus, a capability is a token that can be granted to a client by a manager and then presented to a data storage system to authorize service. Clients may not create their own capabilities (this can be assured by using known cryptographic techniques), but rather receive them from managers and pass them along to the data storage systems.
Logically speaking, various system “agents” (i.e., the clients 24, 26, the managers 14, 22 and the data storage systems 12) are independently-operating network entities. Day-to-day services related to individual files and directories are provided by file managers (FM) 14. The file manager 14 may be responsible for all file- and directory-specific states. In this regard, the file manager 14 may create, delete and set attributes on entities (i.e., files or directories) on clients' behalf When clients want to access other entities on the network 28, the file manager performs the semantic portion of the security work—i.e., authenticating the requester and authorizing the access—and issuing capabilities to the clients. File managers 14 may be configured singly (i.e., having a single point of failure) or in failover configurations (e.g., machine B tracking machine A's state and if machine A fails, then taking over the administration of machine A's responsibilities until machine A is restored to service).
The primary responsibility of a storage manager (SM) 16 is the aggregation of data storage systems 12 for performance and fault tolerance. “Aggregate” objects are objects that use data storage systems in parallel and/or in redundant configurations, yielding higher availability of data and/or higher I/O performance. Aggregation is the process of distributing a single data file or file directory over multiple data storage system objects, for purposes of performance (parallel access) and/or fault tolerance (storing redundant information). The aggregation scheme associated with a particular object may optionally be stored as an attribute of that object on a data storage system 12. A system administrator (e.g., a human operator or software) may choose any layout or aggregation scheme for a particular object. The SM 16 may also serve capabilities allowing clients to perform their own I/O to aggregate objects (which allows a direct flow of data between a data storage system 12 and a client). The storage manager 16 may also determine exactly how each object will be laid out—i.e., on what data storage system or systems that object will be stored, whether the object will be mirrored, striped, parity-protected, etc. This distinguishes a “virtual object” from a “physical object”. One virtual object (e.g., a file or a directory object) may be spanned over, for example, three physical objects (i.e., multiple data storage systems 12 or multiple data storage devices of a data storage system 12). In one embodiment, a new file or directory inherits the aggregation scheme of its immediate parent directory, by default. Storage Manager 16 may be allowed to make layout changes for purposes of load or capacity balancing.
The storage manager 16 may also allow clients to perform their own I/O to aggregate objects (which allows a direct flow of data between a data storage system and a client), as well as providing proxy service when needed. As noted earlier, individual files and directories in the file system network 10 may be represented by unique storage systems objects. Manager 16 may also determine exactly how each object will be laid out—i.e., on which data storage system(s) that object will be stored, whether the object will be mirrored, striped, parity-protected, etc. Manager 16 may also provide an interface by which users may express minimum requirements for an object's storage (e.g., “the object must still be accessible after the failure of any one data storage system”).
Each manager may be a separable component in the sense that the manager may be used for other file system configurations or data storage system architectures. In one embodiment, the topology for the system network 10 may include a “file system layer” abstraction and a “storage system layer” abstraction. The files and directories in the system network 10 may be considered to be part of the file system layer, whereas data storage functionality (involving the data storage systems 12) may be considered to be part of the storage system layer. In one topological model, the file system layer may be on top of the storage system layer.
The storage access module (SAM) is a program code module that may be compiled into the managers as well as the clients. The SAM may include an I/O execution engine that implements simple I/O, mirroring, and map retrieval algorithms. The SAM generates and sequences the data storage system-level operations necessary to implement system-level I/O operations, for both simple and aggregate objects. A performance manager 22 may run on a server that is separate from the servers for other managers (as shown, for example, in
Each manager 10 may maintain global parameters, notions of what other managers are operating or have failed, and provides support for up/down state transitions for other managers. A benefit to the present system is that the location information describing at what data storage system 12 (e.g., OSD) or systems the desired data is stored may optionally be located at a plurality of data storage systems in the network. In such an embodiment, a client 30 need only identify one of a plurality of data storage systems 12 containing location information for the desired data to be able to access that data. The data may be returned to the client directly from the data storage systems 12 without passing through a manager.
A further discussion of various managers shown in
The installation of the manager and client software to interact with data storage systems 12 and perform object-based data storage in the file system 10 may be called a “realm.” The realm may vary in size, and the managers and client software may be designed to scale to the desired installation size (large or small). A realm manager 18 is responsible for all realm-global states. That is, all states that are global to a realm state are tracked by realm managers 18. A realm manager 18 maintains global parameters, notions of what other managers are operating or have failed, and provides support for up/down state transitions for other managers. Realm managers 18 keep such information as realm-wide file system configuration, and the identity of the file manager 14 responsible for the root of the realm's file namespace. A state kept by a realm manager may be replicated across all realm managers in the data storage network 10, and may be retrieved by querying any one of those realm managers 18 at any time. Updates to such a state may only proceed when all realm managers that are currently functional agree. The replication of a realm manager's state across all realm managers allows making realm infrastructure services arbitrarily fault tolerant—i.e., any service can be replicated across multiple machines to avoid downtime due to machine crashes.
The realm manager 18 identifies which managers in a network contain the location information for any particular data set. The realm manager assigns a primary manager (from the group of other managers in the data storage network 10) which is responsible for identifying all such mapping needs for each data set. The realm manager also assigns one or more backup managers (also from the group of other managers in the system) that also track and retain the location information for each data set. Thus, upon failure of a primary manager, the realm manager 18 may instruct the client 24, 26 to find the location data for a data set through a backup manager.
Generally, the clients may directly read and write data, and may also directly read metadata. The managers, on the other hand, may directly read and write metadata. Metadata may include, for example, file object attributes as well as directory object contents, group inodes, object inodes, and other information. The managers may create other objects in which they can store additional metadata, but these manager-created objects may not be exposed directly to clients.
In some embodiments, clients may directly access data storage systems 12, rather than going through a server, making I/O operations in the object-based data storage networks 10, 30 different from some other file systems. In one embodiment, prior to accessing any data or metadata, a client must obtain (1) the identity of the data storage system(s) 12 on which the data resides and the object number within the data storage system(s), and (2) a capability valid on the data storage systems(s) allowing the access. Clients may learn of the location of objects by directly reading and parsing directory objects located on the data storage system(s) identified. Clients obtain capabilities by sending explicit requests to file managers 14. The client includes with each such request its authentication information as provided by the local authentication system. The file manager 14 may perform a number of checks (e.g., whether the client is permitted to access the data storage system, whether the client has previously misbehaved or “abused” the system, etc.) prior to granting capabilities. If the checks are successful, the FM 14 may grant requested capabilities to the client, which can then directly access the data storage system in question or a portion thereof. Additional details regarding network communications and interactions, commands and responses thereto, among other information, may be found in U.S. Pat. No. 7,007,047, which is incorporated by reference herein in its entirety.
As noted, the processor 310 manages data storage in the storage devices. In this regard, it may execute routines to receive data and write that data to the storage devices and to read data from the storage devices and output that data to the network or other destination. The processor 310 also perform other storage-related functions, such as providing data regarding storage usage and availability, and creating, storing, updating and deleting metadata related to storage usage and organization, and managing data security.
The storage device(s) 320 may be divided into a plurality of blocks for storing data for a plurality of volumes. For example, the blocks may correspond to sectors of the data storage device(s) 320, such as sectors of one or more storage disks. The volumes may correspond to blocks of the data storage devices directly or indirectly. For example, the volumes may correspond to groups of blocks or a Logical Unit Number (LUN) in a block-based system, to object groups or object partitions of an object-based system, or files or file partitions of a file-based system. The processor 310 manages data storage in the storage devices 320. The processor may, for example, allocate a volume, modify a volume, or delete a volume. Data may be stored in the data storage system 12 according to one of several protocols.
As described herein, an error-correcting code is applied to a group of sectors or logical blocks on a single disk drive addressed by the disk drive interface software (such as, in a RAID controller or software RAID engine, or an OSD software stack on an object-based controller, or a disk device driver in an operating system or system library). This differs from the application of an error-correcting code to a RAID array, since in this case the code is applied over blocks from a single disk, rather than over blocks from multiple disks.
Described generally, the processor 310 operates as a disk device driver to arrange the addressable blocks on the raw storage disk device 320 into error correction groups, each of which may be a group of N data blocks and M coding blocks, and then computes an error-correcting code (such as XOR-based parity) over the N data blocks. The computed error-correcting code coding data is stored in the M additional blocks (coding blocks). N is referred to as the “block group size”. M may be determined based on the error correcting code used and the desired failure tolerance of the block group.
An example is illustrated in
In accordance with the principles described herein, the size of the block group may be selected so as to balance capacity overhead, minimum update size, and the expected frequency and distribution of UREs. Large block groups amortize the overhead cost of the coding blocks over more data blocks, increasing the number of blocks that are usable for user data. However, each block group defines a URE “failure domain”, so the larger the block group, the higher the chances that multiple UREs will occur in the same block group and result in an unrecoverable failure. Moreover, a write that modifies a region of the disk smaller than the block group, or is not aligned on a block group boundary, will impose a read-modify-write style update that is less efficient than simply writing the new data and its coding block(s). For example, when using XOR parity as the error correcting code, writing less data than the full block group may require either reading the old data and parity, or reading the remainder of the block group, in order to compute the new contents of the coding block.
One possible implementation would use XOR (parity) as the error correcting code, with a block group size of 8 (N=8, M=1). Assuming a common disk sector size of 512 bytes, this would lead to a block group of 4 KB and error correcting code overhead of ( 1/9)=11%. These parameters would allow for recovery of any single URE in a group of eight sectors.
In addition to the N=8, M=1 encoding described above, there are other specific encodings that may be useful in common applications. These include:
(1) N=16, M=1, using parity as the error correcting code over 8 KB of data;
(2) N=256, M=1, using parity as the error correcting code parity over 128 KB of data;
(3) N=8, M=2, using Reed-Solomon as the error correcting code over 4 KB of data, which is tolerant of up to 2 failures in this region;
(4) N=16, M=2, using Reed-Solomon as the error correcting code over 8 KB of data, which is tolerant of up to 2 failures in this region;
(5) N=64, M=2, using Reed-Solomon as the error correcting code over 32 KB of data, which is tolerant of up to 2 failures in this region; and
(6) N=256, M=4, using Reed-Solomon as the error correcting code over 128 KB of data, which is tolerant of up to 4 failures in this region.
Any of these encodings may be combined with a block group interleaving technique in order to increase resilience to multiple failures on sequential disk blocks. The illustration of specific encodings should not be interpreted to limit the utility of this invention only to these example parameter values; other parameter values may be used with this algorithm depending on the I/O characteristics of the application and the desired tolerance for media defects.
In addition to unrecoverable read errors, in which the storage device, e.g., disk drive, signals an error rather than returning the requested data, storage devices also occasionally suffer from silent read errors. In a silent read error, the storage device returns data instead of an error status from a READ command, but the data does not match the expected contents. This can be due to a variety of causes, including returning the incorrect block (i.e., block 20 was requested but the drive returned the contents of block 21 instead) and random data corruption inside the storage device data path (e.g., bit-flip errors in the disk drive's cache memory). The error-correcting code described above can also be used to detect (and correct, in some cases) silent read errors.
Analysis of error patterns on real-world storage devices shows that bad blocks are not randomly and uniformly distributed. Instead, it is common for more than one block in the same region of the disk to go bad at the same time. This pattern is due to some of the underlying root causes of unrecoverable read errors (i.e., high-fly writes, particulate contamination, physical defects on the media, etc.) which affect more than one block in a small region of the disk. For this reason, an error-correction method which only allows recovery from a single block error in a contiguous sequence of logical blocks may not adequately address instances of UREs seen in the field.
One solution to this is to use an error-correcting code which can tolerate a larger number of errors in a coding group (i.e., Reed-Solomon coding). However, these codes are often significantly more mathematically complex to compute than single-fault tolerant codes. Another solution is to interleave multiple block groups, such that blocks which are sequential on the disk belong to different block groups. A simple interleaved assignment of blocks to correction groups is shown, by way of example, in
The disadvantage of interleaving block groups is that a write operation which updates sequential blocks will touch different block groups, and require updating multiple coding blocks. In addition, the size and alignment constraints to avoid a read-modify-write cycle are larger. In the non-interleaved case of
One possible implementation uses an interleave factor of 4, a block group size of 8, and XOR parity as the error-correcting code (M=1). This arrangement would allow correcting all sequential runs of four UREs in a group of (8*4)=32 blocks. The minimum write size and alignment to avoid a read-modify-write cycle would be 16 KB, assuming a common 512-byte sector size. As with the non-interleaved example above, the error correcting code overhead is ( 1/9)=11%.
An alternate mechanism to interleaving block groups is to divide the block space up into groups of (N+1)2−1 blocks, and compute both row and column parity in that group of blocks. The example of
In this arrangement, any consecutive run of N UREs in the group can be repaired, as well as two non-consecutive UREs anywhere in the group, and many combinations of 3 or more non-consecutive UREs. Generally the error correcting code overhead is 2N/(N2+2N). In an example implementation where N=8 (i.e., each row and column is 4 KB, assuming 512-byte sectors), the error correcting code overhead would be 20%. A write that touches fewer than N2 data blocks will involve between 1 and N read-modify-write updates to the parity blocks.
The error correcting code may be provided to a client reading data, for the purpose of allowing the client to detect errors between the disk drive and the client. For example, the client may use the error correcting codes to determine whether a network device positioned between the client and the disk drive has corrupted data.
The error-correcting code arrangement and data recovery techniques described herein may be used in conjunction with a RAID-X implementation. For example, a RAID-X implementation may involve distributing data across multiple storage devices, using striping and/or an error-correcting code, such as XOR-based parity or a Reed-Solomon code. Some examples of specific RAID-X formats include RAID-0, RAID-1, RAID-3, RAID-4, RAID-5, RAID-10, RAID-50, etc. In accordance with a RAID-X implementation, the file's data for a given file may be broken-up into separate components (or stripe units), each component may be allocated on a physical storage device with the separate components of the file being stored on different storage devices, and the RAID parity for the file may be computed in accordance with the physical boundaries of the separate components of the file on the different storage devices. Each file can have different RAID parameters (for example, stripe unit size, stripe width, etc.) and can be stored on a different combination of the available storage devices. A file system (implemented, e.g., on manager 10 and client(s) 30 in the example of
As will be appreciated by those skilled in the art that changes could be made to the embodiments described above without departing from the broad inventive concept thereof. It is understood, therefore, that this invention is not limited to the particular embodiments disclosed, but is intended to cover modifications within the spirit and scope of the present invention as defined in the appended claims.
This application is a continuation of copending U.S. application Ser. No. 12/219,323, filed Jul. 18, 2008 which is hereby incorporated herein by reference in its entirety. This application claims the benefit of priority under 35 U.S.C. §119(e) based on U.S. provisional application No. 60/950,433, filed on Jul. 18, 2007, which is incorporated herein in its entirety.
| Number | Date | Country | |
|---|---|---|---|
| 60950433 | Jul 2007 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 12219323 | Jul 2008 | US |
| Child | 13436168 | US |
