Patent Grant
7277986
The invention relates generally to data processing systems employing hard drive storage technology. More particularly, the invention relates to a sector-edge cache for improving system performance during write operations to disk.
Low cost has made ATA (Advanced Technology Attachment) disk drives a popular choice for data storage in various data processing systems. The development of the Serial ATA (SATA) protocol has furthered this popularity by providing features such as hot-pluggability, flexible cabling, and CRC (cyclic redundancy check) for all data, command, and status communications. In addition, the storage density of ATA disk drives makes them particularly advantageous for large-capacity storage systems. Accordingly, Redundant Array of Independent Disks (RAID) implementations, which have traditionally employed SCSI or Fibre Channel disk drive technologies, are currently adopting the use of ATA disk drives.
Current ATA disk drives, however, only support a standard-sized sector format of 512 bytes, whereas applications or other software that interact with the disk drives, such as a RAID implementation, can require sectors of a different size. A sector is the smallest unit for which an access can be made to the disk. The difference in sector size requires address translation between the addresses and sectors specified by an application, referred to as logical addresses and logical sectors, and the physical addresses and physical sectors of the disk drive. This sector-size difference also produces a misalignment between logical and physical sectors of the disk drives. For example, if a logical sector is larger (in bytes) than a physical sector, then the logical sector overlaps multiple physical sectors. Consequently, a write operation to a logical sector is, in effect, a write operation to multiple physical sectors, although portions of one or of all of these physical sectors are not being written to. These portions are located at the edges of the overlap between the logical sector and the physical sectors; such edges are referred to as sector edges. This write operation can require an additional read I/O operation to obtain data from the sector edges. Additional read I/O operations of sector edges thus have performance implications for applications that perform small write or large misaligned write operations. To capitalize on the various advantages of ATA technology, therefore, there is a need for a system and method that can avoid or mitigate the performance penalty incurred by reading the sector edges during write operations.
In one aspect, the invention features a method of handling input/output (I/O) requests to a storage disk, wherein the I/O requests use a different sector size format that that of the storage disk. The method comprises receiving an I/O request corresponding to a read operation at a logical address of a virtual disk. A logical sector associated with the logical address is determined and mapped to a portion of a physical sector of the storage disk. A sector edge of the physical sector is determined and stored in a cache memory.
In another aspect, the invention features a data processing system having a storage disk with physical sectors. The data processing system includes a host system executing application software that performs input/output (I/O) operations to a virtual disk. The data processing system also includes an input/output (I/O) controller for issuing I/O requests to the storage disk based on the I/O operations of the application software. The I/O requests use logical sectors having a different sector size format than that of the storage disk. The I/O controller has a sector mapper for mapping logical sectors to physical sectors of the storage disk and for identifying one or more sector edges of physical sectors associated with each logical sector. The I/O controller reads each sector edge associated with a logical sector referenced in an I/O request. A sector-edge cache stores each sector edge read by the I/O controller.
In still another aspect, the invention features an apparatus for performing input/output (I/O) operations to a storage disk. The apparatus includes means for executing application software that performs I/O operations to a virtual disk. The apparatus also includes means for issuing I/O requests to the storage disk based on the I/O operations of the application software, means for mapping logical sectors of the I/O requests to physical sectors of the storage disk. The logical sectors have a different sector size format than that of the storage disk. The apparatus also has means for identifying one or more sector edges of physical sectors associated with each logical sector, means for reading each sector edge associated with a logical sector addressed in an I/O request, and means for caching each sector edge.
The above and further advantages of this invention may be better understood by referring to the following description in conjunction with the accompanying drawings, in which like numerals indicate like structural elements and features in various figures. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention.
The invention features a system and method for avoiding or mitigating a performance penalty associated with pre-reading sector edges from disk, as described herein. Sector edges result when software of a data processing system interacts with physical storage using a differently sized sector format than that of the physical storage. The software issues read and write requests addressed to logical sectors, which are mapped to physical sectors of storage disks. Aligned I/O operations to groups of sectors, called segments, can avert needing to pre-read sector edges; however, small (i.e., smaller in size than one segment) or misaligned write operations require a pre-reading of sector edges so that the pre-read sector edges can be combined with the new data to be written. The combination is then written to disk.
Use of sector-edge caches of the present invention avoids needing to pre-read sector edges for every write operation. A write operation to a logical block may find a previously pre-read sector edge already stored in the cache, thus avoiding the need to perform an I/O operation to obtain the sector edge from the disk. In addition, sector-edge caches can be beneficially combined with a RAID-5 implementation: typically, write operations in a RAID-5 implementation entail a read-modify-write process; thus, sector edges can be read and cached during the read phase of the read-modify-write process. No additional pre-reads are then necessary when combining sector edges with new data during the write phase of the process. The read phase of the read-modify-write process thus absorbs the performance penalty associated with mapping logical sectors to physical sectors.
The host system 14 is in communication with the I/O controller 18 over a host storage interconnect 28 (e.g., a Fibre Channel bus, SCSI bus). The I/O controller 18 implements an I/O subsystem (e.g., a RAID-0, RAID-1, RAID-5 implementation) for controlling access to the disks 26 in response to read and write I/O operations initiated by the host application program 30. Through the I/O controller 18, this host application program 30 interacts with the disks 26 of the disk array 22, which the I/O controller 18 presents to the host application program 30 as one or more virtual disks 32. In addition, the host application program 30 issues I/O requests to the virtual disks 32 and the I/O controller 18 transforms these I/O requests into I/O requests to the physical disks 26 within the disk array 22.
Each disk 26 in the disk array 22 has a plurality of sectors, referred to herein as physical sectors. The physical sectors have a uniform size in bytes. In accordance with the invention, the sectors referenced in I/O requests issued by the I/O controller 18, hereafter referred to as logical sectors, have a different size than that of the physical sectors on the disks 26. (In general, the size difference may be attributable to the use of additional information (e.g., metadata) by the I/O controller in order to implement the functionality of a particular I/O subsystem). For converting logical sectors referenced by the I/O controller 18 into physical sectors of the disks 26, the I/O controller 18 has a sector mapper 34.
A plurality of sector-edge caches 38 resides, in this embodiment, at the I/O controller 18 for storing sector edges. Each sector-edge cache 38 is associated with one disk 26 in the disk array 22 and caches sector edges obtained from that disk 26. In other embodiments, one cache can support multiple disks or all disks in the disk array. The caches 38 are write-through caches. In general, each sector-edge cache 38 is relatively small in size, e.g., 1 Mb of memory, and can be implemented in high-speed memory, e.g., dynamic RAMs, static RAMs, and synchronous dynamic RAMs. An advantage is that the sector-edge cache 38 saves a significant amount of memory by caching just the sector edges and avoiding having to cache an entire I/O request.
The sector edge data in the caches 38 are never dirty (i.e., contain data not already stored on disk) when the cache entry is valid, the sector edge data mirrors the data physically present on the disk from which that data was obtained. In addition to the sector edge data, each cache entry in a sector-edge cache 38 also has other associated data, for example, an address tag (to indicate the physical address), control bits (i.e., to indicate whether the cache entry is valid and when the cache entry was most recently used). During a read operation from disk, the I/O controller 18 adds one or more sector edges to the sector-edge cache 38 or overwrites one or more sector edges currently in the cache 38. During a write operation to disk, the I/O controller 18 may invalidate one or more sector edges currently in the cache to indicate that the data no longer matches the data on the disk(s) 26.
In a preferred embodiment, the data processing system 10 is a networked storage system in which the I/O controller 18 operates as a storage processor, having a Fibre Channel front end and a Serial ATA back end (i.e., disks 26 are ATA disk drives), and provides a RAID-5 implementation. The RAID-5 implementation performs a parity calculation on data written to disk. This parity calculation occurs in real time and is part of the typical read-modify-write process of RAID-5. The written data and parity data are “striped” across the disks 26 in the disk array 22; corresponding sectors on the disks 26 define a stripe. If one disk 26 in the disk array fails, RAID-5 recreates the lost data on the failing disk from the parity data and data blocks on the remaining disks.
For each write request, the RAID-5 system performs multiple I/O operations as part of the read-modify-write process. One I/O operation reads in the data blocks of a stripe to be updated with new data, and another reads the parity block associated with the data blocks. In accordance with the principles of the invention, the read I/O operation also causes the sector edges associated with the data and parity blocks to be cached in the sector-edge cache 38. Then, the parity is modified to reflect the new data. Subsequently, another I/O operation writes the updated parity to the parity block and the new data to the data blocks.
Also, in the preferred embodiment, ATA disk drives have 512-byte physical sectors and the RAID-5 implementation employs 520-byte sectors. Principles of the invention also apply, however, when the logical sectors have fewer bytes than the physical sectors, as described further below. It is to be understood that this preferred embodiment is but one example of the various system configurations in which the invention may be practiced. For example, other levels of RAID, such as RAID-0 and RAID-1, can practice the invention, although not as beneficially as a RAID-5 implementation, because RAID-0 and RAID-1 do not perform read-modify-write processes like RAID-5. Further, when issuing I/O requests to the virtual disks 32, as presented by the I/O controller 18, different embodiments of the host application program 30 can use sectors (which can also be considered logical sectors) having a sector size format that is the same as or differs from that of the physical disks 26.
LBA512=(LBA520*520)/512—this is integer math that discards the remainder.
OFFSET512=(LBA520*520)%512—this is a modulo function.
where LBA512 is the physical sector that contains the start of the data corresponding to the logical sector LBA520, and where OFFSET512 represents the byte offset within physical sector LBA512 that contains the first byte of logical sector LBA520. The form (Sector, Offset) represents the address of the 520-byte sector data in the 512-byte sector address space. To read all of the data for sector LBA520 requires reading the data from physical address (LBA512, OFFSET512) through physical address (LBA512+1, (OFFSET512+520-1)%512). This algorithm applies to single sectors (i.e., not to multiple sector requests).
As shown, the logical sectors 50 includes data blocks labeled block A, block B, and block N; the physical sectors 54 include data blocks labeled block 0, block 1, block 2, and block n. Indicated by ellipses, but not shown, are the other intermediate logical and physical sectors. Hereafter, the data blocks of logical sectors 50 are also called logical blocks and those of physical sectors 54, physical blocks. As an illustrative example, logical block A maps to the entirety of physical block 0 and to a leading portion 58 of physical block 1. The portion 62 of physical block 1 to which the logical block A does not map is referred to as a trailing sector edge because this unmapped portion 62 trails (i.e., follows in address) the mapped leading portion 58. Accordingly, logical block A has an associated trailing sector edge, but no associated leading sector edge.
I/O access of uniformly sized segments that achieve a last-address alignment between a logical sector and a physical sector, e.g., a segment comprised of 64 520-byte logical sectors mapped to 65 512-byte physical sectors, have an advantage of not needing to pre-read sector edges during an aligned write operation to a segment. Another advantage is that such segment sizes fully utilize the storage capacity of the disk 26, in comparison with other segment sizes as described in Table 1 below.
Also shown is an exemplary mapping of 520-byte logical blocks to the 512-byte physical blocks. Three logical blocks, comprised of block A, block B, and block C, map to the four physical blocks: block 0, block 1, block 2, and block 3. The mapping ensures that there is an alignment with the start address of a logical block with the starting address of the first block of each segment (e.g., block A and block 0, block D and block 4). As shown, a segment size of four physical sectors consequently does not produce a last-address alignment between any logical block and physical block. Accordingly, several physical addresses are unusable, as indicated by portion 82. Notwithstanding, the loss of address space may be an acceptable tradeoff in order to benefit from improved write performance (e.g., not having to pre-read sector-edges). Other segment sizes—again, with reference to 520-byte logical sectors and 512-byte physical sectors—also result in unusable physical addresses. For example, segment sizes of 8, 16, and 32 logical blocks also have a degree of lost address space. No address loss occurs when the segment size is an integer multiple of 64 (520-byte) logical blocks (i.e., 65 (512-byte) physical blocks). Table 1 below compares exemplary segment sizes with the percentage of unused storage capacity:
Data blocks are read (step 82) from each identified physical block. At step 86, the I/O controller 18 identifies each sector edge resulting from the logical-to-physical sector mapping. An identified sector edge can be a leading sector edge, trailing sector edge, or both, depending upon the referenced logical sector and the particular mapping of that logical sector to the physical sectors. In accordance with the invention, the I/O controller 18 stores (step 90) each identified sector edge in the sector-edge cache 38 associated with the given disk. If, at the time of caching a sector edge, the sector-edge cache 38 is currently full of valid cache entries, the oldest entry (i.e., least recently used) is removed to make room for the new sector edge. If the sector-edge cache 38 currently contains an entry for the sector edge being cached, in one embodiment that entry is overwritten. Overwriting the cache entry, in effect, changes the aging status of the existing entry for subsequent determinations of a least recently used entry and, if the existing entry is currently invalid, restores a valid status.
With reference also to
If the sector-edge cache 38 has an entry for each sought-for sector edge, and that entry is valid, the I/O controller 18 combines (step 120) a portion of the new data to be written with that sector edge. For instance, any leading sector edge is prepended to and any trailing sector edge is appended to the new data. The combination of each sector edge with an appropriate portion of the new data is then written (step 124) to the physical sector. Thus, the caching of sector edges can avoid the need to pre-read the contents of a physical block in order to perform a write operation to that physical block.
For example, consider a write request to the logical block B of
The writing of new data to one or more physical blocks can cause changes within the sector-edge cache 38. Continuing with the previous example, writing new data to logical block B results in new data being written to physical blocks 1 and 2. Specifically, as shown in
Referring again to
Because a sector edge associated with one logical sector can also hold sector data for associated with another logical sector, read-modify-write operations are atomic so that the sector edges cannot be modified during the write operation. Accordingly, the data processing system 10 of the invention employs a locking strategy that covers physical sectors. Locks on physical sectors are given to I/O requests on a first-come-first-serve basis. In addition, if the data processing system 10 has multiple storage processors that can access the same disk, no more than one storage processor at a time can modify a given physical sector of that disk.
The read-modify-write process of a RAID-5 implementation suits well the process of pre-reading and caching of sector edges. Because a RAID-5 write operation pre-reads data and parity from a disk in order to recalculate parity before the write to disk occurs, this pre-read can also serve to obtain and cache sector edges. Thus, embodiments that combine a RAID-5 implementation with sector-edge caches can absorb the performance penalty associated with pre-reading sector edges during the logical sector-to-physical sector conversion. Further, employing a segment size that achieves no address loss, e.g., sixty-four 520-byte/sixty-five 512-byte segments, can optimize a design of the data processing system 10 based on cost and storage capacity.
Other levels of RAID implementations, e.g. RAID-1, which do not employ a read-modify-write process when performing a write operation, can incur the performance penalty associated with pre-reading sector edges. Notwithstanding, such RAID implementations can benefit during a subsequent write to physical blocks for which sector edges are already cached and currently valid.
As noted above, in some embodiments of the invention, logical sectors can be smaller in bytes than physical sectors. For example,
While the invention has been shown and described with reference to specific preferred embodiments, it should be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the following claims. For instance, the principles of caching sector edges apply generally to any software that interacts with physical storage using a differently sized sector format than that of the physical storage, and not just specifically between the logical sectors as specified by the I/O controller and the physical sectors of the disk.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5522049 | Kimura et al. | May 1996 | A |
| 5813025 | Murphy et al. | Sep 1998 | A |
| 6191712 | Still | Feb 2001 | B1 |
| 6725342 | Coulson | Apr 2004 | B1 |
| 7215496 | Kupferman et al. | May 2007 | B1 |
| 20050036381 | Hassner et al. | Feb 2005 | A1 |
| Number | Date | Country |
|---|---|---|
| 0589664 | Mar 1994 | EP |
| WO0013180 | Mar 2000 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20060224823 A1 | Oct 2006 | US |
