© 2003-2004 Netcell Corporation. A portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever. 37 CFR §1.71(d).
This invention pertains to digital data storage systems and, more specifically, pertains to improvements in disk array controller technology for digital data storage and retrieval.
Mapping Register
The typical RAID controller for a small computer system includes an interface to a host system and an interface to a drive array.
A physical port is required for the attachment of a mass storage device such as a disk drive to a system. While some interfaces are capable of supporting concurrent data transfers to multiple devices, the physical ports tend to become a bottleneck. For this reason, a high performance RAID controller may have a physical port per mass storage device as shown in
One of the performance benefits of RAID comes from the striping of data across the drives of the array. For example, reading data from four drives at once yields a four times improvement over the transfer rate of a single drive. For the example shown in
The 100 MBPS transfer rate from each of the drives becomes a 400 MBPS transfer rate to the buffer. Dashed box 26 conceptually represents a data path switch described later in detail. The data path switch 26 provides dynamically configurable data paths between logical data ports and physical data ports.
One of the features of the Synchronous Redundant Data Transfers described in U.S. Pat. No. 6,018,778 is that it allows redundant data to be processed “On-The-Fly” as described in U.S. Pat. No. 6,237,052.
The 16-bit wide Bus XOR shown in the figure is equivalent to sixteen XOR gates, each with four inputs. The use of the XOR function is also very symmetrical between the disk read and disk write operations as can be seen in
The preceding paragraphs demonstrate some examples of the various relationships that might exist between a set of logical ports and a set of physical device ports in a RAID Controller. In general, a high performance RAID controller is forced to deal with multiple arrays made up of various sub-groups of the mass storage devices connected to its physical ports. One aspect of the present invention employs a novel mapping register and associated logic to enable software configuration of storage device arrays, and improve performance as further explained below.
In accordance with one embodiment of the invention, a Mapping Register 24, the structure of which is shown in
The Mapping Register fields can be of almost any size. An eight-bit field, for example, would support an array of up to 256 physical ports. In the illustrative embodiment, with only five physical ports, a three bit field is sufficient. The five fields pack nicely into a sixteen bit register with a bit to spare noted by an “r” in the Figures for “reserved”. Any type of non-volatile memory can be used to store the mapping register information.
To demonstrate the use of the Mapping Register, we will briefly revisit each of the configurations described so far. In
Referring again to
Data Path Switch
In the preceding discussion, we have demonstrated that four values loaded into the fields of a Mapping Register may be used to represent all of the possible configurations between four logical data ports, and 1, 2, or 4 drive arrays attached to five physical ports, with or without a redundant drive; and for the arrays with redundant drives, with or without a failed drive. The following will describe how the contents of the Mapping Register is used to configure the hardware blocks and the data paths. The following discussion, in other words, presents the details of a presently preferred implementation of the data path switch 26, and how it is configured by the mapping register contents.
Referring now to
The data path to each of the physical ports may come from any of the four logical data ports, or from the Disk Write XOR. Examples were shown with reference to
Referring now to
At this point, there are two open issues to resolve. In a two-drive array, a given physical port received data from two different logical ports, though on different cycles. Referring back to
In a single-drive array where a single physical port receives data from all four logical ports (See
Global Read & Writes
In accordance with the ATA/ATAPI specifications, sending commands to the drives requires the use of Programmed IO or PIO mode that may be as slow as 600 nS per access for devices that support only PIO Mode 0 and no better than 120 nS per access for devices supporting Mode 4. A single command requires eight or more accesses. If all of the drives have to be commanded sequentially, this time is multiplied by the number of drives and adds considerable latency to the entire process. The commands could be issued concurrently by an independent controller per port, but this adds considerably to the complexity and cost.
When data is striped over an array of drives, portions of a given stripe will be located at the same relative position on each of the drives. This makes the address of the data, the Logical Buffer Address or LBA, the same for each of the drives. As a result, the commands to read a given stripe are identical for all of the drives of the array. And the commands to write a given stripe would be identical as well. This makes it possible for the local processor (e.g. 20 in
As noted earlier, a drive array may consist of a subset of the attached drives. (One of the advantages of the present invention is the ability to easily configure, or reconfigure, the organization of attached drives into defined arrays simply by storing appropriate configuration bytes into the mapping register.) In the case where an array consists of a subset of the attached drives, commands (such as read and write) may only be “broadcast” to the selected subset. Either the drives must be commanded one at a time, or some means must be provided to “mask” the physical data ports not participating in the current array.
Referring to
It may seem that a “global read” does not make any sense as it implies that potentially conflicting data values are returned on a common bus. In the current embodiment, a “global read” causes a read strobe,
The local processor will then read each of the ports one at a time using a different address which does not cause a repeat of the Pn_DIOR# strobe cycle and without changing any of the latched data. These cycles do allow the local processor to fetch the potentially unique values stored in each of the data latches. The Pn_DIOR# cycle which may require up to 600 nS is only executed once. The values latched in each of the ports may be fetched in 15 ns each for a significant time savings over repeating the Pn_DIOR# cycle five times.
The “global read” and “global write” apparatus allows the local processor to send commands to and receive control status from the currently selected array in the minimum possible amount of time. When a different sub-array is selected by loading a new value in the Mapping Register, the control interface updates automatically without other code changes.
Status Ordering
The preceding discussion dealt with generating many of the physical port outputs and showed how they were steered by the Mapping Register. Each of these ports has a number of input signals as well. Once again, associating these signals with logical drives can minimize the software overhead. For example, each of the drives has an interrupt output used to signal the need for service from the controller.
Interrupts ANY and ALL
The selected interrupts from the logical data ports can be logically ANDed 94 and ORed 96 as shown in
Logical Address Mapping
While the bulk of the run-time software takes advantage of global commands and status described above there is still the requirement to access individual devices for initialization and for handling errors within specific devices. For this purpose, each of the physical data ports appears at unique location within the local processor address space. When an access to any of these locations is decoded, the decoded output if remapped according to the contents of the Mapping Register. During initialization, the Mapping Register is loaded with an “identity” pattern, i.e. logical device 0 points to physical port 0, logical device 1 points to physical port 1, etc. This makes the physical ports appear in order starting with first physical port location in the processor's address space. In normal operation the Mapping Register will be loaded with a logical to physical drive map. If an interrupt is then received from logical port 2, the local processor may access the interrupting drive through the unique address space that accessed physical port 2 when the identity map is loaded. This makes the servicing of logical drives independent of the physical data port to which they are attached.
One hardware implementation of the logical addressing feature is shown in
Each of the port select signals is then steered by the PP_Ln values from the Mapping Register. The one-of-eight decoder 104 takes the P2_SEL signals and routes it according to the PP_L2 value from the Mapping Register producing a set of signals of the form L2_P0_CS indicating a chip select from physical port zero from logical port two. The one-of-eight decoders for the other four logical ports are identical (not shown).
Each physical port has a five-input OR gate, for example 106. The OR gate 106 for physical port #2 is shown. It ORs together the five different sources for a chip select to physical port #2. Note that for a single-drive sub-array, the chip select will be asserted by all four logical devices and for a dual drive sub-array, the chip select is asserted by two of the logical devices.
In the foregoing description and in the drawings we illustrated several examples of one type of mapping register; it can be called a logical mapping register. As explained, it provides a field for each logical drive in a defined array, and in that field, a value indicates a corresponding physical port number. In an alternative embodiment, called a physical mapping, a register provides a field for each physical port or attached drive, and in each field, a value indicates a corresponding logical port number. This alternative mapping register is illustrated in the following example.
Assume an array is to be defined for striped data over four drives. Blocks of the stripe width are stored on each of the available drives in a specific sequence. This process is then repeated. For example, the first block of data (as well as the 5th, 9th, etc) is stored on the drive connected to physical port #1. The second block (as well as 6th, 10th, etc) is stored on the drive connected to physical port #2. The third block of data (as well as 7th, 11th, etc) is stored on the drive connected to physical port #4. The first block of data goes on logical drive 0, the second on logical drive 1, the third on logical drive two and the fourth on logical drive 3. The two alternative types of mapping registers for this case are as follows:
Logical Mapping
Physical Mapping:
It will be obvious to those having skill in the art that many changes may be made to the details of the above-described embodiments without departing from the underlying principles of the invention. The scope of the present invention should, therefore, be determined only by the following claims.
This is a continuation of and claims priority from U.S. Provisional Application No. 60/464,892 filed Apr. 21, 2003. Said provisional application is incorporated herein by this reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4514823 | Mendelson et al. | Apr 1985 | A |
| 5003558 | Gregg | Mar 1991 | A |
| 5038320 | Heath et al. | Aug 1991 | A |
| 5072378 | Manka | Dec 1991 | A |
| 5151977 | Fredericks et al. | Sep 1992 | A |
| 5185862 | Casper et al. | Feb 1993 | A |
| 5257391 | DuLac et al. | Oct 1993 | A |
| 5268592 | Bellamy et al. | Dec 1993 | A |
| 5392425 | Elliott et al. | Feb 1995 | A |
| 5428649 | Cecchi | Jun 1995 | A |
| 5471581 | Munier et al. | Nov 1995 | A |
| 5491804 | Heath et al. | Feb 1996 | A |
| 5537567 | Galbraith et al. | Jul 1996 | A |
| 5568629 | Gentry et al. | Oct 1996 | A |
| 5581715 | Verinsky et al. | Dec 1996 | A |
| 5608891 | Mizuno et al. | Mar 1997 | A |
| 5619723 | Jones et al. | Apr 1997 | A |
| 5765186 | Searby | Jun 1998 | A |
| 5771372 | Pham et al. | Jun 1998 | A |
| 5794063 | Favor | Aug 1998 | A |
| 5801859 | Yamamoto | Sep 1998 | A |
| 5845319 | Yorimitsu | Dec 1998 | A |
| 5890014 | Long | Mar 1999 | A |
| 5964866 | Durham et al. | Oct 1999 | A |
| 6018778 | Stolowitz | Jan 2000 | A |
| 6098114 | McDonald et al. | Aug 2000 | A |
| 6098128 | Velez-McCaskey et al. | Aug 2000 | A |
| 6105146 | Tavallaei et al. | Aug 2000 | A |
| 6151641 | Herbert | Nov 2000 | A |
| 6151685 | Li et al. | Nov 2000 | A |
| 6161165 | Solomon et al. | Dec 2000 | A |
| 6237052 | Stolowitz | May 2001 | B1 |
| 6513098 | Allingham | Jan 2003 | B2 |
| 6665773 | McCombs | Dec 2003 | B1 |
| 6772108 | Stolowitz | Aug 2004 | B1 |
| 6886068 | Tomita | Apr 2005 | B2 |
| 6999476 | Lerman et al. | Feb 2006 | B2 |
| 7743275 | Tormasov et al. | Jun 2010 | B1 |
| 7913148 | Stolowitz | Mar 2011 | B2 |
| 8065590 | Stolowitz | Nov 2011 | B2 |
| 8074149 | Stolowitz | Dec 2011 | B2 |
| 20020066050 | Lerman et al. | May 2002 | A1 |
| 20030066010 | Acton | Apr 2003 | A1 |
| 20030200478 | Anderson | Oct 2003 | A1 |
| 20040205269 | Stolowitz | Oct 2004 | A1 |
| 20050223269 | Stolowitz | Oct 2005 | A1 |
| 20100251073 | Stolowitz | Sep 2010 | A1 |
| 20100257401 | Stolowitz | Oct 2010 | A1 |
| Number | Date | Country |
|---|---|---|
| 0 578 139 | Jul 1993 | EP |
| 0578139 | Jan 1994 | EP |
| Number | Date | Country | |
|---|---|---|---|
| 20040264309 A1 | Dec 2004 | US |
| Number | Date | Country | |
|---|---|---|---|
| 60464892 | Apr 2003 | US |