Patent Grant
9268695
1. Field of the Invention
The invention relates generally to storage controllers and more specifically relates to methods and structure for using region lock structures to divert I/O requests to one of multiple I/O processing stacks of the storage controller.
2. Related Patents
This patent is related to the following commonly owned United States patents and/or patent applications (collectively referred to herein as the “Related Patents”):
All of which are hereby incorporated by reference.
3. Discussion of Related Art
A storage controller is a device adapted to receive I/O requests from one or more host systems and to process such received requests to store or retrieve data on storage devices coupled with the storage controller. For example, a RAID (Redundant Array of Independent Drives) storage controller manages one or more logical volumes that each comprises portions of one or more physical storage devices. The RAID controller maps logical block addresses of the logical volumes to corresponding physical blocks of the storage devices and provides redundancy and/or striping of the data for enhanced reliability and/or performance (in accordance with redundancy and striping policies of a particular RAID storage management technique defined for the logical volume).
It is common for high performance storage controllers to utilize a cache memory to enhance performance of the storage controller in processing I/O requests. For example, data written to a logical volume by a host system may be stored in the storage controller's cache memory such that subsequent read I/O requests may be satisfied by retrieving the requested data from the cache memory far more rapidly than by retrieving the requested data from the storage devices that comprise the logical volume. Some storage controllers may manage the cache memory as a “write-through” cache in which data is written to both the cache memory and to the storage devices. Some controllers may manage the cache memory in a “write-back” mode in which data is written to the cache memory and only posted/flushed to the storage devices at some later time (e.g., when the storage controller is less busy processing I/O requests). In the write-back mode, there could be data (referred to as “dirty data”) that resides in the cache memory but is not yet stored on the storage devices.
Some storage controllers (e.g., high-performance RAID storage controllers) provide for multiple processing pathways or stacks within the storage controller for processing I/O requests. For example, some storage controllers from LSI Corporation provides for a “fast path” processing circuit (e.g., a first processing pathway/stack) operable in conjunction with a firmware processing stack (e.g., a second/conventional processing pathway generally implemented as firmware instructions executed by a general or special purpose processor of the controller). The LSI Corporation fast path processing circuit works in conjunction with specialized drivers on the host systems to more rapidly process I/O requests formatted to allow the fast path circuit to rapidly process the request (e.g., the host system enhanced driver understands the RAID mapping and generates fast path requests to directly access the underlying physical storage devices thus allowing the fast path circuit to rapidly process the I/O request). The firmware processing pathway is adapted to process any type of I/O request received from any host system (e.g., for processing I/O requests regardless of its formatting as a fast path I/O request). The firmware stack handles more generalized processing using a general purpose processor executing instructions and providing full RAID mapping and management. Thus, though more flexible, the firmware processing stack processes I/O requests more slowly than does the fast path circuit.
In such storage controllers having multiple processing stacks (e.g., having a fast path circuit pathway and a firmware processing stack), some initial processing by the storage controller is necessary to determine which of the multiple processing stacks is appropriate for processing a particular I/O request. This initial processing can be a complex process depending on a number of factors. For example, if the storage controller utilizes its cache memory in a write-back mode, use of a fast path processing circuit may give rise to data integrity issues if there is dirty data presently in the cache memory of the storage controller that has not yet been flushed or posted to the storage devices. The fast path circuit may be optimally tuned in such a controller so that it is unaware of the contents of the cache memory and thus could write data to the storage devices that could later be overwritten if the dirty data is flushed. Or, the fast path circuit could read data from the storage devices that is not up to date because dirty data (more up to date data) is presently stored in the cache memory for the blocks accessed by the fast path circuit. Further, for example, if the firmware processing stack maintains a list of bad blocks to be avoided in storing or retrieving data from a logical volume or from particular storage devices, the fast path processing circuit may be unaware of the list of bad blocks and may again give rise to data integrity problems by attempting to access the potentially bad blocks. The fast path circuit may attempt to read or write data to such a bad block that only the firmware processing stack is aware may be bad. Other situations may arise where a firmware I/O request processing stack is presently engaged in certain optimizations such as coalescing smaller I/O requests to form full-stripe writes, optimization for streaming I/O requests, etc. These and other reasons may give rise to a preference for one I/O request processor of a storage controller over another despite an encoding of the request by the host to designate or prefer a particular processor.
Accounting for these various conditions in determining which of the multiple processing stacks is best suited for processing an I/O request can be a time-consuming, complex procedure. Thus, present techniques may simply disable the use of one of the multiple processing stacks (e.g., disable use of the fast path I/O processing circuit of LSI Corporation storage controllers) to avoid these and other potential problems if certain conditions may arise. For example, if any dirty data is presently stored in the write-back cache memory (regardless of whether a particular fast path I/O request may access that dirty data) or if any blocks are designated as “bad blocks” by the firmware processing stack (regardless of whether a particular fast path I/O request may access those bad blocks), the firmware processing stack disables the fast path processing circuit and chooses to process all I/O requests through its slower, but more generalized, processing pathway. In a conservative design approach, the LSI fast path I/O processing circuit may simply be disabled to avoid these and other problems arising by processing of an I/O request directed to the fast path circuit. Disabling use of the fast path I/O processing circuit may degrade performance of the storage controller.
Thus it is an ongoing challenge to effectively and efficiently determine which I/O processing stack is best suited for processing a particular I/O request and to divert a received I/O request to the preferred, selected I/O processing stack without risk of data integrity problems.
The present invention solves the above and other problems, thereby advancing the state of the useful arts, by providing methods and structure within a storage controller for using region locks to efficiently divert an I/O request received from an attached host system to one of multiple processing stacks in the controller. A region lock module within the controller allows each processing stack to request a region lock for a range of block addresses of the storage devices. A divert-type lock request may be established to identify a range of block addresses for which I/O requests should be diverted to a particular one of the multiple processing stacks.
In one exemplary embodiment of a controller having a fast path circuit for processing fast path formatted I/O requests and a firmware processing stack for processing any form of I/O request, the firmware processing stack may request that the region lock module establish a divert-type region lock for a range of block addresses. The range of block addresses are associated with dirty data in cache and/or potential bad block addresses. When the fast path circuit attempts to lock a range of blocks to process an I/O request, a divert-type lock that covers any portion of the range of blocks for the I/O request will cause the I/O request to be diverted to the firmware processing stack.
In one aspect hereof, a storage controller is provided that is adapted to couple with one or more host systems and adapted to manage data on one or more storage devices coupled with the storage controller. The storage controller comprises a fast path circuit adapted to process an I/O request formatted for fast path processing received from an attached host system and a firmware processing stack operating on a processor of the controller, the firmware processing stack adapted to process any format of I/O request. The controller further comprises a region lock module coupled with the fast path circuit and coupled with the firmware processing stack. The region lock module is adapted to receive lock request signals to lock a specified range of block addresses and is further adapted to determine whether a received fast path formatted I/O request should be diverted to the firmware processing stack based on the status of presently locked ranges of block addresses. The storage controller is further adapted to divert the fast path formatted I/O request to the firmware processing stack for processing in response to determining that the fast path formatted I/O should be diverted to the firmware processing stack.
Other aspects hereof provide a computer readable medium embodying a method and the method itself. The method is operable in a storage controller adapted to couple with one or more host systems and adapted to couple with one or more storage devices. The storage controller has a fast path circuit for processing of fast path formatted I/O requests received from a host system and has a firmware processing stack for processing I/O requests of any format. The method comprises receiving a fast path formatted I/O request and determining, by operation of a region lock module of the controller, whether the received fast path formatted I/O should be diverted to the firmware processing stack based on the status of presently locked ranges of block addresses. The method then diverts the fast path formatted I/O request to the firmware processing stack for processing in response to the determination that the fast path formatted I/O request should be diverted to the firmware processing stack.
Storage controller 100 comprises fast path circuit 102 and firmware processing stack 104 both adapted to process received I/O requests from an attached host system by accessing storage devices 130. Both circuit 102 and stack 104 may be referred to as processing “stacks”. Thus, controller 100 comprises multiple stacks for processing I/O requests. Further, any number of such processing stacks may be provided in an embodiment of controller 100 as a matter of design choice. Controller 100 is generally operable to divert an I/O request from one processing stack to another based on diversion information comprising region locks as discussed further herein below. Such diversions by region locks may be used to avoid potential corruption of data should one processing stack be incapable of correctly processing an I/O request. The diversion region locks may also be used, for example, to prefer one processing stack over another for purposes of performance enhancements and optimizations.
Firmware processing stack 104 comprises a general or special purpose processor (not shown) and associated program and data memory (not shown) storing programmed instructions and data for execution by the general or special purpose processor. The programmed instructions of firmware processing stack 104 enable stack 104 to receive I/O requests from host system 120 formatted in accordance with any suitable command structure and to process the received I/O request to access information on storage devices 130.
As noted above, although stack 104 provides the flexibility to process any and all I/O requests, including RAID storage management associated with I/O requests directed to a logical volume, stack 104 may be slower than desired for processing of certain types of I/O requests. Thus, storage controller 100 further comprises fast path circuit 102. Fast path circuit 102 comprises an electronic circuit specifically designed to handle I/O requests formatted for processing by the fast path circuit 102 received from an enhanced driver within the system 120. Some storage controllers from LSI Corporation exemplify the use of such a fast path circuit in combination with an enhanced driver module to improve the performance of storage controller 100 in accessing data. In general, an enhanced driver (not shown) of host system 120 has access to mapping information for RAID logical volumes managed by storage controller 100 similar to the mapping information maintained by controller 100 for RAID management. The enhanced driver of host 120 may selectively perform requisite logical to physical mapping more quickly and efficiently than storage controller 100 utilizing firmware processing stack 104. In such cases, the enhanced driver of host system 120 prepares a fast path formatted I/O request utilizing its own processing power for any required logical to physical mapping or other required logical volume management computations. The fast path formatted I/O request is then forwarded to storage controller 100 for processing by fast path circuit 102.
Firmware processing stack 104 is coupled with cache memory 108 and bad block memory 110 by any suitable processor or system bus 154. Each of cache memory 108 and bad block memory 110 may comprise any suitable memory components including, for example, dynamic random access memory (DRAM), flash memory, etc. As is generally known in the art, firmware processing stack 104 utilizes cache memory 108 to store data associated with write I/O requests destined to storage devices 130. In a write-through mode of managing cache memory 108, firmware processing stack 104 stores write data associated with a write I/O request in cache memory 108 and also (essentially simultaneously) stores the write data on storage devices 130. The data so stored in cache memory 108 may then be utilized by firmware processing stack 104 to more rapidly complete subsequent read requests requesting data from block addresses for which data is presently stored in cache memory when awake. In addition, firmware processing stack 104 may identify a particular block address or ranges of block addresses that may be potentially bad blocks. Any suitable techniques within firmware processing stack 104 may be utilized to identify blocks that may be considered “bad”. Firmware processing stack 104 records such block addresses in bad block memory 110 so as to note the blocks for avoidance in its processing of I/O requests. These and other functions within controller 100 may benefit from the divert-type region lock features and aspects hereof. In any situation within controller 100 where requests directed to one request processing stack or circuit should be diverted to a different processing stack or circuit, the divert-type region lock features and aspects hereof provide for making such determination efficiently with minimal overhead processing.
Both fast path circuit 102 and firmware processing stack 104 interact with region lock module 106 of controller 100 (e.g., via communication paths 156 and 158, respectively). Region lock module 106 comprises any suitable circuit for receiving lock requests and region lock release requests from fast path circuit 102 and from firmware processing stack 104. Region lock requests generally identify a range of block addresses for which an I/O request processing stack requires access (i.e., temporary shared access for a read, exclusive access for some operations, etc.). For example, when fast path circuit 102 identifies a range of affected block addresses required for processing a received fast path formatted I/O request, it issues a lock request via path 156 to region lock module 106. Region lock module 106 maintains data structures in an associated region lock memory (not shown) identifying ranges of blocks that are presently locked to either of fast path circuit 102 or firmware processing stack 104. Responsive to a new lock request, region lock module 106 determines whether the range of block addresses identified by the new request overlaps any portion of the ranges of block addresses associated with a presently granted region lock. If such an overlap is detected, region lock module 106 denies the lock or delays granting the lock until such time as the range of block addresses for the new request does not overlap any portion of block addresses associated with a presently granted lock. In like manner, fast path processing circuit 102 and firmware processing stack 104 also issue lock release requests to region lock module 106 when their need for access is completed (i.e., when the associated I/O request has been processed). In some exemplary embodiments, region lock module 106 may also serve as an initial routing component to detect whether a received I/O request is formatted as a fast path I/O request or not so formatted. In such exemplary embodiments, module 106 may be directly coupled with host 120 (through an appropriate communication medium and protocol) while circuit 102 and stack 104 are then coupled directly with module 106 (and thus indirectly with host 120). These and other configuration and arrangements of the elements of controller 100 will be apparent to those of ordinary skill in the art as a matter of design choice.
In accordance with features and aspects hereof, firmware processing stack 104 may issue divert-type lock requests to region lock module 106 (via path 158). A divert-type lock request identifies a range of block addresses for which all I/O requests that may affect the identified a range of block addresses should be diverted to one or another of the multiple I/O request processing stacks (e.g., fast path circuit 102 or firmware processing stack 104) to avoid possible data corruption and/or to permit optimizations for performance enhancement. More specifically, for example, firmware processing stack 104 may manage use of cache memory 108 and bad block of memory 110. As noted above, when firmware processing stack 104 manages cache memory 108 in a write-back mode, write data associated with a received write I/O request may be stored in cache memory 108 and marked as “dirty data”. When firmware processing stack 104 stores such dirty data in cache memory 108, it may also issue a divert-type lock request to region lock module 106 to identify the corresponding range of block addresses for which further I/O requests should be diverted to firmware processing stack 104. When fast path circuit 102 next attempts to lock a range of block addresses for a fast path formatted I/O request, region lock module 106 will detect the overlap of the granted divert-type lock and force diversion of the fast path formatted request to firmware processing stack 104 for further processing. Thus, fast path formatted I/O requests destined to the range of block addresses for which the firmware processing stack has already acquired a divert-type region lock, will be diverted away from fast path circuit 102 for processing by firmware processing stack 104. As noted above, region locks may be employed by firmware processing stack 104 for any number of reasons to divert fast path formatted I/O requests away from fast path circuit 102 for processing by firmware processing stack 104. Besides the avoidance of data corruption as noted above, diversion region locks may be utilized by firmware processing stack 104 to assure proper operation of various optimizations that may be available such as, coalescing of smaller operations into larger operations, load balancing among the I/O processing stacks, streaming I/O optimizations, etc.
Bolded dashed arrows 160 and 162 represent the diversion of a fast path formatted I/O request received by fast path circuit 102 for further processing by firmware processing stack 104. In some exemplary embodiments, region lock module 106 may perform the diversion (as indicated by dashed arrow 162) in response to fast path circuit 102 attempting to lock a range of block addresses before processing a fast path formatted I/O request. In other exemplary embodiments, fast path circuit 102 may request its access lock request for a range of block addresses to be accessed by a fast path formatted I/O request and receive a denial signal from region lock module 106 indicating that the affected range of block addresses should be diverted to firmware processing stack 104 for further processing. In such an exemplary embodiment, fast path circuit 102 may itself perform the processing to divert the fast path formatted I/O request to firmware processing stack 104 as indicated by dashed arrow 160.
In some exemplary embodiments, region lock module 106 maintains information regarding presently granted locks (access locks as well as divert-type locks) in a tree structure to permit rapid searching to locate potential overlapping lock requests. The tree structure entries are organized by ranges of block addresses to allow for rapid searching by region lock module 106. The tree structure may be maintained by suitably designed custom circuitry within region lock module 106 or, in other exemplary embodiments, may be provided by additional custom circuit logic associated with the circuitry of region lock module 106. The Related Patents noted above provide exemplary details for implementation of such a tree structure and for custom circuits adapted to manage such a tree structure. It will be understood that such a tree structure represents one possible embodiment. Numerous other data structures and corresponding control log may be employed to record and retrieve information regarding the locks. For example, a hash table structure may be employed for these purposes.
Exemplary additional details of methods of operation of storage controller 100 as regards the diversion of fast path formatted I/O requests and the associated management of divert-type region locks are discussed herein below with respect to other figures. Those of ordinary skill in the art will readily recognize that divert-type region lock features and aspects hereof may be applicable in any storage controller architecture having at least two processing stacks/processors (e.g., such as LSI Corporation's fast path circuit in conjunction with its firmware/conventional processing stack). Features and aspects hereof may beneficially apply to any such storage controller architecture where a rapid decision is desired to divert a request from one processing stack (to which a request is nominally directed) to another processing stack of the controller. Further, those of ordinary skill in the art will readily recognize numerous additional and equivalent elements that may be present in a fully functional storage controller. Such additional and equivalent elements are omitted herein for simplicity and brevity of this discussion
Following step 408 or if step 406 determines that the new lock request does not overlap a granted divert-type lock, processing continues at step 410 to determine whether any other (e.g., non-divert-type such as an access lock) may overlap the range of block addresses of the received lock request from the fast path circuit. If so, step 412 waits for all such other previously granted locks to be released.
Following step 412 or if step 410 determines that no other such locks are overlapped, processing continues at step 414 to determine whether the newly added entry in the lock tree has been marked as now diverted. If not, step 404 marks the lock entry has now granted to the fast path circuit for the identified range of blocks and signals the fast path circuit of the grant thus allowing the fast path circuit to continue its processing of the fast path formatted I/O request.
Step 414 determines that the new lock entry has been marked as now diverted and step 416 marks the newly added lock entry as granted to the firmware processing stack for the identified range of block addresses. The fast path formatted I/O request is thus diverted to the firmware processing stack for eventual processing in due course. No further processing of the request is performed by the fast path circuit. In other exemplary embodiments, as noted above, the region lock module may simply signal the fast path circuit that its lock request has been denied and should be diverted to the firmware processing stack thus allowing the fast path circuit, per se, to perform the processing to divert the request to the fast firmware processing stack.
Also depicted in the method of
Those of ordinary skill in the art will readily recognize numerous additional and equivalent steps that may be present in fully functional methods such as the methods described with respect to
Embodiments disclosed herein can take the form of software, hardware, firmware, or various combinations thereof. In one particular embodiment, software is used to direct a processing system of a storage device (e.g., a storage controller) to perform the various operations disclosed herein.
For the purposes of this description, computer readable storage medium 912 can be anything that can contain or store the program for use by the computer. Computer readable storage medium 912 can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor device. Examples of computer readable storage medium 912 include a solid state memory (e.g., a flash memory), a magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk, and an optical disk. Current examples of optical disks include compact disk-read only memory (CD-ROM), compact disk-read/write (CD-R/W), and DVD.
Processing system 900, being suitable for storing and/or executing the program code, includes at least one processor 902 coupled to program and data memory 904 and to cache memory subsystem 908 through a system bus 950. Program and data memory 904 can include local memory employed during actual execution of the program code, bulk storage, and cache memories that provide temporary storage of at least some program code and/or data in order to reduce the number of times the code and/or data are retrieved from bulk storage during execution.
Input/output or I/O devices 906 may further be included (e.g., devices for coupling with the storage devices and host systems. Fast path circuit 908 and region lock module 914 may also be integrated with the system to enable processing system 900 to rapidly process fast path formatted I/O requests and to coordinate locking of block addresses as discussed herein above.
While the invention has been illustrated and described in the drawings and foregoing description, such illustration and description is to be considered as exemplary and not restrictive in character. One embodiment of the invention and minor variants thereof have been shown and described. In particular, features shown and described as exemplary software or firmware embodiments may be equivalently implemented as customized logic circuits and vice versa. Protection is desired for all changes and modifications that come within the spirit of the invention. Those skilled in the art will appreciate variations of the above-described embodiments that fall within the scope of the invention. As a result, the invention is not limited to the specific examples and illustrations discussed above, but only by the following claims and their equivalents.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5285528 | Hart | Feb 1994 | A |
| 5761659 | Bertoni | Jun 1998 | A |
| 6032216 | Schmuck et al. | Feb 2000 | A |
| 6574749 | Parsons | Jun 2003 | B1 |
| 6675157 | Mitchell | Jan 2004 | B1 |
| 7082390 | Bergsten | Jul 2006 | B2 |
| 7167960 | Kodama | Jan 2007 | B2 |
| 7260703 | Moore et al. | Aug 2007 | B1 |
| 7269588 | Verma et al. | Sep 2007 | B1 |
| 7290105 | Jeter, Jr. et al. | Oct 2007 | B1 |
| 7305537 | Moore et al. | Dec 2007 | B1 |
| 7370128 | Douglas | May 2008 | B2 |
| 7386692 | Moore et al. | Jun 2008 | B1 |
| 7478179 | Moore et al. | Jan 2009 | B2 |
| 7562200 | Chatterjee et al. | Jul 2009 | B1 |
| 7627744 | Maher et al. | Dec 2009 | B2 |
| 7685462 | Leong | Mar 2010 | B1 |
| 7730222 | Passerini | Jun 2010 | B2 |
| 7734616 | Mogi et al. | Jun 2010 | B2 |
| 7774569 | Chatterjee et al. | Aug 2010 | B1 |
| 7827362 | Passerini | Nov 2010 | B2 |
| 20030135783 | Martin et al. | Jul 2003 | A1 |
| 20030140209 | Testardi | Jul 2003 | A1 |
| 20030200398 | Harris | Oct 2003 | A1 |
| 20050220112 | Williams, Jr. et al. | Oct 2005 | A1 |
| 20050228924 | Marushak et al. | Oct 2005 | A1 |
| 20060047902 | Passerini | Mar 2006 | A1 |
| 20060095658 | Marushak et al. | May 2006 | A1 |
| 20070088928 | Thangaraj et al. | Apr 2007 | A1 |
| 20070136341 | Schopp | Jun 2007 | A1 |
| 20070233684 | Verma et al. | Oct 2007 | A1 |
| 20080024413 | Minami et al. | Jan 2008 | A1 |
| 20080181229 | Williams et al. | Jul 2008 | A1 |
| 20080244136 | Radulescu | Oct 2008 | A1 |
| 20090254774 | Chamdani et al. | Oct 2009 | A1 |
| 20100268904 | Sheffield et al. | Oct 2010 | A1 |
| 20120042101 | Young et al. | Feb 2012 | A1 |
| 20120089753 | Pelis et al. | Apr 2012 | A1 |
| Entry |
|---|
| Brian Randell. 1985. Hardware/software tradeoffs: a general design principel?. SIGARCH Comput. Archit. News 13, 2 (Jun. 1985), 19-21. |
| Number | Date | Country | |
|---|---|---|---|
| 20140164715 A1 | Jun 2014 | US |
