1. Field of the Invention
The present invention relates to storage area networks, and more particularly, to monitoring events in storage area networks.
2. Background of the Invention
Storage Area Networks (“SANs”) provide multiple paths to host computing systems to access memory devices (or storage devices). The multiple paths allow host systems to access data in the event of a failure.
Various standards are used for operating SANs. One such standard is Fibre Channel. Fibre channel is a set of American National Standard Institute (ANSI) standards, which provide a serial transmission protocol for storage and network protocols such as HIPPI, SCSI, IP, ATM and others. Fibre channel provides an input/output interface to meet the requirements of both channel and network users.
Fibre channel supports three different topologies: point-to-point, arbitrated loop and fibre channel fabric. The point-to-point topology attaches two devices directly. The arbitrated loop topology attaches devices in a loop. The fibre channel fabric topology attaches host systems directly to a fabric, which are then connected to multiple devices. The fibre channel fabric topology allows several media types to be interconnected.
Fibre channel is a closed system that relies on multiple ports to exchange information on attributes and characteristics to determine if the ports can operate together. If the ports can work together, they define the criteria under which they communicate.
In fibre channel, a path is established between two nodes where the path's primary task is to transport data from one point to another at high speed with low latency, performing only simple error detection in hardware.
Fibre channel fabric devices include a node port or “N_Port” that manages fabric connections. The N_port establishes a connection to a fabric element (e.g., a switch) having a fabric port or F_port. Fabric elements include the intelligence to handle routing, error detection, recovery, and similar management functions.
A fibre channel switch is a multi-port device where each port manages a simple point-to-point connection between itself and its attached system. Each port can be attached to a server, peripheral, I/O subsystem, bridge, hub, router, or even another switch. A switch receives messages from one port and automatically routes it to another port. Multiple calls or data transfers happen concurrently through the multi-port fibre channel switch.
Fibre channel switches use memory buffers to hold frames received and sent across a network. Associated with these buffers are credits, which are the number of frames a Fibre Channel port can transmit without overflowing the receive buffers at the other end of the link. Receiving an R_RDY primitive signal increases the credit, and sending a frame decreases the credit.
In conventional SAN systems, if an input/output operation is “timed out”, (i.e. it takes longer than say a threshold value) it results in an application time-out. This results in a prolonged recovery operation and reduces the overall availability of data to a user using a host computing system. Hence, in conventional systems, recovery occurs after a failure has already occurred.
As SANs become more complex with higher bandwidth requirements there is a need for detecting failure and re-routing data requests before the actual failure occurs and disrupts information flow within the SAN.
In one embodiment of the present invention, a system for monitoring at least one event and detecting at least one indicator that precedes a failure in a storage area network is provided. The system includes, a switch element having a port-monitoring module that detects when a fibre channel port link indicator varies from a threshold value for the fibre channel port link parameter.
The port-monitoring module sends a message to a performance-monitoring module to process an event when the fibre channel port link indicator varies from the threshold value. The performance-monitoring module may notify a host computer when the fibre channel port link indicator varies from the threshold value, and the fibre channel port whose port link parameter varies from the threshold value may be taken off-line.
In yet another embodiment of the present invention, the system includes a fabric monitoring module that detects when a remote fabric switch and a local fabric switch cannot exchange information and then sends a message to a threshold monitoring module.
The host system is notified that the remote and the local fabric switch cannot exchange information.
In yet another embodiment of the present invention, the system includes a chassis monitoring module that detects when an indicator varies from a threshold value and notifies the threshold monitoring module of the variation.
In yet another embodiment of the present invention, the threshold-monitoring module receives a message from at least one monitoring agent indicating if an indicator varies from a threshold value, and in response to the message, the threshold monitoring module coordinates an event. The threshold values are stored and updated in a threshold table. The monitoring agent includes the Chassis monitoring module, the fabric monitoring module, the port monitoring module, and/or an Nx_port event monitoring module.
The threshold-monitoring module notifies an event response module when an indicator value varies from a threshold value and an event is generated in response to the notification based on an event table.
In yet another embodiment of the present invention, a method for monitoring at least one event and detecting at least one indicator that precedes a failure in a storage area network is provided. The method includes, monitoring at least one event by using at least one monitoring agent in a fabric switch; comparing an indicator value to a threshold value for the indicator; and notifying a threshold-monitoring module if the parameter value varies from the threshold value.
In one aspect of the present invention, various events are monitored in real time, which can result in the failure of certain SAN components/services. This allows a system to be intelligent and preemptive, which avoids disruption of SANs.
This brief summary has been provided so that the nature of the invention may be understood quickly. A more complete understanding of the invention can be obtained by reference to the following detailed description of the preferred embodiments thereof concerning the attached drawings.
The foregoing features and other features of the present invention will now be described with reference to the drawings of a preferred embodiment. In the drawings, the same components have the same reference numerals. The illustrated embodiment is intended to illustrate, but not to limit the invention. The drawings include the following Figures:
Definitions:
The following definitions are provided as they are typically (but not exclusively) used in the fibre channel environment, implementing the various adaptive aspects of the present invention.
“E-Port”: A fabric expansion port that attaches to another Interconnect port to create an Inter-Switch Link.
“F-Port”: A port to which non-loop N_Ports are attached to a fabric and does not include FL_Ports.
“Fibre channel ANSI Standard”: The standard describes the physical interface, transmission and signaling protocol of a high performance serial link for support of other high level protocols associated with IPI, SCSI, IP, ATM and others.
“FC-1”: Fibre channel transmission protocol, which includes serial encoding, decoding and error control.
“FC-2”: Fibre channel signaling protocol that includes frame structure and byte sequences.
“FC-3”: Defines a set of fibre channel services that are common across plural ports of a node.
“FC-4”: Provides mapping between lower levels of fibre channel, IPI and SCSI command sets, HIPPI data framing, IP and other upper level protocols.
“Fabric”: A system which interconnects various ports attached to it and is capable of routing fibre channel frames by using destination identifiers provided in FC-2 frame headers.
“Fabric Topology”: This is a topology where a device is directly attached to a fibre channel fabric that uses destination identifiers embedded in frame headers to route frames through a fibre channel fabric to a desired destination.
“FL_Port”: A L_Port that is able to perform the function of a F_Port, attached via a link to one or more NL_Ports in an Arbitrated Loop topology.
“Inter-Switch Link”: A Link directly connecting the E_port of one switch to the E_port of another switch.
Port: A general reference to N. Sub.--Port or F.Sub.--Port.
“L_Port”: A port that contains Arbitrated Loop functions associated with the Arbitrated Loop topology.
“N-Port”: A direct fabric attached port.
“NL_Port”: A L_Port that can perform the function of a N_Port.
“Switch”: A fabric element conforming to the Fibre Channel Switch standards.
To facilitate an understanding of the preferred embodiment, the general architecture and operation of a fibre channel system will be described. The specific architecture and operation of the preferred embodiment will then be described with reference to the general architecture of the fibre channel system.
The devices of
The fabric element of the present invention is presently implemented as a single CMOS ASIC, and for this reason the term “fabric element” and ASIC are used interchangeably to refer to the preferred embodiments in this specification. Although
ASIC 20 has 20 ports numbered in
For illustration purposes only, all GL ports are drawn on the same side of ASIC 20 in
Each port GL0–GL19 has transmit and receive connections to switch crossbar 50. One connection is through receive buffer 52, which functions to receive and temporarily hold a frame during a routing operation. The other connection is through a transmit buffer 54.
Switch crossbar 50 includes a number of switch crossbars for handling specific types of data and data flow control information. For illustration purposes only, switch crossbar 50 is shown as a single crossbar. Switch crossbar 50 is a connectionless crossbar (packet switch) of known conventional design, sized to connect 21×21 paths. This is to accommodate 20 GL ports plus a port for connection to a fabric controller, which may be external to ASIC 20.
In the preferred embodiments of switch chasis described herein, the fabric controller is a firmware-programmed microprocessor, also referred to as the input/out processor “IOP”). IOP 66 is shown in
Control register 62 receives and holds control information from IOP 66, so that IOP 66 can change characteristics or operating configuration of ASIC 20 by placing certain control words in register 62. IOP 66 can read status of ASIC 20 by monitoring various codes that are placed in status register 64 by monitoring modules, as discussed below with respect to
Each GL port has an optical-electric converter, designated as OE0–OE19 connected with its SERDES through serial lines, for providing fibre optic input/output connections, as is well known in the high performance switch design. The converters connect to switch channels C0–C19. It is noteworthy that the ports can connect through copper paths or other means instead of optical-electric converters.
It is noteworthy that the present invention is not limited to any particular number of remote switches, host computers, fabric switches or number of ports. For example, although only one remote fabric switch 204 is shown to illustrate the adaptive aspects of the present invention, other remote switches can be used with system 200.
It is noteworthy that some of the components in modules 201–204 have not been shown, as they are well known in the art. For example, a host computer 202 includes a central processing unit, storage devices, random access memory, read-only memory, keyboard, video interfaces and other devices that have not been described.
Fabric switch 201 is of the type discussed above with respect to
Fabric switch 201 is coupled to a host computer 202. I/O Port 205 and 221 provide connectivity between host computer 202 and fabric switch 201. It is noteworthy that plural I/O ports 205 and 221 may be used to couple plural host computing systems.
Fabric Switch 201 is also coupled to storage controller 203 using ports 210 and 225. Once again, plural number of ports 210 and 225 may be used to couple fabric switch 201 to more than one storage controller 203. In one aspect of the present invention, storage controller 203 is a redundant array of independent disks (“RAID”) controller and/or host bus adapter (“HBA”).
RAID controllers control access to plural disks. An HBA is an I/O adapter that sits between a host computer's bus and the fibre channel loop and manages the transfer of information between the two channels. Both the RAID controller and HBAs have various components, for example, a processor, memory, and arbitration modules that allow them to operate with host computers, storage systems, and fabric switches. The present invention is not limited to any particular structure or type of storage controller 203.
Turning now in more detail to fabric switch 201, it includes a Performance Threshold Monitor module 209, which monitors the threshold status and events related to plural monitoring agents (discussed below) and compares the monitored status with threshold values maintained in Threshold Table 214. The monitoring agents include Chassis Monitor 218, Nx_Port Event Monitor 212, Port Monitor 216, and Fabric Monitor 215, which are described below.
Chassis Monitor 218 tracks the temperature of the switch chassis (not shown), the power supply status, and fan (not shown) speed using sensors 219. When a status value exceeds the threshold configured in Threshold Table 214, a control signal (this term as used in the specification includes commands and/or message packets) is sent from Chassis Monitor 218 to Performance Threshold Monitor 209 for further processing of the event.
Performance threshold monitor 209 may send a signal to host computer 202 and/or remote fabric switch 204 when an operational parameter varies from a threshold value in table 214. Host computer 202 may perform a shut down of fabric switch 201 upon such a signal and may select an alternative path for subsequent I/O transfers.
Nx_Port Event Monitor 212 receives events generated by SAN targets such as Storage Controller 203. These events include failed internal path notifications and other internal threshold errors. Nx_Port Event Monitor 212 supports a plurality of instances of I/O Port B1 modules 210 to monitor a plurality of Storage Controller 203 SAN targets.
When an event is received, Nx_Port Event Monitor 212 sends a control signal to Performance Threshold Monitor 209 for further processing of the event. Nx_Port Event Monitor 212 is also coupled to a diagnostic control module 228 that performs diagnostics based on certain events and threshold variations. The diagnostic information may be sent to host computer 202 via I/O port 205.
Storage controller 203 includes an event generator 224 that can issue events/message based on internal errors.
Port Monitor 216 tracks port statistics for all FC ports 217 on Fabric Switch 201. FC ports 217 are used to couple plural devices to switch 201. The statistics monitored including total words transmitted and received, CRC errors, link resets, invalid transmission words, loss of sync and excessive congestion and other link error conditions.
When a status value exceeds the threshold values configured/stored in Threshold Table 214, a control signal is sent from Port Monitor 216 to Performance Threshold Monitor 209 for further processing of the event. In one aspect of the present invention, a port is taken offline and/or a Performance Threshold Monitor 209 notifies host computer 202 of the threshold variation/violation.
Fabric Monitor 215 tracks the operation of Remote Fabric Switch 204 in a SAN. Fabric Monitor 215 issues keep-alive handshake messages through I/O Port C1211 to other switches (for example, switch 204) in the SAN. Fabric Monitor 215 supports a plurality of instances of I/O Port C1 modules 211 to monitor a plurality of Remote Fabric Switch 204 in the SAN. The I/O Port C1 modules 211 may be based on Ethernet or Fibre Channel standards.
When messages/information cannot be exchanged with a given Remote Fabric Switch 204, Fabric Monitor 215 sends a control signal to Performance Threshold Monitor 209. Fabric Monitor 215 also receives Remote Fabric Switch events such as forced fail-over for I/O blade failure and for switches being taken offline for servicing. Fabric Monitor 215 issues control signals to Performance Threshold Monitor 209 for further processing of these events.
In one aspect of the present invention, when a remote fabric switch 204 does not respond, the E_Port connecting to the remote fabric switch 204 is taken off-line. An event command may also be sent to host 202. In case of a partial failure, host computer 202 may take over a portion of the remote switch data path.
Performance Threshold Monitor 209 consolidates the threshold events and control signals sent by the monitoring agents and if the thresholds defined in Threshold Table 214 have been exceeded, a control signal is sent to Event Response Director 208 for further processing.
Event Response Director 208 uses the control signals sent by Performance Threshold Monitor 209 to select an event (routing and handling) as defined in Event Response Table 213. Based on the selected event routing and handling definition, Event Response Director 208 sends control signals to Local Event Handler 207 and External Event Generator 206.
Local Event Handler 207 performs control actions on the local Fabric Switch 201. This includes operations such as taking an E_Port down and forcing new inter-switch routing as well as shutting down the switch in an over-temperature situation.
External Event Generator 206 prepares and sends I/O PathGuard RSCNs (Registered State Change Notice) to registered Host Computers (202) through the I/O Port A1 interface 205. External Event Generator 206 supports a plurality of instances of I/O Port A1 modules 205 to notify a plurality of Host Computers 202. Host Computer I/O Path Failover Controller 222 receives the I/O PathGuard RSCN and uses this event to trigger a path failover whereby SCSI traffic is redirected to be sent over an alternative path.
At the completion of the path failover processing, Host Computer I/O Path Failover Controller 202 sends a control signal back to the Fabric Switch via I/O Port A2221. External Event Controller 206 forwards Host Computer 202 response to Event Response Director 208 to complete the event handling.
Threshold Event Configuration module 220 updates the Event Response Table 213 and the Threshold Table 214 based on user requests using host computer 202. Tables 213 and 214 can be updated based on history of certain failures and any other reliability data that may be useful to predict failures.
Host Computer 202 may initiate diagnostic action via I/O Path Diagnostic Controller 223 by sending a control signal to the Fabric Switch Diagnostic Control module (not shown). The Diagnostic Control may initiate loop back diagnostics and supports diagnostic probes to the attached devices.
Turning in detail to
In step S301, the process compares an events status value with a threshold value for the event. For example, in the case of a Chassis Monitor 218, as described above with respect to
In step S302, the process consolidates the various control signals from plural monitoring agents and then processes the events. Performance Threshold Monitor 209 processes the events, as described above.
In one aspect of the present invention, various events are monitored in real time, which can result in the failure of certain SAN components/services. This allows a system to be intelligent and retroactive, which avoids disruption of SANs.
Although the present invention has been described with reference to specific embodiments, these embodiments are illustrative only and not limiting. For example, the various modules shown in
Number | Name | Date | Kind |
---|---|---|---|
4162375 | Schlichte | Jul 1979 | A |
4425640 | Philip et al. | Jan 1984 | A |
4546468 | Christmas et al. | Oct 1985 | A |
4569043 | Simmons et al. | Feb 1986 | A |
4725835 | Schreiner et al. | Feb 1988 | A |
4821034 | Anderson et al. | Apr 1989 | A |
5144622 | Takiyasu et al. | Sep 1992 | A |
5367520 | Cordell | Nov 1994 | A |
5598541 | Malladi | Jan 1997 | A |
5610745 | Bennett | Mar 1997 | A |
5687172 | Cloonan et al. | Nov 1997 | A |
5748612 | Stoevhase et al. | May 1998 | A |
5818842 | Burwell et al. | Oct 1998 | A |
5894560 | Carmichael et al. | Apr 1999 | A |
5987028 | Yang et al. | Nov 1999 | A |
5999528 | Chow et al. | Dec 1999 | A |
6014383 | McCarty | Jan 2000 | A |
6021128 | Hosoya et al. | Feb 2000 | A |
6047323 | Krause | Apr 2000 | A |
6081512 | Muller et al. | Jun 2000 | A |
6118776 | Berman | Sep 2000 | A |
6128292 | Kim et al. | Oct 2000 | A |
6144668 | Bass et al. | Nov 2000 | A |
6160813 | Banks et al. | Dec 2000 | A |
6292483 | Kerstein | Sep 2001 | B1 |
6308220 | Mathur | Oct 2001 | B1 |
6324181 | Wong et al. | Nov 2001 | B1 |
6330236 | Ofek et al. | Dec 2001 | B1 |
6353612 | Zhu et al. | Mar 2002 | B1 |
6411599 | Blanc et al. | Jun 2002 | B1 |
6424658 | Mathur | Jul 2002 | B1 |
6449274 | Holden et al. | Sep 2002 | B1 |
6597691 | Anderson et al. | Jul 2003 | B1 |
6597777 | Ho | Jul 2003 | B1 |
6697359 | George | Feb 2004 | B1 |
6760302 | Ellinas et al. | Jul 2004 | B1 |
20010038628 | Ofek, et al. | Nov 2001 | A1 |
20030016683 | George et al. | Jan 2003 | A1 |
20030091062 | Lay et al. | May 2003 | A1 |
20030120983 | Vieregge et al. | Jun 2003 | A1 |
20030179748 | George et al. | Sep 2003 | A1 |
20030189935 | Warden et al. | Oct 2003 | A1 |
20040013092 | Betker et al. | Jan 2004 | A1 |
20040013125 | Betker et al. | Jan 2004 | A1 |
20040028038 | Anderson et al. | Feb 2004 | A1 |
20040141521 | George | Jul 2004 | A1 |
Number | Date | Country |
---|---|---|
0649098 | Sep 1994 | EP |
0856969 | Jan 1998 | EP |
WO-9836537 | Aug 1998 | WO |