Various embodiments of the present invention concern storage-area networks, especially storage routers and Fibre Channel switches suitable for such networks.
In recent years, the rapid growth of the Internet and other computer networks has fueled an equally fantastic growth in the use of computers as everyday communications devices for both individuals and businesses. Such widespread and growing use has led to the generation and accumulation of vast amounts of digital data. This, in turn, has spurred scientists and engineers to develop specialized subsystems, such as storage-area networks, for managing and storing data.
A storage-area network (SAN) is a high-speed subnetwork of shared data-storage devices, such as disk and tape drives. These networks are particularly advantageous not only because they spare other servers in a larger network, such as corporate intranet, from the burden of storing and managing large amounts of data, and thus allow use of these servers for other higher priority uses, but also because they facilitate data consolidation, which allows users to exploit economies of scale. Moreover, a group of storage devices can be logically partitioned into private storage areas, called zones, with the access to each zone restricted to certain users or computer systems.
Some storage-area networks (SANs) are structured so that an end-user or client-computer can access data on one or more target storage devices through a storage router and a separate Fibre Channel switch. (Fibre Channel generally refers to a serial data-transfer architecture and communications standard developed by a consortium of computer and storage-device manufacturers for use with high-speed mass-storage devices and other peripherals, particularly via optical fiber interconnects.) The Fibre Channel (FC) switch converts data received from the storage router to a Fibre Channel-compliant protocol, such as FC-AL (Fibre Channel Arbitrated Loop) standard, and directs the converted data via high-speed electrical or optical fiber lines to the proper target devices.
However, the present inventor has recognized that conventional storage-area networks, such as those that use separate FC switches and storage routers, present increased complexities and costs related to maintaining and restoring a zoning configuration. For example, administrators of these networks may be forced to manually and separately reconfigure the zoning defined in a FC switch in the event of a system failure. This can be problematic because the zoning configuration may be configured by different entities independently of the administrator, making it even more difficult, time-consuming, and costly to perform the restoration.
Accordingly, the present inventor has recognized a need to reduce the cost and complexities associated with maintaining storage-area networks, particularly restoring zoning configurations.
To address this and other needs, the present inventor devised methods, software, and related devices and systems that automatically maintain and synchronize a copy of a zone configuration for at least one FC switch in another device, such an iSCSI storage router, to facilitate restoration of the zone configuration. In one embodiment, the FC switch is integrated onto a common circuit board with the iSCSI storage router. Through these and other features, the exemplary embodiment ultimately reduces the cost and complexities associated with maintaining storage-area networks.
This description, which references and incorporates the above-identified figures and the appended claims, describes one or more specific embodiments of one or more inventions. These embodiments, offered not to limit but only to exemplify and teach the one or more inventions, are shown and described in sufficient detail to enable those skilled in the art to implement or practice the invention. Thus, where appropriate to avoid obscuring the invention, the description may omit certain information known to those of skill in the art.
iSCSI-compliant hosts 110 and 112, which take the exemplary forms of servers or other computer systems, and are substantially identical in this embodiment, include respective features such as displays, user interfaces, processors, memories, operating systems, and host bus adapters.
The exemplary host bus adapters comply with a version of the Internet Small Computer System Interface (iSCSI) protocol for storage of data over an IP network. The host bus adapters facilitate encapsulation and communication of Small Computer System Interface (SCSI) input-output commands using an Internet Protocol, such as TCP/IP. In the exemplary embodiment, each host bus adapter includes a network interface card or a Gibabit Ethernet server card, which includes a Gigabit Ethernet driver. Other embodiments use any IP-capable interfaces, for example, 10/100 Ethernet, wireless local-area network (LAN), Asynchronous Transfer Mode (ATM), etc. In some embodiments, the host bus adapters are virtual in the sense that iSCSI drivers are provided to effectively enable non-iSCSI network interface cards to function as iSCSI-compliant devices. For details regarding a virtual host bus adapter and related software modules, see co-pending and co-owned U.S. patent application Ser. No. 10/143,561 entitled System, Method, and Software for Target ID Binding in a Storage-area Network and U.S. patent application Ser. No. 10/143,456 entitled System, Method, and Software for a Virtual Host Bus Adapter in a Storage-area Network. Both of these applications were filed on May 9, 2002 and are incorporated herein by reference
Management station 114 includes user interfaces 114.1, such as a graphical-user interface and/or a command-line interface (neither shown separately), that facilitate monitoring, configuring, and control of router-switches 130 and 132.
Integrated storage router-switches 130 and 132, which communicate with hosts 110 and 112 and management station 114 via IP network 120, are substantially identical in the exemplary embodiment. Router-switch 130 includes GE ports 130.1 and 130.2, a management (M) port 130.3, a high-availability (HA) port 130.4, and a FC port set 130.5. Router-switch 132 similarly includes GE ports 132.1 and 132.2, a management (M) port 132.3, a high-availability (HA) port 132.4, and a FC port set 132.5.
More particularly, GE ports 130.1, 130.2, 132.1 and 132.2 are coupled to IP network 120. M ports 130.3 and 132.3 are coupled to each other, and HA ports 130.4 and 132.4 are also coupled to each other.
FC port set 130.5 include ports F1-F8, which are operatively coupled via router-switch interface 130.6 to GE ports 130.1 and/or 130.2. FC port set 132.5 includes optical ports FA-FH, which are operatively coupled via router-switch interface 132.6 to GE ports 132.1 and/or 132.2.
In the exemplary embodiment, all the Fibre-Channel ports are general ports, which can function as fabric, fabric-looped, or N ports. Router-switch interface 130.6 includes software for making GE ports 132.1 and 132.2 and FC port set 130.5 appear to hosts 110 and 112 as a single integrated device. FC ports in port sets 130.5 and 132.5 are coupled via FC-compliant communication channels 140, for example, optical fiber cables, to FC hosts and devices 150. In the exemplary embodiment, router-switches 130 and 132 conform to a version of the Fibre Channel standard, such as FC-AL standard.
FC hosts and devices 150, which together with router-switches 130 and 132 and channels 140 constitute a FC-compliant storage-area network, includes disk arrays 151 and 152, a tape library 153, and FC hosts 154 and 155. Disk array 151 is accessible via FC ports FJ and FK, which are coupled respectively to ports F1 and FA of router-switches 130 and 132. Disk array 152 is accessible via FC ports FL, FM, FN, FO, which are coupled respectively to ports F2, FB, F3, and FC. (Disk arrays 151 and 152 are RAIDs (Redundant Array of Independent (or Inexpensive) Disks) or JBODs (Just a Bunch of Disks.) Tape library 153 is accessible via FC ports FP and FQ, which are coupled respectively to ports F4 and FD. Each of the target storage devices has an associated logical unit number (not shown.)
FC host 154 includes FC ports FR and FS, which are coupled respectively to port F5 and FE. And, FC host 155 includes FC ports FT and FU, with port FT coupled to port F6 of switch-router 130 and port FU coupled to port FF of switch-router 132. FC hosts 154 and 155 can access disk arrays 151 and 152 and a tape library 153 through router-switch 130 and/or 132. In the exemplary embodiment, the storage devices are logically organized into one or more zone, such as zones A and B, with components within zone A accessible by iSCSI-compliant host 110 and components within zone B accessible by iSCSI-compliant host 112. As described in further detail below, the router-switches each maintain one or more sets of zone configuration data (not shown in this figure), which are subject to modification by authorized modifiers, such as management station 114, iSCSI-compliant hosts 110, 112, and/or FC hosts 154, 155.
Operation of the router-switch includes system initialization. In the exemplary embodiment, system initialization includes executing start-up diagnostics, starting and initializing applications, initializing the switch, and verifying switch operation. The initialization of the switch and the router portion are synchronized so that the entire system is made operational in a controlled fashion.
After initialization, host 110 or 112 accesses one or more of target storage devices 151, 152, and/or 153 through its operating system. The operating system forwards a block-level input-output command through the respective HBA, which in turn communicates the command and any associated data in an appropriate format (with target and LUN addresses or names) through a TCP/IP socket and IP network 120 to one of GE ports 130.1, 130.2, 132.1 and 132.2. The router portion of switch-router 130 or 132 strips off the TCP/IP and iSCSI headers, maps the logical iSCSI targets to SCSI addresses, adds FC and FCP headers, and routes the logical command and data onto the switch portion that includes FC port set 130.5 (or 132.5). The switch portion then transfers the logical command and data via one of its FC ports and a Fibre Channel link to one or more of FC hosts and devices 150 based on the SCSI addresses.
In response to the command and any associated data, the appropriate target storage device communicates data and/or status information through the appropriate channel and through one of the FC ports of switch-routers 130 and 132. The receiving switch-router formats the data and status according to the iSCSI and TCP/IP protocols and transmits across an IP network to the host bus adapter of the iSCSI host (110 or 112) that initiated the input-output command.
Router portion 210 includes GE ports 211.1 and 211.2, console port 212, management port 213, high-availability (HA) port 214, bridge-and-buffer module 215, software 216, router processor 217, and router-to-switch interface 218. In the exemplary embodiment, router portion 210 complies with a version of the iSCSI protocol, such as draft 08 or later, and incorporates commercially available router technology, such as the 5420 Storage Router from Cisco System, Inc. of San Jose, Calif.
More particularly, GE ports 211.1 and 211.2 couple the router switch to an IP network for access by one or more servers or other computers, such as servers or iSCSI hosts (in
Console port 212 couples to a local control console (not shown). In the exemplary embodiment, this port takes the form of an RS-232 interface.
Management port 213 provides a connection for managing and/or configuring router-switch 110. In the exemplary embodiment, this port takes the form of a 10/100 Ethernet port and is assigned the base MAC address for the router-switch. As such, this port communicates via an IP protocol, for example, TCP/IP.
HA port 214 provides a physical connection for high-availability communication with another router-switch. In the exemplary embodiment, this port takes the form of a 10/100 Ethernet port, and is assigned the base MAC address plus 1. A failover occurs in the exemplary embodiment when: all devices used by the router-switch are inaccessible; the GE port used by the router-switch fails; router-to-switch interface 218 fails; or a switch failure is detected. A switch failure may cause the entire system, that is, the router-switch, to reboot.
Bridge-and-buffer module 215, which is coupled to GE ports 211.1 and 211.2, provides SCSI router services which are compliant with a version of the iSCSI protocol. In the exemplary embodiment, module 215 incorporates a Peripheral Component Interface (PCI) bridge, such as the Galileo GT64260 from Marvell Semiconductor, Inc., of Sunnyvale, Calif. Also module 215 includes a 64-megabyte flash file system, a 1-megabyte boot flash, and a 256-megabyte non-volatile FLASH memory (not shown separately.) In addition to data and other software used for conventional router operations, module 215, in this exemplary embodiment, includes router-switch interface software 216.
Software 216 performs iSCSI routing between servers and the storage devices. In the exemplary embodiment, the software includes not only an integrated router-switch command-line-interface module CLI and a web-based graphical-user-interface module GUI which facilitate operation, configuration, administration, maintenance, and support of the router-switch. Both the command-line interface and the graphical user interface are accessible from a terminal via one or both of the ports 212 and 213.
Additionally, software 216 includes a synchronization module 216.1 which maintains one or more backup copies of zone configuration for the FC switch, detects zone configuration changes, and updates or synchronizes one or more of the backup copies of the zone configuration data, such as backup zone data 216.2. In the exemplary embodiment, backup zone data 216.2 takes the form of XML files in a backup-zone directory structure and includes zone configuration data that specifically concerns switch portion 220 and that is defined or modified through one or more of ports 211.1, 211.2, 212, and 213 (non-storage-side ports). However, in other embodiments, the backup zone data includes zone configuration data for one or more FC switches other than that in switch portion 220 and/or zone configuration data that is defined or modified through one or more non-storage-side ports, but also through one or more storage-side ports, such as the FC ports. Examples of zone configuration data include zone, zoneset, and alias listings or definitions. Generally, a zone definition generally includes ports, FC addresses, device WWN, and aliases; a zoneset includes a listing of zones; and an alias is a convenient grouping of ports, FC addresses, and device WWN. However, some embodiments may restrict the items that may be used or included within a zone, zoneset, or alias. For example, one embodiment may prevent use of FC addresses or ports in an alias or zone definition.
Various embodiments may store the backup zone data in any convenient storage device or location, including, for example, in a volatile or non-volatile memory on router portion 210 or in a non-volatile memory on a management station, such as management station 114 in
Router Processor 217, in the exemplary embodiment, is implemented as a 533-MHz MPC7410 PowerPC from Motorola, Inc. of Schaumburg, Ill. This processor includes 1-megabyte local L2 cache (not shown separately). In the exemplary embodiment, router processor 217 runs a version of the VX Works operating system from WindRiver Systems, Inc. of Alameda, Calif. To support this operating system, the exemplary embodiment also provides means for isolating file allocations tables from other high-use memory areas (such as areas where log and configuration files are written).
Coupled to router processor 217 as well as to bridge-and-buffer module 215 is router-to-switch (RTS) interface 218. RTS interface 218 includes N/NL switch-interface ports 218.1 and 218.2 and management-interface port 218.3.
Switch-interface ports 218.1 and 218.2 are internal FC interfaces through which the router portion conducts input-output (I/O) operations with the switch portion. When a mapping to a FC storage device is created, the router-switch software automatically selects one of the switch-interface port to use when accessing the target device. The internal interfaces are selected at random and evenly on a per-LUN (logical unit number) basis, allowing the router-switch to load-balance between the two FC interfaces. The operational status of these internal FC interfaces is monitored by each active SCSI Router application running on the switch-router. The failure of either of these two interfaces is considered a unit failure, and if the switch-router is part of a cluster, all active SCSI Router applications will fail over to another switch-router in the cluster. Other embodiments allow operations to continue with the remaining switch-interface port. Still other embodiments include more than two switch-interface ports with various allocation algorithms.
In the exemplary embodiment, the N/NL switch-interface ports can each use up to 32 World Wide Port Names (WWPNs). The WWPNs for port 218.1 are computed as 28+virtual port+base MAC address, and the WWPNs for port 218.2 are computed as 29+virtual port+base MAC address. Additionally, management of switch-interface ports 218.1 and 218.2 is hidden from the user. One exception is the WWPN of each internal port. The internal WWPNs are called “initiator” WWPNs. Users who set up access control by WWPN on their FC devices set up the device to allow access to both switch-router initiator WWPNs.
Switch-interface port 218.3 is used to exchange configuration data and get operational information from switch portion 220 through its management-interface port 224. In the exemplary embodiment, switch-interface port 218.3 is an 10/100 Ethernet port. In the exemplary embodiment, this exchange occurs under the control of a switch management Application Program Interface (API) that is part of interface software 216. One example of a suitable commercially available API is from QLogic Corporation of Aliso Viejo, Calif. Ports 218.1, 218.2, and 218.3 are coupled respectively to FC interface ports 221.1 and 221.2 and interface port 224 of switch portion 220.
Switch portion 220, which in the exemplary embodiment incorporates commercially available technology and supports multiple protocols including IP and SCSI, additionally includes internal FC interface ports 221.1 and 221.2, an FC switch 222, external FC ports (or interfaces) 223, a management interface port 224, and a switch processor module 225.
FC interface ports 221.1221.2 are coupled respectively to ports of 218.1 and 218.2 of the router-to-switch interface via copper traces on or within board 230 or internal optical fiber links, thereby forming internal FC links. (In the exemplary embodiment, each external FC interface supports auto-negotiation as either an F or FL port, and the internal FC interfaces are fixed as either an F or FL port.)
FC switch 222, in the exemplary embodiment, incorporates a SANbox2-16 FC switch from QLogic Corporation. This SANbox2 switch includes QLogic's Itasca switch ASIC (application-specific integrated circuit). Among other things, this switch supports Extended Link Service (ELS) frames that contain manufacturer information. FC switch 222 also includes one or more sets of switch configuration data 222.1, which are subject to modification not only via commands from “storage-side” components, such as FC servers, hosts, and controllers, but also via commands from “router-side” components, such as a management station, command-line interface, or graphical user interface. Various embodiments may store these sets of switch configuration data in any convenient storage location, including for example, in one of the target storage devices or in a volatile or non-volatile memory anywhere on switch portion 220.
FC ports 223.1-223.8, which adhere to one or more FC standards or other desirable communications protocols, can be connected as point-to-point links, in a loop or to a switch. For flow control, the exemplary embodiment implements a FC standard that uses a look-ahead, sliding-window scheme, which provides a guaranteed delivery capability. In this scheme, the ports output data in sets of operatively related frames called sequences, with each frame having a header, a checksum, and a maximum length, such as 2148 bytes.
Moreover, the FC ports are auto-discovering and self-configuring and provide 2-Gbps full-duplex, auto-detection for compatibility with 1-Gbps devices. For each external FC port, the exemplary embodiment also supports: Aribtated Loop (AL) Fairness; Interface enable/disable; Linkspeed settable to 1 Gbps, 2 Gbps, or Auto; Multi-Frame Sequence bundling; Private (Translated) Loop mode.
Switch processor module 225 operates the FC switch and includes a switch processor (or controller) 225.1, and associated memory which includes a switch management agent 225.2. In the exemplary embodiment, switch processor 225.1 includes an Intel Pentium processor and a Linux operating system. Additionally, processor 225 has its own software image, initialization process, configuration commands, command-line interface, and graphical user interface (not shown). (In the exemplary embodiment, this command-line interface and graphical-user interface are not exposed to the end user.) A copy of the switch software image for the switch portion is maintained as a tar file 226 in bridge-and-buffer module 215 of router portion 210.
The exemplary router-switch implements a communications method to make router portion 210 and switch portion 220 appear as an integrated device, particularly as viewed through management interface 213. To this end, software 216 includes an SNMP router-management agent and an MIB router handler (not shown). (SNMP denotes the Simple Network Management Protocol, and MIB denotes Management Information Base (MB)). The agent and handler cooperate with counterparts in switch portion 220 (also not shown) to provide integrated management and control of router and switching functions in router-switch 200.
Specifically, the exemplary embodiment implements or supports Fibre Alliance MIB 3.0, which entails loading MIB objects as separate modules, and adding them to a global MIB database that the SNMP router-management agent uses when resolving objects to a particular set of MIB handlers. In the exemplary embodiment, the SNMP router-management agent is based on the Wind River WindNet v1/v2c SNMP Agent code base, and an SNMP switch-management agent is based on the Linux UCD-SNMP code base.
More specifically,
Block 310 shows that the exemplary method begins with router-management agent AGT (in
In block 320, the router-management agent processes the management inquiry, which in the exemplary embodiment takes the form of an SNMP get or get-next request, by generating at least one SNMP management-request object.
In block 330, the router-management agent determines whether the management-request object relates to router or switch functions. If the object concerns router functions or can otherwise be fulfilled by the router-management agent, execution branches to block 340, which entails the router-management agent processing the request object, and then to block 380.
If, at block 330 the request object is determined to concern switch functions or otherwise cannot be fulfilled by the router-management agent, execution branches first to block 350 which entails mapping the request to a switch request and then sending the request object to the switch management agent, second to block 360 which entails the switch-management agent processing the request object, and third to block 370, which entails sending the response to the request back to the router-management agent. In some embodiments, the router-management agent, at block 360, may also perform other scheduled maintenance tasks, such as retrieve all or one or more portions of a set (or sets) of switch configuration data stored on the FC switch.
In the exemplary embodiment, the request object and its response are communicated between the router-management agent and the switch-management agent via the internal Ethernet link defined by ports 218.3 and 224. Also note that in the exemplary embodiment, the FC-management handler will try as many as three times to get a valid response from the switch-management agent, with a timeout of 0.3 seconds per request. If it does not get a valid response after three tries, an SNMP error signal is passed back to the router-management agent and ultimately onto the management station. Additionally, entries for non-user (internal) FC ports are filtered out from any responses passed back to the router-management agent.
The exemplary embodiment uses two data structures to facilitate communications between the FC-management handler and the switch-management agent. For each group of MIB objects, the exemplary embodiment maintains a branch information table, and for each object within a group, there also exists a leaf information object. The exemplary branch information data structure (FcSwSnmpBranchInfo) includes the following fields:
After execution of block 370 as well as block 340, exemplary execution continues with block 380. In block 380, the router-management agent, which may be regarded as a master management agent to the switch- or slave-management agent for the switch, sends the response back to the management station (or other requestor.) The exemplary embodiment sends the response back via management port 213. In some embodiments, the router-management agent may also perform scheduled or event-driven maintenance-related tasks, such as retrieving one or more portions of a source set (or sets) of switch configuration data stored on the switch.
Next, block 390 determines whether there are further requests to be answered. If there are additional requests, execution of the exemplary method branches back to block 310. And, if there are no additional requests execution branches to block 399, which generally represents any other operations.
The exemplary method begins at block 410, which entails detecting a zone change. In the exemplary embodiment, this detection occurs as a result of detecting receipt of a command to create or modify zone data, such as a zone, a zoneset, or an alias, and receiving acknowledgment that the command was successfully executed. (Note that commands that are successfully executed result in changes to the relevant portion of the zone configuration data on the FC switch; commands that fail result in transmission of a error message to the command sender.) A zone change can also be detected independently of any received zone-changing commands by comparing zone data stored on one or more FC switches (or in memories associated with FC switches) to a reference set of zone data, such as backup zone data 216.2. For example, some embodiments schedule regular comparisons of zone data (or portion thereof) to the backup zone data. Still other embodiments perform such comparisons based on a set of events, for example, receipt of zone merge commands. With detection of a zone change, exemplary execution advances to block 420.
Block 420 entails determining whether to update the backup zone data based on the detected change. In the exemplary embodiment, this determination entails determining whether the detected zone change was an authorized type of change and whether the change was made or initiated by an authorized changer. For example, some embodiments treat any zone changes made to FC switch 222 through use of the graphical user interface or command line interface for the router-switch as being authorized types of changes and as being by an authorized changer. In some embodiments, any changes to FC switch 222 made by or through one or more of ports 211.1, 211.2, 212, 213, and 214 are deemed to be authorized changes by an authorized changer. In still other embodiments, changers, such as other FC switches, acting through one or more of the FC ports on switch-router 200 are authorized to make one or more types of changes. For example, in one embodiment, another FC switch acting through one of FC ports 223 on switch-router 200 can clear the zone configuration for FC switch 222, which would in turn result in clearing of the backup zone data. However, other embodiments may be more or less restrictive in defining the sets of authorized changes and changers. For example, some embodiments consider changes made to the zone configuration data for other FC switches or components to be authorized changes.
If the update determination is negative, then execution branches back to block 410 to await detection of another zone change. However, if the update determination is affirmative, execution advances to block 430.
Block 430 updates the backup zone data. In the exemplary embodiment, this update entails updating the backup zone data by appending or overwriting a complete, date-stamped copy of the zone configuration data from FC switch 222 to any existing backup zone data. The copy may be in the form of a date-stamped file added to a backup zone directory structure. In some embodiments, this zone configuration data includes not only the zone configuration data for FC switch 222, but also the (or at least a version of the) zone-configuration data of other FC switches that may be coupled to FC switch 222.
However, other embodiments append a date-stamped copy of zone changes, which when applied to the previous backup zone data, yield complete zone data. This form of update creates a form of audit trail that facilitates restoration of the zone configuration to any previous state represented in the backup zone data.
Still other embodiments append both changes and a complete zone data set to any previous local backup zone data.
Update in some instances, such as one made in response to a clearing of zone configuration data on the switch, entails erasing or deleting the contents of the backup zone configuration. (If the clearing is the result of a command from one of a predetermined set of authorized changers, for example, a clear command issued through the graphical user interface or command line interface for the router-switch, then FC switch 222 will be isolated from the fabric.) (Note that the exemplary embodiment automatically creates an “iscsi” alias in the backup zone data if it contains no zone configuration data. Creation of the “iscsi” alias ensures that both internal Fibre Channel ports are added to a zone and that the IP side of the network does not become isolated from the Fibre Channel devices.) After update of the backup zone data, execution returns to block 410 to await detection of another zone change.
The exemplary restoration method begins at block 510, which entails receipt of a restoration command. In the exemplary embodiment, this command is received from the command-line or graphical user interface on a management station, such as management station 114 in
Block 520 entails isolating the FC switch that is the target of the restoration. In the exemplary embodiment, this entails clearing the current zone configuration data on the FC, using a “clear FC zones” command, which results in isolation of the FC switch. Execution proceeds to block 530.
Block 530 entails writing entries from the backup zone data to the FC switch. In the exemplary embodiment, this entails writing the specified set of backup zone data to the FC switch and updating the backup zone data pursuant to the method outlined in
In furtherance of the art, the inventor has presented various exemplary systems, methods, and software related to creating, maintaining, and synchronizing backup zone configuration data for one or more components in a storage-area network.
The embodiments described above and in the following claims are intended only to illustrate and teach one or more ways of practicing or implementing the present invention, not to restrict its breadth or scope. The actual scope of the invention, which embraces all ways of practicing or implementing the teachings of the invention, is defined only by the following claims and their equivalents.
Number | Name | Date | Kind |
---|---|---|---|
4495617 | Ampulski et al. | Jan 1985 | A |
5390326 | Shah | Feb 1995 | A |
5461608 | Yoshiyama | Oct 1995 | A |
5473599 | Li et al. | Dec 1995 | A |
5535395 | Tipley et al. | Jul 1996 | A |
5544077 | Hershey | Aug 1996 | A |
5579491 | Jeffries et al. | Nov 1996 | A |
5600828 | Johnson et al. | Feb 1997 | A |
5666486 | Alfieri et al. | Sep 1997 | A |
5732206 | Mendel | Mar 1998 | A |
5812821 | Sugi et al. | Sep 1998 | A |
5832299 | Wooten | Nov 1998 | A |
5850573 | Wada | Dec 1998 | A |
5870571 | Duburcq et al. | Feb 1999 | A |
5909544 | Anderson et al. | Jun 1999 | A |
5951683 | Yuuki et al. | Sep 1999 | A |
5991813 | Zarrow | Nov 1999 | A |
5996024 | Blumenau | Nov 1999 | A |
5996027 | Volk et al. | Nov 1999 | A |
6006259 | Adelman et al. | Dec 1999 | A |
6009476 | Flory et al. | Dec 1999 | A |
6009480 | Pleso | Dec 1999 | A |
6018765 | Durana et al. | Jan 2000 | A |
6041381 | Hoese | Mar 2000 | A |
6078957 | Adelman et al. | Jun 2000 | A |
6108300 | Coile et al. | Aug 2000 | A |
6108699 | Moiin | Aug 2000 | A |
6131119 | Fukui | Oct 2000 | A |
6134673 | Chrabaszcz | Oct 2000 | A |
6145019 | Firooz et al. | Nov 2000 | A |
6163855 | Shrivastava et al. | Dec 2000 | A |
6178445 | Dawkins et al. | Jan 2001 | B1 |
6185620 | Weber et al. | Feb 2001 | B1 |
6195687 | Greaves et al. | Feb 2001 | B1 |
6195760 | Chung et al. | Feb 2001 | B1 |
6209023 | Dimitroff et al. | Mar 2001 | B1 |
6219771 | Kikuchi et al. | Apr 2001 | B1 |
6268924 | Koppolu et al. | Jul 2001 | B1 |
6269396 | Shah et al. | Jul 2001 | B1 |
6314526 | Arendt et al. | Nov 2001 | B1 |
6343320 | Fairchild et al. | Jan 2002 | B1 |
6363416 | Naeimi et al. | Mar 2002 | B1 |
6378025 | Getty | Apr 2002 | B1 |
6393583 | Meth et al. | May 2002 | B1 |
6400730 | Latif et al. | Jun 2002 | B1 |
6449652 | Blumenau et al. | Sep 2002 | B1 |
6470382 | Wang et al. | Oct 2002 | B1 |
6470397 | Shah et al. | Oct 2002 | B1 |
6473803 | Stern et al. | Oct 2002 | B1 |
6480901 | Weber et al. | Nov 2002 | B1 |
6484245 | Sanada et al. | Nov 2002 | B1 |
6574755 | Seon | Jun 2003 | B1 |
6591310 | Johnson | Jul 2003 | B1 |
6597956 | Aziz et al. | Jul 2003 | B1 |
6640278 | Nolan et al. | Oct 2003 | B1 |
6654830 | Taylor et al. | Nov 2003 | B1 |
6658459 | Kwan et al. | Dec 2003 | B1 |
6678721 | Bell | Jan 2004 | B1 |
6683883 | Czeiger et al. | Jan 2004 | B1 |
6691244 | Kampe et al. | Feb 2004 | B1 |
6697924 | Swank | Feb 2004 | B2 |
6701449 | Davis et al. | Mar 2004 | B1 |
6718361 | Basani et al. | Apr 2004 | B1 |
6721907 | Earl | Apr 2004 | B2 |
6724757 | Zadikian et al. | Apr 2004 | B1 |
6738854 | Hoese et al. | May 2004 | B2 |
6748550 | McBrearty et al. | Jun 2004 | B2 |
6757291 | Hu | Jun 2004 | B1 |
6760783 | Berry | Jul 2004 | B1 |
6763195 | Willebrand et al. | Jul 2004 | B1 |
6763419 | Hoese et al. | Jul 2004 | B2 |
6766520 | Rieschl et al. | Jul 2004 | B1 |
6771663 | Jha | Aug 2004 | B1 |
6771673 | Baum et al. | Aug 2004 | B1 |
6799316 | Aguilar et al. | Sep 2004 | B1 |
6807581 | Starr et al. | Oct 2004 | B1 |
6823418 | Langendorf et al. | Nov 2004 | B2 |
6856591 | Ma et al. | Feb 2005 | B1 |
6859462 | Mahoney et al. | Feb 2005 | B1 |
6877044 | Lo et al. | Apr 2005 | B2 |
6886171 | MacLeod | Apr 2005 | B2 |
6895461 | Thompson | May 2005 | B1 |
6920491 | Kim | Jul 2005 | B2 |
6938092 | Burns | Aug 2005 | B2 |
6944785 | Gadir et al. | Sep 2005 | B2 |
6980525 | Banks et al. | Dec 2005 | B2 |
7039870 | Takaoka et al. | May 2006 | B2 |
7433300 | Bennett et al. | Oct 2008 | B1 |
20020010750 | Baretzki | Jan 2002 | A1 |
20020042693 | Kampe et al. | Apr 2002 | A1 |
20020049845 | Sreenivasan et al. | Apr 2002 | A1 |
20020055978 | Joon-Bo et al. | May 2002 | A1 |
20020059392 | Ellis | May 2002 | A1 |
20020065872 | Genske et al. | May 2002 | A1 |
20020103943 | Lo et al. | Aug 2002 | A1 |
20020116460 | Treister et al. | Aug 2002 | A1 |
20020126680 | Inagaki et al. | Sep 2002 | A1 |
20020156612 | Schulter et al. | Oct 2002 | A1 |
20020188657 | Traversat et al. | Dec 2002 | A1 |
20020188711 | Meyer et al. | Dec 2002 | A1 |
20020194428 | Green | Dec 2002 | A1 |
20030002492 | Gallagher | Jan 2003 | A1 |
20030005068 | Nickel et al. | Jan 2003 | A1 |
20030018756 | Nguyen et al. | Jan 2003 | A1 |
20030018813 | Antes et al. | Jan 2003 | A1 |
20030018927 | Gadir et al. | Jan 2003 | A1 |
20030058870 | Mizrachi et al. | Mar 2003 | A1 |
20030084209 | Chadalapaka | May 2003 | A1 |
20030097607 | Bessire | May 2003 | A1 |
20030137941 | Kaushik et al. | Jul 2003 | A1 |
20030182455 | Hetzler et al. | Sep 2003 | A1 |
20030208579 | Brady et al. | Nov 2003 | A1 |
20030210686 | Terrell et al. | Nov 2003 | A1 |
20030237016 | Johnson et al. | Dec 2003 | A1 |
20040022256 | Green | Feb 2004 | A1 |
20040024778 | Cheo | Feb 2004 | A1 |
20040064553 | Kjellberg | Apr 2004 | A1 |
20040078599 | Nahum | Apr 2004 | A1 |
20040088508 | Ballard et al. | May 2004 | A1 |
20040141468 | Christensen | Jul 2004 | A1 |
20040233910 | Chen et al. | Nov 2004 | A1 |
20050038836 | Wang | Feb 2005 | A1 |
20050055418 | Blanc et al. | Mar 2005 | A1 |
20050063313 | Nanavati et al. | Mar 2005 | A1 |
20050195736 | Matsuda | Sep 2005 | A1 |
20050216788 | Mani-Meitav et al. | Sep 2005 | A1 |
20050262233 | Alon et al. | Nov 2005 | A1 |
20050268151 | Hunt et al. | Dec 2005 | A1 |
20060117211 | Matsunami et al. | Jun 2006 | A1 |
20060117212 | Meyer et al. | Jun 2006 | A1 |
20060206494 | Ibrahim et al. | Sep 2006 | A1 |
20060265529 | Kuik et al. | Nov 2006 | A1 |