System and method for operating a SCSI bus with redundant SCSI adaptors

Information

  • Patent Grant
  • 6735715
  • Patent Number
    6,735,715
  • Date Filed
    Thursday, April 13, 2000
    24 years ago
  • Date Issued
    Tuesday, May 11, 2004
    20 years ago
Abstract
A computer system includes a port duplex driver (PDD) that creates a “virtual SCSI adaptor,” to control the operations of one or more redundant SCSI adaptors. During boot-up operations or when the status of a device on a SCSI bus changes, the PDD identifies the virtual SCSI adaptor as the only adaptor that provides access to particular storage devices on the SCSI bus. System components then direct data transfer operations through the virtual SCSI adaptor to the storage devices. The PDD intercepts commands that are directed through the virtual SCSI adaptor, and redirects the commands to a selected one of the actual SCSI adaptors. The selected SCSI adaptor then operates in a conventional manner, to translate the generic commands from the system components to device-specific commands for the storage devices on the SCSI bus. If the selected SCSI adaptor fails, the PDD redirects the data transfer operations instead through one of the redundant SCSI adaptors. The failure does not affect the remainder of the system, which continues to conduct data transfer operations through the virtual SCSI adaptor.
Description




FIELD OF THE INVENTION




The invention relates generally to fault tolerant computer systems and, more particularly, to mechanisms for fault tolerant access to storage subsystems.




BACKGROUND OF THE INVENTION




Storage devices, such as disk drives, typically attach to and communicate with a computer system over a SCSI bus. Other devices, such as the key board and system processor, communicate with the storage devices over a system I/O bus such as a PCI bus. To facilitate communication between the devices on the two busses, a SCSI adaptor plugs into both busses. The SCSI adaptor is essentially a translator, which receives generic commands generated by the system components, and translates the commands to specialized commands for the particular storage devices that are attached to the SCSI bus. The system components thus have to know only the types, or classes, of storage devices available, and not the specifics of the devices.




During system boot-up, a SCSI port driver, either alone or in conjunction with a plug and play manager, identifies the devices that are accessible through the SCSI adaptor by relative location and class, e.g., disk or tape drive. If the system is operating in, for example, a Windows NT environment, the operating system then creates for the identified devices a set of device objects that can be used by the system components, to direct the generic data transfer commands to the respective storage devices.




To add fault tolerance to such a system, it is desirable to include in the system one or more additional SCSI adaptors that provide redundant paths from the PCI bus to the SCSI bus, and thus, to the storage devices. Adding the redundant SCSI adaptors to a system that uses Windows NT, however, causes the system to create additional sets of device objects for the storage devices. Specifically, the system creates one set that corresponds to the storage devices that are accessible through the first SCSI adaptor, a second set for the storage devices that are accessible through the second SCSI adaptor and so forth. Accordingly, each storage device is identified by multiple device objects, which may confuse or cause other problems for data transfer operations.




SUMMARY OF THE INVENTION




The inventive system includes a port duplex driver (PDD) that creates a “virtual SCSI adaptor,” to control the operations of one or more conventional SCSI adaptors.




During boot-up operations or when the status of a device on the SCSI bus changes, the PDD identifies the virtual SCSI adaptor as the only adaptor that provides access to the storage devices on the SCSI bus. The system then assigns a single set of device objects to the storage devices and data transfer commands are directed to the storage devices through the virtual adaptors using the device objects. The PDD thereafter intercepts the commands that are directed through the virtual SCSI adaptor, and redirects them to a selected one of the actual SCSI adaptors. The selected SCSI adaptor then operates in a conventional manner, to translate the generic commands to device-specific commands for the storage devices, and so forth.




If the selected SCSI adaptor fails, the PDD redirects the data transfer operations instead through one of the redundant SCSI adaptors. The failure does not affect the remainder of the system, which continues to conduct data transfer operations through the virtual SCSI adaptor.




The PDD may create the virtual SCSI adaptor even for configurations in which a single SCSI adaptor is connected to the SCSI bus. In this way, one or more redundant SCSI adaptors may be added at a later time and mapped by the PDD to the existing virtual SCSI adaptor, without affecting the way in which the remainder of the system conducts data transfer operations.











BRIEF DESCRIPTION OF THE DRAWINGS




The invention description below refers to the accompanying drawings, of which:





FIG. 1

is a functional block diagram of a system constructed in accordance with the invention;





FIG. 2

is a flow chart of boot-up operations of the system;





FIG. 3

is a more detailed functional block diagram of the system;





FIGS. 4-5

are more detailed flow charts of the operations system of

FIG. 1

;





FIG. 6

is a functional block diagram of an alternative SCSI adaptor in the systems of

FIG. 1

;





FIG. 7

is a flow chart of the operations of the system when a SCSI adaptor fails; and





FIG. 8

is a flow chart of the operations of a port duplex driver when a SCSI adaptor fails;











DETAILED DESCRIPTION OF AN ILLUSTRATIVE




EMBODIMENT Referring to

FIG. 1

, a computer system includes an I/O bus


10


that handles communications for various system components


12


, such as, for example, a system processor. The I/O bus


10


in the exemplary system is a PCI bus. One or more storage devices


14


, such as disk drives and/or tape drives, that participate in data transfer operations with the system components


12


connect to the system over a SCSI bus


16


. Two SCSI adaptors


20


and


22


interconnect the I/O bus


10


and the SCSI bus


16


, to provide redundant paths for communications between the system components


12


and the storage devices


14


. This configuration of the SCSI bus is referred to herein as“dual initiated” SCSI bus. If only a single SCSI adaptor


20


is included in the system, the configuration is instead referred to as a “single initiated” SCSI bus. As depicted in the drawing, the system includes redundant system processors


25


and redundant I/O busses


10


, to provide fault tolerant operation of the system processors and the SCSI adaptors. the system processors in the exemplary system operate in lock-step, and thus, perform the same operations at the same times.




Referring also to

FIGS. 2 and 3

, during system boot-up, a plug-and-play manager


32


that is connected to or resident on the system processor


12


calls on a PCI bus driver


34


to enumerate the devices on the PCI bus


10


(step


200


). The PCI bus driver queries the devices on the PCI bus and identifies the SCSI adaptors


21


and


22


to the plug and play manager (step


202


). The PCI bus driver will also identify any other devices (not shown) that are connected to the bus, however, for the following discussion these devices are not of interest.




Referring also to

FIGS. 2 and 3

, during system boot-up, a plug-and-play manager


32


that is connected to or resident on the system processor


25


calls on a PCI bus driver


34


to enumerate the devices on the PCI bus


10


(step


200


). The PCI bus driver queries the devises on the PCI bus and identifies the SCSI adaptors


20


and


22


to the plug and play manager (step


202


). The PCI bus driver will also identify any other devices (not shown) that are connected to the bus, however, for the following discussion these devices are not of interest.




The plug-and-play manager


32


consults a stored look-up table


33


, to determine which driver to assign to each SCSI adaptor


20


and


22


. Based on the contents of the table, the plug-and-play manager then assigns a port driver


36


, which is referred to herein also as a mini-port driver, to each SCSI adaptor


20


and


22


.




In steps


204


and


206


, the plug and play manager asks the mini-port driver to determine what is attached, for example, to the SCSI adaptor


20


, that is, to enumerate the devices on the SCSI bus through that adaptor. In response to the request from the plug-and-play manager


32


, the mini-port driver


36


sends queries to the devices on the SCSI bus


16


, by location through the SCSI adaptor


20


. The individual devices respond by sending back a response that indicates that the device is a storage device. The mini-port driver next returns to the plug and play manager a list of the devices associated with the SCSI adaptor


20


, and identifies each storage device on the list as a “SCSI other” device. The plug and play manager similarly requests that the mini-port driver


36


enumerate the devices on the SCSI bus through the SCSI adaptor


22


. The mini-port driver then queries the storage devices


14


through that adaptor. Each device responds to the query by sending back a response that indicates that the device is a storage device. The mini-port driver then returns a list of devices associated with the SCSI adaptor


20


, and similarly identifies each storage device on the list as a SCSI other device.




The plug and play manager


32


next enters the stored look-up table


33


to determine which drivers to assign to the various SCSI other devices. Based on the contents of the table


33


, the plug-and-play manager


32


assigns to each of the SCSI other devices a port duplex driver


38


(step


208


). The port duplex driver, or PDD, may be loaded at this time or it may have been pre-loaded during an earlier step in the boot-up operations When the PDD is loaded, it creates a “virtual SCSI adaptor” and identifies the virtual device to the plug-and-play manager.




As a further step in the device enumeration operation, the plug-and-play manager


32


issues for each SCSI other device a request that the PDD


38


list what is connected to the device (step


210


). In the example, the plug-and-play manager first requests enumeration of a particular SCSI other device that is connected to SCSI adaptor


20


.




The operations of the PDD


38


are discussed with reference to FIG.


4


. In response to a request from the plug-and-play manager


32


, the PDD enumerates the particular SCSI other device by sending a query to the corresponding storage device


14


through the associated SCSI adaptor


20


(step


400


). The storage device responds by returning its serial number and any other requested information to the PDD through the adaptor


20


. When the PDD receives the response, the PDD determines if it has already queried this device through another path to the device (steps


402


,


404


). The PDD thus checks if the serial number contained in the response matches a serial number contained in any previously received response.




If the serial number does not match any of the previously received serial numbers, the PDD creates a new SCSI-class device and “attaches” the device to the virtual SCSI adaptor (step


406


). The PDD then tells the plug and play manager that the configuration of the virtual SCSI adaptor has changed (step


408


). The PDD also internally maps the storage device


14


to the newly created SCSI-class device, and maps the SCSI adaptor


20


to the virtual adaptor (steps


410


,


412


).




If the serial number in the response from the associated storage device matches a serial number in a previous response, the PDD determines that the SCSI bus is dual initiated (step


414


). The PDD then internally maps the storage device to the corresponding SCSI-class device attached to the virtual SCSI adaptor (step


410


). Further, the PDD internally maps the associated SCSI adaptor, in the example, adaptor


20


, to the virtual SCSI adaptor (step


412


).




After mapping the SCSI other device to the appropriate SCSI-class device that is attached to the virtual SCSI adaptor, the PDD notifies the plug and play manager that no devices are attached to the SCSI other device (step


416


).




The PDD operates in the same manner in response to the requests by the plug and play manager to enumerate each of remaining the SCSI other devices attached to the SCSI adaptor


20


. The PDD also operates in the same manner in response to the requests by the plug and play manager to enumerate each of the SCSI other devices attached to the SCSI adaptor


22


. When the PDD has responded to the last enumeration request, the PDD has internally mapped the SCSI other devices to a single set of SCSI-class devices that are attached to the virtual SCSI adaptor.




Referring also to

FIG. 5

, when the plug and play manager is notified by the PDD that the configuration of the virtual SCSI adaptor ahs changed, the plug and play manager requests that the PDD enumerate the devices connected to the virtual SCSI adaptor (step


500


). In response to the request from the plug and play manager, the PDD provides a list of the SCSI-class devices then known to the PDD (step


502


). The plug and play manger then consults its table


33


to assign to the listed SCSI-class devices the appropriate SCSI-class drivers


40


(

FIG. 3

) (step


504


). The plug and play manager thus assigns an appropriate disk device driver to each SCSI-disk device and an appropriate tape device driver to each SCSI-tape device. The plug and play manager the, in a conventional manner, asks the assigned SCSI-class drivers


40


to separately enumerate each SCSI-class device (step


506


).




In response to a request from the plug and play manager, a given SCSI-class driver directs a request to read partition information and so forth to a particular SCSI-class device on the list. The request is directed to the SCSI-class device through the virtual SCSI adaptor. The PDD thus intercepts the request and re-directs it through a selected SCSI adaptor


20


or


22


to the associated storage device


14


. The selected SCSI adaptor then acts in a conventional manner to provide the requested access to the disk or tape device. The SCSI-class drivers


40


continue to enumerate each of the SCSI-class devices listed as attached to the virtual SCSI adaptor, and the PDD continues to direct associated requests through the selected SCSI adaptor to the corresponding storage devices


14


.




When the SCSI-class drivers


40


have enumerated the SCSI-class devices and provided the necessary information to the plug and play manager


32


and the mini-port drivers


36


, as appropriate, the plug and play manager and/or the mini-port driver then operate in a conventional manner to identify the SCSI disk devices and SCSI tape devices to the operating system (step


508


). The operating system then assigns a set of device objects to the SCSI disk and tape devices in a conventional manner. Non-storage devices on the SCSI bus, depicted in

FIG. 3

as blocks


15


, are treated in a conventional manner by the mini-port driver. These devices are similarly assigned appropriate SCSI class drivers


42


(

FIG. 3

) and device objects, also in a conventional manner.




The operations of the mini-port driver and the PDD ensure that each storage device will be assigned a single device object, regardless of how many SCSI adaptors provide access to the storage device. When a generic data transfer command is addressed to a given storage device


14


using the assigned device object, the PDD


38


intercepts the command and redirects it to the corresponding storage device through the selected one of the SCSI adaptors


20


or


22


. The selected SCSI adaptor then operates in a conventional manner, to translate the command to a device-specific command, and so forth.




Referring now to

FIG. 6

, the SCSI adaptors


20


and


22


may be single-ported and connect, respectively, to one SCSI bus as depicted in FIG.


1


. Alternatively, as depicted in

FIG. 6

, the SCSI adaptors may be dual-ported and thus each connect to redundant SCSI busses


16


and


17


. The two SCSI adaptors


50


and


52


depicted in the drawing each have primary and secondary ports


50




0


and


50




1


and


52




0


and


52




1


, respectively. The primary port for each adaptor is, for example, the port with the lowest address, namely,


50




0


or


52




0


. The hardware is configured such that the primary port


50




0


of the SCSI adaptor


50


connects to one SCSI bus, in the example, the bus


16


, and the secondary port


50




1


of the adaptor connects to the second SCSI bus


17


. The second of the SCSI adaptors has its primary port


52




0


connected to the SCSI bus


17


and its secondary port


52




1


connected to the SCSI bus


16


.




With dual-ported SCSI adaptors, the PDD


38


selects one SCSI adaptor for use on one bus and another for use on the other bus, in order to load share between the two SCSI adaptors. In the example, the PDD selects the SCSI adaptor


50


for use on the SCSI bus


16


, which is connected to the primary port


50




0


of the adaptor. Similarly, the PDD selects the SCSI adaptor


52


for use on the SCSI bus


17


. The PDD could instead select to use the adaptors on the busses that are connected to the secondary port of each adaptor, with the same load sharing result.




Referring now to

FIGS. 3 and 7

, if the selected SCSI adaptor, for example, adaptor


20


, fails, the plug-and-play manager


32


receives a message from a hardware monitoring subsystem (not shown) that something in the storage subsystem has changed at the PCI bus level. Alternatively, the error message may be sent by the PDD


38


, which receives error messages in response to requests sent through the failed adaptor (step


700


). In response, the plug-and-play manager


32


generates an interrupt and asks the PCI bus driver


32


to again enumerate the PCI bus (step


702


). The PCI bus driver now identifies only one SCSI adaptor, namely, the operative adaptor


22


(step


704


).




The plug-and-play manager


32


next removes its prior listing of the failed SCSI adaptor


20


and the associated SCSI other devices (step


706


). The plug and play manager then notifies the PDD


38


of the removal of each SCSI other device attached to the failed adaptor, and notifies the mini-port driver of the removal of the failed SCSI adaptor (steps


708


-


710


).




The plug and play manager still sees the storage devices


14


as attached to the virtual SCSI adaptor, and thus, the plug and play manager does not see any change in the way data transactions are made with those devices. Accordingly, the plug and play manager determines that adaptor failure does not affect the data transfer operations, and no error message is required to be sent.




Referring now also to

FIG. 8

, when the SCSI other devices attached to the SCSI adaptor


20


are removed by the plug and play manager, the PDD


38


selects the operative SCSI adaptor


22


for use in the data transfer operations (steps


800


-


802


). Further, for those devices that are disk devices, the PDD controls a reissue, through the operative SCSI adaptor


22


, of any requests that were directed through the failed adaptor


20


are now associated with error messages (step


804


). For those devices that are tape devices, the PDD passes information relating to the failed requests back through the system to the applications that issued the requests (step


806


). The applications then re-issued the requests in a conventional manner.




After failures inoperative SCSI adaptor may be rep , replaced and/or removed, without altering the way in which the system components


12


perform future data transfer operations. If the failed SCSI adaptor is repaired or replaced, the adaptor is enumerated and the PDD changes its internal mappings to the virtual SCSI adaptor and the associated SCSI-class devices attached to the virtual SCSI adaptor, to again include the redundant path to the devices. The PDD may then change its adaptor selection, to revert to the repaired or replaced adaptor for load sharing.




Instead of using the mini-port driver essentially to disguise the storage devices


14


as SCSI other devices, the system may use a filter driver that is inserted into the driver stack of the storage devices to intercept and respond to SCSI bus enumeration requests. In the response, the filter driver identifies the storage devices as a special class of device that is handled by the PDD, and the PDD then operates in the manner described above. This allows the mini-port driver to operate in a conventional manner, to identify various non-storage devices that fall into the conventional SCSI other class. In any event, conventional SCSI adaptors are used.




The foregoing description has been limited to a specific embodiment of this invention. It will be apparent, however, that variations and modifications may be made to the invention, such as the redundant SCSI bus adaptors may be bus adaptors for busses such as the USB, IEEE 1394, IBM serial storage architecture bus, Infiniband, IBM ESCON, or fiber channel, with the attainment of some or all of its advantages. Therefore, it is the object of the appended claims to cover all such variations and modifications as come within the true spirit and scope of the invention.



Claims
  • 1. A computer system including:A. one or more storage devices connected to a first bus; B. one or more system components that connect to a second bus and communicate with the storage devices; C. one or more bus adaptors for transferring communications to and from the second bus and the storage devices on the first bus; D. a port duplex driver for mapping one or more communications paths to the storage devices through the one or more bus adaptors to a communication path through a virtual bus adaptor, E. object means for assigning to each storage device on the first bus a device object that corresponds to the communications path through the virtual bus adaptor, the object means informing the system components of the assigned device objects; and F. the port duplex driver further re-directing to a selected one of the bus adaptors communications from the system components that include the assigned device objects.
  • 2. The computer system of claim 1 further comprising:a. a mini-port driver that identifies the storage devices as belonging to a given special class of device, and b. a plug and play manager that associates the given special class of device with the port duplex driver.
  • 3. The computer system of claim 1 wherein the first bus is a SCSI bus and the second bus is a system I/O bus.
  • 4. The computer system of claim 3 wherein the system I/O bus is a PCI bus.
  • 5. The computer system of claim 1 wherein the port duplex driver changes the selection of the bus adaptor from a previously selected bus adaptor that has failed to an operative bus adaptor.
  • 6. The computer system of claim 5 wherein the port duplex driver re-issues to the operative bus adaptor the communications that were directed to the failed bus adaptor after the failure.
  • 7. The computer system of claim 1 further comprising:G. additional storage devices connected to another first bus, H. the one or more bus adaptors are dual ported, and I. the port duplex driver selects one bus adaptor for use on one first bus and a different bus adaptor for use on the other first bus.
  • 8. A method of operating a computer system, the method comprising the steps of:A. determining that one or more storage devices are accessible over a first communications bus through one or more bus adaptors; B. mapping the one or more bus adaptors to a virtual bus adaptor and presenting to system components that communicate with the storage devices a communication path through the virtual bus adaptor to the storage devices; C. assigning to each storage device on the first communications bus a device object that corresponds to the communications path through the virtual bus adaptor; D. informing the system components of the assigned device object, and E. re-directing to a selected one of the bus adaptors the communications that are directed through the virtual bus adaptor to the storage devices.
  • 9. The method of claim 8 wherein the step of determining that the one or more storage devices are accessible over the first communication bus includes:i. sending queries to the storage devices through each of the one or more bus adaptors and receiving responses from the storage devices through each of the one or more bus adaptors, ii. identifying the storage devices as devices that are controlled by a port duplex driver, iii. reporting by the port duplex driver that the storage devices are accessible through the virtual bus adaptor. 10.The method of claim 9 wherein the step of reporting by the port duplex driver includesa. determining that the one or more bus adaptors provide redundant paths to the storage devices, and b. mapping the bus adaptors that provide the redundant path to the same virtual bus adaptor.
  • 11. The method of claim 10 wherein the step of determining redundant paths further includesc. identifying the storage devices as special devices that are accessible through each of the bus adaptors; and d. associating the special devices with the port duplex driver.
  • 12. The method of claim 11 wherein the port duplex driver:e. in the step of reporting further includes reporting the virtual bus adaptor and the storage devices as devices to which the system assigns device objects, and f. in the step of re-directing further includes re-directing to a selected bus adaptor the communications that include the device objects that are assigned to the storage devices.
  • 13. The method of claim 9 wherein the step of mapping further includes mapping a newly added bus adaptor to the virtual bus adaptor if the newly added bus adaptor provides a redundant path to the storage devices.
US Referenced Citations (142)
Number Name Date Kind
3544973 Borck, Jr. et al. Dec 1970 A
3548176 Shutler Dec 1970 A
3641505 Artz et al. Feb 1972 A
3710324 Cohen et al. Jan 1973 A
3736566 Anderson et al. May 1973 A
3795901 Boehm et al. Mar 1974 A
3805039 Stiffler Apr 1974 A
3820079 Bergh et al. Jun 1974 A
3840861 Amdahl et al. Oct 1974 A
3997896 Cassarino, Jr. et al. Dec 1976 A
4015246 Hopkins, Jr. et al. Mar 1977 A
4032893 Moran Jun 1977 A
4059736 Perucca et al. Nov 1977 A
4228496 Katzman et al. Oct 1980 A
4245344 Richter Jan 1981 A
4263649 Lapp, Jr. Apr 1981 A
4275440 Adams, Jr. et al. Jun 1981 A
4309754 Dinwiddie, Jr. et al. Jan 1982 A
4366535 Cedolin et al. Dec 1982 A
4434463 Quinquis et al. Feb 1984 A
4449182 Rubinson et al. May 1984 A
4453215 Reid Jun 1984 A
4467436 Chance et al. Aug 1984 A
4484273 Stiffler et al. Nov 1984 A
4486826 Wolff et al. Dec 1984 A
4503496 Holzner et al. Mar 1985 A
4543628 Pomfret Sep 1985 A
4590554 Glazer et al. May 1986 A
4597084 Dynneson et al. Jun 1986 A
4608631 Stiffler et al. Aug 1986 A
4628447 Cartret et al. Dec 1986 A
4630193 Kris Dec 1986 A
4633394 Georgiou et al. Dec 1986 A
4654857 Samson et al. Mar 1987 A
4669056 Waldecker et al. May 1987 A
4669079 Blum May 1987 A
4700292 Campanini Oct 1987 A
4703420 Irwin Oct 1987 A
4750177 Hendrie et al. Jun 1988 A
4805091 Thiel et al. Feb 1989 A
4809169 Sfarti et al. Feb 1989 A
4816990 Williams Mar 1989 A
4827409 Dickson May 1989 A
4866604 Reid Sep 1989 A
4914580 Jensen et al. Apr 1990 A
4916695 Ossfeldt Apr 1990 A
4926315 Long et al. May 1990 A
4931922 Baty et al. Jun 1990 A
4939643 Long et al. Jul 1990 A
4974144 Long et al. Nov 1990 A
4974150 Long et al. Nov 1990 A
4985830 Atac et al. Jan 1991 A
4994960 Tuchler et al. Feb 1991 A
5083258 Yamasaki Jan 1992 A
5117486 Clark et al. May 1992 A
5138257 Katsura Aug 1992 A
5179663 Iimura Jan 1993 A
5243704 Baty et al. Sep 1993 A
5247522 Reiff Sep 1993 A
5251303 Fogg, Jr. et al. Oct 1993 A
5270699 Signaigo et al. Dec 1993 A
5295258 Jewett et al. Mar 1994 A
5313627 Amini et al. May 1994 A
5317726 Horst May 1994 A
5428766 Seaman Jun 1995 A
5463755 Dumarot et al. Oct 1995 A
5465340 Creedon et al. Nov 1995 A
5537535 Maruyama et al. Jul 1996 A
5546535 Stallmo et al. Aug 1996 A
5550986 DuLac Aug 1996 A
5555372 Tetreault et al. Sep 1996 A
5574865 Hashemi Nov 1996 A
5584030 Husak et al. Dec 1996 A
5600784 Bissett et al. Feb 1997 A
5613162 Kabenjian Mar 1997 A
5627965 Liddell et al. May 1997 A
5630056 Horvath et al. May 1997 A
5659681 Ojima Aug 1997 A
5671443 Stauffer et al. Sep 1997 A
5696905 Reimer et al. Dec 1997 A
5701410 BeMent et al. Dec 1997 A
5701457 Fujiwara Dec 1997 A
5737601 Jain et al. Apr 1998 A
5751955 Sonnier et al. May 1998 A
5758065 Reams et al. May 1998 A
5790397 Bissett et al. Aug 1998 A
5790775 Marks et al. Aug 1998 A
5809256 Najemy Sep 1998 A
5809285 Hilland Sep 1998 A
5812748 Ohran et al. Sep 1998 A
5815647 Buckland et al. Sep 1998 A
5828903 Sethuram et al. Oct 1998 A
5838899 Leavitt et al. Nov 1998 A
5838900 Horvath et al. Nov 1998 A
5838993 Riley et al. Nov 1998 A
5862145 Grossman et al. Jan 1999 A
5875308 Egan et al. Feb 1999 A
5875351 Riley Feb 1999 A
5881251 Fung et al. Mar 1999 A
5892928 Wallach et al. Apr 1999 A
5894560 Carmichael et al. Apr 1999 A
5903778 Chang May 1999 A
5928339 Nishikawa Jul 1999 A
5944800 Mattheis et al. Aug 1999 A
5953538 Duncan et al. Sep 1999 A
5956476 Ranson et al. Sep 1999 A
5978866 Nain Nov 1999 A
5982672 Moon et al. Nov 1999 A
5983289 Ishikawa et al. Nov 1999 A
5991844 Khosrowpour Nov 1999 A
5991900 Garnett Nov 1999 A
5996035 Allen et al. Nov 1999 A
6000043 Abramson Dec 1999 A
6009481 Mayer Dec 1999 A
6009535 Halligan et al. Dec 1999 A
6012119 Ninomiya et al. Jan 2000 A
6012120 Duncan et al. Jan 2000 A
6021456 Herdeg et al. Feb 2000 A
6026458 Rasums Feb 2000 A
6032201 Tillery, Jr. et al. Feb 2000 A
6032271 Goodrum et al. Feb 2000 A
6041375 Bass et al. Mar 2000 A
6047343 Olarig Apr 2000 A
6049894 Gates Apr 2000 A
6052795 Murotani et al. Apr 2000 A
6055584 Bridges et al. Apr 2000 A
6062480 Evoy May 2000 A
6073196 Goodrum et al. Jun 2000 A
6085333 DeKoning et al. Jul 2000 A
6098137 Goodrum et al. Aug 2000 A
6125417 Bailis et al. Sep 2000 A
6128750 Espy et al. Oct 2000 A
6141769 Petivan et al. Oct 2000 A
6154802 Khosrowpour Nov 2000 A
6185697 Shiraishi Feb 2001 B1
6202115 Khosrowpour Mar 2001 B1
6324627 Kricheff et al. Nov 2001 B1
6363448 Waller et al. Mar 2002 B1
6430710 Moriyama et al. Aug 2002 B1
6449731 Frey, Jr. Sep 2002 B1
6519671 Kondou et al. Feb 2003 B1
6563595 Galloway May 2003 B1
Foreign Referenced Citations (8)
Number Date Country
0 299 700 Jul 1987 EP
0 301 499 Feb 1989 EP
0 428 330 May 1991 EP
0 475 005 Nov 1995 EP
0 293 860 Feb 1996 EP
0772127 May 1997 EP
0 390 567 Jun 1999 EP
2 060 229 Apr 1981 GB