BRIEF DESCRIPTION OF THE DRAWINGS
The invention, as well as a preferred mode of use and further objectives and advantages thereof, will best be understood by reference to the following detailed description of illustrative embodiments when read in conjunction with the accompanying drawings, wherein:
FIG. 1 depicts a pictorial representation of an exemplary blade server system in which aspects of the illustrative embodiments may be implemented;
FIG. 2 illustrates a configuration and connectivity for attaching external storage via a storage area network in accordance with an illustrative embodiment;
FIG. 3 illustrates a fabric connection of various blade implementations in accordance with an illustrative embodiment;
FIG. 4 illustrates a configuration for partitioning connectors in a high-speed switch module in accordance with an illustrative embodiment;
FIG. 5 illustrates a topology for interfacing processor blades and a boot disk enclosure blade (B-DEB) in accordance with an exemplary embodiment;
FIG. 6 illustrates a topology for interfacing processor blades and a switched bunch of disks enclosure blade (S-DEB) in accordance with an exemplary embodiment;
FIG. 7 illustrates a topology for interfacing processor blades and an integrated RAID controller disk enclosure blade (I-DEB) in accordance with an exemplary embodiment; and
FIG. 8 is a flowchart illustrating operation of configuring various types of blades in a blade server system with high-speed switch fabrics in accordance with an illustrative embodiment.
DETAILED DESCRIPTION OF THE ILLUSTRATIVE EMBODIMENTS
With reference now to the figures and in particular with reference to FIGS. 1 and 2, exemplary diagrams of a data processing environment are provided in which illustrative embodiments of the present invention may be implemented. It should be appreciated that FIGS. 1 and 2 are only exemplary and is not intended to assert or imply any limitation with regard to the environments in which aspects or embodiments of the present invention may be implemented. Many modifications to the depicted environments may be made without departing from the spirit and scope of the present invention.
With reference now to the figures, FIG. 1 depicts a pictorial representation of an exemplary blade server system in which aspects of the illustrative embodiments may be implemented. Blade server system 100 may include a plurality of blade servers 102, each having a corresponding daughter card 104. In the depicted example, blade server system includes fourteen blade servers; however, more or fewer blade slots may be included depending upon the implementation. A server blade may be a processor blade, a storage blade, or the like.
Switch modules 1-4112-118 provide connectivity to Ethernet, Fibre Channel, or the like. Blade servers 102 and daughter cards 104 connect to switch modules 112-118 through one lane (1X) fabrics 146 (14 deep for the 14 blade servers in this example).
High speed switch module 1122 and high speed switch module 2124 provide connectivity to InfiniBand (IB), Ethernet, or the like. High speed switch module 3126 and high speed switch module 4128 provide connectivity to IB, Ethernet, or serial attached small computer systems interface (serial attached SCSI or SAS).
Blade servers 102 and daughter cards 104 connect to high speed switch modules 122-128 via four lane (4X) fabrics 142. These 4X fabrics are 14 deep for the 14 blade servers in this example). Also, 4X fabrics 144 connect switch modules 116, 118 and high speed switch modules 122-128.
Blade server system 100 also includes management module 110. Management module 110 performs various management functions for blade server system 100, such as monitoring the system and its associated blade servers 102, daughter cards 104, switch modules 112-118, 122-128, and the like. The components of blade server system 100 are connected to management module 110 through management fabric 148.
Those of ordinary skill in the art will appreciate that the hardware in FIG. 1 may vary depending on the implementation. Other internal hardware or peripheral devices, such as fans, power supplies, and the like, may be used in addition to, or in place of, the hardware depicted in FIG. 1. Also, the processes of the illustrative embodiments may be applied to a data processing system other than the blade server system mentioned above, without departing from the spirit and scope of the illustrative embodiments.
FIG. 2 illustrates a configuration and connectivity for attaching external storage via a storage area network in accordance with an illustrative embodiment. Blade server system 200 may connect to storage area network (SAN) 250 through high speed switch modules 226, 228, for example. SAN 250 may connect to switch modules 226 and 228 through InfiniBand, Ethernet, or SAS; however, in the illustrative embodiment, switch modules 226 and 228 connect to SAN 250 through SAS connections. Storage enclosures 252, 254, 256 also connect to SAN 250 using SAS connections.
FIG. 3 illustrates a fabric connection of various blade implementations in accordance with an illustrative embodiment. Processor blade 310 provides one or more server processors. Processor blade 310 comprises two 4X high-speed connections 312, 314. The fabric connections 312, 314 of processor blade 310 are partitioned into dual two lane (2X) sub-fabrics to separate the host adapter (HA) traffic from the device adapter (DA) traffic. Host adapter traffic is the data that flows between the processor blade and the storage and/or RAID controller, via the high speed SAS switch. Device adapter traffic is the data that flows between the RAID controller and storage via the SAS switch. As seen in the depicted example, the DA interface is not connected (-nc-).
FIG. 3 also depicts three variants on storage blade implementations. FIGS. 5-7 illustrate these different blade implementations, as will be discussed in further detail below.
I-DEB blade 320 provides a “switched bunch of disks” (SBOD) with an integrated RAID controller. I-DEB blade 320 comprises four 4X high-speed connections 322, 324, 326, 328. On I-DEB blade 320, connectors 322, 324, 326, 328 are partitioned into dual 2X interfaces, one for HA traffic and one for DA traffic. I-DEB blade 320 has an integrated RAID controller in conjunction with multiple disk drives, as will be shown in FIG. 7 and described in further detail below.
S-DEB blade 330 provides a SBOD without an integrated RAID controller. S-DEB blade 330 has four 4X high-speed connectors 332, 334, 336, 338. On S-DEB blade 330, connectors 332, 334, 336, 338 use only a single 2X interface for DA traffic. S-DEB blade 330 is essentially a SBOD that provides simple shared storage for the processor blades.
B-DEB blade 340 provides boot drives for the processor blades in the blade server system. B-DEB blade 340 is a simple SBOD, but provides redundant disk drives for the processor blades boot function (non-customer data). B-DEB blade 340 comprises four 4X high-speed connectors 342, 344, 346, 348. On B-DEB blade 340, connectors 342, 344, 346, 348 use only a single 1X interface for HA traffic.
FIG. 4 illustrates a configuration for partitioning connectors in a high-speed switch module in accordance with an illustrative embodiment. Serial attached SCSI (SAS) switch 400 comprises 36-port expander 430 for host adapter (HA) traffic and 36-port expander 440 for device adapter (DA) traffic. SAS switch 400 supports connection to 14 blade slots through 4X high speed connectors 410-423. Each 4X connector 410-423 is partitioned into dual 2X interfaces.
Also, each 4X connector is associated with a blade slot 0-13. Blade slots may be populated with any combination of processor blades, B-DEB blades, S-DEB blades, or I-DEB blades. Dual SAS expanders 430, 440 are implemented in SAS switch 400 to accommodate the separate HA and DA traffic.
FIG. 5 illustrates a topology for interfacing processor blades and a boot disk enclosure blade (B-DEB) in accordance with an exemplary embodiment. Processor blades 502 connect to serial attached SCSI (SAS) switch A high-speed fabric 510 and SAS switch B high-speed fabric 560. Switch modules, such as switches 510, 560, may be used in pairs for redundancy. Processor blades 502 and SAS switches 510, 560 may be internal to a blade server system, such as blade server system 100 shown in FIG. 1. More particularly, processor blades 502 may be one of blade servers 102, and SAS switches 510, 560 may be high speed switch modules 126, 128, for example.
Boot disk enclosure blade (B-DEB) 530 may also be a blade itself, such as one of blade servers 102 in FIG. 1. B-DEB 530 includes storage controllers 532 and 534. Storage controllers 532, 534 connect to disk drives 521-528 via internal non-blocking SAS switches 542, 544. Non-blocking SAS switches 542, 544 connect to SAS switch 510 via high-speed connectors 552, 554. Non-blocking SAS switches 542, 544 connect to SAS switch 560 via high-speed connectors 556, 558. High-speed connectors 552, 554, 556, 558 are 4-port connectors.
In B-DEB 530, high-speed connectors 552, 554 connect to non-blocking switch 512 in SAS switch A high-speed fabric 510 through 1X interfaces, as described above with respect to FIG. 3. Similarly, high-speed connectors 556, 558 connect to non-blocking switch 562 in SAS switch B high-speed fabric 560 through 1X interfaces. Host adapter (HA) traffic occurs through these 1X interfaces. Also, processor blades 502 connect to non-blocking SAS switch 512 and non-blocking SAS switch 562 through 1X interfaces. HA traffic occurs through these interfaces. Non-blocking SAS switches 512, 562 may also provide dual 4X connectors for external devices.
FIG. 6 illustrates a topology for interfacing processor blades and a switched bunch of disks enclosure blade (S-DEB) in accordance with an exemplary embodiment. Switch modules, such as switches 610, 660, may be used in pairs for redundancy. SAS switches 610, 660 may be internal to a blade server system, such as blade server system 100 shown in FIG. 1. More particularly, SAS switches 610, 660 may be high speed switch modules 126, 128, for example.
Switched bunch of disks (SBOD) enclosure blade (S-DEB) 630 may be a blade itself, such as one of blade servers 102 in FIG. 1. S-DEB 630 includes storage controllers 632 and 634. Storage controllers 632, 634 connect to disk drives 621-628 via internal non-blocking SAS switches 642, 644. Non-blocking SAS switches 642, 644 connect to SAS switch 610 via high-speed connectors 652, 654. Non-blocking SAS switches 642, 644 connect to SAS switch 660 via high-speed connectors 656, 658. High-speed connectors 652, 654, 656, 658 are 4-port connectors.
In S-DEB 630, high-speed connectors 652, 654 connect to non-blocking switch 612 in SAS switch A high-speed fabric 610 through 2X interfaces, as described above with respect to FIG. 3. Similarly, high-speed connectors 656, 658 connect to non-blocking switch 662 in SAS switch B high-speed fabric 660 through 2X interfaces. Device adapter (DA) traffic occurs through these 2X interfaces. Non-blocking SAS switches 612, 662 may also provide dual 4X connectors for external devices.
FIG. 7 illustrates a topology for interfacing processor blades and an integrated RAID controller disk enclosure blade (I-DEB) in accordance with an exemplary embodiment. Processor blades 702 connect to serial attached SCSI (SAS) switch A high-speed fabric 710 and SAS switch B high-speed fabric 760. Switch modules, such as switches 710, 760, may be used in pairs for redundancy. Processor blades 702 and SAS switches 710, 760 may be internal to a blade server system, such as blade server system 100 shown in FIG. 1. More particularly, processor blades 702 may be one of blade servers 102, and SAS switches 710, 760 may be high speed switch modules 126, 128, for example.
Integrated RAID controller disk enclosure blade (I-DEB) 730 may also be a blade itself, such as one of blade servers 102 in FIG. 1. I-DEB 730 includes redundant array of independent disk (RAID) controllers 732 and 734. RAID controllers 732, 734 connect to disk drives 721-728 via internal non-blocking SAS switches 742, 744. Non-blocking SAS switches 742, 744 connect to SAS switch 710 via high-speed connectors 752, 754. Non-blocking SAS switches 742, 744 connect to SAS switch 760 via high-speed connectors 756, 758. High-speed connectors 752, 754, 756, 758 are 4-port connectors.
In I-DEB 730, high-speed connectors 752, 754 connect to non-blocking switch 714 in SAS switch A high-speed fabric 710 through 2X interfaces, as described above with respect to FIG. 3. Similarly, high-speed connectors 756, 758 connect to non-blocking switch 762 in SAS switch B high-speed fabric 760 through 2X interfaces. Device adapter (DA) traffic occurs through these 2X interfaces.
RAID controllers 732, 734 connect to 2X interfaces of non-blocking SAS switch 712 in SAS switch A high-speed fabric 710 through HS connectors 752, 754. RAID controllers 732, 734 connect to 2X interfaces of non-blocking SAS switch 764 of SAS switch B high-speed fabric 760 through high-speed connectors 756, 758. Also, processor blades 702 connect to non-blocking SAS switch 712 and non-blocking SAS switch 764 through 2X interfaces. HA traffic occurs through these interfaces. Non-blocking SAS switches 712, 714, 762, 764 may also provide dual 4X connectors for external devices.
FIG. 8 is a flowchart illustrating operation of configuring various types of blades in a blade server system with high-speed switch fabrics in accordance with an illustrative embodiment. Operation begins and provides a serial attached SCSI (SAS) switch with a SAS expander for host adapter (HA) traffic and a SAS expander for device adapter (DA) traffic (block 802). Then, one provides in the SAS switch a 4X connector for each blade slot, where each 4X connector is partitioned into dual 2X interfaces for HA traffic and DA traffic (block 804). Thereafter, one connects the 2X interfaces for HA traffic to the SAS expander for HA traffic (block 806) and connects the 2X interfaces for DA traffic to the SAS expander for DA traffic (block 808).
One provides a processor blade with one or more 4X connectors, where each 4X connector is partitioned into dual 2X interfaces for HA and DA traffic (block 810). One provides a B-DEB blade with one or more 4X connectors, where each 4X connector is reduced to a single 1X interface for HA traffic (block 812). One provides an S-DEB blade with one or more 4X connectors, where each 4X connector is reduced to a single 2X interface for DA traffic (block 814). And, one provides an I-DEB blade with one or more 4X connectors, where each 4X connector is partitioned into dual 2X interfaces for HA and DA traffic (block 816).
Then, one connects a blade to the dual 2X interfaces on the SAS switch where HA traffic occurs on the 2X HA interface and DA traffic occurs on the 2X DA interface (block 818). Thereafter, operation ends. One may connect any combination of processor blades, B-DEBs, S-DEBs, and/or I-DEBs in block 818. The configuration of the switches and the connectors, particularly the manner in which the connectors are partitioned to separate the HA and DA traffic, allows any blade to be connected to any slot. The configuration uses existing high-speed SAS fabric topology.
Thus, the illustrative embodiments solve the disadvantages of the prior art by providing a high speed fabric that partitions the fabric into sub-fabrics for host adapter traffic and device adapter traffic. A high-speed switch includes an expander for host adapter traffic and an expander for device adapter traffic. The high-speed switch also includes a plurality of connectors, where each connector is partitioned into dual interfaces. For each connector, one interface is connected to the host adapter traffic expander and the other interface is connected to the device adapter traffic expander. Blades are provided with one or more connectors, where each connector is partitioned into dual interfaces for host adapter traffic and device adapter traffic. The blades may be any combination of processor blades, boot drive enclosure blades, switched bunch of disks enclosure blades, and/or integrated RAID controller disk enclosure blades.
The description of the present invention has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the invention in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiment was chosen and described in order to best explain the principles of the invention, the practical application, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated.
Patent Application
20080091810
