Patent Grant
11863377
The present disclosure relates to computer networks, and more particularly to network discovery and configuration.
A computer network 104 (
This section summarizes some features of the present disclosure. Other features are described in subsequent sections. The invention is defined by the appended claims.
Some embodiments of the present invention completely or partially automate certain aspects of network configuration, such as configuration of a server's network ports connected to the network. This is particularly helpful if a server 110 has multiple network ports used for different purposes (different roles). For example, a server may be configured to load specific software (e.g. operating system) through a specific port. Also, it may be desirable to use faster ports for data traffic, while slower ports for management traffic. Typically, each port's role is manually configured by a human administrator. However, manual configuration is a labor-intensive, error-prone task. Therefore, some embodiments of the present invention at least partially automate configuring the ports for their roles.
Other features are within the scope of the invention, as defined by the appended claims.
This section describes some embodiments in detail. The invention is defined by the appended claims.
Computer 200 includes a subsystem 204 connected to baseboard management controller (BMC) 206. Subsystem 204 includes one or more computer processors 210 executing computer programs stored in memory 220. Memory 220 is also used for data storage. Memory 220 may include volatile and/or non-volatile memories implemented as semiconductor, magnetic, optical, or other technologies.
The computer programs stored in memory 220 include BIOS (Basic Input/Output System) 230, which is a boot-strapping program executed automatically when the subsystem 204 is powered up. In this disclosure, the term “BIOS” is used broadly, to include any bootstrapping technology, for example UEFI (Unified Extensible Firmware Interface).
Memory 220 stores an operating system (OS) 234. In some embodiments, the OS is loaded into memory 220 in response to BIOS instructions. The OS is loaded from a storage location specified by OS image location identifier 236. The OS image storage can be part of computer 200, or can be remote (i.e. accessible over a network).
BMC 206 provides remote access to computer 200 even when the processors 210 are down and/or memory 220 is corrupt. BMC 206 may include its own computer processors and/or memory (not shown), and/or may share processors or memory or other resources with subsystem 204. Exemplary BMC types are Dell Remote Access Manager (DRAC) and integrated DRAC (iDRAC), that are available from Dell Corporation of the Unites States of America. See for example the following documents incorporated herein by reference: US Pre-Grant Patent Application US 2019/0020540 A1, published Jan. 17, 2019 (inventors: Yen et al.); US 2014/0215030 A1, published Jul. 31, 2014 (inventors: Terwilliger et al.); US 2014/0208133 A1, published Jul. 24, 2014 (inventors: Gopal et al.).
Subsystem 204 and BMC 206 are connected to the network via one or more ports P1, P2, . . . each of which is implemented by a Network Interface Card (NIC) 250. A port may include multiple subports, and a subport may include multiple slots, and each slot may provide a separate physical connection to the network. Also, a single NIC 250 may implement multiple ports or subports or slots. For simplicity, the term “port” will be used herein to refer to a port, a subport, a slot, or any other physical interface to the network, unless a different meaning is indicated. Also, for ease of description, we will assume that each NIC 250 corresponds to a single port; but the invention is not so limited.
Different ports may be used for different roles. For example, port P1 may be used for out-of-band (OOB) communications, i.e. communications with BMC 206. Ports P2, P3, P4 may be used for data communications (i.e. client communications and/or non-management communications). In the example of
One or more of ports P2, P3, P4 can also be used for in-band management, e.g. to transmit statistical or other data to a management computer (not shown), or receive configuration data for configuring the computer 200, e.g. to configure VLANs on the ports 250, or configure Virtual Machines (VMs), or to load the OS 234 from an OS image, for other types of configuration. An in-band management role and other roles can be defined in any way meaningful for a particular application of the network.
Network 310 includes a management station (“Management Solution” or MS) 320 and Virtual Machine Manager (VMM) 330. Each of nodes 110, 120, 320, 330 may have some or all of the components of computer system 200 of
Network links 335 include links 3350 (shown by thick dashed lines) carrying OOB traffic; links 335d carrying data traffic; and links 335i (thin dashed lines) carrying in-band management traffic. A link may have multiple uses, e.g. carry both in-band management and data traffic. A link 335 may be a physically link (electric or optical cable for example, or a string of cables). A link 335 may be a virtual link, possibly traversing different networks. The ports interconnected by a virtual link communicate as if they were interconnected by a physical link. For example, if the ports execute a Link Layer Discovery Protocol (LLDP), they treat each other as neighbors.
Switches 120.1 and 120.2 are part of the OOB network, i.e. dedicated to OOB traffic. (A switch may or may not be dedicated to OOB traffic or some other kind of traffic.) Switch 120.1 is immediately (directly) connected to ports P1 of servers 110.1, 110.2, 110.3. As used herein, immediate (direct) connection is a connection by a link 335.
Switches 120.3 through 120.6 are used for in-band management and data traffic. The switches 120.3 and 120.4 are immediately (directly) connected to servers 110. In some embodiments, servers 110 and switches 120.1, 120.3, 120.4 are mounted on a single rack (not shown), and switches 120.3 and 120.4 are “top of the rack” switches (TORs). In some embodiments, one or more servers 110 are configured for Storage Spaces Direct operation; see e.g. “Dell EMC Solutions for Microsoft Azure Stack HCl”, Dell Inc., 2019, Rev. A05, incorporated herein by reference. In some embodiments, one or more servers are configured for VSAN operation; see e.g. U.S. Pat. No. 8,862,799, issued Oct. 14, 2014, incorporated herein by reference. These details are exemplary and not limiting.
In this example, servers 110 will be configured to use their ports P30 for in-band management traffic (and possibly data traffic); ports P40 for data traffic on a VLAN 10; and ports P50 for data traffic on a VLAN 20. Such configuration can be at least partially automated by a process shown in
The process of
In this example, the ports are assigned logical names (such as “FastEth1/3” for port P30) to simplify port management for human administrators. In-band management ports P30, P40, P50 are configured with MTU=1500. The other ports are configured for data traffic with MTU=9162.
At step 420, server 110.1 is powered up.
At step 430, switches 120 execute a discovery protocol, e.g. Link Layer Discovery Protocol (LLDP), possibly over the in-band network (which may include the in-band management links 335i and/or data links 335d and/or other links, and the ports and computers interconnected by such links). The servers' NICs 250 can be pre-configured to respond to discovery protocol messages from the switches even if the servers have not yet been deployed. During discovery, each server's NIC 250 informs the immediately connected switch port of the server NIC's MAC address and possibly other properties, for example MTU etc. Some properties, including the MTUs, may be negotiated between the switch port and the server port in the discovery process, possibly changing the server settings obtained in step 410.
At the end of step 430, switch 120.3 may store, in its memory 220 (
At step 440, MS 320 communicates with other nodes' BMCs 206 (
At step 444, MS 320 uses a discovery protocol (e.g. LLDP) to identify switches 120 and the entire topology of network 310. MS 320 then requests the switches 120, possibly using SNMP or some other network protocol, possibly over the in-band network, to provide operation parameters for each switch port. The operation parameters may include, for each switch port, the Adjacent MACs, the VLAN IDs, and the Properties (see Table 2). In sending these requests to the switches, MS 320 may use the switches' MAC addresses as destination addresses.
At step 450, for each server port MAC address obtained at step 440, MS 320 identifies the immediately connected switch port (“SW's Adjacent Port”) and the Properties configured on the switch port. For example, MS 320 may look up the server port MAC address in the Adjacent MAC column in Table 2, and obtain the corresponding entries in the same row, which include the immediately connected switch port's MAC address (in the “MAC address” column), the VLAN ID, and the Properties (“MTU and other properties” column).
At step 460, MS 320 obtains a solution blueprint for network 310. The blueprints may be stored in any suitable database 340 (
The blueprint does not necessarily associate a role with a specific server port or switch port. The blueprint may associate a role with properties and/or VLANs. For example, the blueprint may associate the data traffic role with the MTU property value of 9162, and the in-band management role with MTU of 1500. In another example, the blueprint may associates the data traffic role with a VLAN ID of 10.
At step 470, for a role specified in the blueprint, the MS 320 reads the blueprint's corresponding parameters such as VLAN IDs or Properties, and MS 320 matches these parameters against the data received from the switches (step 450). If parameters match, MS 320 assigns the role to the corresponding switch port(s), and to the adjacent server ports. For example, if the blueprint associates a role with a VLAN ID, then all the switch ports associated with the same VLAN ID at step 450, and their adjacent server ports, will be assigned the same role.
Many implementations of this process are possible. For example, in some embodiments, instead of looping through the roles, MS 320 may loop through the server ports (obtained at step 444), and for each server port, may determine the Adjacent switch port and the corresponding VLAN ID and/or Properties (as in step 450; steps 450 and 470 may be merged). If the blueprint specifies a role for the VLAN ID, and/or for any of the Properties, MS 320 will assign the role to the server port.
For example, suppose the blueprint associates the data traffic (or corresponding VLAN IDs) with MTU of 9162, and the in-band management traffic (or corresponding VLAN IDs) with MTU of 1500. In configuring the server 110.1 (
If the roles are inconsistent, e.g. one role is associated with a VLAN ID configured on a switch port, and a different role is associated with an MTU value on the same switch port, the inconsistency may be resolved by assigning multiple roles to the Adjacent server port, and/or assigning the roles based on some priority or other default mechanism, and/or by getting a human administrator's input. Alternatively or in addition, MS 320 may resolve the inconsistency using the history data for the server port or switch port roles. For example, if the server port or the adjacent switch port had the data traffic role in the most recent use, MS 320 may assign the data traffic role to the server port, or may show the history data to the user to help the user determine the role. MS 320 may keep the history in its memory 220 for example, or the history may be kept, for each switch or server, in the switch's or server's memory and provided to MS 320 upon request from the MS. In some embodiments, the administrator may be shown (e.g. on a computer monitor) different roles as possible candidates based on the inconsistent roles and/or history roles, and may be requested to pick the role using a user interface device (keyboard, mouse, touch-screen technology, voice recognition, or other suitable types).
The same techniques can be used if the blueprint specifies the same parameter values (e.g. the same MTU value) for different roles, so the parameters cannot be used to determine a port's role. For example, if all the roles are associated with the MTU value of 9162 and with no other parameters, and if all the switch ports have the MTU value of 9162, the server ports' roles cannot be determined from the process of
At step 480, the server port roles are used to configure the server 110, possibly by conventional techniques. For example, in some embodiments, MS 320 writes the server ports' roles in the OS image to be loaded into the server's memory 220. MS 320 then automatically powers down the server 110 (or powers the server subsystem 204) via an OOB command to BMC 206. Then MS 320 powers the server (or subsystem 204) back up to cause BIOS execution. BIOS 230 then loads the OS image containing the updated server port roles into memory 220.
In another embodiment, BIOS 230 is designed to load the OS 234 through an in-band management port. Upon discovering the server ports' roles, MS 320 writes the roles to server memory 220 at a location coordinated with the BIOS. Then MS 320 powers down each server and powers it up again, to cause the server to execute the BIOS 230. BIOS 230 uses the roles written by MS 320 to identify the in-band management port, and loads the OS through this port.
Some embodiments of the present invention are defined by the following clauses:
The invention includes switches, servers, management computer systems, and other computers for performing the methods described herein, and includes computer readable media with software for causing the computers to perform methods described herein. The invention is defined by the appended claims.
Although illustrative embodiments have been shown and described, a wide range of modification, change and substitution is contemplated in the foregoing disclosure and in some instances, some features of the embodiments may be employed without a corresponding use of other features. Accordingly, it is appropriate that the appended claims be construed broadly and in a manner consistent with the scope of the embodiments disclosed herein.
| Number | Name | Date | Kind |
|---|---|---|---|
| 7380025 | Riggins | May 2008 | B1 |
| 7466996 | Carballo | Dec 2008 | B2 |
| 8862799 | Rastogi et al. | Oct 2014 | B2 |
| 8880817 | Krishnan et al. | Nov 2014 | B2 |
| 9805322 | Kelkar et al. | Oct 2017 | B2 |
| 10097620 | Reddy et al. | Oct 2018 | B2 |
| 10169100 | Barzik et al. | Jan 2019 | B2 |
| 10419311 | Laribi et al. | Sep 2019 | B2 |
| 10445695 | Chaganti | Oct 2019 | B2 |
| 10798014 | Krishnamurthy | Oct 2020 | B1 |
| 20040255154 | Kwan | Dec 2004 | A1 |
| 20060168152 | Soluk | Jul 2006 | A1 |
| 20060274774 | Srinivasan | Dec 2006 | A1 |
| 20110093579 | Koizumi | Apr 2011 | A1 |
| 20140208133 | Gopal et al. | Jul 2014 | A1 |
| 20140215030 | Terwilliger et al. | Jul 2014 | A1 |
| 20140376558 | Rao | Dec 2014 | A1 |
| 20150317169 | Sinha et al. | Nov 2015 | A1 |
| 20170286164 | Kinoshita | Oct 2017 | A1 |
| 20190020501 | Yu | Jan 2019 | A1 |
| 20190020540 | Yen et al. | Jan 2019 | A1 |
| 20200106862 | Chugtu | Apr 2020 | A1 |
| Entry |
|---|
| “Dell EMC Solutions for Microsoft Azure Stack HCI”, Dell Inc., 2019, Rev. A05. |
| Number | Date | Country | |
|---|---|---|---|
| 20210243078 A1 | Aug 2021 | US |
