Patent Application
20180167282
This invention relates to the field of network address assignment in a Software-Defined Network which is controlled by a centralized controller.
This invention considers the problem of assigning addresses in a software-defined network (SDN) that comprises multiple independent network devices such as network switches, network interface cards (NICs) and servers.
The network devices are connected together and placed under the control of a (logically) centralized controller.
The network under consideration is mostly static with servers, and switches subject to infrequent changes.
The devices that comprise the SDN need to be assigned communication network addresses in order to communicate with each other within the SDN, to receive traffic and in order to statically enforce tight security. Each device's address should reflect where in the topology said device connects to allow routing and network access controls to be pre-configured.
Each network switch in the network has many ports. Some ports face external networks that are not part of the SDN, called external ports, while all other ports are called internal ports.
In a SDN, no routing, not even L2 is available after power-up unless the controller installs it. In legacy networks L2 and L3 connectivity would normally be established by distributed protocols such as STP [STP], which runs on the network devices themselves, and client server protocols such as BOOTP and DHCP [BOOTP, DHCP], or neighbor discovery [IPV6NEIGH]. Even the simple operation of a network switch resolving the mapping of L3-to-L2 addresses called ARP does not work in an SDN unless programmed. All functionality is software-defined, the flip-side is that it also means that there is no functionality such as communication unless it is explicitly specified by software. In an SDN all of a network's behaviors are specified at runtime by the centralized controller. This invention defines a new alternative for address assignment.
It is possible to adapt legacy network address assignment protocols to operate in an SDN environment or to program the SDN to allow the legacy protocol to operate, but that is cumbersome because those protocols are not designed for use in a software-designed network. They are complex in features as they must operate in nearly all networks under arbitrary conditions, and they need to be tweaked so that their network address assignments become known to the SDN controller. This creates a long, fragile dependency chain. This present invention exploits that fact that SDNs are specific instances of networks for which a software stack is instantiated. It is acceptable that the mechanism for address assignment that is described here is unrelated to and incompatible with all known legacy protocols and that it only works in switched networks that can be statically enumerated.
The invention takes into account the following requirements:
First, a network address assignment mechanism must provide to each network device Layer 2 and Layer 3 addresses for communication.
Second, in order to minimize controller load, addresses should only change if SDN topology changes.
Third, spoofing on the network, i.e., a network device assuming the identity of another, should be prevented by design.
The following paragraphs describe related inventions and published works of prior art that are applicable to the same or variants of this problem, solutions that seem to relate to this invention but for subtle reasons fail to address the problems described above, and other inventions upon which this invention builds. A list of detailed document references is provided following this discussion of Prior Art.
Traditionally computers and network devices receive their network address based on client-server protocols upon the network device broadcasting its unique identity, typically embedded in the device, to an address server. Most common is that the device address assignment bootstraps by using the device's MAC address which is uniquely created by the manufacturer of the network interface card (NIC) and hard-coded into the NIC as a unique ID an address request. The address server looks up its mapping table for the unique-id, if it finds one, it provides it to the requesting device, if not, it creates a new one, updates its own state, and replies with that new mapping to the requesting device.
The previous paragraph is an abstract description of how addresses are assigned in an IPv4 or IPv6 infrastructure with a DHCP server: a client broadcasts a request for address assignment [BOOTP, DHCP] to a central server; this broadcast contains the client's MAC address. The server responds with an address assignment for the device that is based on a lookup table that maps MAC addresses to layer 3 addresses or it assigns an address from a flexible pool. This method is obviously very difficult to secure. In particular, the switches that connect network devices cannot be configured as to which physical network port will receive traffic from a specific MAC, IP or IPv6. This is by design, because the protocols are designed to support client devices that are dynamically plugged in and unplugged from network switches.
IPv6 implements a new neighbor discovery protocol [IPV6NEIGH] solution in which devices self-assign an address suffix and then probe if any other device in the same LAN segment shares the same address. Devices receive an address prefix by asking the nearest gateway for said prefix. This doesn't require an address server program, but it requires broadcasting, multicasting, and other protocol interactions to be implemented in the network and routers. It is also difficult to secure, depends on correct support from all of the network participants. Logic is divided among many entities, which this invention aims to avoid.
In U.S. Pat. No. 8,856,384 the inventors define an alternate method to resolve the DHCP server bottleneck in which the controller simply rewrites legacy DHCP requests to go to alternate DHCP servers based on the requesters network. This method both enables a legacy protocol to be used in an SDN and it provides a method for scaling it, but it is substantially different from the method of this present invention which does not use any legacy protocols.
In U.S. Pat. No. 5,276,813 a method for assigning addresses in a bus system is described which, for the purpose of address assignment is nearly indistinguishable from the [BOOTP] protocol in which a client sends a request and receives a reply message.
U.S. Pat. No. 9,032,054 extends the above U.S. Pat. No. 5,276,813. U.S. Pat. No. 9,032,054, as part of its provisioning process, describes another method for assigning addresses to virtual components in a network, when said components are installed and started by virtual machine managers. The cited patent describes a virtual machine being launched on some machine by a management module, and the virtual network device sending an address request message as described in U.S. Pat. No. 5,276,813 and being granted such an address for a virtual machine. The address assignment problem is solved by creating a database that maps a requestor ID (physical port) to a device ID, which the network management module consults in order to assign an address. U.S. Pat. No. 9,032,054 invention does not address the problem of enumerating an entire software-defined network for the purpose of assigning physical addresses to the physical functions of a network.
The following paragraphs list the documents that are cited in this present invention as related art in the technical field of the invention:
Other Publications are listed with the reference labels that are used in this present invention.
This invention makes the simplifying assumption of a hierarchical interconnect for the devices in the SDN. Hierarchical is defined as follows. Switches of tier N only connect to switches of tier N−1 and N+1. A switch is of tier 0 if it only connects to external ports and tier 1 ports. tier 0 switches are known in advance and can be specified in a configuration file.
This invention relies on all switches connecting through a separate management network to the controller and having established communication with the controller in a legacy manner on the management network. The connection to the controller is required in order to execute the protocol of this present invention.
All devices in the SDN connect to a logically centralized SDN controller.
This invention treats the address assignment to network devices in a manner that is conceptually related to, but technically vastly different from shared I/O bus enumeration methods such as that of U.S. Pat. No. 5,276,813. What is similar is that the enumerated bus addresses reflect the path to the device, which is an idea that is also present in the hierarchical MAC addresses that are created by this invention. What is different is everything practical, the signalling, the protocols, the exact message exchanges, the namespaces, the methods of enumeration, and the types of addresses created.
In order to scan for new devices, this invention defines a probe packet type that is compatible with the recognized protocols of the SDN network infrastructure but that is intentionally not used in the SDN.
The SDN controller installs rules at each switch that redirect incoming probe messages to the SDN controller.
The SDN controller uses the SDN control protocol to inject probe messages into every port of the network. Those messages will be transmitted on the output port named in the SDN control message.
If the above injected packet flows to another SDN switch, which is prepared with the SDN rule of [2-17], the switch will receive and successfully match the probe message and, in accordance with the installed rule, immediately sends this message back to the controller which injected it per [2-18].
Upon receipt of its own probe message, the controller can infer connection topology as it learns which physical switch received the probe packet from which other switch. The message that returns the probe packet to the controller will contain enough information about the receiving switch itself, in particular its datapath ID, thus is enabling the controller to understand device topology.
If the receiving device is not a switch but a server within the SDN then the device it will also detect the probe packets and optionally derive from the intercepted probe packet its own position in the topology. This invention specifies a formula that the receiving host can use to infer a MAC and IP address for its own network interface within the SDN, purely based on the received probe packet.
The controller continuously injects probe packets to provide a simple method for recovery from failure and detection of changes in the physical network topology of the SDN.
Furthermore, the network is configured to prevent any node other than the controller from inserting probe packets.
The network is configured to drop any packets on SDN ports that do not originate from the MAC and IP that is assigned to that port (or devices behind that port).
Since addresses are computed by a formula, it is possible for every host to automatically populate its ARP tables with each IP and MAC address pair that is valid within the SDN.
The SDN controller learns the connectivity of all switches and hosts that are in the SDN.
The system uses a very minimal set of SDN protocol features, which is a benefit because different hardware vendors support different subsets, and often very restricted subsets, of SDN features.
Each computing device receives an L2 and L3 address that directly corresponds to its position in the topology of the SDN.
Neither the L2 nor the L3 address on the SDN can be spoofed from any port other than the port that is configured for the given L2 and L3 address.
The system is very scalable as the controller enumerates devices rather than having devices apply for addresses on their own which could lead to congestion implosion at the SDN controller.
Liveness is established through continuous pings, thus nodes can infer when they are disconnected from the fabric and enter fail-safe mode.
The addresses adjust automatically on physical reconnection to different ports in the SDN hierarchy.
The system does not require the SDN internal network to be configured in any workable L2 or L3 configuration before bootstrapping.
The addressing system is secure against remote attacks.
The addressing system is secure from attacks from compute devices that constitute the SDN.
This solution does not require ARP, consequently ARP spoofing is eliminated.
The solution is robust against spoofing in general.
All drawings are numbered. Sub-elements within each figure are labeled with a three digit number with the first digit representing the number of the figure in which the sub-element first appeared. The second and third digit uniquely identify the sub-element within the figure of its first appearance. The following drawings are provided to aid in the understanding of the detailed description of the invention.
Whenever a switch 102 connects to the controller 103, as shown in
The controller installs a rule on every switch 102 that matches a specific type of packet, called “probe packet,” and forces each matched packet to be sent back to the controller. The probe packet format used in this present invention is derived from the ARP packet, the format is shown in
The method of injecting probe packets is depicted in
It is obvious that the controller can conclude that switch 102 port 302 connects to switch 301 via port 309. Once this operation has completed for all ports and datapaths, the controller 103 can infer the topology of the system via a topological sort.
The sorting operation starts from a known set of top-tier (tier 0) switches, switches that only connect to SDN external networks 110 and tier 1 switches. All switches of tiers greater than 0 are assumed to be internal with regard to the SDN. The controller finds all switches which received a probe packet from a switch that was classified as tier 0, those switches are labeled as tier 1, and switches that have received probe packets from tier 1 are labeled as tier 2, and so on. This procedure is preferably executed recursively, interleaved with the sending of probe packets. For example, first inject probe packets into every tier 0 switch, process the identities of the tier 1 switches, assign numeric IDs to all those tier 1 switches and then inject probe packets into all tier 1 switches and so on, interleaving numbering and probing at every step.
The numbering of switches may be arbitrary, but consecutive and the maximal switch number must not overflow the bits set aside for switch enumeration 502 shown in
The entire probe packet format itself is shown in
The most important field of the probe packet is DL_DST 401, which is of a special fixed form 500 shown in
The addressing format 500 of
Each NIC must be configured in such a way that an incoming packet ending with a queue suffix 504 corresponding to the number Q, should be routed to the queue number Q on the receiving NIC. Some NICs allow bitmask matching, in which case such configuration can be performed on every network device 104-107 at boot time to route MAC address suffix to queue Q. Otherwise, these assignments are made when the first probe packet is detected as the probe packet defines the entire prefix and suffix of the address for the receiving NIC. The receiving host, knowing MAX_QUEUE, only needs to enumerate its queues, replacing 504 in the received packet and install fixed MAC-to-queue mappings on the receiving NIC.
The length of the individual fields are shown in the address format 500 as octets in the preferred embodiment but these numbers are deployment specific.
Returning to the controller side of the invention, the nested for-loop (
Each switch is configured with rules of the form:
dl_dst=0:0:0:switch:PORT:queue,action=output:PORT,
so that any packet destined for the given MAC address will always flow to the intended physical port
If a switch port PORT connects to another switch 302 SWITCH then an appropriate forwarding rule forwarding must be installed at the upstream switch 102.
dl_dst=0:0:0:SWITCH:0:0/ff:ff:ff:ff:0:0,action=output:PORT,
which means that all ports under SWITCH switch should be forwarded out of PORT.
The above rules ensure forwarding but they do not prevent spoofing, i.e., a switch port injecting packets from a fictitious source MAC or source IP address. To prevent spoofing, ingress packets must be filtered. This can be done by adding an anti-spoofing clause to every forwarding rule that matches a given ingress port PORT on switch SWITCH. Every such rule will be augmented with the additional non-spoofing match restriction,
M1: dl_src=0:0:0:SWITCH:PORT:0/ff:ff:ff:ff:ff:0.
The “added non-spoofing match restriction” approach of [4-16] can be modified on some OpenFlow switches that provide table-type pattern matches TTP [TTP]. Such TTP matches allow installing pure L2 filters on a per-port basis. If such a facility is available, then the source match M1 is installed as an ingress filter in the ingress TCAM in order to permit only traffic matching the non-spoofing match restriction M1.
At all external ports that separate the SDN (under the controller) from other networks 110 that do not obey the controller, the following two filtering rules are installed at highest match priority, in order to drop all specially formatted addresses per format 500 that are being sent to or received from external networks.
These rules ensure non-interference from remote attacks.
Each computer device with an operating system that is part of the SDN and receives a probe packet may interpret the received packet as follows. Since it received a packet for a specific destination MAC 401, from a source of the form 0:0:0:*:0:0, the receiver can be assured that based on the safety configuration [4-16]-[4-18] of the network, this packet could have only been originated from the controller. Therefore, the DL_DST 401 in the probe packet is the address that per controller should be assigned to the NIC that received the packet. If the receiving NIC is a multi-queue adapter, then it may enumerate the last octet of the received DL_DST MAC address with a number for each queue on the receiving host's NIC and assign the so-enumerated MAC address to the corresponding queue on the NIC using packet steering primitives such as U.S. Pat. No. 7,660,306.
The full probe packet 400 of
OFFSET:=(SWITCH*MAX_PORT*MAX_QUEUE)+(PORT*MAX_QUEUE)+QUEUE.
The sum of OFFSET and BASE_IP is a valid IP address for the receiving NIC. This method can be analogously applied to IPv6 addresses.
The receiver of a probe packet, if it is a host, will send back a reply packet. The reply packet swaps the DL_SRC 402 and DL_DST fields 401, and replaces the CTL_IP 411 field with its own (the host's) management IP address, only if the receiver has an interface connected to the management network. If the receiver does not have such a network interface, then the CTL_IP field will be zeroed out in the reply.
The reply to the probe automatically flows to the controller because the controller installs the following rule on every switch:
dl_dst=0:0:0:switch:0:0/ff:ff:ff:0:0:0,action=output:CONTROLLER
In this manner the controller is able to learn which physical ports respond to probe packets, and how the responding device is reached on the management network via a CTL_IP 411.
Any host receiving and replying to the probe message, will note its IP address and port number based on the above. It will also extract SWITCH 502 from the received probe packet DL_DST field and execute the algorithm of
The SDN of this example is small (see
A firewall can be implemented as a collection of several OpenVSwitch software instances, each running on a server (104-107), with a front-end physical OpenFlow switch 102. The physical OpenFlow 102 switch is configured to split all incoming traffic across all OpenVSwitch instances. This present invention assigns ports and MAC addresses to each of the software OpenVSwitch instances on physical hosts 104-107. Each software OpenVSwitch instance can be configured to receive packets on exactly one CPU and each MAC that is supplied to the host by the probe packets is tied to exactly one NIC queue which is tied to exactly one CPU core by packet steering. This method of traffic to CPU allocation delivers superior processing performance to a default address and resource assignment of resources as is customary in common server deployments.
The absence of probe replies from a well-formed MAC address 500 means that there is no OpenVSwitch and no part of the to-be-firewalled traffic will be forwarded to that MAC address.
All live ports (as evidenced by a reply to a probe message) receive a slice of all incoming traffic from the external network 110 because the controller installs rules at the switch to match incoming packets at the external ports, and rewrites their destination L2 address to a well-formed MAC address 500 representing a live port in the system. The manner of rewriting destination MAC addresses in order to affect load-balancing is outside the scope of this present invention.
