Patent Application
20210126812
The present disclosure relates to computer networks, and more particularly to anycast address management in extended Local Area Networks (LANs).
Local Area Networks have limited geographical size, but are popular because they are fast and are easy to manage. Wide Area Networks (WANs) may have a larger size, but are slower, and may require more network management, needed to configure the WAN routers for example. Due to LAN advantages and easily available hardware and software to meet wide variety of needs, LANs have been extended to large areas by interconnecting different LAN regions via a WAN. An exemplary extended LAN is a VXLAN architecture described in RFC 7348, “Virtual eXtensible Local Area Network (VXLAN): A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks” (IETF, Mahalingam et al., August 2014), incorporated herein by reference.
In a VXLAN network, individual LAN regions are connected to gateways that communicate with each other via tunnels through an IP network. The tunnels terminate at Virtual Tunnel End Points (VTEPs) provided by the gateways. The gateways have interfaces connected to LAN regions, and have other interfaces connected to the IP network.
In some VXLAN implementations, the VTEP interfaces connected to the LAN regions are assigned the same, anycast address. See e.g. “VXLAN Network with MP-BGP EVPN Control Plane Design Guide” (Cisco, 2016). If a network node or virtual machine (VM) in one LAN region sends data to a VTEP (for forwarding to another LAN region or outside the VXLAN network), the network node or machine specifies the anycast address. Since the anycast address is the same in each LAN region, the network nodes or VMs do not need to be configured with a new VTEP address when moving from one LAN region to another. Therefore, moving between the LAN regions is facilitated.
This section summarizes some features of the invention. Other features are described in the subsequent sections. The invention is defined by the appended claims.
Configuring an anycast address in a large extended LAN, possibly having hundreds of VTEPs, is a time-consuming and error prone task for a human network administrator. Some embodiments of the present disclosure facilitate anycast address configuration in VXLANs or other types of extended LANs. For example, in some embodiments, if an anycast address is configured on one VTEP, the VTEP sends the anycast address to other VTEPs, and the other VTEPs automatically configure the anycast address on their interfaces connected to the LAN regions. If a VTEP receives conflicting anycast addresses from an administrator or other VTEPs, the VTEP resolves the conflict to select an anycast address according to a predefined protocol. In some embodiments, all VTEPs use the same conflict resolution protocol, so the same anycast address is installed at each VTEP despite the conflict. (In this disclosure, the phrases “configure an address on an interface” and “install an address on an interface” are synonymous, and denote causing an interface to have said address.)
The invention is not limited to the features described above. Other features of the invention are described below. The invention is defined by the appended claims.
The switch populates the forwarding table 150 using a number of techniques, including MAC learning based on the source addresses 120sa: if, for example, the switch receives a data frame with a source address MAC1 on a port P1, the switch associates the address MAC1 with the port P1 in table 150. This is called “data plane” learning, because the addresses are learned from the data traffic and not from management frames (control frames).
Forwarding table 150 has a limited maximum size, and is not suitable for a large network, e.g. the Internet. Large networks use network layer (layer 3) addresses, e.g. IP addresses. Network layer addresses are logical addresses, and they can be organized to assign a range of consecutive addresses to a subnet. Within the subnet, the ports Pi have the same layer-3 address prefix. Hence, packets can be forwarded to the subnet (to the subnet's edge router) based on the address prefix. The forwarding table size is therefore reduced.
Also, in a layer-3 network, flooding can be reduced by using control-plane address-learning techniques (routing protocols), such as OSPF, RIP, BGP, and others.
However, LANs are often faster, and are easier to manage, than layer-3 networks. Therefore, there is a demand to use LAN technology in large networks.
One factor increasing LAN popularity is the ease and efficiency of network virtualization. If a network operator (e.g. operating a data center or cloud) provides network services to multiple tenants (clients), the operator could provide a separate LAN for each tenant, but it is more economical to use the same LAN 104 for multiple tenants if different tenants' data can be isolated from each other. Tenant isolation can be accomplished by defining virtual LANs (VLANs) in a LAN. A data frame 120 (
Also, rather than using separate nodes 114 or separate servers for each tenant, the network operator may create a Virtual Machine (VM) 114vm, shown in
In VELAN network 310, each LAN region 104 may be as in
Each gateway 110gw has one or more interfaces P.IP connected to IP network 314, and one or more interfaces P.VL connected to one or more LAN regions 104. In
When a gateway 110gw.i receives a frame 120 from LAN region 104.i, and determines, from the frame's destination address 120da (and possibly the VLAN ID 120vid and/or other packet fields), that the destination (110 or 114 or 114vm) is in a different LAN region 104.j, the gateway 110gw.i encapsulates the frame 120 into an IP packet 320, with the IP destination address 320da being the address of gateway 110gw.j, possibly of an interface P.IP of gateway 110gw.j. The IP addresses of interfaces P.IP are shown in
The gateway 110gw.i sends the packet 320 to gateway 110gw.j via IP network 314. Gateway 110gw.j decapsulates the frame 120 and sends the frame to the destination address 120da in LAN region 104.j.
Each gateway 110gw implements one or more Virtual Tunnel End Points (VTEPs) for the virtual tunnels connecting the gateway's P.IP interfaces to other gateways. Each VTEP 110gw can run on a distributed or non-distributed computer system. We use the same numeral “110gw” for the VTEPs and the gateways. In fact, a VTEP can be viewed as a virtual gateway in VELAN communications.
For virtualization, packet 320 may include a Virtual Network Identifier (VNI) 320vni, similar to VLAN ID, identifying the corresponding virtual extended LAN (VELAN). Each VELAN is a separate layer-2 broadcast domain, and appears as a separate LAN to a tenant. The frames 120 are not allowed to cross VELAN boundaries, except possibly in a controlled manner as defined by the gateways 110gw and/or switches 110. The VNI can be longer than the VLAN ID. For example, for VXLANs, the VNI is 24 bits, while an Ethernet VLAN ID is 12 bits. VTEPs 110gw translate between VLAN IDs and VNIs as needed.
Each VTEP 110gw is configured, possibly by an administrator, to have MAC and/or IP addresses on its P.VL interfaces. For example, in VTEP 110gw.1, the MAC and IP addresses on the P.VL interface are denoted respectively as VL.MAC1 and VL.IP1.
Similarly, in VTEP 110gw.2, the MAC and IP addresses on the P.VL interface are denoted respectively as VL.MAC2 and VL.IP2.
In each LAN region 104, the physical and virtual machines 114, 110, 114vm should store the VLAN interface addresses VL.IP, VL.MAC of one or more VTEPs 110gw in the LAN region, so as to avoid wasteful address learning when a machine needs to communicate with a VTEP. In some systems, all VTEPs' P.VL interfaces are assigned the same, anycast addresses VL.IP, VL.MAC. For example, VL.MAC1 and VL.MAC2 can be the same anycast MAC address (VL.MAC); and VL.IP1 and VL.IP2 can be the same anycast IP address (VL.IP). Therefore, the need to re-configure the addresses on physical or virtual machines 114, 110, 114vm is reduced or eliminated when the machines move from one LAN region to another.
A VTEP may have multiple P.VL interfaces, connected to the same or different LAN regions 104. If so, all of them may share the anycast MAC address and the anycast IP address.
In some embodiments, different anycast addresses can be assigned for different VELANs, i.e. different VNIs, even on the same physical P.VL interface. A P.VL interface anycast address can be viewed as a VELAN interface address.
A VELAN network may have thousands or millions of VELANs and thousands of VTEPs. Manual configuration of each P.VL interface is a labor-intensive and error-prone task for a human administrator. Therefore, in some embodiments of the present invention, the administrator can configure anycast addresses on just one VTEP, or just one P.VL interface, and the VTEP can advertise the anycast addresses to other VTEPs. If a VTEP 110gw receives inconsistent anycast addresses from other VTEPs and/or administrator, the VTEP resolves the conflict according to some established rule. In some embodiments, all VTEPs use the same rule, to minimize the probability of inconsistent anycast addresses on different P.VL interfaces.
At step 414, the VTEP i sends a command to all other VTEPs, or at least to those VTEPs whose P.VL interfaces are connected to LAN regions including at least one node in the VELAN(s) specified by the VNI(s). The command commands the VTEPs to configure the P.VL interfaces with the anycast address or addresses received at step 410, for the VNI(s) received at step 410. This command can be sent in any suitable way, e.g. via a unicast transmission to each VTEP, or a multicast transmission. In particular, in some embodiments, the VTEPs or gateways form a multicast group. In some embodiments, a separate group is formed for each VNI. In some embodiments, in each group formed for a VNI, the group membership is reserved only for VTEPs whose corresponding LAN regions have nodes 110 or 114 or 114vm in the corresponding VELAN. Other embodiments have just one multicast group, that includes all VTEPs. Exemplary multicast protocols include bidirectional PIM (Protocol Independent Multicast) or PM-SM (sparse mode). Other VTEP-to-VTEP communication schemes are also possible.
At step 418, each VTEP j receives the command of step 414. Each VTEP j may install the anycast address(es) for the specified VNI(s) on all of its P.VL interface(s), or at least on those interfaces that handle traffic in one or more of the specified VNI(s).
In some embodiments, before installing the anycast addresses, the VTEP j checks, at step 422, whether any received anycast address is inconsistent with other anycast addresses that the VTEP j may have received for the P.VL interface for the same VNI(s). VTEP j may have received the other addresses from an administrator (as in step 410) or another VTEP (as step 418). In some embodiments, the other addresses are ignored if they were received some time ago greater than some time-to-live configured on the VTEP, e.g. 10 minutes. In other embodiments, each command from an administrator (step 410) or a VTEP (step 414) is time stamped, and at step 422 the VTEP j ignores the commands with time stamps older than the most recent time stamp by a predefined time-to-live value.
If a conflict exists, the VTEP j resolves the conflict using any suitable rule, possibly the same rule for all VTEPs. For example, a conflict can be resolved by selecting the lowest address. For example, suppose VTEP j received two or more different addresses VL.IP for the same VNI. Then the lowest of the VL.IP addresses can be selected. Alternatively, the highest address can be selected. In some embodiments, if the selected VL.IP address was received together with a VL.MAC address, then this VL.MAC address is configured for the P.VL interface. Alternatively, a separate check is made by VTEP j as to whether there are conflicting VL.MAC addresses, and if so then a VL.MAC address is selected using some predefined rule, the same for all VTEPs, e.g. as the lowest or highest of the conflicting VL.MAC addresses. In still another example, the IP.MAC address is selected first, and if the selected VL.MAC address was received together with a VL.IP address, then this VL.IP address is configured on the P.VL interface.
Other conflict resolution techniques are possible.
In some embodiments, the VL.MAC address is anycast but the VL.IP addresses are not, or vice versa.
Inter-VTEP communication for step 414 (anycast address advertisement) can be performed using any suitable protocol. Some embodiments use a modified BGP advertisement, e.g. using a modified EVPN (Ethernet Virtual Private Network) advertisement. EVPNs are described in the aforementioned RFC 7432. In particular, BGP requires participating routers to advertise Network Layer Reachability Information (NLRI) which defines routes available through the advertising router. These routes specify IP address prefixes reachable via the advertising router. See RFC 4271, “A Border Gateway Protocol 4 (BGP-4)”, Network Working Group, Rekhter et al., January 2006, incorporated herein by reference. The BGP EVPN specification of RFC 7432 defines additional NLRI routes, advertised by a gateway 110gw to specify IP and MAC addresses reachable via the advertising gateway in LAN regions 104. A modified NLRI can be used to specify the anycast addresses for the P.VL interfaces; such NLRI, transmitted at step 414, does not advertise any routes but merely informs the other VTEPs of the anycast addresses.
The BGP EVPN VTEPs communicate via TCP, so the NLRIs are transmitted in TCP payloads. A suitable NLRI for an anycast address advertisement is shown in Table 1. This format is similar to the NLRI for the EVPN Type-2 route, also called “MAC/IP Advertisement Route” in RFC 7432. Most fields are conventional or “don't care”. A new extended community attribute can be added to specify that this Mac/IP is anycast address and need to be installed for the pertinent VNI. The “Mac Address” and “IP Address” fields can carry the anycast addresses. A BGP advertisement (a message of type UPDATE) may carry multiple NLRIs. Therefore, multiple anycast addresses, for respective multiple VNIs, can be transmitted in a single BGP advertisement.
Embodiments of the present invention can be implemented using a wide variety of computer systems for physical and/or virtual nodes 110, 114, 110vm, 110gw. A computer system may include any instrumentality or aggregate of instrumentalities operable to compute, calculate, determine, classify, process, transmit, receive, retrieve, originate, switch, store, display, communicate, manifest, detect, record, reproduce, handle, or utilize any form of information, intelligence, or data for business, scientific, control, or other purposes. For example, this may be a personal computer (e.g., desktop or laptop), tablet computer, mobile device (e.g., personal digital assistant (PDA) or smart phone), server (e.g., blade server or rack server), a network storage device, or any other suitable device and may vary in size, shape, performance, functionality, and price. The information handling system may include random access memory (RAM), one or more processing resources such as a central processing unit (CPU) or hardware or software control logic, ROM, and/or other types of nonvolatile memory. Additional components of the information handling system may include one or more disk drives, one or more network ports for communicating with external devices as well as various input and output (I/O) devices, such as a keyboard, a mouse, touchscreen and/or a video display. The information handling system may also include one or more buses operable to transmit communications between the various hardware components.
One suitable computer system architecture is illustrated in
The invention covers a non-transitory computer readable medium (e.g. compact disk, flash memory, or other type) comprising one or more computer instructions which, when executed by a processor or processors of system 110 or 110gw or 114, cause the system to perform the methods within the scope of the present invention. The instructions can be stored in the system's memory.
Although illustrative embodiments have been shown and described, a wide range of modification, change and substitution is contemplated in the foregoing disclosure and in some instances, some features of the embodiments may be employed without a corresponding use of other features. The invention is not limited to virtual LANs or servers or nodes: in some embodiments, no virtual networks or VMs are defined. Other variations are also possible.
