Static routes for policy-based VPN

Abstract
Some embodiments provide a method for configuring a gateway datapath that processes data messages between a logical network implemented in a datacenter and an external network. The method receives configuration data including security policy rules for a logical router implemented by the datapath that indicate whether to apply a security protocol to certain data messages transmitted from a particular interface of the logical router. The method identifies a particular security policy rule that applies to data messages that (i) have a destination address in a set of destination addresses and (ii) meet at least one additional criteria. The method generates a static route, for a routing table used by the datapath to implement the logical router, that routes data messages with destination addresses in the set of destination addresses to the particular interface. The datapath applies the security policy rules for data messages transmitted from the particular interface.
Description
BACKGROUND

Virtual private networks (VPNs) are often used to protect data traffic that traverses public networks (e.g., the Internet) between datacenters (e.g., enterprise datacenters, branch offices, public cloud datacenters, etc.). These VPNs often use Internet Protocol Security (IPsec) or other security protocols in order to securely encrypt the data traffic before it leaves the source datacenter. In certain cases, a logical network implemented in one datacenter will send secure traffic to multiple other datacenters (e.g., on-premises enterprise datacenters, branch offices, virtual private clouds in public clouds, etc.), using multiple separate VPNs. To ensure that this traffic is protected and sent to the correct destination datacenter, techniques are required to avoid any potential conflicts between the multiple VPNs in use.


BRIEF SUMMARY

Some embodiments provide a gateway datapath for a logical network that uses static routes to route data messages with destination addresses governed by security protocol rules to an interface at which the security protocol rules are applied. By using these static routes, the gateway datapath implements both route-based virtual private networks (VPNs) as well as policy-based VPNs without conflict. To configure the gateway datapath, in some embodiments a network controller (e.g., a local controller executing on the same host computer as the gateway datapath) receives policy-based VPN rules (i.e., rules based on more criteria than just destination addresses) and generates the static routes for the gateway datapath based on the policy-based VPN rules.


The gateway datapath, in some embodiments, processes data messages between a logical network implemented in a datacenter and networks external to the logical network. These external networks may include public networks (e.g., for communication from public client machines through the Internet), as well as other datacenters to which the gateway datapath securely tunnels traffic via a VPN using a security protocol (e.g., IPsec). A logical network implemented in a first enterprise datacenter might be connected (via separate VPNs) to, e.g., one or more networks in public multi-tenant datacenters, one or more branch offices, etc.


Based on the configuration, these multiple VPNs might include both traditional policy-based VPNs as well as route-based VPNs. For a policy-based VPN, the gateway datapath applies security policy rules that indicate whether to encrypt data messages based on any number of criteria (e.g., source and/or destination network address, source and/or destination transport layer port number, etc.). In some embodiments, these security protocol rules are applied by the gateway datapath in a separate stage of processing after the data messages have been routed to a particular interface to which the rules apply. That is, the set of rules for a particular policy-based VPN are applied only to traffic that has been routed to the interface for which the VPN rules are configured.


To implement a route-based VPN, typically a separate interface (e.g., a virtual interface) is created for the VPN and the gateway datapath encrypts all traffic sent through that interface. To route traffic to the virtual interface, routes (i.e., based exclusively on the destination network address) are configured for the routing table used by the gateway datapath directing the appropriate traffic to that interface. In many cases, a default route will be installed in the routing table such that all routes not meeting a higher-priority route are routed to the virtual interface for the VPN. This default route could be configured based on administrator input (e.g., to a network management and control system for the datacenter) or based on a gateway on the other end of the VPN connection advertising the default route in order to attract data traffic from the logical network.


However, because the gateway datapath executes the routing stage prior to the security protocol stage for outgoing data messages, then a data message that should be processed according to the security protocol rules of a policy-based VPN will be routed to the virtual interface for the route-based VPN based on the default route and securely transmitted to the wrong destination. These data messages will either be dropped at the destination or routed in a loop (and eventually dropped).


Thus, some embodiments generate static routes for each applicable security policy rule of the policy-based VPN based on a destination address (or group of addresses) for which the rule specifies a security policy. For instance, if the security policy for a particular policy-based VPN specifies that all data from IP0 to IP2 is to be encrypted (where IP1 and IP2 may be prefixes that cover a range of IP addresses), some embodiments generate a static route that routes data messages sent to IP2 to the logical router interface for which the security policy rules are applied. This way, at the security protocol stage, the gateway datapath can then apply the security policy rules and encrypt the data message if required.


The preceding Summary is intended to serve as a brief introduction to some embodiments of the invention. It is not meant to be an introduction or overview of all inventive subject matter disclosed in this document. The Detailed Description that follows and the Drawings that are referred to in the Detailed Description will further describe the embodiments described in the Summary as well as other embodiments. Accordingly, to understand all the embodiments described by this document, a full review of the Summary, Detailed Description and the Drawings is needed. Moreover, the claimed subject matters are not to be limited by the illustrative details in the Summary, Detailed Description and the Drawing, but rather are to be defined by the appended claims, because the claimed subject matters can be embodied in other specific forms without departing from the spirit of the subject matters.





BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth in the appended claims. However, for purpose of explanation, several embodiments of the invention are set forth in the following figures.



FIG. 1 conceptually illustrates an example of a logical network that might be implemented in a datacenter of some embodiments.



FIG. 2 conceptually illustrates a first enterprise datacenter in which one or more logical networks are implemented, as well as several additional datacenters that connect to this logical network via VPNs.



FIG. 3 conceptually illustrates the operation of a datapath on a gateway host computer.



FIG. 4 conceptually illustrates a process for generating static routes for policy-based security protocol rules and inserting these static routes into the SR routing table.



FIG. 5 illustrates an example of policy-based IPSec rules and the resultant static routes for those rules.



FIG. 6 conceptually illustrates an electronic system with which some embodiments of the invention are implemented.





DETAILED DESCRIPTION

In the following detailed description of the invention, numerous details, examples, and embodiments of the invention are set forth and described. However, it will be clear and apparent to one skilled in the art that the invention is not limited to the embodiments set forth and that the invention may be practiced without some of the specific details and examples discussed.


Some embodiments provide a gateway datapath for a logical network that uses static routes to route data messages with destination addresses governed by security protocol rules to an interface at which the security protocol rules are applied. By using these static routes, the gateway datapath implements both route-based virtual private networks (VPNs) as well as policy-based VPNs without conflict. To configure the gateway datapath, in some embodiments a network controller (e.g., a local controller executing on the same host computer as the gateway datapath) receives policy-based VPN rules (i.e., rules based on more criteria than just destination addresses) and generates the static routes for the gateway datapath based on the policy-based VPN rules.


The gateway datapath, in some embodiments, processes data messages between a logical network implemented in a datacenter and networks external to the logical network. These external networks may include public networks (e.g., for communication from public client machines through the Internet), as well as other datacenters to which the gateway datapath securely tunnels traffic via a VPN using a security protocol (e.g., IPsec). A logical network implemented in a first enterprise datacenter might be connected (via separate VPNs) to, e.g., one or more networks in public multi-tenant datacenters, one or more branch offices, etc.



FIG. 1 conceptually illustrates an example of such a logical network 100 that might be implemented in a datacenter of some embodiments. As shown, the logical network 100 includes a logical router 105 and two logical switches 110 and 115. Two logical network endpoints (in this case, virtual machines) connect to each of the logical switches 110 and 115. In some embodiments, this logical network is configured by an administrator (e.g., an administrator of the datacenter in which the logical network is implemented, a tenant of the datacenter, etc.). The administrator configures these logical forwarding elements 105-115 by defining logical ports of the logical forwarding elements (e.g., the logical switch ports to which the VMs connect, the logical switch ports that connect to the logical router and corresponding logical router ports, uplink ports of the logical router that connect to external networks, etc.), configuring policies for the logical forwarding elements, etc.


As shown, the logical router 105 is defined to include multiple routing components 120-130 connected by a transit logical switch 135. These routing components include a distributed routing component (also referred to as a distributed router, or DR) 120 as well as multiple centralized routing components (also referred to as service routers, or SRs) 125 and 130. In some embodiments, a network management and control system receives the administrator configuration of the logical network and defines these routing components for the logical router 105. Each of the routing components 120-130 has a separate set of interfaces as well as a separate routing table. The DR 120 includes one northbound interface to the transit logical switch 135 to forward data messages to the SRs 125 and 130, which each include one southbound interface to the transit logical switch 135. In addition, the DR includes one southbound interface for each logical switch connected to the logical router 105 (as well as a southbound interface for any additional logical routers that connect to the logical router 105). The SRs 125 each include one or more uplink interfaces corresponding to uplinks configured by the administrator for the logical router 105 in some embodiments.


The logical forwarding elements 105-115 are implemented within the datacenter by physical managed forwarding elements of the datacenter (e.g., software forwarding elements such as virtual switches, etc.). In some embodiments, the logical switches 110 and 115 as well as the DR 120 (and the transit logical switch 135) are implemented in a distributed manner by numerous managed forwarding elements. That is, each of a set of the managed forwarding elements (MFEs) is configured (e.g., by the network management and control system) to perform processing on data messages according to the configuration of these logical forwarding elements. In some embodiments that use first-hop processing, and MFE executing on a host machine that hosts a logical network VM will receive a data message transmitted by that VM and perform logical processing for all (or many) of the distributed logical forwarding elements. For instance, the MFE receiving a data message sent by VM1 to an external endpoint would process the data message according to the logical switch 110, the DR 120, and the transit logical switch 135. In some embodiments, the SRs are each implemented centrally by a single host machine, and thus the MFE would transmit the data message to one of these SRs (based on the DR routing table).


The SRs 125 and 130 operate as gateways between the logical network 100 and external networks, such that all data messages between the endpoints of the logical network 100 and endpoints external to the logical network are processed by one of the SRs. These external networks may include public networks such as the Internet (via which client devices send data messages to and receive data messages from the VMs of the logical network) as well as other datacenters. In some cases, the SRs also implement stateful services for the logical network (e.g., a stateful firewall, load balancer, etc.). In different embodiments, the SRs may be implemented by a virtual machine or other data compute node, an MFE that also implements the other logical forwarding elements (e.g., a DPDK-based datapath), or another context.


It should be understood that this is a simplistic example of a logical network, and many logical networks include numerous virtual machines or other types of endpoints, additional logical switches, multiple tiers of logical routers, etc. For example, in some embodiments the logical router 105 that includes SRs connecting to external networks is a first tier of logical router managed by a datacenter administrator, and several different logical routers of a second tier (e.g., for different tenant logical networks that are managed by different tenant administrators) connect to this first-tier logical router.



FIG. 2 conceptually illustrates a first enterprise datacenter 200 in which one or more logical networks (such as the logical network 100) are implemented, as well as several additional datacenters that connect to this logical network via VPNs. As shown, the enterprise datacenter 200 includes several host machines, including a gateway host 205 and multiple additional host machines 210 that host logical network data compute nodes (DCNs), such as VMs. In some embodiments, the gateway host 205 implements an SR for a logical network (e.g., one of the SRs 125 and 130), while the VMs of that logical network execute on the hosts 210. In addition, MFEs executing on the hosts 210 as well as the gateway host 205 would implement the other (distributed) logical forwarding elements of the logical network, such as the logical switches 110, 115, and 135 as well as the DR 120). The enterprise datacenter 200 of some embodiments also includes additional gateway hosts for hosting other SRs of the same logical router or of additional logical networks.


The gateway host 205 connects the logical network endpoints executing on the host computers 210 of the datacenter to first and second branch offices 215 and 220 as well as a network in a public cloud 225. In some embodiments, these connections are configured the logical network administrator as VPN connections between the logical router (and thus an SR) and the different networks 215-225. It should be understood that these are examples of the types of networks that might be connected, and different embodiments could have different configurations of datacenter networks (e.g., a gateway connecting to multiple public clouds, a gateway in a branch office connecting to multiple other datacenters, etc.).


In this example, the multiple VPNs that the gateway host 205 implements include both traditional policy-based VPNs as well as route-based VPNs. Specifically, the first branch office 215 uses a policy-based VPN while the second branch office 220 and the public cloud 225 use route-based VPNs. For a policy-based VPN, the administrator configures (and the gateway applies) a set of security policy rules that indicate whether to encrypt data messages based on any number of criteria (e.g., source and/or destination network address, source and/or destination transport layer port number, etc.).


To implement a route-based VPN, typically a separate interface (e.g., a virtual tunnel interface) is created for the VPN and the gateway encrypts all traffic sent through that interface. To route traffic to the virtual interface, routes (i.e., based exclusively on the destination network address) are configured for the SR routing table (used by the gateway) to direct the appropriate traffic to that interface. In some cases, one or more of the networks to which the logical network connects via a VPN will have a legacy system that only uses policy-based VPNs; however, other systems (e.g., a public cloud) might use route-based VPNs. Due to the manner in which the gateway processes outgoing data packets, this combination of different types of VPNs creates the possibility of conflict between the two.



FIG. 3 conceptually illustrates the operation of a datapath 305 on a gateway host computer 300. As shown, the gateway datapath 305, a local controller 310, and a routing protocol application 315 all execute on the gateway host 300. In some embodiments, some or all of these modules execute in virtualization software (e.g., a hypervisor) of the gateway host 300. The gateway datapath 305 handles the packet processing for data messages entering and exiting the datacenter through the gateway host 300 in some embodiments (e.g., data messages sent from this datacenter to another datacenter via VPN, data messages between external devices and logical network endpoints, etc.). In this example, the gateway datapath implements the SR as well as the other logical forwarding elements of one or more logical networks, though in other embodiments the SR is implemented by a separate entity on the gateway host (e.g., a VM or other DCN) than the MFE that implements the other logical forwarding elements.


The local controller 310 configures the gateway datapath 305 to implement the logical networks in some embodiments, including the SRs and the various stateful services that the administrator configures for those SRs. In some embodiments, the local controller 310 configures the gateway datapath 305 based on configuration data that the local controller receives from a centralized network manager and/or controller 325 of a network management and control system. This configuration data includes routes for the gateway datapath, security policy (e.g., IPSec) rules for policy-based VPNs, firewall and load balancer configurations, etc. Though shown as a single box, the central network manager/controller 325 may include separate management plane and central control plane functions in some embodiments, one or both of which may operate in clusters (rather than as a single machine).


In some embodiments, the SR exchanges routes with external routers to which its uplinks connect as well as gateways with route-based VPNs using a routing protocol such as Border Gateway Protocol (BGP) or Open Shortest Path First (OSPF). The routing protocol application 315 is configured to generate route advertisement messages to advertise routes for logical network addresses (e.g., for the subnets of the logical switches, for public virtual IP addresses, etc.) to external public network routers and/or to other datacenters (e.g., via the VPN connections). In addition, when the gateway datapath 305 receives a routing protocol message from one of these peers of the SR, the datapath provides this message to the routing protocol application 315. The routing protocol application identifies additional routes for the SR and provides these routes to the local controller 310, which can use the routes to modify the SR configuration for the datapath 305.


This figure also shows a set of processing stages 320 implemented by the gateway datapath 305 when processing outgoing data messages for a particular logical network. In some embodiments, the gateway datapath 305 handles data messages for multiple logical networks (and multiple SRs) that it implements, and may have a different set of stages for different logical networks (e.g., not all logical networks will necessarily use a load balancer stage, or have multiple VPN connections).


As shown, the gateway datapath 305 initially performs logical network processing to logically forward such an outgoing data message to the SR. In some embodiments, the majority of this logical processing would have previously been performed by the first-hop MFE at the source of the data message, and the only remaining logical network processing involves forwarding the data message to the southbound SR interface based on a previously-determined output port of a transit logical switch that connects the SRs and DR of the logical router.


Next, the gateway datapath routes the data message according to the SR routing table. The SR routing table, in some embodiments, includes routes for incoming data messages (e.g., to send such data messages to the DR), as well as static routes configured for outgoing data messages and routes learned from external routers or gateways on the other end of a VPN connection using a routing protocol (e.g., BGP or OSPF). These are the routes that the routing protocol application 315 receives via routing protocol messages and provides to the local controller 310 to be installed in the SR routing table. In some embodiments, outgoing messages may be routed either to an uplink interface of the SR (in which case various additional processing stages associated with that uplink will be applied) or to a virtual tunnel interface for a route-based VPN connection. The gateway datapath (or a separate module on the gateway host) automatically applies a security protocol (e.g., IPSec) to all data messages routed to the virtual tunnel interface, then encapsulates and transmits these data messages to the datacenter at the other end of the VPN connection.


Data messages that are not routed to the virtual tunnel interface are processed by additional stages of the gateway datapath (e.g., the stages associated with the SR uplink interface to which the data messages are routed). In this case, these stages include load balancing and firewall stages as well as a policy-based IPSec stage. The policy-based IPSec stage applies the policy rules for any policy-based VPNs associated with the uplink. Thus, the set of rules for a particular policy-based VPN are applied only to traffic that has been routed to the interface for which the policy-based VPN rules are configured.


In many cases, a default route (e.g., in CIDR IPV4 notation, a route for 0.0.0.0/0) will be installed in the SR routing table such that all routes not meeting a higher-priority route are routed to a virtual tunnel interface for a route-based VPN. This default route could be configured based on administrator input or based on a gateway on the other end of the VPN connection advertising the default route in order to attract traffic from the logical network.


However, because the gateway datapath 305 executes the SR routing stage prior to the IPSec stage for outgoing data messages (for incoming data messages, the order of the stages is reversed), then a data message that should be processed according to the security protocol rules of a policy-based VPN will be routed to the virtual tunnel interface for the route-based VPN based on the default route and securely transmitted to the wrong destination. These data messages will either be dropped at the destination or routed in a loop (and eventually dropped).


Thus, some embodiments generate static routes for each applicable security policy rule of the policy-based VPN based on a destination address (or group of addresses) for which the rule specifies a security policy. For instance, if the security policy for a particular policy-based VPN specifies that all data from IP1 to IP2 is to be encrypted (where IP1 and IP2 may be prefixes that cover a range of IP addresses), some embodiments generate a static route that routes data messages sent to IP2 to the logical router interface for which the security policy rules are applied. This way, at the security protocol stage, the gateway datapath can then apply the security policy rules and encrypt the data message if required.



FIG. 4 conceptually illustrates a process 400 for generating static routes for policy-based security protocol rules and inserting these static routes into the SR routing table. In some embodiments, the process 400 is performed by the local controller on the gateway at which the SR is implemented (e.g., the local controller 310), to configure the gateway datapath that implements the SR routing table. In other embodiments, the process 400 (or a similar process) is performed by a central network controller or manager (e.g., the centralized network manager/controller 325), which then provides the data (e.g., via the local controller) to the gateway datapath. The process 400 will be described in part by reference to FIG. 5, which illustrates an example of policy-based IPSec rules and the resultant static routes for those rules.


As shown, the process 400 begins by receiving (at 405) configuration data for a policy-based VPN through which a logical network sends and receives data traffic. In some embodiments, the process 400 is only performed when the logical network (and thus the SR) also sends and receives data traffic through at least one route-based VPN, thereby necessitating the injection of static routes for the policy-based VPN. The configuration data could be initial policy rules for the VPN or updates to the policy rules for the VPN. These policy rules, in some embodiments, specify one or more match conditions for data messages and whether to encrypt such data messages (and in some cases how to encrypt the data messages or references to data that specifies how to encrypt the data messages). The match conditions may include source and/or destination IP address ranges, source and/or destination transport layer port numbers, or other packet header fields.



FIG. 5 illustrates an example set of such IPSec rules 500 for a policy-based VPN. These include two rules specifying to protect (encrypt) certain data messages, and one rule specifying to send other data messages unencrypted. Specifically, data messages with source IP addresses in the subnet 10.0.0.0/28 and destination IP addresses in the subnet 10.0.1.0/24 or with source IP addresses in the subnet 10.0.2.0/28 and destination IP addresses in the subnet 10.0.3.0/28 are to be encrypted. In addition, data messages with source IP addresses in the subnet 11.0.0.0/24 and destination IP addresses in the subnet 12.0.0.0/24 are to be sent unencrypted (but still encapsulated using the VPN tunnel headers so that the data message will reach the datacenter at the other end of the VPN connection).


Next, the process 400 determines (at 410) whether all of the policy rules specify a destination IP address range. In some embodiments, if a policy rule does not specify any destination IP address range (i.e., the rule applies to data messages with any destination IP address that also meet other criteria), then it is not possible to generate a non-default static route for the policy rule. Generating a default route would conflict with a default route for a route-based VPN and/or result in routing all traffic that does not match any higher-priority routes to the uplink interface.


As such, if the security policy rules include one or more rules that do not specify destination IP addresses, the process configures (at 415) the gateway datapath to use a policy-based routing stage. In some embodiments, the policy-based routing stage is performed prior to the SR routing stage for outgoing data messages, but after the other logical network processing has been applied (i.e., after a data message has been logically forwarded to the SR). The policy-based routing stage, in some embodiments, uses the policy-based VPN rules to forward data messages to the uplink interface with which the VPN is associated. While policy-based routing is another technique that avoids the conflict between default rules for route-based VPNs and the later application of policy-based VPN rules, applying policy-based rules is more resource- and time-intensive, and thus some embodiments only use a policy-based routing stage when the generation of static routes will not work because one or more of the policy-based rules does not limit the IP address range.


On the other hand, if all of the policy-based rules specify a destination IP address range, then the process 400 proceeds to generate static routes for the policy rules and provide these routes to the SR. As mentioned, the network management and control system (either the centralized manager and/or controller or the local controller on the gateway host computer) automatically generates these static routes and provides the routes to the SR without any additional user intervention (beyond the administrator providing the VPN policy rules).


Thus, the process 400 selects (at 420) one of the rules and generates (at 425) a route for the destination IP address range of the selected rule to the uplink with which the VPN is associated. FIG. 5 illustrates a subset of the SR routing table 505 that includes routes based on the policy-based IPSec rules 500. In addition to a first default route that routes data messages to a virtual tunnel interface for a route-based VPN, the illustrated portion of the SR routing table 505 includes routes based on each of the three policy rules 500. These routes specify that data messages in the IP address ranges 10.0.1.0/24, 10.0.3.0/28, and 12.0.0.0/24 should be output to a particular uplink interface of the SR (Uplink 1), with which this VPN is associated. As such, data messages with destination IP addresses in any of these subnets (and thus to which the policy-based security rules might apply) will be routed to this uplink rather than the virtual tunnel interface for the route-based VPN. After the intermediate service stages (e.g., load balancing, firewall) of the gateway datapath, the IPSec policy rules will be applied to these data messages to determine whether the data messages are sent over the VPN connection and whether they are encrypted.


The process 400 then determines (at 430) whether additional policy rules remain for which static routes need to be generated. If additional rules remain, the process 400 returns to 420 to select the next rule and generate a static route for that rule. Once all of the routes have been generated, the process provides (at 435) the routes to the routing table for the SR. In different embodiments, this entails having the local controller (or a different controller) performing route traversal including these new static routes in order to determine the actual forwarding information base (FIB) used by the gateway datapath to implement the SR, directly providing these routes to the configuration data used by the gateway datapath to implement the SR, or other mechanisms. The process 400 then ends. It should be understood that this is a conceptual process, and that the actual implementation of this process may deviate from the specific operations shown. For instance, some embodiments provide each route to the routing table as the route is generated rather than waiting for all of the static routes to be generated for the (potentially many) policy rules.



FIG. 6 conceptually illustrates an electronic system 600 with which some embodiments of the invention are implemented. The electronic system 600 may be a computer (e.g., a desktop computer, personal computer, tablet computer, server computer, mainframe, a blade computer etc.), phone, PDA, or any other sort of electronic device. Such an electronic system includes various types of computer readable media and interfaces for various other types of computer readable media. Electronic system 600 includes a bus 605, processing unit(s) 610, a system memory 625, a read-only memory 630, a permanent storage device 635, input devices 640, and output devices 645.


The bus 605 collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of the electronic system 600. For instance, the bus 605 communicatively connects the processing unit(s) 610 with the read-only memory 630, the system memory 625, and the permanent storage device 635.


From these various memory units, the processing unit(s) 610 retrieve instructions to execute and data to process in order to execute the processes of the invention. The processing unit(s) may be a single processor or a multi-core processor in different embodiments.


The read-only-memory (ROM) 630 stores static data and instructions that are needed by the processing unit(s) 610 and other modules of the electronic system. The permanent storage device 635, on the other hand, is a read-and-write memory device. This device is a non-volatile memory unit that stores instructions and data even when the electronic system 600 is off. Some embodiments of the invention use a mass-storage device (such as a magnetic or optical disk and its corresponding disk drive) as the permanent storage device 635.


Other embodiments use a removable storage device (such as a floppy disk, flash drive, etc.) as the permanent storage device. Like the permanent storage device 635, the system memory 625 is a read-and-write memory device. However, unlike storage device 635, the system memory is a volatile read-and-write memory, such a random-access memory. The system memory stores some of the instructions and data that the processor needs at runtime. In some embodiments, the invention's processes are stored in the system memory 625, the permanent storage device 635, and/or the read-only memory 630. From these various memory units, the processing unit(s) 610 retrieve instructions to execute and data to process in order to execute the processes of some embodiments.


The bus 605 also connects to the input and output devices 640 and 645. The input devices enable the user to communicate information and select commands to the electronic system. The input devices 640 include alphanumeric keyboards and pointing devices (also called “cursor control devices”). The output devices 645 display images generated by the electronic system. The output devices include printers and display devices, such as cathode ray tubes (CRT) or liquid crystal displays (LCD). Some embodiments include devices such as a touchscreen that function as both input and output devices.


Finally, as shown in FIG. 6, bus 605 also couples electronic system 600 to a network 665 through a network adapter (not shown). In this manner, the computer can be a part of a network of computers (such as a local area network (“LAN”), a wide area network (“WAN”), or an Intranet, or a network of networks, such as the Internet. Any or all components of electronic system 600 may be used in conjunction with the invention.


Some embodiments include electronic components, such as microprocessors, storage and memory that store computer program instructions in a machine-readable or computer-readable medium (alternatively referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, read-only compact discs (CD-ROM), recordable compact discs (CD-R), rewritable compact discs (CD-RW), read-only digital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), a variety of recordable/rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD cards, mini-SD cards, micro-SD cards, etc.), magnetic and/or solid state hard drives, read-only and recordable Blu-Ray® discs, ultra-density optical discs, any other optical or magnetic media, and floppy disks. The computer-readable media may store a computer program that is executable by at least one processing unit and includes sets of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as is produced by a compiler, and files including higher-level code that are executed by a computer, an electronic component, or a microprocessor using an interpreter.


While the above discussion primarily refers to microprocessor or multi-core processors that execute software, some embodiments are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions that are stored on the circuit itself.


As used in this specification, the terms “computer”, “server”, “processor”, and “memory” all refer to electronic or other technological devices. These terms exclude people or groups of people. For the purposes of the specification, the terms display or displaying means displaying on an electronic device. As used in this specification, the terms “computer readable medium,” “computer readable media,” and “machine readable medium” are entirely restricted to tangible, physical objects that store information in a form that is readable by a computer. These terms exclude any wireless signals, wired download signals, and any other ephemeral signals.


This specification refers throughout to computational and network environments that include virtual machines (VMs). However, virtual machines are merely one example of data compute nodes (DCNs) or data compute end nodes, also referred to as addressable nodes. DCNs may include non-virtualized physical hosts, virtual machines, containers that run on top of a host operating system without the need for a hypervisor or separate operating system, and hypervisor kernel network interface modules.


VMs, in some embodiments, operate with their own guest operating systems on a host using resources of the host virtualized by virtualization software (e.g., a hypervisor, virtual machine monitor, etc.). The tenant (i.e., the owner of the VM) can choose which applications to operate on top of the guest operating system. Some containers, on the other hand, are constructs that run on top of a host operating system without the need for a hypervisor or separate guest operating system. In some embodiments, the host operating system uses name spaces to isolate the containers from each other and therefore provides operating-system level segregation of the different groups of applications that operate within different containers. This segregation is akin to the VM segregation that is offered in hypervisor-virtualized environments that virtualize system hardware, and thus can be viewed as a form of virtualization that isolates different groups of applications that operate in different containers. Such containers are more lightweight than VMs.


Hypervisor kernel network interface modules, in some embodiments, is a non-VM DCN that includes a network stack with a hypervisor kernel network interface and receive/transmit threads. One example of a hypervisor kernel network interface module is the vmknic module that is part of the ESXi™ hypervisor of VMware, Inc.


It should be understood that while the specification refers to VMs, the examples given could be any type of DCNs, including physical hosts, VMs, non-VM containers, and hypervisor kernel network interface modules. In fact, the example networks could include combinations of different types of DCNs in some embodiments.


While the invention has been described with reference to numerous specific details, one of ordinary skill in the art will recognize that the invention can be embodied in other specific forms without departing from the spirit of the invention. In addition, a number of the figures (including FIG. 5) conceptually illustrate processes. The specific operations of these processes may not be performed in the exact order shown and described. The specific operations may not be performed in one continuous series of operations, and different specific operations may be performed in different embodiments. Furthermore, the process could be implemented using several sub-processes, or as part of a larger macro process. Thus, one of ordinary skill in the art would understand that the invention is not to be limited by the foregoing illustrative details, but rather is to be defined by the appended claims.

Claims
  • 1. A method for configuring a gateway datapath that processes data messages between a logical network implemented in a datacenter and an external network, the method comprising: receiving configuration data comprising a set of security policy rules for a logical router implemented by the gateway datapath, the security policy rules indicating whether to apply a security protocol to certain data messages transmitted from a particular interface of the logical router;identifying a particular security policy rule in the set of security policy rules that applies to data messages that (i) have a destination address in a set of destination addresses and (ii) meet at least one additional criteria; andgenerating a static route, for a routing table used by the gateway datapath to implement the logical router, that routes data messages with destination addresses in the set of destination addresses to the particular interface, wherein the gateway datapath applies the set of security policy rules for data messages transmitted from the particular interface.
  • 2. The method of claim 1, wherein the additional criteria comprises a data message having a source address in a set of source addresses.
  • 3. The method of claim 1, wherein the additional criteria comprises a data message having at least one of a particular source port and a particular destination port.
  • 4. The method of claim 1, wherein the static route routes data messages with destination addresses in the set of destination addresses irrespective of whether the data messages meet the additional criteria.
  • 5. The method of claim 1, wherein the routing table further comprises a default route that routes data messages, that are not routed by a higher-priority route, to a tunnel interface that applies the security protocol to the data messages and sends the data messages to a particular destination.
  • 6. The method of claim 5, wherein the particular destination is different from a destination for the data messages transmitted from the particular interface.
  • 7. The method of claim 6, wherein the logical router comprises a plurality of interfaces corresponding to a plurality of different destinations.
  • 8. The method of claim 7, wherein the logical network is implemented in a first datacenter and the plurality of different destinations comprises a set of additional datacenters.
  • 9. The method of claim 5, wherein the gateway datapath performs routing for the logical router prior to applying any security policy rules to data messages.
  • 10. The method of claim 1, wherein the security protocol is an encryption protocol for setting up a virtual private network (VPN).
  • 11. The method of claim 10, wherein the encryption protocol is Internet Protocol Security (IPSec).
  • 12. A non-transitory machine readable medium storing a program which when executed by at least one processing unit configures a gateway datapath that processes data messages between a logical network implemented in a datacenter and an external network, the program comprising sets of instructions for: receiving configuration data comprising a set of security policy rules for a logical router implemented by the gateway datapath, the security policy rules indicating whether to apply a security protocol to certain data messages transmitted from a particular interface of the logical router;identifying a particular security policy rule in the set of security policy rules that applies to data messages that (i) have a destination address in a set of destination addresses and (ii) meet at least one additional criteria; andgenerating a static route, for a routing table used by the gateway datapath to implement the logical router, that routes data messages with destination addresses in the set of destination addresses to the particular interface, wherein the gateway datapath applies the set of security policy rules for data messages transmitted from the particular interface.
  • 13. The non-transitory machine readable medium of claim 12, wherein the additional criteria comprises a data message having a source address in a set of source addresses.
  • 14. The non-transitory machine readable medium of claim 12, wherein the additional criteria comprises a data message having at least one of a particular source port and a particular destination port.
  • 15. The non-transitory machine readable medium of claim 12, wherein the static route routes data messages with destination addresses in the set of destination addresses irrespective of whether the data messages meet the additional criteria.
  • 16. The non-transitory machine readable medium of claim 12, wherein the routing table further comprises a default route that routes data messages, that are not routed by a higher-priority route, to a tunnel interface that applies the security protocol to the data messages and sends the data messages to a particular destination.
  • 17. The non-transitory machine readable medium of claim 16, wherein the particular destination is different from a destination for the data messages transmitted from the particular interface.
  • 18. The non-transitory machine readable medium of claim 17, wherein the logical router comprises a plurality of interfaces corresponding to a plurality of different destinations.
  • 19. The non-transitory machine readable medium of claim 18, wherein the logical network is implemented in a first datacenter and the plurality of different destinations comprises a set of additional datacenters.
  • 20. The non-transitory machine readable medium of claim 16, wherein the gateway datapath performs routing for the logical router prior to applying any security policy rules to data messages.
US Referenced Citations (350)
Number Name Date Kind
5504921 Dev et al. Apr 1996 A
5550816 Hardwick et al. Aug 1996 A
5751967 Raab et al. May 1998 A
6006275 Picazo et al. Dec 1999 A
6104699 Holender et al. Aug 2000 A
6219699 McCloghrie et al. Apr 2001 B1
6359909 Ito et al. Mar 2002 B1
6456624 Eccles et al. Sep 2002 B1
6512745 Abe et al. Jan 2003 B1
6539432 Taguchi et al. Mar 2003 B1
6680934 Cain Jan 2004 B1
6785843 McRae et al. Aug 2004 B1
6914907 Bhardwaj et al. Jul 2005 B1
6941487 Balakrishnan et al. Sep 2005 B1
6950428 Horst et al. Sep 2005 B1
6963585 Pennec et al. Nov 2005 B1
6999454 Crump Feb 2006 B1
7046630 Abe et al. May 2006 B2
7107356 Baxter et al. Sep 2006 B2
7197572 Matters et al. Mar 2007 B2
7200144 Terrell et al. Apr 2007 B2
7209439 Rawlins et al. Apr 2007 B2
7260648 Tingley et al. Aug 2007 B2
7283473 Amdt et al. Oct 2007 B2
7342916 Das et al. Mar 2008 B2
7391771 Drava et al. Jun 2008 B2
7447197 Terrell et al. Nov 2008 B2
7450598 Chen et al. Nov 2008 B2
7463579 Lapuh et al. Dec 2008 B2
7478173 Delco Jan 2009 B1
7483411 Weinstein et al. Jan 2009 B2
7555002 Amdt et al. Jun 2009 B2
7606260 Oguchi et al. Oct 2009 B2
7630358 Lakhani et al. Dec 2009 B1
7643488 Khanna et al. Jan 2010 B2
7649851 Takashige et al. Jan 2010 B2
7653747 Lucco et al. Jan 2010 B2
7710874 Balakrishnan et al. May 2010 B2
7742459 Kwan et al. Jun 2010 B2
7764599 Doi et al. Jul 2010 B2
7778268 Khan et al. Aug 2010 B2
7792097 Wood Sep 2010 B1
7792987 Vohra et al. Sep 2010 B1
7802000 Huang et al. Sep 2010 B1
7818452 Matthews et al. Oct 2010 B2
7826482 Minei et al. Nov 2010 B1
7839847 Nadeau et al. Nov 2010 B2
7885276 Lin Feb 2011 B1
7936770 Frattura et al. May 2011 B1
7937438 Miller et al. May 2011 B1
7948986 Ghosh et al. May 2011 B1
7953865 Miller et al. May 2011 B1
7987506 Khalid Jul 2011 B1
7991859 Miller et al. Aug 2011 B1
7995483 Bayar et al. Aug 2011 B1
8027260 Venugopal et al. Sep 2011 B2
8027354 Portolani et al. Sep 2011 B1
8031633 Bueno et al. Oct 2011 B2
8046456 Miller et al. Oct 2011 B1
8054832 Shukla et al. Nov 2011 B1
8055789 Richardson et al. Nov 2011 B2
8060875 Lambeth Nov 2011 B1
8131852 Miller et al. Mar 2012 B1
8149737 Metke et al. Apr 2012 B2
8155028 Abu-Hamdeh et al. Apr 2012 B2
8166201 Richardson et al. Apr 2012 B2
8194674 Pagel et al. Jun 2012 B1
8199750 Schultz et al. Jun 2012 B1
8223668 Allan et al. Jul 2012 B2
8224931 Brandwine et al. Jul 2012 B1
8224971 Miller et al. Jul 2012 B1
8239572 Brandwine et al. Aug 2012 B1
8259571 Raphel et al. Sep 2012 B1
8265075 Pandey Sep 2012 B2
8281067 Stolowitz Oct 2012 B2
8312129 Miller et al. Nov 2012 B1
8339959 Moisand et al. Dec 2012 B1
8339994 Gnanasekaran et al. Dec 2012 B2
8345650 Foxworthy et al. Jan 2013 B2
8351418 Zhao et al. Jan 2013 B2
8370834 Edwards et al. Feb 2013 B2
8416709 Marshall et al. Apr 2013 B1
8456984 Ranganathan et al. Jun 2013 B2
8504718 Wang et al. Aug 2013 B2
8559324 Brandwine et al. Oct 2013 B1
8565108 Marshall et al. Oct 2013 B1
8600908 Lin et al. Dec 2013 B2
8611351 Gooch et al. Dec 2013 B2
8612627 Brandwine Dec 2013 B1
8625594 Sakai et al. Jan 2014 B2
8625603 Ramakrishnan et al. Jan 2014 B1
8625616 Vobbilisetty et al. Jan 2014 B2
8627313 Edwards et al. Jan 2014 B2
8644188 Brandwine et al. Feb 2014 B1
8660129 Brendel et al. Feb 2014 B1
8705513 Merwe et al. Apr 2014 B2
8724456 Hong et al. May 2014 B1
8745177 Kazerani Jun 2014 B1
8958298 Zhang et al. Feb 2015 B2
9021066 Singh et al. Apr 2015 B1
9032095 Traina et al. May 2015 B1
9036639 Zhang May 2015 B2
9059999 Koponen et al. Jun 2015 B2
9137052 Koponen et al. Sep 2015 B2
9313129 Ganichev et al. Apr 2016 B2
9419855 Ganichev et al. Aug 2016 B2
9485149 Traina et al. Nov 2016 B1
9503321 Neginhal et al. Nov 2016 B2
9559980 Li et al. Jan 2017 B2
9647883 Neginhal et al. May 2017 B2
9749214 Han Aug 2017 B2
9787605 Zhang et al. Oct 2017 B2
10057157 Goliya et al. Aug 2018 B2
10075363 Goliya et al. Sep 2018 B2
10079779 Zhang et al. Sep 2018 B2
10095535 Dubey et al. Oct 2018 B2
10110431 Ganichev et al. Oct 2018 B2
10129142 Goliya et al. Nov 2018 B2
10129180 Zhang et al. Nov 2018 B2
10153973 Dubey Dec 2018 B2
10230629 Masurekar et al. Mar 2019 B2
10270687 Mithyantha Apr 2019 B2
10341236 Boutros et al. Jul 2019 B2
10382321 Boyapati et al. Aug 2019 B1
10411955 Neginhal et al. Sep 2019 B2
10454758 Boutros et al. Oct 2019 B2
10601700 Goliya et al. Mar 2020 B2
10623322 Nallamothu Apr 2020 B1
10700996 Zhang et al. Jun 2020 B2
10749801 Dubey Aug 2020 B2
10795716 Dubey et al. Oct 2020 B2
10797998 Basavaraj et al. Oct 2020 B2
10805212 Masurekar et al. Oct 2020 B2
20010043614 Viswanadham et al. Nov 2001 A1
20020067725 Oguchi et al. Jun 2002 A1
20020093952 Gonda Jul 2002 A1
20020194369 Rawlins et al. Dec 2002 A1
20030041170 Suzuki Feb 2003 A1
20030058850 Rangarajan et al. Mar 2003 A1
20030067924 Choe et al. Apr 2003 A1
20030069972 Yoshimura et al. Apr 2003 A1
20040013120 Shen Jan 2004 A1
20040073659 Rajsic et al. Apr 2004 A1
20040098505 Clemmensen May 2004 A1
20040267866 Carollo et al. Dec 2004 A1
20050018669 Arndt et al. Jan 2005 A1
20050027881 Figueira et al. Feb 2005 A1
20050053079 Havala Mar 2005 A1
20050083953 May Apr 2005 A1
20050120160 Plouffe et al. Jun 2005 A1
20050132044 Guingo et al. Jun 2005 A1
20060002370 Rabie et al. Jan 2006 A1
20060018253 Windisch et al. Jan 2006 A1
20060026225 Canali et al. Feb 2006 A1
20060029056 Perera et al. Feb 2006 A1
20060050719 Barr Mar 2006 A1
20060056412 Page Mar 2006 A1
20060092940 Ansari et al. May 2006 A1
20060092976 Lakshman et al. May 2006 A1
20060174087 Hashimoto et al. Aug 2006 A1
20060187908 Shimozono et al. Aug 2006 A1
20060193266 Siddha et al. Aug 2006 A1
20060291387 Kimura et al. Dec 2006 A1
20060291388 Amdahl et al. Dec 2006 A1
20070043860 Pabari Feb 2007 A1
20070064673 Bhandaru et al. Mar 2007 A1
20070140128 Klinker et al. Jun 2007 A1
20070156919 Potti et al. Jul 2007 A1
20070165515 Vasseur Jul 2007 A1
20070201357 Smethurst et al. Aug 2007 A1
20070206591 Doviak et al. Sep 2007 A1
20070297428 Bose et al. Dec 2007 A1
20080002579 Lindholm et al. Jan 2008 A1
20080002683 Droux et al. Jan 2008 A1
20080013474 Nagarajan et al. Jan 2008 A1
20080049621 McGuire et al. Feb 2008 A1
20080049646 Lu Feb 2008 A1
20080059556 Greenspan et al. Mar 2008 A1
20080071900 Hecker et al. Mar 2008 A1
20080086726 Griffith et al. Apr 2008 A1
20080151893 Nordmark et al. Jun 2008 A1
20080159301 Heer Jul 2008 A1
20080189769 Casado et al. Aug 2008 A1
20080225853 Melman et al. Sep 2008 A1
20080240122 Richardson et al. Oct 2008 A1
20080253366 Zuk et al. Oct 2008 A1
20080291910 Tadimeti et al. Nov 2008 A1
20090031041 Clemmensen Jan 2009 A1
20090043823 Iftode et al. Feb 2009 A1
20090064305 Stiekes et al. Mar 2009 A1
20090083445 Ganga Mar 2009 A1
20090092137 Haigh et al. Apr 2009 A1
20090122710 Bar-Tor et al. May 2009 A1
20090150527 Tripathi et al. Jun 2009 A1
20090161547 Riddle et al. Jun 2009 A1
20090249470 Litvin et al. Oct 2009 A1
20090249473 Cohn Oct 2009 A1
20090262741 Jungck Oct 2009 A1
20090279536 Unbehagen et al. Nov 2009 A1
20090292858 Lambeth et al. Nov 2009 A1
20090300210 Ferris Dec 2009 A1
20090303880 Maltz et al. Dec 2009 A1
20100002722 Porat et al. Jan 2010 A1
20100046531 Louati et al. Feb 2010 A1
20100107162 Edwards et al. Apr 2010 A1
20100115101 Lain et al. May 2010 A1
20100131636 Suri et al. May 2010 A1
20100153554 Anschutz et al. Jun 2010 A1
20100153701 Shenoy et al. Jun 2010 A1
20100162036 Linden et al. Jun 2010 A1
20100165877 Shukla et al. Jul 2010 A1
20100169467 Shukla et al. Jul 2010 A1
20100192225 Ma et al. Jul 2010 A1
20100205479 Akutsu et al. Aug 2010 A1
20100214949 Smith et al. Aug 2010 A1
20100275199 Smith et al. Oct 2010 A1
20100290485 Martini et al. Nov 2010 A1
20100318609 Lahiri et al. Dec 2010 A1
20100322255 Hao et al. Dec 2010 A1
20110016215 Wang Jan 2011 A1
20110022695 Dalai et al. Jan 2011 A1
20110026537 Kolhi et al. Feb 2011 A1
20110032830 Merwe et al. Feb 2011 A1
20110032843 Papp et al. Feb 2011 A1
20110075664 Lambeth et al. Mar 2011 A1
20110075674 Li et al. Mar 2011 A1
20110085557 Gnanasekaran et al. Apr 2011 A1
20110085559 Chung et al. Apr 2011 A1
20110103259 Aybay et al. May 2011 A1
20110119748 Edwards et al. May 2011 A1
20110134931 Merwe et al. Jun 2011 A1
20110142053 Merwe et al. Jun 2011 A1
20110149964 Judge et al. Jun 2011 A1
20110149965 Judge et al. Jun 2011 A1
20110194567 Shen Aug 2011 A1
20110205931 Zhou et al. Aug 2011 A1
20110261825 Ichino Oct 2011 A1
20110283017 Alkhatib et al. Nov 2011 A1
20110299534 Koganti et al. Dec 2011 A1
20110310899 Alkhatib et al. Dec 2011 A1
20110317703 Dunbar et al. Dec 2011 A1
20120014386 Xiong et al. Jan 2012 A1
20120014387 Dunbar et al. Jan 2012 A1
20120131643 Cheriton May 2012 A1
20120155467 Appenzeller Jun 2012 A1
20120182992 Cowart et al. Jul 2012 A1
20120236734 Sampath et al. Sep 2012 A1
20130007740 Kikuchi et al. Jan 2013 A1
20130044636 Koponen et al. Feb 2013 A1
20130044641 Koponen et al. Feb 2013 A1
20130051399 Zhang et al. Feb 2013 A1
20130058225 Casado et al. Mar 2013 A1
20130058229 Casado et al. Mar 2013 A1
20130058335 Koponen et al. Mar 2013 A1
20130058350 Fulton Mar 2013 A1
20130058353 Koponen et al. Mar 2013 A1
20130060940 Koponen et al. Mar 2013 A1
20130094350 Mandal et al. Apr 2013 A1
20130103817 Koponen et al. Apr 2013 A1
20130103818 Koponen et al. Apr 2013 A1
20130132536 Zhang et al. May 2013 A1
20130142048 Gross, IV et al. Jun 2013 A1
20130148541 Zhang et al. Jun 2013 A1
20130148542 Zhang et al. Jun 2013 A1
20130148543 Koponen et al. Jun 2013 A1
20130148656 Zhang et al. Jun 2013 A1
20130151661 Koponen et al. Jun 2013 A1
20130151676 Thakkar et al. Jun 2013 A1
20130208621 Manghirmalani et al. Aug 2013 A1
20130212148 Koponen et al. Aug 2013 A1
20130223444 Liljenstolpe et al. Aug 2013 A1
20130230047 Subrahmaniam et al. Sep 2013 A1
20130266007 Kumbhare et al. Oct 2013 A1
20130266015 Qu et al. Oct 2013 A1
20130266019 Qu et al. Oct 2013 A1
20130268799 Mestery et al. Oct 2013 A1
20130329548 Nakil et al. Dec 2013 A1
20130332602 Nakil et al. Dec 2013 A1
20130332619 Xie et al. Dec 2013 A1
20130339544 Mithyantha Dec 2013 A1
20140003434 Assarpour et al. Jan 2014 A1
20140016501 Kamath et al. Jan 2014 A1
20140059226 Messerli et al. Feb 2014 A1
20140146817 Zhang May 2014 A1
20140173093 Rabeela et al. Jun 2014 A1
20140195666 Dumitriu et al. Jul 2014 A1
20140229945 Barkai et al. Aug 2014 A1
20140241247 Kempf et al. Aug 2014 A1
20140269299 Koornstra Sep 2014 A1
20140328350 Hao et al. Nov 2014 A1
20140372582 Ghanwani et al. Dec 2014 A1
20140376550 Khan et al. Dec 2014 A1
20150009831 Graf Jan 2015 A1
20150016300 Devireddy et al. Jan 2015 A1
20150063360 Thakkar et al. Mar 2015 A1
20150063364 Thakkar et al. Mar 2015 A1
20150089082 Patwardhan et al. Mar 2015 A1
20150092594 Zhang et al. Apr 2015 A1
20150103838 Zhang et al. Apr 2015 A1
20150188770 Naiksatam et al. Jul 2015 A1
20150222550 Anand Aug 2015 A1
20150263897 Ganichev Sep 2015 A1
20150263946 Tubaltsev et al. Sep 2015 A1
20150263952 Ganichev et al. Sep 2015 A1
20150271011 Neginhal et al. Sep 2015 A1
20150271303 Neginhal et al. Sep 2015 A1
20150299880 Jorge et al. Oct 2015 A1
20160105471 Nunes et al. Apr 2016 A1
20160119229 Zhou Apr 2016 A1
20160182287 Chiba et al. Jun 2016 A1
20160191374 Singh et al. Jun 2016 A1
20160226700 Zhang et al. Aug 2016 A1
20160226754 Zhang et al. Aug 2016 A1
20160226762 Zhang et al. Aug 2016 A1
20160261493 Li Sep 2016 A1
20160294612 Ravinoothala et al. Oct 2016 A1
20160344586 Ganichev et al. Nov 2016 A1
20170005923 Babakian Jan 2017 A1
20170048129 Masurekar et al. Feb 2017 A1
20170048130 Goliya et al. Feb 2017 A1
20170063632 Goliya et al. Mar 2017 A1
20170063633 Goliya et al. Mar 2017 A1
20170064717 Filsfils et al. Mar 2017 A1
20170070425 Mithyantha Mar 2017 A1
20170126497 Dubey et al. May 2017 A1
20170180154 Duong et al. Jun 2017 A1
20170230241 Neginhal et al. Aug 2017 A1
20170288981 Hong Oct 2017 A1
20170317919 Fernando et al. Nov 2017 A1
20180006943 Dubey Jan 2018 A1
20180062914 Boutros et al. Mar 2018 A1
20180097734 Boutros et al. Apr 2018 A1
20180367442 Goliya et al. Dec 2018 A1
20190018701 Dubey et al. Jan 2019 A1
20190020580 Boutros et al. Jan 2019 A1
20190020600 Zhang et al. Jan 2019 A1
20190109780 Nagarkar Apr 2019 A1
20190124004 Dubey Apr 2019 A1
20190190885 Krug et al. Jun 2019 A1
20190199625 Masurekar et al. Jun 2019 A1
20190245783 Mithyantha Aug 2019 A1
20190281133 Tomkins Sep 2019 A1
20190312812 Boutros et al. Oct 2019 A1
20190334767 Neginhal et al. Oct 2019 A1
20190372895 Parthasarathy Dec 2019 A1
20200021483 Boutros et al. Jan 2020 A1
20200169496 Goliya et al. May 2020 A1
20200186468 Basavaraj et al. Jun 2020 A1
20200220802 Goliya et al. Jul 2020 A1
20200267095 Zhang et al. Aug 2020 A1
Foreign Referenced Citations (32)
Number Date Country
1442987 Sep 2003 CN
1714548 Dec 2005 CN
102780605 Nov 2012 CN
103890751 Jun 2014 CN
103947164 Jul 2014 CN
104335553 Feb 2015 CN
105791412 Jul 2016 CN
1653688 May 2006 EP
2838244 Feb 2015 EP
3013006 Apr 2016 EP
2000244567 Sep 2000 JP
2003069609 Mar 2003 JP
2003124976 Apr 2003 JP
2003318949 Nov 2003 JP
2004134967 Apr 2004 JP
2011139299 Jul 2011 JP
2011228864 Nov 2011 JP
2014534789 Dec 2014 JP
1020110099579 Sep 2011 KR
2005112390 Nov 2005 WO
2008095010 Aug 2008 WO
2013020126 Feb 2013 WO
2013026049 Feb 2013 WO
2013055697 Apr 2013 WO
2013081962 Jun 2013 WO
2013143611 Oct 2013 WO
2013184846 Dec 2013 WO
2015015787 Feb 2015 WO
2015142404 Sep 2015 WO
2016123550 Aug 2016 WO
2017027073 Feb 2017 WO
2018044746 Mar 2018 WO
Non-Patent Literature Citations (32)
Entry
Non-Published commonly Owned U.S. Appl. No. 16/447,939, filed Jun. 20, 2019, 37 pages, Nicira, Inc.
Non-Published commonly Owned U.S. Appl. No. 16/506,782, filed Jul. 9, 2019, 91 pages, Nicira, Inc.
Author Unknown, “Cisco Border Gateway Protocol Control Plane for Virtual Extensible LAN,” White Paper, Jan. 23, 2015, 6 pages, Cisco Systems, Inc.
Author Unknown, “Cisco Data Center Spine-and-Leaf Architecture: Design Overview,” White Paper, Apr. 15, 2016, 27 pages, Cisco Systems, Inc.
Moreno, Victor, “VXLAN Deployment Models—A Practical Perspective,” Cisco Live 2015 Melbourne, Mar. 6, 2015, 72 pages, BRKDCT-2404, Cisco Systems, Inc.
Non-published commonly owned U.S. Appl. No. 16/581,118 (N310.C1), filed Sep. 24, 2019, 36 pages, Nicira, Inc.
Non-published commonly owned U.S. Appl. No. 16/823,050 (N201.01.C2), filed Mar. 18, 2020, 79 pages, Nicira, Inc.
Non-published commonly owned U.S. Appl. No. 16/868,524 (N182.02.C2), filed May 6, 2020, 105 pages, Nicira, Inc.
Agarwal, Sugam, et al., “Traffic Engineering in Software Defined Networks,” 2013 Proceedings IEEE INFOCOM, Apr. 14, 2013, 10 pages, Bell Labs, Alcatel-Lucent, Holmdel, NJ, USA.
Aggarwal, R., et al., “Data Center Mobility based on E-VPN, BGP/MPLS IP VPN, IP Routing and NHRP,” draft-raggarwa-data-center-mobility-05.txt, Jun. 10, 2013, 24 pages, Internet Engineering Task Force, IETF, Geneva, Switzerland.
Author Unknown, “Defining Security Policies for Policy-based and Route-based VPNs,” Month Unknown 2018, 5 pages, Fortinet, Inc., retrieved at http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-ipsecvpn-54/Defining_VPN_Policies/Defining_Policies_for Policy_and_Route.htm.
Author Unknown, “VMware® NSX Network Virtualization Design Guide,” Month Unknown 2013, 32 pages, Item No. VMW-NSX-NTWK-VIRT-DESN-GUIDE-V2-101, VMware, Inc., Palo Alto, CA, USA.
Ballani, Hitesh, et al., “Making Routers Last Longer with ViAggre,” NSDI '09: 6th USENIX Symposium on Networked Systems Design and Implementation, Apr. 2009, 14 pages, USENIX Association.
Caesar, Matthew, et al., “Design and Implementation of a Routing Control Platform,” NSDI '05: 2nd Symposium on Networked Systems Design & Implementation , Apr. 2005, 14 pages, Usenix Association.
Dumitriu, Dan Mihai, et al., (U.S. Appl. No. 61/514,990), filed Aug. 4, 2011.
Fernando, Rex, et al., “Service Chaining using Virtual Networks with BGP,” Internet Engineering Task Force, IETF, Jul. 7, 2015, 32 pages, Internet Society (ISOC), Geneva, Switzerland, available at https://tools.ietf.org/html/draft-fm-bess-service-chaining-01.
Handley, Mark, et al., “Designing Extensible IP Router Software,” Proc. of NSDI, May 2005, 14 pages.
Koponen, Teemu, et al., “Network Virtualization in Multi-tenant Datacenters,” Technical Report TR-2013-001E, Aug. 2013, 22 pages, VMware, Inc., Palo Alto, CA, USA.
Lakshminarayanan, Karthik, et al., “Routing as a Service,” Report No. UCB/CSD-04-1327, Month Unknown 2004, 16 pages, Computer Science Division (EECS), University of California—Berkeley, Berkeley, California.
Lowe, Scott, “Learning LSX, Part 14: Using Logical Routing,” Scott's Weblog: The weblog of an IT pro specializing in cloud computing, virtualization, and networking, all with an open source view, Jun. 20, 2014, 8 pages, available at https://blog.scottlowe.org/2014/06/20/learning-nsx-part-14-using-logical-routing/.
Maltz, David A., “Routing Design in Operational Networks: A Look from the Inside,” SIGCOMM '04, Aug. 30-Sep. 3, 2004, 14 pages, ACM, Portland, Oregon, USA.
Non-Published commonly owned U.S. Appl. No. 16/210,410, filed Dec. 5, 2018, 29 pages, VMware, Inc.
Non-Published Commonly Owned U.S. Appl. No. 16/216,936 (N281.C1), filed Dec. 11, 2018, 47 pages, Nicira, Inc.
Rosen, E., “Applicability Statement for BGP/MPLS IP Virtual Private Networks (VPNs),” RFC 4365, Feb. 2006, 32 pages, The Internet Society.
Sajassi , Ali, et al., “Integrated Routing and Bridging in EVPN draft-sajassi-12vpn-evpn-inter-subnet-forwarding-04”, Jul. 4, 2014, 24 pages.
Shenker, Scott, et al., “The Future of Networking, and the Past of Protocols,” Dec. 2, 2011, 30 pages, USA.
Wang, Anjing, et al., “Network Virtualization: Technologies, Perspectives, and Frontiers,” Journal of Lightwave Technology, Feb. 15, 2013, 15 pages, IEEE.
Wang, Yi, et al., “Virtual Routers on the Move: Live Router Migration as a Network-Management Primitive,” SIGCOMM '08, Aug. 17-22, 2008, 12 pages, ACM, Seattle, Washington, USA.
Wang, Yushun, et al., “Connect Azure VPN gateways to multiple on-premises policy-based VPN devices using PowerShell,” VPN Gateway Documentation, Apr. 18, 2018, 5 pages, retrieved at https://docs.microsoft.com/en-us/azure/vpn-gateway/vpn-gateway-connect-multiple-policybased-rm-ps.
Non-published commonly owned U.S. Appl. No. 16/945,910 (N281.C2), filed Aug. 2, 2020, 46 pages, Nicira, Inc.
Non-Published commonly owned U.S. Appl. No. 17/062,531 (N242.C2), filed Oct. 2, 2020, 75 pages, Nicira, Inc.
Non-published commonly owned U.S. Appl. No. 17/068,588 (N179.01.C1), filed Oct. 12, 2020, 75 pages, Nicira, Inc.
Related Publications (1)
Number Date Country
20200195607 A1 Jun 2020 US