This application relates generally to overlay network routing over the publicly-routed Internet.
Distributed computer systems are well-known in the prior art. One such distributed computer system is a “content delivery network” (CDN) or “overlay network” that is operated and managed by a service provider. The service provider typically provides the content delivery service on behalf of third parties (customers) who use the service provider's shared infrastructure. A distributed system of this type typically refers to a collection of autonomous computers linked by a network or networks, together with the software, systems, protocols and techniques designed to facilitate various services, such as content delivery, web application acceleration, or other support of outsourced origin site infrastructure. A CDN service provider typically provides service delivery through digital properties (such as a website), which are provisioned in a customer portal and then deployed to the network. A digital property typically is bound to one or more edge configurations that allow the service provider to account for traffic and bill its customer.
A wide area network (WAN) is a telecommunications network e.g., with links across metropolitan, regional, national or international boundaries, that covers a broad geographical area, typically using leased telecommunication lines. Enterprises and government entities utilize WANs to relay data among employees, clients, buyers, and suppliers from various geographical locations. For example, a WAN commonly is used to connect local area networks (LANs) and other types of networks together, so that users and computers in one location can communicate with users and computers in other locations. Many WANs are built for one particular organization and are private. Other types of WANs include those built by Internet service providers, and these can be used to provide connections from an organization's LAN to the Internet. When a WAN is built using leased lines, a router positioned at each end of the leased line connects the LANs on each side to each other.
One common WAN approach using leased lines implements Multi-Protocol Label Switching (MPLS). MPLS is a standard-based technology for speeding up network traffic flow. In MPLS, a specific path (identified by a label) is set up for a given packet sequence, thereby obviating router look-up of a next address to which to forward the packet. MPLS works with various types of network protocols, such as IP, ATM and frame relay. While delivery over MPLS is efficient and secure, it also is expensive, primarily due to the cost of the leased line. As an alternative, WANs also can be built using less costly packet switching methods such as those that can take full advantage of the Internet's packet-switched network.
MPLS providers often must provide support for customers with branch offices that are not within reach of the provider's MPLS cloud. One common solution is for the MPLS provider to place VPN (IPsec) concentrators at the edge of their MPLS cloud. The provider may then provide the customer with a Customer Premises Equipment (CPE) device (e.g., a router) that will connect to a standard broadband Internet connection to connect to their MPLS services via the VPN concentrator. The number and location of the VPN concentrators, however, is often limited, resulting in varying performance depending on a branch office customer's location.
More generally, enterprises now desire to effectively utilize Internet links as an optimized wide area network (WAN), connecting branches, data centers, teleworkers and mobile users to applications over the Internet. Driven also by the impact of cloud computing and mobility, enterprises need a network service that can deliver an optimal and predictable cloud experience to users, preferably a network that is low-cost, easy-on, and global with security and optimization built-in.
The techniques herein provide for enhanced overlay network-based transport of traffic, such as IPsec traffic, e.g., to and from customer branch office locations, facilitated through the use of the Internet-based overlay routing infrastructure. This disclosure in particular describes a preferred method of selecting an ingress edge region of the Internet-based overlay network. The overlay network has an authoritative domain name service (DNS). Preferably, the method begins by receiving and mapping a service hostname to an IKEv2 destination of an outer IPsec tunnel associated with a first overlay network edge. The service hostname is received at the authoritative DNS from a DNS resolver associated with a customer router. An IKEv2 session is then established from the first overlay network edge to the customer router. Upon tunnel establishment, a secondary lookup is performed to determine whether the first overlay network edge is an appropriate ingress region. The secondary lookup is initiated to the authoritative DNS by the first overlay network edge. Based on a response to the secondary lookup, a IKEv2 redirect is issued from the first overlay network edge to a second overlay network edge. A new tunnel is then established from the second overlay network edge to the customer router. Thereafter, an additional lookup may also be performed to determine whether the second overlay network edge remains an appropriate ingress region. This additional lookup is initiated to the authoritative DNS by the second overlay network edge. Based on a response to the additional lookup, responses to one or more liveness probes otherwise being received over the new tunnel from the customer router may then be dropped, thereby triggering the customer router to initiate another service hostname lookup.
The foregoing has outlined some of the more pertinent features of the subject matter. These features should be construed to be merely illustrative. Many other beneficial results can be attained by applying the disclosed subject matter in a different manner or by modifying the subject matter as will be described.
For a more complete understanding of the subject matter and the advantages thereof, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:
In a known system, such as shown in
As illustrated in
A CDN edge server is configured to provide one or more extended content delivery features, preferably on a domain-specific, customer-specific basis, preferably using configuration files that are distributed to the edge servers using a configuration system. A given configuration file preferably is XML-based and includes a set of content handling rules and directives that facilitate one or more advanced content handling features. The configuration file may be delivered to the CDN edge server via the data transport mechanism. U.S. Pat. No. 7,111,057 illustrates a useful infrastructure for delivering and managing edge server content control information, and this and other edge server control information can be provisioned by the CDN service provider itself, or (via an extranet or the like) the content provider customer who operates the origin server.
The CDN may include a storage subsystem, such as described in U.S. Pat. No. 7,472,178, the disclosure of which is incorporated herein by reference.
The CDN may operate a server cache hierarchy to provide intermediate caching of customer content; one such cache hierarchy subsystem is described in U.S. Pat. No. 7,376,716, the disclosure of which is incorporated herein by reference.
The CDN may provide secure content delivery among a client browser, edge server and customer origin server in the manner described in U.S. Publication No. 20040093419. Secure content delivery as described therein enforces SSL-based links between the client and the edge server process, on the one hand, and between the edge server process and an origin server process, on the other hand. This enables an SSL-protected web page and/or components thereof to be delivered via the edge server.
In a typical operation, a content provider identifies a content provider domain or sub-domain that it desires to have served by the CDN. The CDN service provider associates (e.g., via a canonical name, or CNAME) the content provider domain with an edge network (CDN) hostname, and the CDN provider then provides that edge network hostname to the content provider. When a DNS query to the content provider domain or sub-domain is received at the content provider's domain name servers, those servers respond by returning the edge network hostname. The edge network hostname points to the CDN, and that edge network hostname is then resolved through the CDN name service. To that end, the CDN name service returns one or more IP addresses. The requesting client browser then makes a content request (e.g., via HTTP or HTTPS) to an edge server associated with the IP address. The request includes a host header that includes the original content provider domain or sub-domain. Upon receipt of the request with the host header, the edge server checks its configuration file to determine whether the content domain or sub-domain requested is actually being handled by the CDN. If so, the edge server applies its content handling rules and directives for that domain or sub-domain as specified in the configuration. These content handling rules and directives may be located within an XML-based “metadata” configuration file.
By way of further background, CDN customers may subscribe to a “behind the firewall” managed service product to accelerate Intranet web applications that are hosted behind the customer's enterprise firewall, as well as to accelerate web applications that bridge between their users behind the firewall to an application hosted in the internet cloud. To accomplish these two use cases, CDN software may execute on virtual machines hosted in one or more customer data centers, and on virtual machines hosted in remote “branch offices.” The CDN software executing in the customer data center typically provides service configuration, service management, service reporting, remote management access, customer SSL certificate management, as well as other functions for configured web applications. The software executing in the branch offices provides last mile web acceleration for users located there. The CDN itself typically provides CDN hardware hosted in CDN data centers to provide a gateway between the nodes running behind the customer firewall and the service provider's other infrastructure (e.g., network and operations facilities. This type of managed solution provides an enterprise with the opportunity to take advantage of CDN technologies with respect to their Company's intranet.
As an overlay, the CDN resources such as described above also may be used to facilitate wide area network (WAN) acceleration services between enterprise data centers (which may be privately-managed) and third party software-as-a-service (SaaS) providers. The following provides additional details regarding this type of solution.
In particular,
Many of the machines in the overlay are servers located near the edge of the Internet, i.e., at or adjacent end user access networks. As has been described above, e.g.,
A known OIP routing mechanism comprises a representative set of components, as illustrated in
In one known use scenario of the overlay network, one or more clients desire to send packets to a single IP address. This is illustrated in
The various connections used in the overlay network and as described typically are secured via SSL or other transport layer security (TLS) techniques.
A virtual private network (VPN)-as-a-service (or more generally, “network-as-a-service”) can be facilitated using an overlay IP (OIP) routing mechanism such as shown in
A description of this network-as-a-service approach is provided in U.S. Publication No. 2015/0188943.
With the above as background, the techniques herein provide for enhanced overlay network-based transport of traffic, such as IPsec traffic, e.g., to and from customer branch office locations, facilitated through the use of the Internet-based overlay routing infrastructure described above.
The following describes a new type of edge region, referred to a NEdge, which is used as an entry point for delivery of IPSec tunnel traffic using the overlay network's existing routing and loss-mitigation capabilities to provide improved reliability when compared to the standard Internet. An NEdge typically will include a set of edge machines, such as the machine shown in
By way of background, a typical router is a dynamic multipoint VPN (DMVPN) router, although this is not a limitation. A router of this type typically includes a network controller component, and it allows for secure connectivity from branch offices to hub routers (spoke to hub), as well as for branch to branch (spoke to spoke) connections. The overlay network transports DMVPN traffic between branch routers and hub routers (spoke to hub), as well as between branch to branch (spoke to spoke) routers. Typically, IKEv2 is used to authenticate routes with the overlay network, and the established IPsec tunnel is then used to authenticate DMVPN data traffic within the network. In this approach, an IKEv2 identifier payload is used to correlate IKEv2 traffic with a particular customer and to locate an appropriate secret to use for the IKEv2 authentication. When DMVPN routers communicate with one another, they establish an end-to-end IPsec tunnel using the Internet Key Exchange (IKE). This control plane IKE negotiation is performed over a pre-established outer authentication tunnel into the overlay network. Once the end-to-end (router-to-router) IPsec tunnels are established via IKE, data-plane IPsec packets for those tunnels are routed out a DMVPN GRE interface on the router, and are then routed to a best overlay network edge over the pre-established outer authentication tunnel. This edge selection is described in more detail below as the primary aspect of this disclosure. The notion of “best” is not meant to refer to some absolute requirement, but may be relative.
In one approach, the overlay network provider does not have visibility into the customer end-to-end traffic. Data packets transmitted between a router and the overlay network edge (and vice versa) use the additional layer of IPsec to restrict access to the overlay network. This additional layer of IPsec authenticates customer traffic with the service. This additional IPsec authentication layer is applied at the ingress to the overlay network but then stripped off at the edge before being forwarded to the rest of the overlay network.
In an alternative, and according to one aspect of this disclosure, the outer IPsec tunnel is only established between the branch office router and the NEdge.
As further background, the customer's routers (typically located behind the firewall) periodically send liveness probes to the NEdge, typically to facilitate best route selection as will be seen. DMVPN liveness probes are essentially ICMP echo packets. These packets are not encrypted within the inner IPsec tunnel. To a router then, receipt of replies from the NEdge indicates that the authentication tunnel is functioning properly. The NEdge machine turns these packets around (into an ICMP response), re-applies authentication, and then sends the response back to the edge router via the authentication tunnel. Should a router not receive a valid response from the NEdge, it will cease using the tunnel as a valid route for data packets, and will failover to another route option (such as the public Internet of a dedicated circuit).
According to this disclosure, CPE devices (e.g., customer edge routers) are configured to perform initial first mile edge selection, typically using a DNS request. The IKEv2 destination for the outer IPsec tunnels into the overlay network is configured with an overlay network hostname for the service. The CPE routers are typically configured to use a standard corporate DNS resolver to perform hostname resolution.
Preferably, both the DNS request and IKEv2 redirect mechanisms are only repeated at the time of tunnel establishment, or at the time of an IKE re-key event. Thus, there may be operating scenarios where there is a need for additional mechanisms to drive the router to reconnect to a different NEdge region. This additional aspect of the disclosure is now described.
In order to monitor the reachability of the active NEdge being used, IP-SLA' s (liveness probes) are used to detect any failure of the tunnel. As noted above, IP-SLA probes are sent over the tunnel to ensure connectivity using the tunnel. In addition, preferably NEdge performs periodic Mapper lookups on behalf of the routers connected into that edge region. Should the edge determine via this lookup that it is no longer a valid choice for a given router, it will begin to drop the IP-SLA probes sent over the tunnel from that router. If IP-SLA probes fail for any reason (either unintentional or deliberate), typically an event manager running in the router triggers an event that executes a local router action to use an alternate tunnel. As such, the NEdge can trigger this result by intentionally dropping the IP-SLA to indirectly signal the router to look for a new edge region.
Each above-described process preferably is implemented in computer software as a set of program instructions executable in one or more processors, as a special-purpose machine.
Representative machines on which the subject matter herein is provided may be Intel Pentium-based computers running a Linux or Linux-variant operating system and one or more applications to carry out the described functionality. One or more of the processes described above are implemented as computer programs, namely, as a set of computer instructions, for performing the functionality described.
While the above describes a particular order of operations performed by certain embodiments of the invention, it should be understood that such order is exemplary, as alternative embodiments may perform the operations in a different order, combine certain operations, overlap certain operations, or the like. References in the specification to a given embodiment indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic.
While the disclosed subject matter has been described in the context of a method or process, the subject matter also relates to apparatus for performing the operations herein. This apparatus may be a particular machine that is specially constructed for the required purposes, or it may comprise a computer otherwise selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer readable storage medium, such as, but is not limited to, any type of disk including an optical disk, a CD-ROM, and a magnetic-optical disk, a read-only memory (ROM), a random access memory (RAM), a magnetic or optical card, or any type of media suitable for storing electronic instructions, and each coupled to a computer system bus. A given implementation of the present invention is software written in a given programming language that runs in conjunction with a DNS-compliant name server (e.g., BIND) on a standard Intel hardware platform running an operating system such as Linux. The functionality may be built into the name server code, or it may be executed as an adjunct to that code. A machine implementing the techniques herein comprises a processor, computer memory holding instructions that are executed by the processor to perform the above-described methods.
While given components of the system have been described separately, one of ordinary skill will appreciate that some of the functions may be combined or shared in given instructions, program sequences, code portions, and the like.
While given components of the system have been described separately, one of ordinary skill will appreciate that some of the functions may be combined or shared in given instructions, program sequences, code portions, and the like. Any application or functionality described herein may be implemented as native code, by providing hooks into another application, by facilitating use of the mechanism as a plug-in, by linking to the mechanism, and the like.
The techniques herein generally provide for the above-described improvements to a technology or technical field, as well as the specific technological improvements to various fields including distributed networking, Internet-based overlays, WAN-based networking (using MPLS or otherwise), secure utilization of Internet links, and the like, all as described above.
Having described our invention, what we claim is set forth below.
Number | Date | Country | |
---|---|---|---|
62273479 | Dec 2015 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 15392649 | Dec 2016 | US |
Child | 17335121 | US |