Systems and methods for scaling down cloud-based servers handling secure connections

Abstract

The disclosed technology relates to systems and methods for automatically scaling down network resources, such as servers or gateway instances, based on predetermined thresholds. A system is configured to detect a reduction in one or more network metrics related to a first server, and instruct the first server to issue a rekey request to a plurality of devices connected to the first server. The system is further configured to instruct a load balancer to route to at least one other server responses from the plurality of devices to the rekey request, and determine a number of connections remaining between the first server and the plurality of devices. The system may be further configured to instruct the load balancer to terminate the first server based on the detected number of connections remaining between the first server and the plurality of devices.

Description

TECHNICAL FIELD

This present disclosure relates in general to the field of computer networks, and more specifically to systems and methods for scaling down cloud-based servers handling secure connections.

BACKGROUND

In a cloud-managed network or cloud-based system, such as an enterprise private network or a data center network, devices such as endpoint machines, access points, routers, switches, servers, firewalls, gateways, other computing devices, virtual machines, containers (an instance of container-based virtualization), or resources (e.g., applications, endpoint groups, etc.) may connect to the cloud-based system over a secure connection, such as by a TLS, DTLS or IPSEC connection.

Cloud-based systems may utilize a virtual machine to serve as a scalable security gateway to manage secure connections between devices and cloud-based servers. Additional instances of the security gateway (e.g., TLS GW, IPSEC GW) may be spun up based on increased network traffic. Cloud-based systems, however, may not be capable of scaling down network resources due to the presence of secure connections that prevent termination of a network resource (e.g., security gateway instance or server) without disconnecting from connected devices.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to describe the manner in which the above-recited and other advantages and features of the disclosure can be obtained, a more particular description of the principles briefly described above will be rendered by reference to specific embodiments that are illustrated in the appended drawings. Understanding that these drawings depict only embodiments of the disclosure and are not therefore to be considered to be limiting of its scope, the principles herein are described and explained with additional specificity and detail through the use of the accompanying drawings in which:

FIG. 1 is a conceptual block diagram illustrating an example network environment, in accordance with various aspects of the subject technology.

FIG. 2 depicts a sequence diagram showing the communications between devices, a load balancer, servers, and a monitoring module, in accordance with various aspects of the subject technology.

FIG. 3 depicts an example method for scaling down a resource in a network environment, in accordance with various aspects of the subject technology.

FIG. 4 illustrates an example of a system in accordance with some aspects.

DESCRIPTION OF EXAMPLE EMBODIMENTS

The detailed description set forth below is intended as a description of various configurations of embodiments and is not intended to represent the only configurations in which the subject matter of this disclosure can be practiced. The appended drawings are incorporated herein and constitute a part of the detailed description. The detailed description includes specific details for the purpose of providing a more thorough understanding of the subject matter of this disclosure. However, it will be clear and apparent that the subject matter of this disclosure is not limited to the specific details set forth herein and may be practiced without these details. In some instances, structures and components are shown in block diagram form in order to avoid obscuring the concepts of the subject matter of this disclosure.

A cloud-managed network or cloud-based system may utilize secure connection protocols, such as by Transport Layer Security (TLS), Datagram Transport Layer Security (DTLS) or Internet Protocol Security (IPSEC), to connect devices to gateways (GW) or servers. Cloud-based systems may utilize a virtual machine or container to serve as a scalable security gateway to manage secure connections or tunnels between devices and GW instances. Additional GW instances (e.g., TLS GW, IPSEC GW) may be spun up based on increased network traffic.

During non-peak network traffic periods, connections to minimally loaded GW instances cannot be terminated without affecting secure tunnel sessions between the GW instance and connected devices. For example, if a single tunnel remains active on a minimally loaded GW instance, then the minimally loaded GW instance remains active until such time the connection is terminated by the connected device. In particular, where a tunnel is running over a TCP connection, the connection to the minimally loaded GW instance cannot be transitioned to another GW instance without disconnecting or otherwise negatively affecting the tunnel.

In addition, conventional hardware, such as on-premises data gateways, cannot be utilized to scale down GW instances in a cloud-based system because GW instances in a cloud-based system may be located in different clusters, regions, or data centers. Further, scaling down network resources using conventional on-premises data gateways require a central backup of a state of a connection, as well as information about the connection, and thereby require additional overhead.

Accordingly, there is a need in the art for certain embodiments of an intelligent and dynamically down-scalable cloud-based system to address these and/or other issues. Aspects of the subject technology relate to systems and methods for automatically scaling down network resources, such as servers or security GW instances, based on predetermined thresholds, without negatively affecting connectivity. Various embodiments of the disclosure are discussed in detail below. While specific implementations are discussed, it should be understood that this is done for illustrative purposes only. A person skilled in the relevant art will recognize that other components and configurations may be used without departing from the spirit and scope of the disclosure.

FIG. 1 illustrates a conceptual block diagram illustrating an example network environment 100, in accordance with various aspects of the subject technology. Various aspects are discussed with respect to a general wide area network for illustrative purposes, however, these aspects and others may be applied to other types of networks. For example, the network environment 100 may be implemented by any type of network and may include, for example, any one or more of an enterprise private network (EPN), cellular network, a satellite network, a personal area network (PAN), a local area network (LAN), a broadband network (BBN), the Internet, and the like. The network can be a public network, a private network, or a combination thereof. The network environment 100 may be implemented using any number of communications links associated with one or more service providers, including one or more wired communication links, one or more wireless communication links, or any combination thereof. Additionally, the network environment 100 can be configured to support the transmission of data formatted using any number of protocols (e.g., TLS, DTLS, IPSEC).

The network environment 100 may comprise a cloud-managed network or cloud-based system that includes one or more devices 110A-N. A device 110A-N may include machines (e.g., servers, personal computers, laptops), virtual machines, containers, mobile devices (e.g., tablets or smart phones), smart devices (e.g., set top boxes, smart appliances, smart televisions, internet-of-things devices), or network equipment, servers, containers, among other computing devices.

Each device 110A-N is configured to communicate with one or more servers 140A-N via a load balancer 120. For example, devices 110A-N may utilize software applications, browsers, or computer programs that are running on a device such as a desktop computer, laptop computer, tablet computer, server computer, smartphone, or any other apparatus on which an application (e.g., client application) is running that at some point in time, involves a user accessing a service or data provided by the server 140A-N. Devices 110A-N may operate pursuant to the TLS, DTLS, or IPSEC protocol to control how data (e.g., packets) are handled to provide for the data flow of content to the devices 110A-N. Other protocols for provisioning data flow to the devices 110A-N by the load balancer 120 and/or servers 140A-N may also be used.

The load balancer 120 may be configured to manage network traffic to the GW instances or servers 140A-N. In some aspects, an “instance” may refer to a virtual server in a cloud network, such as, for example, servers 140A-N. In cloud deployment of servers 140A-N (e.g., security GW instances), the servers 140A-N may be front-ended by the load balancer 120, which is configured to distribute secure tunnel sessions from devices 110A-N across available servers 140A-N according to one or more distribution schemes. The distribution schemes may, for example, comprise a round-robin distribution, weighted distribution, random distribution, or a combination of load balancing distribution schemes. The distribution schemes may also define a minimal number of instances required for creating a group, and may also define network conditions requiring additional instances to be added to existing groups. Network conditions that may trigger additional instances may include, for example, CPU usage, memory utilization on current instances reaching certain threshold limits, or a number of tunnels or connections reaching certain threshold limits.

The network environment 100 includes a monitoring module 130 connected to the load balancer 120 and servers 140A-N. The network environment 100 may also include additional components, fewer components, or alternative components, such as additional service providers, additional servers, different networks for different devices, and/or additional third-party servers. The network environment 100 may include additional components, such as routers, firewalls, or servers. The load balancer 120 and/or the monitoring module 130 may be implemented as a single machine or distributed across a number of machines in the network, and may comprise one or more servers. In some embodiments, the monitoring module 130 may be implemented as a part or component of another entity such as the load balancer 120 or a network controller.

The network devices (e.g., devices 110A-N, load balancer 120, monitoring module 130, and servers 140A-N) may be connected over links through ports. Any number of ports and links may be used. The ports and links may use the same or different media for communications. Wireless, microwave, wired, Ethernet, digital subscriber lines (DSL), telephone lines, T1 lines, T3 lines, satellite, fiber optics, cable and/or other links may be used.

According to the subject technology disclosed herein, the monitoring module 130 may be configured to scale down or automatically shrink a number of servers 140A-N (e.g., security GW instances) in a cloud-based deployment during non-peak network traffic periods without impacting existing secure connections or tunnels from the devices 110A-N. The monitoring module 130 may be configured to request or detect a number of secure tunnels or connections to a GW instance or server 140A-N, CPU usage, bandwidth utilization, response time, memory utilization, and/or usage of other computing resources. For example, the monitoring module 130 may request from each server 140A-N a number of active connections or tunnels to devices 110A-N. If the number of connections or tunnels for a particular server 140A-N is equal to or less than a predetermined threshold, the monitoring module 130 may run a scaling down or automatic shrink routine, as discussed further below. In another example, the monitoring module 130 may request from each server 140A-N CPU usage, memory usage, and/or other computing resource usage. If CPU or resource usage for a particular server 140A-N is equal to or less than a predetermined threshold, the monitoring module 130 may run a scaling down or automatic shrink routine, as discussed further below.

In some aspects, a number of servers 140A-N (e.g., security GW instances) may be scaled down by transferring secure connections or tunnels to servers 140A-N having connections, tunnel sessions, CPU usage, bandwidth utilization, response time, memory utilization, and/or usage of other computing resources that exceed lower limits of predetermined thresholds (e.g., minimally loaded security GW instances), to other available servers 140A-N (e.g., security GW instances). In one aspect, transfer of secure connections or tunnels may be accomplished without impacting tunnel connectivity by initiating a rekey routine. A rekey routine may refer to a process of changing a session key (e.g., encryption key of an ongoing communication) in order to limit the amount of data encrypted with the same key. A rekey routine may be run after a pre-set volume of data has been transmitted, a given period of time has passed, and/or a command is issued to force new key exchange. For example, the monitoring module 130 may be configured to instruct servers 140A-N (e.g., minimally loaded security GW instances) to initiate a rekey routine to all connected devices 110A-N with secure connections or tunnel sessions. In response, servers 140A-N (e.g., minimally loaded security GW instances) may transmit rekey requests to the connected devices 140A-N. Responses from connected devices 110A-N to the rekey requests may be routed by the load balancer 120 to other available servers 140A-N (e.g., security GW instances) to establish a new secure connection or tunnel session with a different server 140A-N, and thereby replace all secure connections or tunnel sessions to minimally loaded security GW instances. In some aspects, the monitoring module 130 may be configured to instruct the load balancer 120 to route all responses to the rekey requests from devices 110A-N to one or more servers 140A-N, other than minimally loaded servers or security GW instances. After the new secure connections or tunnel sessions are established, the minimally loaded security GW instances may be disconnected from the network thereby scaling down the number of servers 140A-N on the network. In one aspect, by establishing new secure connections or tunnel sessions with other available servers 140A-N (e.g., security GW instances) before terminating the secure connections or tunnel sessions to the minimally loaded servers or security GW instances, transfer of secure connections or tunnel sessions may occur without affecting tunnel connectivity with devices 110A-N.

FIG. 2 depicts a sequence diagram 200 showing the communications between devices 110A-N, load balancer 120, servers 140A-C, and monitoring module 130, in accordance with various aspects of the subject technology. The sequence diagram of FIG. 2 is performed by the devices shown. Devices 110A-N perform acts 205, 230, 245, 250, 260 and 265. The load balancer 120 performs act 235. The server 140A performs acts 240, 255 and 260. The server 140C performs acts 205 and 225. The monitoring module performs acts 210, 215 and 220. Other devices may perform any one or more of the acts, such as a different server. Any of the acts may involve operations by more than one component, such as the determination that threshold limits are met in act 210 by the monitoring module 130, or instruction to not send new session requests to server 140C in act 220 by the monitoring module 130.

Additional, different, or fewer acts may be provided. For example, acts for any one of the devices (e.g., devices 110A-N, load balancer 120, server 140A-C, and monitoring module 130) are performed with or without the other devices performing acts. In yet another example, instruction transmission, rekey processes, routing, or other networking acts are performed in addition to the acts shown in FIG. 2. The acts may be performed in the order shown. The order is listed in numerical sequence and/or from top to bottom in FIG. 2. In alternative aspects, the acts may be performed in other orders.

In act 205, a secure connection or tunnel session (e.g., TLS, DTLS, or IPSEC) is established between each device 110A-N and the server 140C (e.g., security GW instance) to provide for the data flow of content to the devices 110A-N. The monitoring module 130 is configured to carry out policies for detecting network conditions for scaling down servers 140A-C (e.g., containers, virtual machines, security GW instances, etc.). For example, monitoring module 130 may be configured to monitor network metrics, such as number of secure tunnels or connections, CPU usage, bandwidth utilization, response time, memory utilization, and/or usage of other computing resources, and compare values of the metrics with predetermined thresholds to determine whether lower limits of the predetermined thresholds are met or exceeded. By way of example, the monitoring module 130 may be configured to monitor the number of connections or tunnel sessions to all of the security GW instances. In act 210, if the monitoring module 130 determines that the number of tunnel sessions to server 140C is equal to or less than a predetermined threshold (e.g., 10, 100 or 1,000 tunnel sessions), the monitoring module 130 runs a scaling down or auto-shrink routine to transfer all connections or tunnel sessions from server 140C (e.g., minimally loaded security GW instance) to other available servers (e.g., security GW instances), and thereby allow server 140C to be subsequently terminated without negatively affecting secure connections or tunnel sessions from devices 110A-N.

In act 215, the monitoring module 130 instructs the server 140C (e.g., minimally loaded security GW instance) to initiate a rekey request to all secure connections or tunnel sessions connected to the server 140C. In act 220, the monitoring module 130 instructs the load balancer 120 to not send any new secure tunnel session requests (e.g., TCP handshake) to server 140C. In some aspects, the monitoring module 130 may update data associated with the load balancer 120 to cause the load balancer 120 to forward any new secure tunnel session requests (e.g., TCP handshake) to available servers 140A, B (e.g., security GW instances), other than server 140C.

In act 225, in response to the instruction from the monitoring module 130 in act 215, the server 140C transmits a rekey request to all connected devices 110A-N to initiate, for example, a TCP handshake. In act 230, devices 110A-N connected to server 140C that received the rekey request in act 225, transmit a response to the rekey request to initiate a new secure connection (e.g., Security Association (SA)). The response to the rekey request is received by the load balancer 120 and in act 235, routed, distributed or assigned to other available security GW instances, such as server 140A, based on the instruction from the monitoring module in act 220. Devices 110A-N and server 140A may engage in a handshake to negotiate and establish a new secure connection or tunnel session (e.g., TCP handshake) between the devices 110A-N and the server 140A. For example, in act 230, the devices 110A-N may transmit a TCP SYN message that is routed, distributed or assigned to server 140A in act 235 by the load balancer 120. In response, in act 240, the server 140A may transmit a TCP SYN ACK message to the devices 110A-N. In response, in act 245, the devices 110A-N may transmit a TCP ACK message to the server 140A.

In act 250, the devices 110A-N may transmit a “Hello” message to the server 140A and in act 255, the server 140A may respond with a “Hello” message back to the devices 110A-N. In act 260, a secure connection or tunnel session (e.g., TLS, DTLS, or IPSEC) is established between each device 110A-N and the server 140A (e.g., security GW instance) to provide for the data flow of content to the devices 110A-N. In one aspect, the secure connection established in act 205 between each device 110A-N and server 140C remains active and provides the data flow of content to each respective device 110A-N until the new secure connection of act 260 is established with the respective device 110A-N. Once the new secure connection of act 260 is established between the respective device 110A-N and the server 140C, in act 265 the secure connection established in act 205 between the respective device 110A-N and the server 140C may be terminated. In one aspect, because the secure connections established in act 205 between the devices 110A-N and the server 140C are terminated after the new secure connections are established in act 260 between the devices 110A-N and the server 140C, data flow of content to the devices 110A-N is not negatively impacted.

In some aspects, the monitoring module 130 may be further configured to communicate with the server 140C to confirm that there are no active secure connections or tunnel sessions. Once confirmed, the monitoring module 130 may instruct the load balancer 120 to disconnect server 140C to scale down the number of servers 140A-C active in the cloud-based network.

In other aspects, acts 225-265 occur on all connections or tunnel sessions associated with server 140C and may result in transfer of all connections or tunnel sessions from server 140C to server 140A within a timeframe of a few minutes.

FIG. 3 shows an example method 300 for scaling down network resources in a cloud-based network environment. It should be understood that, for any process discussed herein, there can be additional, fewer, or alternative steps performed in similar or alternative orders, or in parallel, within the scope of the various aspects unless otherwise stated. The method 300 can be performed by a network (e.g., the network environment 100 of FIG. 1) or similar system. For example, the method 300 may be performed by monitoring module 130 of FIG. 1.

At operation 302, a determination is made by the monitoring module 130 of FIG. 1 regarding whether one or more network metrics associated with a first server (e.g. first security GW instance) equals or exceeds a predetermined threshold. In other aspects, the determination may be based on detected number of connections or tunnel sessions associated with the first server, CPU usage, bandwidth utilization, response time, memory utilization, and/or usage of other computing resources, as discussed above. If the detected number of connections or tunnel sessions associated with the first sever (e.g. first security GW instance) equals or exceeds the predetermined threshold, an automatic-shrink or scale down routine is commenced indicating that there is excess capacity of security GW instances in a cloud-based network as may be the case during off-peak hours where the network may experience less demand and traffic.

At operation 304, the monitoring module 130 of FIG. 1 instructs the first server to initiate or issue a rekey request or procedure for all connections associated with the first server. In response, the first server may issue a rekey request from the first server to a plurality of devices connected to the first server. In response, the plurality of devices connected to the first server and receiving data via a secure connection or tunnel session from the first server (e.g., first security GW instance), transmit a rekey request (e.g., TCP handshake) to a load balancer.

At operation 306, the monitoring module 130 of FIG. 1 instructs the load balancer 120 of FIG. 1 to not send any subsequent rekey requests (e.g., TCP handshakes) to the first server. In one aspect, the load balancer may be instructed to send, route, or otherwise distribute any subsequent rekey requests (e.g., TCP handshakes) to a second server (e.g., second security GW instance). The load balancer may be configured to manage, distribute, or assign rekey requests received from a plurality of devices to a plurality of servers, including the first server (e.g., first security GW instance) and the second server (e.g., second security GW instance).

At operation 308, rekey requests from the plurality of devices are routed by the load balancer 120 of FIG. 1 to the second server, according to the instruction received at operation 306.

The rekey request forwarded, routed, or otherwise assigned to the second server is received by the second server. A secure connection (e.g., TLS, DTLS, IPSEC) between the second server and each of the plurality of devices may then be established to provide data flow to each of the plurality of devices. After a connection is established between the second server and a respective device of the plurality of devices, the secure connection (e.g., TLS, DTLS, IPSEC) between the respective device and the first server may be terminated, thereby relying solely on the secure connection between the second server and the respective device to provide data flow to the respective device. After each device of the plurality of devices establishes a secure connection with the second server, the connection to the first server may be terminated by the load balancer 120.

At operation 310, the monitoring module 130 of FIG. 1 may be configured to ping or query the first server to obtain information relating to the number of active connections or tunnel sessions associated with the first server. If the monitoring module 130 determines that there are no active connections or tunnel sessions to the first server, the monitoring module 130 may instruct the load balancer 120 to terminate the first server. At operation 312, after all devices of the plurality of devices establish respective secure connections with the second server, the first server may be disconnected from the cloud-based network or otherwise terminated, thereby reducing the number of servers (e.g., security GW instances) on the network.

In some aspects, the method 300 provides a method for scaling down security GW instances without compromising or otherwise negatively impacting data flow to devices. Encrypted application data may flow through existing secure tunnel sessions until after new secure tunnel sessions are established. As such, transitioning encrypted data or traffic from existing tunnel sessions to newly established tunnel sessions is seamless and does not negatively affect any application or data flow.

FIG. 4 depicts an example of a computing system 400 in which the components of the system are in communication with each other using connection 405. Connection 405 can be a physical connection via a bus, or a direct connection into processor 410, such as in a chipset architecture. Connection 405 can also be a virtual connection, networked connection, or logical connection.

In some embodiments, computing system 400 is a distributed system in which the functions described in this disclosure can be distributed within a datacenter, multiple datacenters, a peer network, etc. In some embodiments, one or more of the described system components represents many such components each performing some or all of the function for which the component is described. In some embodiments, the components can be physical or virtual devices.

System 400 includes at least one processing unit (CPU or processor) 410 and connection 405 that couples various system components including system memory 415, such as read only memory (ROM) 420 and random access memory (RAM) 425 to processor 410. Computing system 400 can include a cache 412 of high-speed memory connected directly with, in close proximity to, or integrated as part of processor 410.

Processor 410 can include any general purpose processor and a hardware service or software service, such as services 432, 434, and 436 stored in storage device 430, configured to control processor 410 as well as a special-purpose processor where software instructions are incorporated into the actual processor design. Processor 410 may essentially be a completely self-contained computing system, containing multiple cores or processors, a bus, memory controller, cache, etc. A multi-core processor may be symmetric or asymmetric.

To enable user interaction, computing system 400 includes an input device 445, which can represent any number of input mechanisms, such as a microphone for speech, a touch-sensitive screen for gesture or graphical input, keyboard, mouse, motion input, speech, etc. Computing system 400 can also include output device 435, which can be one or more of a number of output mechanisms known to those of skill in the art. In some instances, multimodal systems can enable a user to provide multiple types of input/output to communicate with computing system 400. Computing system 400 can include communications interface 440, which can generally govern and manage the user input and system output. There is no restriction on operating on any particular hardware arrangement and therefore the basic features here may easily be substituted for improved hardware or firmware arrangements as they are developed.

Storage device 430 can be a non-volatile memory device and can be a hard disk or other types of computer readable media which can store data that are accessible by a computer, such as magnetic cassettes, flash memory cards, solid state memory devices, digital versatile disks, cartridges, random access memories (RAMs), read only memory (ROM), and/or some combination of these devices.

The storage device 430 can include software services, servers, services, etc., that when the code that defines such software is executed by the processor 410, it causes the system to perform a function. In some embodiments, a hardware service that performs a particular function can include the software component stored in a computer-readable medium in connection with the necessary hardware components, such as processor 410, connection 405, output device 435, etc., to carry out the function.

It will be appreciated that computing system 400 can have more than one processor 410, or be part of a group or cluster of computing devices networked together to provide greater processing capability.

For clarity of explanation, in some instances the various embodiments may be presented as including individual functional blocks including functional blocks comprising devices, device components, steps or routines in a method embodied in software, or combinations of hardware and software.

In some aspects the computer-readable storage devices, mediums, and memories can include a cable or wireless signal containing a bit stream and the like. However, when mentioned, non-transitory computer-readable storage media expressly exclude media such as energy, carrier signals, electromagnetic waves, and signals per se.

Methods according to the above-described examples can be implemented using computer-executable instructions that are stored or otherwise available from computer readable media. Such instructions can comprise, for example, instructions and data which cause or otherwise configure a general purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions. Portions of computer resources used can be accessible over a network. The computer executable instructions may be, for example, binaries, intermediate format instructions such as assembly language, firmware, or source code. Examples of computer-readable media that may be used to store instructions, information used, and/or information created during methods according to described examples include magnetic or optical disks, flash memory, USB devices provided with non-volatile memory, networked storage devices, and so on.

Devices implementing methods according to these disclosures can comprise hardware, firmware and/or software, and can take any of a variety of form factors. Typical examples of such form factors include laptops, smart phones, small form factor personal computers, personal digital assistants, rackmount devices, standalone devices, and so on. Functionality described herein also can be embodied in peripherals or add-in cards. Such functionality can also be implemented on a circuit board among different chips or different processes executing in a single device, by way of further example.

The instructions, media for conveying such instructions, computing resources for executing them, and other structures for supporting such computing resources are means for providing the functions described in these disclosures.

Although a variety of examples and other information was used to explain aspects within the scope of the appended claims, no limitation of the claims should be implied based on particular features or arrangements in such examples, as one of ordinary skill would be able to use these examples to derive a wide variety of implementations. Further and although some subject matter may have been described in language specific to examples of structural features and/or method steps, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to these described features or acts. For example, such functionality can be distributed differently or performed in components other than those identified herein. Rather, the described features and steps are disclosed as examples of components of systems and methods within the scope of the appended claims.

Claims

1. A computer-implemented method, comprising: detecting a reduction in one or more network metrics related to a first server;instructing the first server to issue a rekey request to a plurality of devices connected to the first server in response to the detected reduction in the one or more network metrics related to the first server, wherein the rekey request is part of a rekey routine for changing one or more session keys of an ongoing communication with the plurality of devices;instructing, in response to the detected reduction in the one or more network metrics, a load balancer to route responses from the plurality of devices to the rekey request to at least one other server to connect the plurality of devices to at least one of the at least one other server in response to the rekey request;determining a number of connections remaining between the first server and the plurality of devices; andinstructing the load balancer to terminate the first server based on the determined number of connections remaining between the first server and the plurality of devices,wherein instructing the load balancer to route the responses from the plurality of devices to the at least one other server comprises instructing the load balancer to not send responses from the plurality of devices to the first server.

2. The computer-implemented method of claim 1, wherein the one or more network metrics comprises at least one of a number of connections to the first server, CPU usage of the first server, and memory utilization of the first server.

3. The computer-implemented method of claim 1, wherein the determining the number of connections remaining comprises transmitting a query to the first server to solicit a response indicating a number of active connections between the first server and the plurality of devices.

4. The computer-implemented method of claim 1, wherein the load balancer is instructed to terminate the first server when the determined number of connections remaining between the first server and the plurality of devices is zero.

5. The computer-implemented method of claim 1, wherein the connections comprise at least one of a TLS, DTLS, and IPSEC connection.

6. The computer-implemented method of claim 1, wherein the first server comprises a first security gateway instance, and wherein the second server comprises a second security gateway instance.

7. A non-transitory computer-readable medium comprising instructions, the instructions, when executed by a computing system, cause the computing system to: detect a reduction in one or more network metrics related to a first server;instruct the first server to issue a rekey request to a plurality of devices connected to the first server in response to the detected reduction in the one or more network metrics related to the first server, wherein the rekey request is part of a rekey routine for changing one or more session keys of an ongoing communication with the plurality of devices;instruct, in response to the detected reduction in the one or more network metrics, a load balancer to route responses from the plurality of devices to the rekey request to at least one other server to connect the plurality of devices to at least one of the at least one other server in response to the rekey request;determine a number of connections remaining between the first server and the plurality of devices; andinstruct the load balancer to terminate the first server based on the determined number of connections remaining between the first server and the plurality of devices,wherein instructing the load balancer to route the responses from the plurality of devices to the at least one other server comprises instructing the load balancer to not send responses from the plurality of devices to the first server.

8. The non-transitory computer-readable medium of claim 7, wherein the one or more network metrics comprises at least one of a number of connections to the first server, CPU usage of the first server, and memory utilization of the first server.

9. The non-transitory computer-readable medium of claim 7, wherein the determining the number of connections remaining comprises transmitting a query to the first server to solicit a response indicating a number of active connections between the first server and the plurality of devices.

10. The non-transitory computer-readable medium of claim 7, wherein the load balancer is instructed to terminate the first server when the determined number of connections remaining between the first server and the plurality of devices is zero.

11. The non-transitory computer-readable medium of claim 7, wherein the connections comprise at least one of a TLS, DTLS, and IPSEC connection.

12. The non-transitory computer-readable medium of claim 7, wherein the first server comprises a first security gateway instance, and wherein the second server comprises a second security gateway instance.

13. A system comprising: a processor; anda non-transitory computer-readable medium storing instructions that, when executed by the system, cause the system to: detect a reduction in one or more network metrics related to a first server;instruct the first server to issue a rekey request to a plurality of devices connected to the first server in response to the detected reduction in the one or more network metrics related to the first server, wherein the rekey request is part of a rekey routine for changing one or more session keys of an ongoing communication with the plurality of devices;instruct, in response to the detected reduction in the one or more network metrics, a load balancer to route responses from the plurality of devices to the rekey request to at least one other server to connect the plurality of devices to at least one of the at least one other server in response to the rekey request;determine a number of connections remaining between the first server and the plurality of devices; andinstruct the load balancer to terminate the first server based on the determined number of connections remaining between the first server and the plurality of devices,wherein instructing the load balancer to route the responses from the plurality of devices to the at least one other server comprises instructing the load balancer to not send responses from the plurality of devices to the first server.

14. The system of claim 13, wherein the one or more network metrics comprises at least one of a number of connections to the first server, CPU usage of the first server, and memory utilization of the first server.

15. The system of claim 13, wherein the determining the number of connections remaining comprises transmitting a query to the first server to solicit a response indicating a number of active connections between the first server and the plurality of devices.

16. The system of claim 13, wherein the connections comprise at least one of a TLS, DTLS, and IPSEC connection.

17. The system of claim 13, wherein the first server comprises a first security gateway instance, and wherein the second server comprises a second security gateway instance.