Dynamic SD-WAN hub cluster scaling with machine learning

Abstract

Some embodiments of the invention provide a method of dynamically scaling a hub cluster in a software-defined wide area network (SD-WAN) based on particular traffic statistics, the hub cluster being located in a datacenter of the SD-WAN and allowing branch sites of the SD-WAN to access resource of the datacenter by connecting to the hub cluster. A controller of the SD-WAN receives, from the hub cluster, traffic statistics centrally captured at the hub cluster. The controller then analyzes these statistics to identify traffic load fluctuations, and determines that a number of hubs in the hub cluster should be adjusted based on the identified fluctuations. The controller adjusts the number of hubs in the hub cluster based on the determination.

Description

BACKGROUND

Today, SD-WAN hub-clustering is a networking mechanism akin to server load balancing. The hub clusters reside in datacenters and allow branch sites to access resources of the datacenter by connecting to one of the hubs of the cluster in a load-balanced fashion. Determining the number of hubs to be included in the cluster, however, remains a challenge faced by network designers, as the determination is based on multiple factors, such as the number of branches and network usage patterns of those branches. As a result, network operators tend to over-provision hubs, sometimes by as much as a factor of two. The lack of an easy solution can result in unsatisfied and frustrated customers who are increasingly using cluster solutions in cloud deployments and IaaS (infrastructure as a service).

BRIEF SUMMARY

Some embodiments of the invention provide a method of dynamically scaling a hub cluster in a software-defined wide area network (SD-WAN) based on current and historical traffic statistics. The SD-WAN includes a datacenter in which the hub cluster is located, a controller, and multiple branch sites that connect to the hub cluster in order to access resources of the datacenter. In addition to providing access to resources of the datacenter, the hub cluster captures and sends traffic statistics relating to flows to and from the branch sites, and provides these traffic statistics to the controller of the SD-WAN. The controller analyzes the received traffic statistics to identify traffic load fluctuations, and determines that a number of hubs in the hub cluster should be adjusted based on the identified fluctuations. The controller then adjusts the number of hubs in the hub cluster based on the determination.

In some embodiments, each branch site includes an edge forwarding node for sending and receiving traffic flows between the branch site and other elements of the SD-WAN (e.g., between the branch site and the hub cluster). In addition to the traffic statistics received from the hub cluster, the controller in some embodiments also receives additional traffic statistics from the edge forwarding nodes at the branch sites. In some embodiments, the additional traffic statistics received from the edge forwarding nodes include more fine-grained statistics compared to the traffic statistics received from the hub cluster.

The controller, in some embodiments, includes a traffic statistics storage, a learning engine, and a decision-making and evaluation engine. In some embodiments, the traffic statistics storage stores historical traffic statistics (i.e., traffic statistics received over time). The historical traffic statistics and the real-time traffic statistics received by the controller are used as input for the learning engine, according to some embodiments. For example, the real-time traffic statistics of some embodiments are input into the learning engine to identify real-time load fluctuations (e.g., fluctuations happening at short time scales), while the historical traffic statistics in addition to the current traffic statistics are input into the learning engine to identify cyclical load fluctuations (e.g., daily morning/night fluctuations) and seasonal load fluctuations (e.g., fluctuations during peak business operations). By identifying these fluctuations, the learning engine and decision making and evaluation engine of some embodiments are able to adapt and adjust the hub cluster size in response to real-time, cyclical, and seasonal load fluctuations.

In some embodiments, the traffic statistics (both historical and real-time) are divided into two sets. The first set includes flow traffic load statistics that form the traffic parameters used by the learning engine to identify the fluctuations described above, while the second set includes flow performance statistics that form the evaluation parameters of the decision-making and evaluation engine to determine whether to adjust the hub cluster size. Flow traffic load statistics, in some embodiments, include at least the number of branch sites in the SD-WAN, the rate of flow creation per branch site, the total number of flows per branch site, flows further categorized into applications, average bandwidth per flow, total number of fat flows per branch, and average bandwidth per fat flow. The flow performance statistics in some embodiments include at least packet re-transmits per flow, packet drops per flow, and mean opinion score (MOS) of real-time flows (e.g., VoIP calls).

In order to determine that a number of hubs in the hub cluster should be adjusted based on the identified fluctuations (e.g., real-time, cyclical, or seasonal fluctuations), the decision making and evaluation engine in some embodiments uses the identified fluctuations to predict the traffic load over a certain time interval (e.g., the next three hours) and to determine the hub cluster size that can handle the predicted load with the expected flow performance (e.g., based on prior tests on sizing of the hub cluster at different traffic loads). Based on the different between the current hub cluster size and required cluster size, the decision making and evaluation engine decides whether to scale the cluster up (i.e., add hubs) or to scale the cluster down (i.e., remove hubs).

In some embodiments, in addition to deciding whether to scale the cluster up or down, the decision making and evaluation engine evaluates the learning engine's performance by comparing the predicted traffic load and the actual traffic load. This comparison, in some embodiments, can trigger a re-training of the learning engine if the engine's predictions start to deviate significantly from the actual values. In some embodiments, re-training can also be triggered after a certain time-interval to account for new data.

The preceding Summary is intended to serve as a brief introduction to some embodiments of the invention. It is not meant to be an introduction or overview of all inventive subject matter disclosed in this document. The Detailed Description that follows and the Drawings that are referred to in the Detailed Description will further describe the embodiments described in the Summary as well as other embodiments. Accordingly, to understand all the embodiments described by this document, a full review of the Summary, the Detailed Description, the Drawings, and the Claims is needed. Moreover, the claimed subject matters are not to be limited by the illustrative details in the Summary, the Detailed Description, and the Drawings.

BRIEF DESCRIPTION OF FIGURES

The novel features of the invention are set forth in the appended claims. However, for purposes of explanation, several embodiments of the invention are set forth in the following figures.

FIG. 1 conceptually illustrates an example of an SD-WAN in some embodiments.

FIG. 2 conceptually illustrates an example of the machine-learning model of some embodiments.

FIGS. 3A-3B conceptually illustrate an example of an SD-WAN before and after a machine-learning model has adjusted the hub cluster based on identified load fluctuations, according to some embodiments.

FIG. 4 illustrates a process performed by the controller in some embodiments to identify real-time load fluctuations.

FIG. 5 illustrates a process performed by the controller in some embodiments to identify cyclical load fluctuations.

FIG. 6 illustrates a process performed by the controller in some embodiments to identify seasonal load fluctuations.

FIG. 7 illustrates a process performed by the controller to evaluate the performance of the machine-learning model, in some embodiments.

FIG. 8 conceptually illustrates a computer system with which some embodiments of the invention are implemented.

DETAILED DESCRIPTION

In the following detailed description of the invention, numerous details, examples, and embodiments of the invention are set forth and described. However, it will be clear and apparent to one skilled in the art that the invention is not limited to the embodiments set forth and that the invention may be practiced without some of the specific details and examples discussed.

Some embodiments of the invention provide a method of dynamically scaling a hub cluster in a software-defined wide area network (SD-WAN) based on current and historical traffic statistics. The SD-WAN includes a datacenter in which the hub cluster is located, a controller, and multiple branch sites that connect to the hub cluster in order to access resources of the datacenter. In addition to providing access to resources of the datacenter, the hub cluster captures and sends traffic statistics relating to flows to and from the branch sites, and provides these traffic statistics to the controller of the SD-WAN. The controller analyzes the received traffic statistics to identify traffic load fluctuations, and determines that a number of hubs in the hub cluster should be adjusted based on the identified fluctuations. The controller then adjusts the number of hubs in the hub cluster based on the determination.

In some embodiments, each branch site includes an edge forwarding node for sending and receiving traffic flows between the branch site and other elements of the SD-WAN (e.g., between the branch site and the hub cluster). In addition to the traffic statistics received from the hub cluster, the controller in some embodiments also receives additional traffic statistics from the edge forwarding nodes at the branch sites. In some embodiments, the additional traffic statistics received from the edge forwarding nodes include more fine-grained statistics compared to the traffic statistics received from the hub cluster.

The controller, in some embodiments, includes a traffic statistics storage, a learning engine, and a decision-making and evaluation engine. In some embodiments, the traffic statistics storage stores historical traffic statistics (i.e., traffic statistics received over time). The historical traffic statistics and the real-time traffic statistics received by the controller are used as input for the learning engine, according to some embodiments. For example, the real-time traffic statistics of some embodiments are input into the learning engine to identify real-time load fluctuations (e.g., fluctuations happening at short time scales), while the historical traffic statistics in addition to the current traffic statistics are input into the learning engine to identify cyclical load fluctuations (e.g., daily morning/night fluctuations) and seasonal load fluctuations (e.g., fluctuations during peak business operations). By identifying these fluctuations, the learning engine and decision making and evaluation engine of some embodiments are able to adapt and adjust the hub cluster size in response to real-time, cyclical, and seasonal load fluctuations.

In some embodiments, the traffic statistics (both historical and real-time) are divided into two sets. The first set includes flow traffic load statistics that form the traffic parameters used by the learning engine to identify the fluctuations described above, while the second set includes flow performance statistics that form the evaluation parameters of the decision-making and evaluation engine to determine whether to adjust the hub cluster size. Flow traffic load statistics, in some embodiments, include at least the number of branch sites in the SD-WAN, the rate of flow creation per branch site, the total number of flows per branch site, flows further categorized into applications, average bandwidth per flow, total number of fat flows per branch, and average bandwidth per fat flow. The flow performance statistics in some embodiments include at least packet re-transmits per flow, packet drops per flow, and mean opinion score (MOS) of real-time flows (e.g., VoIP calls).

In order to determine that a number of hubs in the hub cluster should be adjusted based on the identified fluctuations (e.g., real-time, cyclical, or seasonal fluctuations), the decision making and evaluation engine in some embodiments uses the identified fluctuations to predict the traffic load over a certain time interval (e.g., the next three hours) and to determine the hub cluster size that can handle the predicted load with the expected flow performance (e.g., based on prior tests on sizing of the hub cluster at different traffic loads). Based on the different between the current hub cluster size and required cluster size, the decision making and evaluation engine decides whether to scale the cluster up (i.e., add hubs) or to scale the cluster down (i.e., remove hubs).

In some embodiments, in addition to deciding whether to scale the cluster up or down, the decision making and evaluation engine evaluates the learning engine's performance by comparing the predicted traffic load and the actual traffic load. This comparison, in some embodiments, can trigger a re-training of the learning engine if the engine's predictions start to deviate significantly from the actual values. In some embodiments, re-training can also be triggered after a certain time-interval to account for new data.

FIG. 1 illustrates an example embodiment of an SD-WAN (also referred to herein as a virtual network) for connecting multiple branch sites to each other and to a controller and at least one datacenter hub. As shown, the SD-WAN 100 includes a controller 110, three branch sites 120-124 that each include an edge forwarding node 130-134 (also referred herein as edge nodes or nodes), a cloud gateway 140, and a datacenter 150 with a hub 145.

The edge nodes in some embodiments are edge machines (e.g., virtual machines (VMs), containers, programs executing on computers, etc.) and/or standalone appliances that operate at multi-computer locations of the particular entity (e.g., at an office or datacenter of the entity) to connect the computers at their respective locations other nodes, hubs, etc. in the virtual network. In some embodiments, the nodes are clusters of nodes at each of the branch sites. In other embodiments, the edge nodes are deployed to each of the branch sites as high-availability pairs such that one edge node in the pair is the active node and the other edge node in the pair is the standby node that can take over as the active edge node in case of failover.

An example of an entity for which such a virtual network can be established includes a business entity (e.g., a corporation), a non-profit entity (e.g., a hospital, a research organization, etc.), and an education entity (e.g., a university, a college, etc.), or any other type of entity. Examples of public cloud providers include Amazon Web Services (AWS), Google Cloud Platform (GCP), Microsoft Azure, etc., while examples of entities include a company (e.g., corporation, partnership, etc.), an organization (e.g., a school, a non-profit, a government entity, etc.), etc. In other embodiments, hubs like the hub 145 can also be deployed in private cloud datacenters of a virtual WAN provider that hosts hubs to establish SD-WANs for different entities.

In the example SD-WAN 100, the hub 145 is a multi-tenant forwarding element that is deployed on the premises of the datacenter 150. The hub 145 can be used to establish secure connection links (e.g., tunnels) with edge nodes at the particular entity's multi-computer sites, such as branch sites 130-134, third party datacenters (not shown), etc. For example, the hub 145 can be used to provide access from each branch site 120-124 to each other branch site 120-124 (e.g., via the connection links 160 that terminate at the hub 145) as well as to the resources 155 of the datacenter 150. These multi-computer sites are often at different physical locations (e.g., different buildings, different cities, different states, etc.), according to some embodiments. In some embodiments, hubs can be deployed as physical nodes or virtual nodes. Additionally, hubs in some embodiments can be deployed on a cloud (e.g., as a set of virtual edges configured as a cluster).

In the SD-WAN 100, the hub 145 also provides access to the resources 155 of the datacenter 150 as mentioned above. The resources in some embodiments include a set of one or more servers (e.g., web servers, database servers, etc.) within a microservices container (e.g., a pod). Conjunctively, or alternatively, some embodiments include multiple such microservices containers, each accessible through a different set of one or more hubs of the datacenter (not shown). The resources, as well as the hubs, are within the datacenter premises, according to some embodiments. While not shown, some embodiments include multiple different SaaS datacenters, which may each be accessed via different sets of hubs, according to some embodiments. In some embodiments, the SaaS datacenters include datacenters for video conferencing SaaS providers, for middlebox (e.g., firewall) service providers, for storage service providers, etc.

Additional examples of resources accessible via the hub 145, in some embodiments, include compute machines (e.g., virtual machines and/or containers providing server operations), storage machines (e.g., database servers), and middlebox service operations (e.g., firewall services, load balancing services, encryption services, etc.). In some embodiments, the connections 160 between the branch sites and the hub 145 are secure encrypted connections that encrypt packets exchanged between the edge nodes 130-134 of the branch sites and the hub 145. Examples of secure encrypted connections used in some embodiments include VPN (virtual private network) connections, or secure IPsec (Internet Protocol security) connection.

In some embodiments, multiple secure connection links (e.g., multiple secure tunnels) can be established between an edge node and the hub 145. When multiple such links are defined between a node and a hub, each secure connection link, in some embodiments, is associated with a different physical network link between the node and an external network. For instance, to access external networks in some embodiments, a node has one or more commercial broadband Internet links (e.g., a cable mode and a fiber optic link) to access the Internet, a wireless cellular link (e.g., a 5G LTE network), etc. The collection of the edge nodes, gateway, datacenter hub, controller, and secure connections between the edge nodes, gateway, datacenter hub, and controller form the SD-WAN 100.

As mentioned above, the controller 110 communicates with each of the nodes 130-134 at the branch sites 120-124, in some embodiments, to send information such as trace monitoring rules and receive information such as trace monitoring results via the connection links 170A-170C. In some embodiments, the controller 110 also provides trace monitoring rules to, and receives trace monitoring results from, the gateway 140 via the connection link 170D and the hub 145 via the connection link 170E. While illustrated as individual connection links, the links 170A-170E are sets of multiple connection links, according to some embodiments.

In addition to the connection links 170A-170E and 160, edge nodes 132 and 134 are connected via connection link 164, while edge nodes 130 and 132 are connected to the gateway 140 via connection links 162. The gateway 140 in this example is responsible for relaying information between edge nodes (e.g., edge nodes 120 and 122, which do not share a direct connection). Also, the gateway 140 in some embodiments is used to set up direct edge-to-edge connections. In some embodiments, the gateway 140 can be used to provide the edge nodes with access to cloud resources (e.g., compute, storage, and service resources of a cloud datacenter).

FIG. 2 illustrates an example of the machine-learning model used in the embodiments described herein. The machine-learning model 200 in some embodiments is a component of the controller (e.g., controller 110). The machine-learning model includes the learning engine component 205 and the decision-making and evaluation component 210. In some embodiments, the machine learning model 200 is a time-series model that captures real-time, cyclical, and seasonal load fluctuations. The model can be a single mathematical model that captures all three fluctuations (e.g., ARIMA (autoregressive integrated moving average model) in some embodiments, or it can include a collection of different mathematical models, each of which capture the relevant fluctuation in some embodiments.

When the model includes a collection of different mathematical models, the cyclical and seasonal parameters are learned from the time-series data, and subsequently subtracted from the time-series data to capture real-time fluctuations, as will be described in further detail below. In some embodiments, the model 200 is trained on historical data and updated periodically based on new data as well as based on the model's decisions are affecting system performance.

The decision-making and evaluation engine 210, in some embodiments, uses fluctuations identified by the learning component 205 to predict traffic loads and adjust the hub cluster accordingly to account for the predicted loads. Additionally, the decision-making and evaluation engine 210 in some embodiments evaluates the performance of the machine learning model 200, and triggers re-training if necessary, as mentioned above.

In some embodiments, the machine-learning model 200 makes both real-time and historical decisions based off of the received traffic statistics. For example, using parameters like the number of branch sites, packet re-transmits or drops per flow, MOS scores as input works by having a statistical co-relation of these parameters across all of the branch sites (e.g., branch sites 120-124), in some embodiments. While one, or a handful, of branch sites reporting increased retransmits or drops may not be indicative of much, the co-relation can indicate that the increase in these parameters are indeed connected to each other and thus probably have the same root cause, according to some embodiments. As a result, the machine-learning model in some embodiments can concur that the hub cluster requires scaling.

An example of a historical decision can include using parameters like flows per second, bandwidth consumed by the flows, and fat flow bandwidth. For example, an increase of fat flows when a few branch sites perform data backup at night can indicate that the cluster needs to be scaled up at time T on days D. As another example, the number of flows per second (i.e., rate of creation of flows) can be a significant parameter that impacts the operation of large networking devices. These flows may not consume bandwidth, but the sheer volume of flows impacts the scale. By using historical data, the hub can be scaled up to account for the sudden increase in the rate of flow creations.

In a third example, a sudden onslaught of traffic for an airport's SD-WAN when several flights land around the same time can bring a sudden burst of network users, thus degrading network connectivity in an unpredictable manner. Such a model is an easy learnable model to scale hubs beforehand to avoid degradation of network connectivity.

The real-time decisions made by the machine-learning model kick in after the problem has occurred, and are meant to mitigate in time to prevent further degradation, while the historical decisions kick in before the problem occurs to avoid the problem completely. As a result, most problems are initially corrected in real-time, while historical decisions will start to fix problems over the course of operation of the network.

In some embodiments, due to the nature the historical decisions, new hubs (e.g., VMs) can be spawned ahead of time and terminated when their need has ended. While spawning new hubs can be time consuming, it does not require other resources while not in use, which works well for historical decisions allow for enough time to spawn these hubs. Real-time needs, however, are more time-constrained and the thus spawning new hubs to solve real-time problems is untenable. Thus, some embodiments have a pool of backup hubs that do not occupy any compute resources, but are fully configured and ready to launch (e.g., from a disk/storage/configuration perspective). Additional details regarding the functionality of the machine learning model 200 will be described in further detail below.

FIGS. 3A-3B illustrate an example embodiment of an SD-WAN before and after the hub cluster size has been altered. In these example, the SD-WAN 300 includes a datacenter 350, branch sites 320-324, and a controller 310. The datacenter 350 includes hub clusters 345A and 345B, respectively. Like the hub cluster 145, the hub cluster 345A/B provides the branch sites 320-324 with access to the datacenter resources 355.

As shown in FIG. 3A, the hub cluster 345A and edge nodes at the branch sites 320-324 provide traffic statistics to the learning engine 312 of the controller 310. The learning engine then identifies fluctuations for the decision making and evaluation engine 314 to use to predict traffic load for a time period and determine the necessary hub cluster size for the predicted load. The decision making and evaluation engine 314 includes a sizing tests storage 316 for storing previous hub cluster size calculations, and a load comparisons storage 318 for storing comparisons of actual and predicted loads.

After the decision-making and evaluation engine 314 performs its calculations, it adjusts the hub cluster size by scaling the cluster up or down to ensure the hub cluster can handle the predicted traffic load and is not using unnecessary bandwidth (e.g., as illustrated by the encircled 3). In some embodiments, the decision-making and evaluation engine also evaluates the performance of the machine learning model (i.e., the performance of the learning engine and decision-making and evaluation engine).

As illustrated in FIG. 3B, the hub cluster 345B includes 5 hubs compared to the 3 hubs in the hub cluster 345A as a result of the adjustment triggered by the decision-making and evaluation engine. Additionally, the hub cluster and branch sites are providing traffic statistics indicative of the actual load to the controller.

The decision-making and evaluation engine 314 then compares the actual load to the predicted load, in some embodiments, and then averages the performance of the machine-learning model over time (e.g., using data from the load comparison storage 318). When the calculations indicate significant deviations over time between the predicted loads and actual loads, model re-training is triggered, as illustrated by the encircled 3. Additional details regarding the processes performed by the machine-learning model of the controller will be described in further detail below with reference to FIGS. 4-7.

FIG. 4 illustrates a process performed by the controller in some embodiments to adjust hub cluster size based on identified real-time load fluctuations. The process 400 starts, at 405, with the controller receiving real-time traffic statistics from the hub cluster and branch sites. For example, FIG. 3A illustrates the learning engine 312 receiving traffic statistics from the edge nodes at the branch sites 320-324, as well as from the hub cluster 345A. In some embodiments, the learning engine only receives these statistics from the hub cluster and not from the branch sites.

After receiving the real-time traffic statistics at 405, the controller analyzes, at 410, the real-time traffic statistics along with historical traffic statistics using a time-series model to determine cyclical and seasonal parameters. The controller then subtracts, at 415, the cyclical and seasonal parameters from the time-series data (i.e., the real-time and historical traffic parameters) to identify potential real-time load fluctuations. Real-time load fluctuations are traffic flow fluctuations that happen at short time scales (e.g., minutes to hours), according to some embodiments. For example, in some embodiments, a burst of users connecting to an application can cause a sudden burst of traffic.

The controller next determines, at 420, whether any real-time load fluctuations have been identified. When the controller determines that no real-time load fluctuations have been identified, the process returns to 405 to receive real-time traffic statistics from the hub cluster and branch sites. Otherwise, when the controller determines at 420 that real-time load fluctuations have been identified, the process transitions to 425 to predict the traffic load for a selected time period (e.g., the next 3 hours). For example, after the learning engine 312 of the controller 310 identifies any load fluctuations, the decision-making and evaluation engine 314 uses the identified load fluctuations to predict traffic load for an upcoming time interval, in some embodiments.

Next, the controller determines, at 425, a hub cluster size that can handle the predicted traffic load with the expected flow performance. In some embodiments, this determination can be based on prior tests on sizing of the hub cluster at different traffic loads. In the example of FIG. 3A, the decision-making and evaluation engine 314 of the controller 310 makes this determination after predicting the traffic load.

After determining the needed hub cluster size at 425, the controller determines at 430 whether to scale-up the hub cluster. For example, when the needed hub cluster size determined by the decision-making and evaluation engine is larger than the current hub cluster size, a scale-up decision is triggered. When the controller determines that the hub cluster should be scaled up, the process transitions to 435 to add hubs to the cluster. The process then returns to 405 to receive real-time traffic statistics.

Alternatively, when the controller determines that the hub cluster should not be scaled up, the process transitions to 440 to determine whether to scale down the hub cluster. For example, when the determined hub cluster size for the predicted traffic load is smaller than the current hub cluster size, a scale-down decision is triggered. Accordingly, when the controller determines at 440 that the hub cluster should be scaled-down, the process transitions to 445 to remove hubs from the hub cluster. The process then transitions back to 405 to receive real-time traffic statistics.

When the controller alternately determines at 440 that the hub cluster should not be scaled-down, the process returns to 405 to receive real-time traffic statistics. In some embodiments, this occurs when the determined hub cluster size is equal to the current hub cluster size, and thus the cluster does not need to be scaled up or down.

FIG. 5 illustrates a process 500 performed by the controller in some embodiments to adjust hub cluster size based on identified cyclical load fluctuations. The process 500 starts at 505 when the controller receives real-time traffic statistics from the hub cluster and branch sites. Next, at 510, the controller analyzes the real-time traffic statistics along with historical traffic statistics using a time-series model to identify potential cyclical load fluctuations. In some embodiments, cyclical load fluctuations are traffic flow fluctuations that occur in a cyclical fashion at the time scale of business operations. For example, morning and night fluctuations can occur daily at an enterprise when users log into the enterprise network in the morning, and subsequently drop off from the network at night, according to some embodiments.

Following the analysis, the controller determines, at 515, whether any cyclical load fluctuations were identified. When the controller determines that no cyclical load fluctuations were identified (e.g., a constant load), the process transitions back to 505 to receive real-time traffic statistics. Otherwise, when the controller determines at 515 that cyclical load fluctuations have been identified, the process transitions to 520 to predict the traffic load for a selected cyclical time period. For example, in some embodiments, the controller predicts the traffic load at the time that users log into the enterprise network in the example above.

Next, the controller determines, at 525, the hub cluster size needed for the predicted traffic load. As described above, the controller can base this determination off of previous load test, in some embodiments. At 530, the controller determines whether the hub cluster should be scaled-up. For example, if the determined hub cluster size is greater than the current hub cluster size, then the hub cluster needs to be scaled-up (i.e., for the cyclical time period). When the controller determines at 530 that the hub cluster needs to be scaled-up, the process transitions to 535 to add hubs to the hub cluster. The process then returns to 505 to receive real-time traffic statistics.

Alternatively, when the controller determines at 530 that the hub cluster should not be scaled up, the process transitions to 540 to determine whether the hub cluster should be scaled down. In other words, the process determines whether the determined hub cluster size is smaller than the current hub cluster size. When the controller determines that the hub cluster should be scaled down, the process transitions to 545 to remove hubs from the hub cluster. The process then returns to 505. Otherwise, when the controller determines at 540 that the hub cluster should not be scaled down (i.e., the determined hub cluster size is equal to the current hub cluster size), the process returns to 505 without removing any hubs.

FIG. 6 illustrates a process 500 performed by the controller in some embodiments to adjust hub cluster size based on identified seasonal load fluctuations. The process 600 starts at 605 when the controller receives real-time traffic statistics from the hub cluster and branch sites. The controller then analyzes, at 610, the real-time traffic statistics along with historical traffic statistics using a time-series model to identify potential seasonal load fluctuations. In some embodiments, seasonal load fluctuations are traffic flow fluctuations that occur on a seasonal basis (i.e., on the order of months). For example, in some embodiments, the network traffic load during peak business operations periods could be much higher than normal (e.g., increased online shopping during the months of November and December).

Next, the controller determines, at 615, whether any seasonal load fluctuations were identified during the analysis. When the controller determines that no seasonal load fluctuations were identified (e.g., a constant load for seasonal periods), the process transitions back to 605 to receive real-time traffic statistics. Otherwise, when the controller determines at 615 that seasonal load fluctuations have been identified, the process transitions to 620 to predict the traffic load for a selected seasonal time period. In some embodiments, for example, the controller predicts the traffic load for shopping vendors during peak sales times each year.

At 625, the controller determines the hub cluster size needed for the predicted traffic load. In some embodiments, the controller can base this determination off of previous load test, as mentioned above. Next, at 630, the controller determines whether the hub cluster should be scaled-up. For example, if the determined hub cluster size is larger than the current hub cluster size, then the hub cluster needs to be scaled-up (i.e., for the seasonal time period). When the controller determines at 630 that the hub cluster needs to be scaled-up, the process transitions to 635 to add hubs to the hub cluster in order to scale up the cluster. The process then returns to 605 to receive real-time traffic statistics.

When the controller alternately determines at 630 that the hub cluster should not be scaled up (i.e., the determined hub cluster size is not larger than the current hub cluster size), the process transitions to 640 to determine whether the hub cluster should be scaled down. In other words, the process determines whether the determined hub cluster size is smaller than the current hub cluster size. When the controller determines that the hub cluster should be scaled down, the process transitions to 645 to remove hubs from the hub cluster. The process then returns to 605. Otherwise, when the controller determines at 640 that the hub cluster should not be scaled down (i.e., the determined hub cluster size is equal to the current hub cluster size), the process returns to 605 without removing any hubs.

In some embodiments, the processes 400-600 are performed concurrently, while in other embodiments, these processes are performed separately or in groups. In addition to the steps described for the processes 400-600, the controller of some embodiments evaluates the performance of the machine learning model by comparing the predicted traffic load to the actual traffic load (i.e., once it occurs). FIG. 7 illustrates a process performed by the controller in some embodiments to evaluate the machine learning model's performance.

The process 700 starts, at 710, by receiving data indicating actual traffic load during a particular time period (e.g., months during a seasonal load fluctuation). In some embodiments, the decision-making and evaluation engine of the machine learning model in the controller performs this process. The controller then compares, at 720, the actual traffic load with the predicted traffic load.

Next, at 730, the controller determines whether there is a significant deviation between the actual traffic load and the predicted traffic load. When the controller determines that there is no significant deviation, the process ends. Otherwise, when the controller determines at 730 that there is a significant deviation between the loads, the process transitions to 740 to calculate the average performance of the machine learning model over a particular time-interval. For example, the decision-making and evaluation engine 314 of the controller 310 includes a storage 318 for storing performance-related data.

Based on the calculations, the controller determines at 750 whether there have been significant deviations over time. When the controller determines that there have not been significant deviations over time, the process ends. Otherwise, when the controller determines that there have been significant deviations over time based on the calculations, the process transitions to 760 to trigger re-training of the machine-learning model. Following 760, the process ends. In addition to the process 700, the machine-learning model re-training in some embodiments can be triggered after a certain time-interval to account for new data.

Many of the above-described features and applications are implemented as software processes that are specified as a set of instructions recorded on a computer readable storage medium (also referred to as computer readable medium). When these instructions are executed by one or more processing unit(s) (e.g., one or more processors, cores of processors, or other processing units), they cause the processing unit(s) to perform the actions indicated in the instructions. Examples of computer readable media include, but are not limited to, CD-ROMs, flash drives, RAM chips, hard drives, EPROMs, etc. The computer readable media does not include carrier waves and electronic signals passing wirelessly or over wired connections.

In this specification, the term “software” is meant to include firmware residing in read-only memory or applications stored in magnetic storage, which can be read into memory for processing by a processor. Also, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program while remaining distinct software inventions. In some embodiments, multiple software inventions can also be implemented as separate programs. Finally, any combination of separate programs that together implement a software invention described here is within the scope of the invention. In some embodiments, the software programs, when installed to operate on one or more electronic systems, define one or more specific machine implementations that execute and perform the operations of the software programs.

FIG. 8 conceptually illustrates a computer system 800 with which some embodiments of the invention are implemented. The computer system 800 can be used to implement any of the above-described hosts, controllers, gateway and edge forwarding elements. As such, it can be used to execute any of the above described processes. This computer system includes various types of non-transitory machine readable media and interfaces for various other types of machine readable media. Computer system 800 includes a bus 805, processing unit(s) 810, a system memory 825, a read-only memory 830, a permanent storage device 835, input devices 840, and output devices 845.

The bus 805 collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of the computer system 800. For instance, the bus 805 communicatively connects the processing unit(s) 810 with the read-only memory 830, the system memory 825, and the permanent storage device 835.

From these various memory units, the processing unit(s) 810 retrieve instructions to execute and data to process in order to execute the processes of the invention. The processing unit(s) may be a single processor or a multi-core processor in different embodiments. The read-only-memory (ROM) 830 stores static data and instructions that are needed by the processing unit(s) 810 and other modules of the computer system. The permanent storage device 835, on the other hand, is a read-and-write memory device. This device is a non-volatile memory unit that stores instructions and data even when the computer system 800 is off. Some embodiments of the invention use a mass-storage device (such as a magnetic or optical disk and its corresponding disk drive) as the permanent storage device 835.

Other embodiments use a removable storage device (such as a floppy disk, flash drive, etc.) as the permanent storage device. Like the permanent storage device 835, the system memory 825 is a read-and-write memory device. However, unlike storage device 835, the system memory is a volatile read-and-write memory, such as random access memory. The system memory stores some of the instructions and data that the processor needs at runtime. In some embodiments, the invention's processes are stored in the system memory 825, the permanent storage device 835, and/or the read-only memory 830. From these various memory units, the processing unit(s) 810 retrieve instructions to execute and data to process in order to execute the processes of some embodiments.

The bus 805 also connects to the input and output devices 840 and 845. The input devices enable the user to communicate information and select commands to the computer system. The input devices 840 include alphanumeric keyboards and pointing devices (also called “cursor control devices”). The output devices 845 display images generated by the computer system. The output devices include printers and display devices, such as cathode ray tubes (CRT) or liquid crystal displays (LCD). Some embodiments include devices such as touchscreens that function as both input and output devices.

Finally, as shown in FIG. 8, bus 805 also couples computer system 800 to a network 865 through a network adapter (not shown). In this manner, the computer can be a part of a network of computers (such as a local area network (“LAN”), a wide area network (“WAN”), or an Intranet), or a network of networks (such as the Internet). Any or all components of computer system 800 may be used in conjunction with the invention.

Some embodiments include electronic components, such as microprocessors, storage and memory that store computer program instructions in a machine-readable or computer-readable medium (alternatively referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, read-only compact discs (CD-ROM), recordable compact discs (CD-R), rewritable compact discs (CD-RW), read-only digital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), a variety of recordable/rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD cards, mini-SD cards, micro-SD cards, etc.), magnetic and/or solid state hard drives, read-only and recordable Blu-Ray® discs, ultra-density optical discs, any other optical or magnetic media, and floppy disks. The computer-readable media may store a computer program that is executable by at least one processing unit and includes sets of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as is produced by a compiler, and files including higher-level code that are executed by a computer, an electronic component, or a microprocessor using an interpreter.

While the above discussion primarily refers to microprocessor or multi-core processors that execute software, some embodiments are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions that are stored on the circuit itself.

As used in this specification, the terms “computer”, “server”, “processor”, and “memory” all refer to electronic or other technological devices. These terms exclude people or groups of people. For the purposes of the specification, the terms “display” or “displaying” mean displaying on an electronic device. As used in this specification, the terms “computer readable medium,” “computer readable media,” and “machine readable medium” are entirely restricted to tangible, physical objects that store information in a form that is readable by a computer. These terms exclude any wireless signals, wired download signals, and any other ephemeral or transitory signals.

While the invention has been described with reference to numerous specific details, one of ordinary skill in the art will recognize that the invention can be embodied in other specific forms without departing from the spirit of the invention. For instance, several of the above-described embodiments deploy gateways in public cloud datacenters. However, in other embodiments, the gateways are deployed in a third party's virtual private cloud datacenters (e.g., datacenters that the third party uses to deploy cloud gateways for different entities in order to deploy virtual networks for these entities). Thus, one of ordinary skill in the art would understand that the invention is not to be limited by the foregoing illustrative details, but rather is to be defined by the appended claims.

Claims

1. A method of dynamically scaling a hub cluster in a software-defined wide area network (SD-WAN) based on particular traffic statistics, wherein the hub cluster is located in a datacenter of the SD-WAN and allows a plurality of branch sites of the SD-WAN to access resources of the datacenter by connecting to the hub cluster, the method comprising: at a controller of the SD-WAN: receiving, from the hub cluster, a plurality of traffic statistics that were captured at the hub cluster;analyzing the plurality of traffic statistics to identify traffic load fluctuations, said analyzing comprising identifying repetitive load fluctuations based on the received plurality of traffic statistics and based on historical traffic statistics previously received by the controller from the hub cluster, said analyzing comprising using a set of one or more machine-trained engines to identify the load fluctuations by subtracting cyclical and seasonal parameters from time-series data derived from the received plurality of traffic statistics and the historical traffic statistics;using the identified load fluctuations to predict traffic loads, and based on the predicted traffic load, adjusting a number of hubs in the hub cluster.

2. The method of claim 1, wherein the plurality of traffic statistics is a first plurality of traffic statistics, the method further comprising receiving a second plurality of traffic statistics from the plurality of branch sites, wherein the second plurality of traffic statistics were captured at each branch site in the plurality of branch sites.

3. The method of claim 2, wherein each branch site in the plurality of branch sites comprises an edge forwarding node, wherein the second plurality of traffic statistics captured at each branch site in the plurality of branch sites were captured by the edge forwarding nodes at the plurality of branch sites.

4. The method of claim 1, wherein the set of one or more machine-trained engines comprises (i) a learning engine for identifying real-time, cyclical, and seasonal load fluctuations based on received traffic statistics and (ii) a decision-making and evaluation engine for computing overall flow performance to determine whether to adjust the number of hubs in the hub cluster.

5. The method of claim 4, wherein the plurality of received traffic statistics is a plurality of real-time traffic statistics, wherein the learning engine and decision-making and evaluation engine are trained on the historical traffic statistics previously received by the controller.

6. The method of claim 5, wherein the learning engine identifies the cyclical and seasonal load fluctuations by using a mathematical model to analyze the plurality of real-time traffic statistics along with the historical traffic statistics.

7. The method of claim 1, wherein using the identified traffic load fluctuations to predict traffic loads further comprises (i) predicting the traffic loads over a particular time interval and (ii) determining a number of hubs needed to handle the predicted traffic load.

8. The method of claim 1, wherein adjusting the number of hubs in the hub cluster further comprises one of increasing the number of hubs in the hub cluster and decreasing the number of hubs in the hub cluster.

9. The method of claim 1, wherein the traffic statistics comprise traffic load statistics and flow performance statistics.

10. The method of claim 9, wherein the traffic load statistics comprise at least (i) a number of branch sites, (ii) rate of flow creation per branch site, (iii) total number of flows per branch site, (iv) flows that are further categorized by application, (v) average bandwidth per flow, (vi) total number of fat flows per branch, and (vii) average bandwidth per fat flow.

11. The method of claim 9, wherein the flow performance statistics comprise at least (i) packet re-transmits per flow, (ii) packet drops per flow, and (iii) mean opinion scores (MOS) of real-time flows.

12. A non-transitory machine readable medium storing a program for execution by a set of processing units, the program for dynamically scaling a hub cluster in a software-defined wide area network (SD-WAN) based on particular traffic statistics, wherein the hub cluster is located in a datacenter of the SD-WAN and allows a plurality of branch sites of the SD-WAN to access resources of the datacenter by connecting to the hub cluster, the program comprising sets of instructions for: at a controller of the SD-WAN: receiving, from the hub cluster, a plurality of traffic statistics that were captured at the hub cluster;analyzing the plurality of traffic statistics to identify traffic load fluctuations, said analyzing comprising identifying repetitive load fluctuations based on the received plurality of traffic statistics and based on historical traffic statistics previously received by the controller from the hub cluster, said analyzing comprising using a set of one or more machine-trained engines to identify the load fluctuations by subtracting cyclical and seasonal parameters from time-series data derived from the received plurality of traffic statistics and the historical traffic statistics;using the identified load fluctuations to predict traffic loads, and based on the predicted traffic load, adjusting a number of hubs in the hub cluster.

13. The non-transitory machine readable medium of claim 12, wherein the plurality of traffic statistics is a first plurality of traffic statistics, the program further comprising a set of instructions for receiving a second plurality of traffic statistics from the plurality of branch sites, wherein the second plurality of traffic statistics were captured at each branch site in the plurality of branch sites.

14. The non-transitory machine readable medium of claim 13, wherein each branch site in the plurality of branch sites comprises an edge forwarding node, wherein the second plurality of traffic statistics captured at each branch site in the plurality of branch sites were captured by the edge forwarding nodes at the plurality of branch sites.

15. The non-transitory machine readable medium of claim 12, wherein the set of one or more machine-trained engines controller comprises (i) a learning engine for identifying real-time, cyclical, and seasonal load fluctuations based on received traffic statistics and (ii) a decision-making and evaluation engine for computing overall flow performance to determine whether to adjust the number of hubs in the hub cluster.

16. The non-transitory machine readable medium of claim 15, wherein the plurality of received traffic statistics is a plurality of real-time traffic statistics, wherein the learning engine and decision-making and evaluation engine are trained on the historical traffic statistics previously received by the controller.

17. The non-transitory machine readable medium of claim 16, wherein the learning engine identifies the cyclical and seasonal load fluctuations by using a mathematical model to analyze the plurality of real-time traffic statistics along with the historical traffic statistics.

18. The non-transitory machine readable medium of claim 12, wherein the set of instructions for using the identified traffic load fluctuations to predict traffic loads further comprises a set of instructions for (i) predicting the traffic loads over a particular time interval and (ii) determining a number of hubs needed to handle the predicted traffic load, andwherein adjusting the number of hubs in the hub cluster further comprises one of increasing the number of hubs in the hub cluster and decreasing the number of hubs in the hub cluster.

19. The non-transitory machine readable medium of claim 12, wherein the traffic statistics comprise traffic load statistics and flow performance statistics.

20. The non-transitory machine readable medium of claim 19, wherein the traffic load statistics comprise at least (i) a number of branch sites, (ii) rate of flow creation per branch site, (iii) total number of flows per branch site, (iv) flows that are further categorized by application, (v) average bandwidth per flow, (vi) total number of fat flows per branch, and (vii) average bandwidth per fat flow.

21. The non-transitory machine readable medium of claim 19, wherein the flow performance statistics comprise at least (i) packet re-transmits per flow, (ii) packet drops per flow, and (iii) mean opinion scores (MOS) of real-time flows.