Patent Application
20180176153
This present invention is used in conjunction with the system described in U.S. patent application Ser. No. 15/367,916, “Parallel Multi-Function Packet Processing System for Network Analytics,” describing a parallelized receiver of flows distributed by the apparatus described in this invention.
This invention pertains generally to the field of network communication and specifically the subfield of centrally controlled and managed networks.
This invention applies to a configuration of a network switch with many ports. The switch ports are classified into two port groups: (i) those that are receiving incoming traffic (external ports), and (ii) those that are not (internal ports) over which all incoming traffic will be balanced, subject to liveness of those ports and configuration. Each TCP or UDP connection arriving on external ports, may be forwarded to any internal port, e.g., all internal ports may respond to HTTP for the same public IP address. A traditional network switch must route each incoming packet and send the connection to only one of the ports for each IP address. If all ports connect to devices that are programmed to respond to the same IP addresses, then it is not obvious how to route incoming connections for said public IP address to the internal ports. This work is traditionally implemented in special load-balancer appliances. Such appliances, however, are too complex for the less constrained problems that are better served by a simpler system, such as the system disclosed herein.
Load-balancing itself is not new U.S. Pat. Nos. 7,774,484, 6,996,615, 7,945,678 all relate to various aspects of it. Most of these inventions require special ASICs to operate at line-rate. This present invention achieves highest forwarding rate using OpenFlow switches without specialized hardware. This approach is known as Software-Defined Networks. Various middlebox applications, including load-balancing, have been ported to this new approach [ASTERIX, MICROTE, NIAGARA].
The OpenFlow switch is configured with match patterns in its ternary content addressable memory (TCAM) that maps external ports to internal ports. When a load-balancing mapping from external to internal ports, that maximizes aggregate use of all internal ports is found, then a second problem arises: adapting to traffic shifts.
The challenge is to produce an adaptive system that (i) produces OpenFlow FlowModification to be installed on a switch such that the load measured on internal ports is approximately the same for every port, and (ii) automatically adjusts to traffic and system status changes, such as links and devices coming up and going down or secular changes in user and device populations.
Furthermore, the system should confine the impact of extremely heavy traffic flows that are typically seen in flooding attempts.
This work is complicated by the fact that commodity OpenFlow switches can only accommodate a very limited number of traffic forwarding rules in their TCAM memories, and even if those memories were large, changing those memories is difficult because each change takes effect slowly, if compared to traffic forwarding, and may induce packet loss.
Finally, an adaptive algorithm must prevent thrashing in which flow-assignments change frequently, possibly during the lifetime of individual TCP connections.
This invention uses OpenFlow matches with output actions (FlowModifications) to distribute traffic from received traffic matches on external ports to internal ports. The load balancer software collects feedback from servers (connected to internal ports), flow status (the per rule OpenFlow statistics) and port status (aggregate port traffic statistics). This feedback is processed into per-target capacity estimates in terms of traffic volume, which forces an update in flow assignments to internal ports because the new volume estimates may indicate imbalance.
In fact, the switch and balancing systems are initialized with hash-based OpenFlow matches and their derived FlowModifications, which assign inbound traffic to the internal ports solely based on a hash value computed on the packet headers. The initial distribution of flows ignores actual load in the system. This is adjusted in later rounds of the load-balancing algorithm.
The load balancing system measures flow status, port status and server load information from the controlled switch and servers that accept traffic from the internal ports and incorporates these measurements into updated capacity estimates. The flow-assignment are updated based on these new measurements of the actual load taking into account the previous flow-assignment that lead to the updated load distribution.
Based on the measurements, the balancing system determines for each target, how much above or below average load they are running and reshuffles traffic flow assignments by reassigning traffic currently allocated to overloaded targets to those that are underloaded relative to the average of all targets' loads.
If no target is actually running above capacity, no changes are made.
If one or more flows are too large to be assigned to any target without exceeding the capacity of the target, such flows are split to smaller flows by removing wildcards from the flows matches.
Some flows may be so large that even after splitting them on their wildcarded fields, the generated partial flows still exceed the capacity of all internal ports and servers in the system. Such unmanageable “large flows” are sent to designated victim servers and/or ports that are intentionally sacrificed in order to keep the rest of the system stable in the presence of large flows.
As load shifts, the system could be left with highly fragmented rules due to rule splitting. Some flows do not match many packets per second. This invention automatically aggregates small flows which are assigned to the same target port if their aggregate packet pers seconds is well below the target's capacity. This aspect of the invention preserves TCAM rule space.
The system balances traffic arriving on the external ports of a common top-of-rack switch over a second set of output ports using no additional hardware beyond the switch.
The system is adaptive to changes in traffic, port status, and load.
Flow matches are loaded into switch TCAMs, therefore this invention achieves very high data rates.
The targets' feedback is based on a reusable API, which allows this invention to be reused in balancing applications with any monotonic load metric not only the CPU and packet load metrics described in the detailed description of this invention.
The system gracefully degrades in the presence of flooding attacks by sacrificing a fixed number of victim servers and/or ports.
The invention uses a small number of load-balancing FlowModifications to achieve a balanced assignment of flows to target ports.
This invention minimizes the rate of TCAM updates.
This disclosure considers the following list of references as prior art and explains the differences with and relationships to those related works.
The Prior work referenced above relates to this present invention as follows.
U.S. Pat. No. 7,290,059 introduces a balancing system driving a set of second layer dispatchers from a top-level router. The dispatchers maintain a fine-grained (per-connection) dispatch to determine the ultimate destination of each packet while the router updates independently. The dispatchers exchange their dispatch tables frequently. This invention does not divide the problem in the same layered approach as it permits L4 information to be considered at the top-layer router level.
U.S. Pat. No. 8,416,692 introduces a balancing system with multiple balancing layers, each consisting of multiple routers, switches and commodity servers. The balancing decision of the cited patent is made through multiple balancing layers with distributed information, which distributing balancing decisions to all involved entities. This present invention, in contrast, makes centralized balancing decision without the need to maintain such a heavily distributed system. U.S. Pat. Nos. 7,613,822 and 7,590,736 introduce balancing systems that rely on frequent updates of routing tables based on the server status to make packet forwarding decision. In contrast, this present invention updates the FlowModification on a switch slowly without involving any routers.
The problem of splitting traffic over many links as done in the above software-based load-balancers can be offloaded to an SDN switch. One simple, commodity OpenFlow switch can be programmed to distribute traffic to many backend services, links, and middleboxes. U.S. Pat. No. 8,959,215 describes a meta switch that provisions FlowModification down to the TCAM's and routing tables of network elements, which captures the idea of using OpenFlow as a universal routing API. The cited patent does not explain, however, how to generate FlowModification that accomplish a task such as load balancing and which FlowModification should be generated for which switch.
The OpenFlow Specification [OFSPEC] is a an API fully incorporating the concepts of U.S. Pat. No. 8,959,215. This API is implemented in a large percentage of commodity packet switches. OpenFlow provides the ability to match one or multiple fields for each packet, and to specify for each match which actions to execute. For example, OpenFlow would allow matching all TCP packets with destination port 80 and to associate such match with the action of forwarding the packet to physical port 1 (irrespective of any layer 3 routing). The concept of flow defined in OpenFlow, as a set of bit-masks matching a header is the concept of flow used throughout the description of this present invention. The actual definition of flow as a set of packets matching a bit mask pre dates OpenFlow.
OpenFlow enables wildcard matches by using bit masks. For most fields in an OpenFlow match, there are both match and mask can be specified (because the match is intended to be executed on a TCAM). If a certain bit of mask is set to 0, it indicates a wildcard on that bit. For example, in an OpenFlow match that matches tcp source port, the field and the mask of the field may be specified. If the match is set to 2 (0000000000000010 in binary) and the mask is set to 65535 (1111111111111111 in binary), the match matches all packets with tcp source port 2. If the match is set to 2 while the mask is set to 65534 (1111111111111110 in binary), the match matches all packets with tcp source port 2 or tcp source port 3.
OpenFlow implements matching priorities: rules with higher priority are matched first, and only if a piece of traffic is not matched by rules of higher priority are lower priority evaluated. This is a feature that can be used to drastically reduce the number of flow FlowModification required because complex traffic classification can be expressed as a series of alternating positive and negative matches of different priorities [NIAGARA]. Niagara's approach produces substantially less matches than most flow-matching methods including the methods of this invention. However, the highly compressed flow-match sets of Niagara do not lend themselves to partial updates.
There are alternatives to using explicit OpenFlow matches to distribute traffic such as Equal-Cost Multi-Path (ECMP) and Weighted-Cost Multi-Path (WCMP), as in U.S. Pat. No. 9,246,821. The mechanisms work well except for when specific customization needs to be performed or outliers need to be handled.
Ananta [ANANTA], Maglev [MAGLEV] and the invention subject of U.S. Pat. No. 8,676,980 implement load-balancing atop ECPM/WCMP. Those load balancers are front-ended by a layer of ECMP and run a L4 connection table as second layer. In contrast, this invention uses a single stage of OpenFlow switching for load-balancing.
The basic approach of using dynamic OpenFlow matches is described in Aster*x [ASTERIX], which directs the first packets of each flow to the controller and installs micro-flow FlowModification to forward the rest of the packets in the flow to a dynamically chosen destination. This approach is not practical in many use cases as it requires frequent updates to routing tables and places the controller logically on the forwarding path, thus exposing it to DoS attacks.
MicroTE [MICROTE] is a data center traffic distribution solution that operates on traffic forecasts. This differs from the present invention which uses current traffic measurements and optimizes flow-assignments subject to the assumption that traffic remains stable.
U.S. Patent application US20150271075A1 also describes the use of commodity switches with dynamic rules to perform load balancing. The cited work depends on virtual address mappings. Address virtualization is not part of this present invention.
U.S. Pat. No. 9,325,564 describes a method to offload forwarding logic from hardware device to software controller through tunneling. Tunneling allows greater hop count distance between the controlled switch and the targets of load-balancing. Whether the next hop is tunneled or directly-attached to the controlled switch is orthogonal to the content of this disclosure because the nature of attachment is virtualized by the OpenFlow port abstraction.
Finally, the system described in this present patent application is substantially different from randomized load-balancing systems such as U.S. Patent application US20140310418A1 which describes a system that randomly selects a backend server for each connection and sends the connection request to that randomly-chosen backend server.
The preferred implementation of this invention, comprises: an OpenFlow switch, backend servers (the targets), internal and external ports on the switch. The load-balancing rules expressed as OpenFlow flow modifications (FlowModifications). The system measurement relies on traffic statistics all of which are collected using OpenFlow's flow and port status messages, and server metrics which are reported as attribute value pairs or vectors of values representing time series, both of which are signalled via Remote Procedure Calls (RPCs).
A rule is an OpenFlow match that specifies certain fields in a packet with match values and masks. A flow is defined as all traffic that is matched by a rule. An action is directive that instructs a switch to handle a packet by, for example, dropping it, rewriting its destination, or sending it to a specific port. A FlowModification is a rule with actions. The OpenFlow switch collects match statistics on a per FlowModification basis called flow status, which contains statistics such as the number of packets, number of bytes, last seen match, and match install time. The set of actual statistics per switch is vendor-dependent. The set of FlowModifications generated at install time, prior to the collection of statistics, is called the initial rule set.
The weight of a flow is the number of bytes per second observed in a flow. Alternatively, other metrics may be chosen to replace the byte count (e.g., packets, cpu load incurred by processing of the flow). In fact, the weight of a flow in this invention is often an indirectly derived metric that takes CPU load implied by a flow. This is measured by taking the CPU load at a server, and proportionately allocating it to the flows directed to said server in proportion to each flow's contribution to the total traffic that is directed to the server.
A load balancing target is an entity in the system that will receive part of the inbound traffic. For example an OpenFlow port defined by the switch can be a balancing target. Such a port can be an actual hardware port, a port-mirror, or a tunnel, or a group, collectively referred to as ports in the scope of this invention. During the load-balancing process, each target is associated with one bucket, which is container for flows that are assigned to the given target. The weight of a bucket is the summation of weights of all flows assigned to the bucket.
Victim targets are those targets that are chosen to absorb excess traffic. Any target that is not a victim target is defined as a normal target. In the description of the algorithm, each victim target is associated with one victim bucket and each normal target is associated with one normal bucket.
To achieve balance in the sense of this invention is to ensure that each bucket is assigned flows such that the bucket weight is close to target weight of a bucket, which could be a fair share (total traffic divided by number of buckets) or a skewed target. If the weight of a bucket is greater than the target weight of a bucket then said bucket is overloaded. In the reverse case it is said to be underutilized. If the bucket is neither underutilized nor overloaded it is said to be balanced. Overload and underload are subject to some thresholding (allowing for measurement errors of a few percent).
The method of this invention (the “algorithm”) operates in a sequence of phases. At the beginning of each phase there is an assignment of flows to targets and at the end of each phase there is a new assignment of flows to targets and possibly a set of unassigned flows, henceforth called residual flows.
The system may start out with residual flows because, for example, some network link went down between iterations of the load-balancing algorithm. The algorithm generates residual flows by classifying flows that are too large for all buckets as residual flows.
The following conditions are repeatedly checked in the system.
The system that is subject of this invention is best understood with the help of
The overall system is shown in
If there are special, high-traffic L4 ports then the system 0402 creates special flows for those Layer 4 ports 0501 and takes one of those matched flows out of the queue 0508 and attempts to split it 0509. For example, a flow that matches Layer 4 port “TCP *1*” could split into two FlowModifications, e.g., “TCP 01*” and the other “TCP 11*.” The two split flows are put back in queue 0509 for later splitting. If there are already enough flows in H 0507, then the initialization exits 0511. If the queue H has no splittable content left 0510, then the system attempts to add more flows by adding flows based on generic matches 0502. An initial wild-card match “*” is repeatedly split as outlined for the port-specific matches before. Take a flow from queue Q 0503, split that flow and reinsert the split results into Q 0504, until there are no more splittable flows in Q 0505 or there are enough flows 0506, at which point initialization exits 0511.
All FlowModifications in the initial set have weight of 1 and in the first round of the load-balancing. The balancer engine distributes these initial flows in a round robin fashion among the buckets as shown in
Once the initial set of FlowModifications is enforced at the switches, the system will start collecting load measurements 0114 and traffic flow status 0115 which enable calibration and flow-reassignment as described in the following paragraphs.
The current set of FlowModifications 0204 or the set created at the end of the initialization 0604 is fetched and the current rules, match definitions and flow assignments are extracted from it.
The process of regenerating FlowModifications is shown in
The steps of this algorithm are shown in
The algorithm queries the switch for flow status of all FlowModifications 0304 parses the those 0302 before merging the current FlowModification 0304 with the buckets that match the output action of this FlowModification 0305. For example, if a FlowModification has actions that specify flow (dl_dst=0:1:2:3:4:5, ip, new_src=128.239.1.3) with action output to port 2, then the flow status matching the flow will be put into the bucket that represents port 2. In addition, the metric impact of the flow assignment at the target (bucket) is measured 0306, e.g., CPU consumption, disk utilization, memory consumption, in order to assign to each flow a weight commensurate with its traffic contribution to the bucket 0307. A flow that contributes 10% to the traffic of bucket B is assigned a weight that is 10% of for instance the CPU load at the target server that is associated with bucket B. This triggers rebalancing flow-assignments 0308 and eventually a new set of flow FlowModifications 0208 which the controller installs on the switch 0103.
Flow assignment 0308 is the algorithm which reassigns flows to buckets based on measured load. The initial check for initialization 0401, C0, is what triggers the already discussed initialization procedure in
The goal of Basic Shuffle 0403 is to achieve a balance with the least amount of flow-reassignment possible.
After each phase the balancer checks again if the normal buckets are still imbalanced, C2, 0702 and retries Basic Shuffle until the imbalance vanishes or until there are no options for local improvement.
The reduction of an overloaded bucket 0706 is shown in greater detail in
All flows that are still in the residual flow set even after backfilling all underutilized normal buckets (described in the previous paragraph) are subsequently allocated to victim buckets 0701 in round robin order starting with the largest residual flows first. This stable sorting-based approach minimizes the total number of flows reassignments.
There may still be underutilized normal buckets per C3, because the overload reduction 0706 freed some buckets of flows parts of which would have comfortably fit into another bucket after the other bucket's own overload reduction 0706 freed up capacity in the other bucket. In this case those partial flows can be retrieved in a final pass from the victim buckets 0406. The move from victim targets to normal buckets proceeds in order of the smallest flows that are currently assigned to victim buckets. This procedure repeats until condition C3 no longer holds or there are no more flows in the victim buckets or the current flow cannot be added to normal buckets without overloading them. Only existing capacity in normal buckets are backfilled in this module; no new capacity is freed up in normal buckets.
Since most flows in the victim buckets will be too large to fit into normal buckets they are split large victim flows into smaller fractional flows by fixing certain bits that wildcarded in the large flow that is currently assigned to the victim bucket. For example, the flow “*1” would become two smaller flows “01” and “11.” Flow splitting itself is not new [WILD] but using flow-splitting to back-fill otherwise underutilized buckets from a set over-sized flows in a load-balancing system is.
The process of splitting larger flows into several smaller ones and using those to back-fill gaps in underutilized normal buckets is shown in
If the normal buckets are now balanced or no improvement is possible, then the victim balancing module 0408 reassigns flows among victims only. The algorithm removes flows from victim buckets, and uses round robin based approach to assign the flows, starting with the largest flow. This step is necessary due to the possible split-induced size reduction of some victims buckets.
The details of the victim balancing algorithm (
After all previous balancing modules complete it is still possible that the buckets are imbalanced, C1 is still false, without any option for local balance improvement. In this case, Basic Shuffle has failed and the algorithm will perform an expensive Complete Reassignment of flows 0410, unless the Complete Reassignment algorithm has already been run on this iteration of the load-balancer.
On first failure of Basic Shuffle the Complete Reassignment algorithm is executed which is the same as the algorithm of
Once Complete Reassignment completes, Basic Shuffle is re-run on the reassignment of flows 0403. If this second invocation of Basic Shuffle fails again then the system will enforce the follow assignment resulting from the first (failed) run of Basic Shuffle during the current iteration of the load-balancer algorithm 0208.
After the flow assignment completes each bucket's flows can be mechanically translated into an OpenFlow FlowModification. The bucket itself corresponds to an output action, while the flow can be directly translated to a match. The translation of a bucket to an action works as follows: each bucket is associated with one or more OpenFlow ports, e.g. port 4. Assume it contains the flow of all traffic that matches“TCP destination port: 80,” Then the combination of the bucket and the flow results becomes FlowModification:
“tcp,tp_dst=80, action=output:4”.
The following description aids in the understanding of flow splitting and aggregation:
Flows are generated by bit masks on packet headers, it is easy to divide large flows to multiple small ones [WILD] or to aggregate small flows to a single large flow. For example, in binary format, for a flow with TCP source port value “011” and source port mask “011,” if it is too large to fit into any bucket, the balancer can split it into two flows: 1. TCP source port value “011” and source port mask “111”; 2. TCP source port value “111” and source port mask “111”. When a flow is split into two, it is assumed that each child flow gets half of the weight of parent flow. Of course, this is a guess, but fortunately not a bad one.
The reverse is also possible. Two flows can be combined into one if the bit vectors of the two matches are adjacent, i.e. there is only one bit difference between the bit vectors of the two FlowModification. The weight of aggregated flow is the summation of the weight of the two small flows. For example, in binary, for two flows with one matches TCP source port “011” with mask “111” and another flow matches TCP source port “111 with mask “111”, they can be aggregated to a single flow that matches TCP source port “011” with mask “011.” The flow aggregation can be performed after flow assignment is done. The flows assigned to the same bucket can be aggregated when their match bit vectors are adjacent.
This present invention contains an enhancement for its use in passive traffic analytics solutions in which the external ports receive both directions of traffic from a fiber tap for online inspection. The problem in these applications is that both directions of a TCP or UDP connection need to be received by the same destination processor. So far, the load-balancing strategies of this invention have ignored the problem of how to assign the reverse flow, as all ports were considered equal. Without the following addition the method would generate flow modifications that send forward and reverse traffic on a single TCP connection to two different devices.
This problem is solved by return flow pinning: For each TCP flow the reverse flow is created by swapping source and destination (both Layer 3 and Layer 4) and then inserting the reversed flow match explicitly with higher priority in the FlowModifications that are generated at the output stage of the load-balancing algorithm at step 0208 in
The reversal is applied to flows where the IP source address is smaller than the IP destination address, or if they are both the same and the protocol source (e.g., TCP source port) is less than the protocol destination.
The relationship between forward and reverse flow match is shown in
The forward source IP address 1203 and protocol source 1205 in the forward flow 1201 are inserted in the destination IP field 1212 and the protocol destination 1214 of the reverse flow 1202. Analogously, the destination IP address 1204 and destination protocol address 1206 in the forward flow 1201 are inserted in the source ip field 1211 and source protocol address field 1213 of the reverse flow. The bit masks for the field are swapped likewise in that source and destination IP masks are swapped (1207 moves to 1216, 1208 moves to 1215) and the source and destination protocol address masks are swapped (1209 moves to 1218 and 1210 moves to 1217) in the reverse flow. Other fields of the packet headers in the flow-definition remain the same in the reverse flow.
The so-generated reverse flow is associated with the same action as the forward flow and inserted as a FlowModification (flow plus action) in the controlled switch.
Upon removal of a forward flow an auto-generated reverse flow is removed as well. This can be automated by ensuring that the priority field of reverse flows is always a unique number reserved for reserved flows, or by labelling such flows with a specific OpenFlow cookie. In either case, the unique label makes re-generation and the deletion of the reverse flow for a forward flow a safe operation.
Occasionally, it may be necessary to add more fields to the direction identification of a flow such as physical source port and physical destination port.
The entire system operates by periodically running the algorithm of
One example use of the methods of this invention is to use a switch controlled by the invention as a load-balancing front-end to a set of identically configured firewall routers.
Another example use of this system as a load-balancing front-end to distribute packets to an Intrusion detection system, as is described in the concurrently submitted related U.S. patent application Ser. No. 15/367,916.
Another example use is one in which the system of this invention is used as a front-end to a conventional Layer 4 load-balancer system as an alternative to some of the multi-tiered load-balancer systems described as prior art.
