The present invention relates to data networks and software defined networking (SDN).
Data networks are comprised of network nodes, and fixed-capacity links that connect the nodes. Capacity of a link is generally measured in units of bandwidth such as bits/sec. A provider network is a data network through which customers access the Internet and other services, including inter alia voice, video, on-line gaming, file sharing, data backup and cloud storage.
Reference is made to
As shown in
Network operators strive to use as much as possible of a network's capacity, yet to avoid congestion in the network. Network congestion degrades quality of service for customers who use the network, and leads to low effective utilization of the network due to re-transmissions. Network operators sign service level agreements (SLA's) with customers, and strive to enforce the SLA's in their network and ensure fairness among their customers.
Downstream traffic is directed from media servers 20A and 20B, and from Internet 30, to CPEs 60A and 60B, through network 100, by a semi-static tree structure. Specifically, as long as a network link does not fail, downstream traffic to a specific CPE always traverses the same path of network nodes. Edge router 10 identifies the destination CPE of each downstream frame that enters edge router 10, and stores each frame in a downstream queue that is associated with that destination CPE. In order to avoid congestion, ensure fairness among customers, and optimize network utilization, edge router 10 employs hierarchical traffic management, using a hierarchical scheduling and policing tree that has the same structure as that of the provider network. I.e., the root of the tree is the edge router, the vertices of the tree are the network nodes, the leaves of the tree are the downstream queues of the edge router, and the edges of the tree are network links through which downstream traffic flows between network nodes. The edge router shapes traffic flowing through a link according to the link's capacity; e.g., according to a percentage of the maximal link capacity, or according to a service level agreement in case the link is connected directly to a customer. The edge router shapes traffic by determining a data traffic rate Redgen,m for the downstream link from node n to node m, for some or all of the linked network nodes n and m, and by ensuring the these traffic rates do not exceed the link capacities.
Upstream traffic is controlled by algorithms such as dynamic bandwidth allocation. However, such control is generally limited to links directly connected to CPEs. Other control algorithms, such as Resource Reservation Protocol, allocate bandwidth along a path between the CPE and the edge router. However, such control generally results in limited network utilization.
A large portion of downstream traffic in provider networks is media—video, in particular. Conventionally, media servers 10A and 10B are located on the upstream side of edge router 10, so that all media traffic passes through the edge router 10. As a result, edge router 10 becomes overloaded. Moreover, as customers expect higher video quality, the bandwidth consumed by video in a provider network becomes larger. In turn, this necessitates enlarging the capacities of edge routers.
In addition to overloading edge routers, directing media traffic through edge routers has other drawbacks.
Flow control mechanisms, referred to variously as back-pressure and congestion indication, are standardized, and have been implemented over the years in various packet/cell communication technologies. Flow control mechanisms perform reasonably well in avoiding congestion for small-scale networks having few flows. However, flow control has several drawbacks.
In fact, the above deficiencies were the reason that hierarchical traffic management, currently used by edge routers, was introduced—the rationale being that since flow control cannot resolve congestion as it occurs, then congestion must be avoided altogether. To accomplish this, all data traffic addressed to any specific broadband branch, undergoes hierarchical traffic management taking into account various bottlenecks along its route.
To sum up the situation,
As such, it would be of advantage to control traffic in a way that overcomes the scalability limitation of flow control, and avoids congestion when multiple traffic sources are present in the network.
Aspects of the present invention relate to novel systems and methods for controlling data traffic to avoid congestion in a network that has multiple sources of traffic. Moreover, the sources may introduce traffic into the network downstream of an edge router. These systems and methods are scalable, and overcome the scalability limitations of flow control.
Embodiments of the present invention provide a novel network controller, which periodically gathers statistical traffic data from networks nodes and from one or more edge routers in a data network, and which uses these statistics to analyze traffic distribution from traffic sources on various network links. The controller calculates permitted capacities, i.e., maximum allowed rates, on links downstream of the edge routers. The thus-calculated permitted capacities are in turn used to dynamically configure the hierarchical scheduling and policing tree of one or more of the edge routers, thereby ensuring that the edge routers prevent traffic congestion in the network, and ensuring fairness among customers—despite the edge routers being located upstream of where the traffic sources enter the network.
The present invention is of particular advantage for software-defined networks (SDNs), which separate the data plane from the control plane.
There is thus provided in accordance with an embodiment of the present invention a traffic controller for a data network that includes a plurality of network nodes, a plurality of network links connecting the network nodes, and one or more edge routers, each edge router being configured to control network traffic based on permitted link capacities, and wherein one or more sources of downstream traffic data enter the network downstream of the one or more edge routers, the traffic controller including a receiver operable to periodically receive downstream transmission byte counts from at least some of the network nodes, a processor coupled with the receiver, operable to periodically update the permitted link capacities based on the network node downstream byte counts received by the receiver, and a transmitter coupled with the processor operable to periodically transmit the thus-updated permitted link capacities to the one or more edge routers for their use in controlling the network traffic.
There is additionally provided in accordance with an embodiment of the present invention a non-transitory computer readable medium storing a computer program with computer program code, which, when read by a controller device, causes the controller device to perform a method for controlling traffic in a data network that includes a plurality of network nodes, a plurality of network links connecting the network nodes, and one or more edge routers, each edge router being configured to control network traffic based on permitted link capacities, and wherein one or more sources of downstream traffic data enter the network downstream of the one or more edge routers, the method including periodically receiving downstream transmission byte counts from at least some of the network nodes, periodically updating permitted link capacities based on the network node downstream byte counts received by the periodically receiving, and periodically transmitting the thus-updated permitted link capacities, calculated by the periodically updating, to the one or more edge routers for their use in controlling the network traffic.
The present invention will be more fully understood and appreciated from the following detailed description, taken in conjunction with the drawings in which:
Aspects of the present invention relate to a novel network controller, which enables edge routers to prevent traffic congestion in the network and ensure fairness among customers, despite the edge routers being located upstream of where network traffic sources such as media servers enter the network.
Reference is made to
As shown in
Edge router 10 cannot perform congestion avoidance and ensure fairness in network 200, since it is not aware of the media traffic generated by media servers 20A and 20B that flows through the network to CPEs 60A and 60B. Indeed, edge router 10 cannot determine the data traffic rates, since the sources of the data traffic do not flow into edge router 10. As such, conventional hierarchical scheduling and shaping cannot be used in system 200 to prevent congestion.
Reference is made to
Controller 70 may be an additional network node added to the system, or alternatively it may be an existing network node that adopts the role of a controller. Controller 70 is a standard management entity, including inter alia a simple network management protocol (SNMP) manager or a software-defined network (SDN) controller, or an application over an SDN controller. Alternatively, controller 70 is a proprietary management entity.
Controller 70 collects information and statistical data from other network nodes, using a standard protocol including inter alia remote network monitoring (RMON), SNMP, operations administration and monitoring (OAM) protocol, and the Broadband Forum TR-69 management protocol. Alternatively, controller 70 collects the information and statistical data using proprietary protocols.
Controller 70 reads information from other network nodes, the information including inter alia, for each network node, one or more of:
Controller 70 writes to the hierarchical scheduling and policing tree of edge router 10, and reads information from the tree, including one or more of:
Controller 70 periodically identifies changes in the topology and link capacity information, and adjust its decisions. When such changes are identified, controller 70 notifies an operator that the discovered topology and link capacities do not match the edge router hierarchical scheduling tree. Further, when such changes are identified, controller 70 updates the edge router hierarchical scheduling, based on the updated topology and link capacity information, and notifies the operator accordingly.
The following notation is introduced.
EQ. 1 may be applied recursively to derive the counters Txn,m(t) from the counters Txl(t). As such, information V suffices to determine information IV.
Upon initialization, controller 70 reads the initial hierarchical scheduling and policing tree configuration, including the tree structure and the original edge capacities Corign,m.
In accordance with an embodiment of the present invention, controller 70 periodically reads available information from the network nodes and from edge router 10, and derives traffic rates Rn,m from node n to downstream node m, in accordance with the formula
EQ. 2 uses information III from the network nodes, and information IV or V from edge router 10. Denoting, as above, the data traffic rates determined by edge router 10 by Redgen,m, it is noted that Redgen,m≦Rn,m, and Redgen,m≦Corign,m. If information III is permanently not available to controller 70, then controller 70 sets the rate Rn,m=Redgen,m. If information III is temporarily not available to controller 70, then controller 70 uses a prediction based on previous information III that was available, to determine the rate Rn,m; e.g., a predictor based on a sliding window average or based on linear approximation.
After calculating the rates Rn,m, controller 70 dynamically updates the current permitted capacities Cn,m of each edge of the hierarchical scheduling and policing tree, according to the formula
C
n,m=max{Corign,m−(Rn,m−Redgen,m),0}. (3)
The updated edge capacities Cn,m in accordance with EQ. 3 are then used to dynamically update the configuration of edge router 10, thereby avoiding traffic congestion in the network nodes that receive traffic from edge router 10. It will be appreciated by those skilled in the art that use of EQ. 3 enables edge router 10 to accommodate sources of data traffic, such as media servers 20A and 20B, which do not flow through edge router 10.
Updating of capacities and updating of the configuration of edge router 10 are preferably performed frequently enough to follow traffic source rate changes, but without overloading the network nodes with statistics requests.
Reference is made to
When two or more edge routers are present in the network, such as edge routers 10A and 10B, the capacity updating procedure of EQ. 3 is performed for each edge router. It is noted, however, that the node calculations need only be performed once.
It is further noted that if one edge router, say edge router 10A, becomes inactive, then controller 70 instructs the other edge router, namely, edge router 10B, to use the entire network capacity. It will be appreciated by those skilled in the art that this serves as a failure protection mechanism for the network.
Reference is made to
Transmitter 76 queries the nodes for their statistics, for the next calculation cycle. In an alternative embodiment of the present invention, controller 70 configures the nodes to periodically send their statistics to transmitter 76.
It will be appreciated by those skilled in the art that the present invention has broad application to any data network that supports two or more network nodes that pass traffic from one or more sources into the network, such that one or more of the traffic sources has a connection to a device capable of performing hierarchical schedule and shaping, and such that some or all of the network nodes are capable of providing statistics regarding traffic passing through them.
In the foregoing specification, the invention has been described with reference to specific exemplary embodiments thereof. It will, however, be evident that various modifications and changes may be made to the specific exemplary embodiments without departing from the broader spirit and scope of the invention as set forth in the appended claims. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.
This application claims priority benefit of U.S. Provisional Application No. 61/899,230, entitled CONGESTION AVOIDANCE IN DATA NETWORKS WITH MULTIPLE TRAFFIC SOURCES, filed on Nov. 3, 2013 by inventors Oren Spector and Menachem Kaplan.
Number | Date | Country | |
---|---|---|---|
61899230 | Nov 2013 | US |