Patent Grant
7418000
The present invention relates generally to communication networks, and specifically to methods and systems for managing the flow of traffic over a network link that is shared by multiple connections.
Network service providers frequently aggregate communication traffic from multiple sources for transmission over a shared, high-bandwidth link. Service providers commonly overbook their available bandwidth in order to maximize utilization of this costly resource. In other words, the sum of the bandwidth that the service provider makes available to all traffic sources may exceed the actual bandwidth available on the shared link. Most of the time, the traffic sources generate less than their full bandwidth allocation, so that the aggregated traffic volume is less than the shared link capacity, and all of the traffic is transmitted over the shared link at the rate at which the traffic is generated by the sources. During periods of peak demand, however, the aggregated traffic volume may exceed the shared link capacity. In this case, at least some of the traffic must be buffered, and frames may be dropped if the buffer overflows.
To determine how bandwidth is to be shared during periods of heavy demand, service providers typically define multiple classes of services, with different bandwidth allocations and priorities. Each class of service (CoS) may have a committed information rate (CIR), which the service provider guarantees to provide, and an excess information rate (EIR), which the service provider provides on a best-efforts basis. (Some classes of service may have only a CIR allocation, or only an EIR allocation.) Each connection served by the service provider belongs to a specific class of service (CoS), which may be specified in a service level agreement (SLA) between the service provider and the customer making the connection. In aggregating traffic for transmission over the shared link, the service provider attempts to ensure that all traffic within the CIR of each connection is transmitted. The service provider then divides any remaining capacity among the connections according to their EIR allocations.
Service providers generally try to allocate their available bandwidth fairly among their customers. For this purpose, incoming data packets arriving from different sources are queued, and the traffic aggregator multiplexes among the queues in accordance with a fair scheduling scheme. Different weights may be assigned to different queues in order to permit different sources to receive different relative shares of the available capacity. Various fair scheduling schemes are known in the art. In Fair Queuing, for example, if there are N active queues at any given time, each queue receives 1/N of the available bandwidth. In Weighted Fair Queuing (WFQ), different queues may have different service shares according to their relative weights. In some applications of WFQ, including some embodiments of the present invention that are described hereinbelow, each source or connection has its own queue, an ill-behaved source (generating more than its fair share of traffic) will only punish itself and not other sources. Weighted Round Robin (WRR) scheduling operates on similar principles.
U.S. Pat. No. 6,157,654, whose disclosure is incorporated herein by reference, describes a method for adaptively assigning queue service weights to an ATM traffic management controller in order to reduce service weight update calculations. The queue weight configuration is performed using a reconfigurable weighted fair queuing controller, wherein bandwidth requirements for each queue may be changed in response to connection setup and release values.
The success of a weighted scheduling method, such as WFQ or WRR, in allocating link bandwidth or other resources fairly and efficiently depends on how well the weights accord with the actual classes of service and other parameters of the connections that are active on the link. When a connection is added or removed from the link, the weights should be adjusted to account for the change. In practice, however, in systems known in the art, there is no systematic way to ensure that the weights are really fair at all times or take into account the relative importance and urgency of different types of service.
Embodiments of the present invention provide methods and systems for automatic computation of weights for use in fair scheduling schemes in a communication network. In some of these embodiments, a multiplexer, such as an aggregator, connects multiple sources to a shared link. The multiplexer has multiple queues, each serving one or more connections originating from one or more respective sources. Each connection is designated to carry a certain category or categories of data traffic, which are typically expressed in terms of classes of service. Each connection or class of service has a specified bandwidth allocation, which may include both committed (CIR) and excess (EIR) bandwidth components.
A controller associated with the multiplexer calculates a weight contribution for each connection, based on its specified bandwidth allocation and on the overbooking ratios of the categories of data traffic that the connection may use. Higher levels of expected service (in terms of reliability and low-latency packet delivery) of certain traffic categories are typically reflected by lower overbooking ratios in these categories, and vice versa. Therefore, the connection weight contributions are biased depending on the overbooking ratios so as to give greater weight to categories with low overbooking than to those with high overbooking. For example, the weight contribution may be biased in inverse proportion to the overbooking ratios. The controller then computes a queue weight for each queue as a sum of the weight contributions of the connections served by the queue. When a connection is added, removed or modified, the controller modifies the queue weights accordingly. Use of the connection parameters, and specifically the overbooking ratio, in this manner permits fair weights to be computed and updated automatically, substantially without operator involvement.
Although the embodiments described hereinbelow relate specifically to aggregation of traffic for transmission over a shared network link, the principles of the present invention may similarly be applied in fair scheduling schemes used to allocate other network and processing resources among multiple traffic flows.
There is therefore provided, in accordance with an embodiment of the present invention, a method for communication, including:
setting respective overbooking ratios for multiple categories of data traffic;
assigning respective bandwidth allocations to a plurality of connections for transmitting the data traffic in one or more of the categories over a network;
coupling the data traffic from the connections into respective queues, such that each of the queues is associated with one or more of the connections;
computing respective weights for the queues responsively to the bandwidth allocations and to the overbooking ratios of the categories of the data traffic to be transmitted on the connections that are associated with each of the queues; and
multiplexing among the queues responsively to the respective weights so as to transmit the data traffic from the connections over a link in the network.
Typically, computing the respective weights includes biasing the weights so as to favor, in determining the weights, the categories of the data traffic for which the respective overbooking ratios are relatively smaller over the categories of the data traffic for which the respective overbooking ratios are relatively greater. In a disclosed embodiment, biasing the weights includes weighting the bandwidth allocations in inverse proportion to the overbooking ratios in order to compute the respective weights.
In disclosed embodiments, computing the respective weights includes computing respective weight contributions of the connections responsively to the categories of the data traffic transmitted on each of the connections, and summing the weight contributions of the connections that are associated with each of the queues. Typically, the multiple categories includes classes of service, each having at least one of a committed information rate (CIR) and an excess information rate (EIR) and having respective CIR and EIR overbooking ratios, and assigning the respective bandwidth allocations includes assigning each of the connections to at least one of the classes of service, and computing the respective weight contributions includes determining the weight contributions responsively to the classes of service of the connections. In one embodiment, computing the respective weight contributions includes setting a respective weight contribution of each of the connections belonging to a given class of service CoSj to be α(CoSj)CIR(CoSj)+β(CoSj)EIR(CoSj), wherein α and β are bias coefficients that are determined responsively to the CIR and EIR overbooking ratios of the given class of service, and α and β may be respectively inversely proportional to the CIR and EIR overbooking ratios of the given class of service.
Additionally or alternatively, multiplexing among the queues includes preferentially serving a guaranteed portion of the CIR, and multiplexing a remainder of the CIR and the EIR, and setting a respective weight contribution of each of the connections belonging to a given class of service CoSj to be α(CoSj)CIR(CoSj)[1−1/OVBCIR(CoSj)]+β(CoSj)EIR(CoSj), wherein OVBCIR is the overbooking ratio of the CIR in the given class of service.
In some embodiments, the multiple categories includes classes of service, each having at least one of a committed information rate (CIR) and an excess information rate (EIR) and having respective CIR and EIR overbooking ratios, and assigning the respective bandwidth allocations includes assigning each of the connections to at least one of the classes of service, and wherein computing the respective weights includes determining the weights responsively to the classes of service of the connections. Typically, computing the respective weights includes biasing the weights, responsively to the overbooking ratios, so that the CIR has a relatively greater impact on the weights than does the EIR.
In one embodiment, computing the respective weights includes applying a non-linear mapping to the computed weights.
There is also provided, in accordance with an embodiment of the present invention, apparatus for conveying data traffic over a plurality of connections on a network, each connection being configured to carry one or more categories of the data traffic, the apparatus including:
a multiplexer, which includes a set of queues, each such queue being configured to receive the data traffic on one or more of the connections that are associated with the queue, and which is adapted to multiplex among the queues responsively to respective weights that are assigned to the queues so as to transmit the data traffic from the queues over a link in the network; and
a controller, which is coupled to receive a definition of respective bandwidth allocations and respective overbooking ratios of the categories of the data traffic carried by each of the connections, and which is adapted, responsively to the definition, to compute the respective weights to be assigned to the queues responsively to the bandwidth allocations and to the overbooking ratios of the categories of the data traffic carried by the connections that are associated with each of the queues.
The present invention will be more fully understood from the following detailed description of the embodiments thereof, taken together with the drawings in which:
In network 20, multiple sources 22 (S1, S2, . . . ) transmit data over respective connections to destinations 26 (D1, D2, . . . ) via a core network 24. Generally speaking, a given source 22 may transmit data flows to multiple different destinations. Each flow is characterized by a class of service (CoS), which defines a priority level of the traffic between the source and destination of the flow. Typically, the CoS priority is defined in terms of guaranteed (CIR) and best-efforts (EIR) bandwidth allocations to the given CoS, which are commonly expressed in bits/sec (bps). Alternatively, bandwidth allocations may be defined for more than two categories of traffic per CoS, i.e., there may be one or more additional categories of bandwidth allocation per CoS besides CIR and EIR. The term “connection,” as used in the present patent application and in the claims, is defined by the combination of a source node, a destination node and a CoS. In other words, one connection may carry multiple flows, and there may be multiple connections between a given pair of source and destination nodes. Connections may include, for example, a VLAN or a Layer 2 or Layer 3 tunnel, as are known in the art.
In the example shown in
A connection admission controller (CAC) 32 manages the bandwidth allocations to the connections passing through aggregator 28 and controls the aggregator accordingly. For this purpose, CAC 32 uses the CoS definitions that are input by the system operator, including the predefined CIR and EIR allocations and overbooking ratio for each category in each CoS. Based on the known throughput of system 20 (which is in this example dictated by the bandwidth of link 30) and the overbooking ratios, the CAC admits or rejects each request from one of sources 22 to establish a new connection. The CAC assigns the connections to queues in aggregator 28 (or is informed of the assignments by the entity responsible for making the assignments), and computes respective queue weights for the queues based on the assigned connections, as described hereinbelow.
Based on the classes of service of the connections that are assigned to each queue, the CAC calculates a respective weight 48 for each queue (labeled W1, W2, W3, . . . , in the figure). An exemplary method for weight calculation is described below with reference to
The bandwidth allocations for the new connection are specified, at an allocation step 62. Typically, each new connection is assigned to a certain CoS, according to which CAC 32 determines the bandwidth allocations and overbooking ratios for the connection. In the present example, it will be assumed that these allocations are expressed in terms of CIR(CoSj) and EIR(CoSj), wherein CoSj refers to the jth class of service, to which the given connection is assigned. The overbooking ratios of the CIR and EIR traffic in CoSj are expressed as OVBCIR(CoSj) and OVBEIR(COSj). Although OVBCIR is often set equal to 1 (no overbooking), we consider here the more general case in which CIR(CoSj) may be overbooked. Furthermore, although the present embodiment is described in terms of CIR and EIR, some or all of connections 40 may alternatively have more than two categories of bandwidth allocations, in addition to or instead of CIR and/or EIR. The method of
Based on the bandwidth allocations and overbooking ratios for the new connection added at step 60, CAC 32 computes the weight contribution of the connection, at a contribution computation step 64. The weight contribution for a given connection in CoSj is α(CoSj)CIR(CoSj)+β(CoSj)EIR(CoSj). Here α and β are bias coefficients for the CIR and EIR allocations, respectively. These coefficients typically depend on OVBCIR(CoSj) and OVBEIR(COSj), in such a way as to give greater weight to allocations that have smaller overbooking ratios, and vice versa. For example, it is possible to set α(CoSj)=1/OVBCIR(CoSj) and β(COSj)=1/OVBEIR(CoSj). Since OVBCIR is generally much less than OVBEIR, α will generally be much greater than β for any given CoS. Alternatively, the bias coefficients may have a different functional dependence on the overbooking ratios. In any case, CAC 32 is able to calculate these values automatically, without further user input.
For each queue i, CAC 32 calculates the queue weight Wi based on the individual weight contributions of the connections sharing the queue, at a queue weight computation step 66. Typically, the queue weight for a given queue is simply the sum of the weight contributions of the individual connections that share the queue:
Wi=α(CoS1)·ΣCIR(CoS1)+β(CoS1)·ΣEIR(CoS1)+α(CoS2)·ΣCIR(CoS2)+β(CoS2)·ΣEIR(CoS2)+ . . . +α(CoSN)·ΣCIR(CoSN)+β(CoSN)·ΣEIR(CoSN) (1)
The summations in equation (1) are taken over all connections that share the same CoSj.
The above calculation assumes that CIR traffic and EIR traffic are aggregated together in the same queue for each connection. Alternatively, aggregator 28 may be configured to serve the guaranteed portion of the CIR traffic preferentially, before it serves the remaining CIR and EIR traffic. The guaranteed CIR bandwidth for CoSj is given by CIR(CoSj)/OVBCIR(COSj), so that the remainder of the CIR bandwidth, to be handled by fair scheduling of aggregator 28, is CIR(CoSj)[1−1/OVBCIR(CoSj)]. In this case, the weights determined by CAC 32 at steps 62 and 64 are applied only to the CIR remainder bandwidth and to the EIR bandwidth. Therefore, the weight contributions of the individual connections are now given by α(CoSj)CIR(CoSj)[1−1/OVBCIR(CoSj)]+β(CoSj)EIR(CoSj), and the queue weights are given by:
Wi=α(CoS1)·ΣCIR(CoS1)[1−1/OVBCIR(CoS1)]+β(CoS1)·ΣEIR(CoS1)+α(CoS2)·ΣCIR(CoS2)[1−1/OVBCIR(CoS2)]+β(CoS2)·ΣEIR(CoS2)+ . . . +α(CoSN)·ΣCIR(CoSN)[1−1/OVBCIR(CoSN)]+β(CoSN)·ΣEIR(CoSN) (2)
The bias coefficients α and β may be calculated as described above. Of course, if the CIR allocations are not overbooked at all (OVBCIR=1), then the CIR terms drop out of equation (2).
Alternatively, we may write α′(CoSj)=α(CoSj)[1−1/OVBCIR(CoSj)]. Using this identity, equation (2) reduces to the same form as equation (1):
Wi=α′(CoS1)·ΣCIR(CoS1)+β(CoS1)·ΣEIR(CoS1)+α′(CoS2)·ΣCIR(CoS2)+β(CoS2)·ΣEIR(CoS2)+ . . . +α′(CoSN)·ΣCIR(CoSN)+β(CoSN)·ΣEIR(CoSN) (3)
If the CIR is not overbooked, α′=α, so that equation (3) is simply a more general form of equation (1).
The results of equation (1) or (2) above generally have the form of floating-point numbers. Switch 50, however, typically comprises a hardware component, which is controlled by fixed-point (integer) weight values stored in control registers. Therefore, CAC 32 normalizes the weights calculated at step 66 in order to generate scheduling weights 48, at a weight normalization step 68. In one embodiment, a simple linear conversion is used:
Wi(Integer)=INT{Wi(Float)*MaxWeight(Integer)/MaxWeight(Float)} (4)
Here MaxWeight(Integer) is the largest integer that the weight registers of switch 50 can hold (typically given by 2RegisterSize), while MaxWeight (Float) is the maximum of Wi(Float) taken over all of queues 42, 44, 46 . . . .
Alternatively, a non-linear mapping may be used at step 68 in order to provide better weight resolution over the available integer range. For example, a histogram equalization procedure may be used to spread the normalized weights evenly over the available integer range. Alternatively or additionally, an arctangent or log function may be used to provide a relatively greater share of the available link bandwidth to queues with low floating-point weights. Other mapping schemes will be apparent to those skilled in the art.
The normalized weights are loaded into the control registers of switch 50, which applies the weights in multiplexing among queues 42, 44, 46, . . . , at a multiplexing step 70.
Although for the sake of simplicity, the embodiments described above relate only to allocation of bandwidth and multiplexing of traffic on a single shared link, the principles of the present invention may similarly be applied in more complex network topologies, and in fair sharing of access to other system resources. It will thus be appreciated that the embodiments described above are cited by way of example, and that the present invention is not limited to what has been particularly shown and described hereinabove. Rather, the scope of the present invention includes both combinations and subcombinations of the various features described hereinabove, as well as variations and modifications thereof which would occur to persons skilled in the art upon reading the foregoing description and which are not disclosed in the prior art.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5461611 | Drake, Jr. et al. | Oct 1995 | A |
| 6021263 | Kujoory et al. | Feb 2000 | A |
| 6157654 | Davis | Dec 2000 | A |
| 6178159 | He et al. | Jan 2001 | B1 |
| 6359861 | Sui et al. | Mar 2002 | B1 |
| 6370121 | Hausman | Apr 2002 | B1 |
| 6400681 | Bertin et al. | Jun 2002 | B1 |
| 6584535 | Ouellet et al. | Jun 2003 | B1 |
| 6647008 | Galand et al. | Nov 2003 | B1 |
| 6678241 | Gai et al. | Jan 2004 | B1 |
| 6757286 | Stone | Jun 2004 | B1 |
| 6795394 | Swinkels et al. | Sep 2004 | B1 |
| 6801506 | Dey | Oct 2004 | B1 |
| 6826147 | Nandy et al. | Nov 2004 | B1 |
| 6826158 | Seaman et al. | Nov 2004 | B2 |
| 6922394 | Kajiwara | Jul 2005 | B2 |
| 6934259 | Klincewicz et al. | Aug 2005 | B2 |
| 7065044 | Wang et al. | Jun 2006 | B2 |
| 7092356 | Rabie et al. | Aug 2006 | B2 |
| 7197008 | Shabtay et al. | Mar 2007 | B1 |
| 20010026535 | Amou et al. | Oct 2001 | A1 |
| 20010032271 | Allen | Oct 2001 | A1 |
| 20020163885 | Assa et al. | Nov 2002 | A1 |
| 20020176358 | Assa et al. | Nov 2002 | A1 |
| 20020186881 | Santiago et al. | Dec 2002 | A1 |
| 20030002443 | Basso et al. | Jan 2003 | A1 |
| 20030055920 | Kakadia et al. | Mar 2003 | A1 |
| 20030147352 | Ishibashi et al. | Aug 2003 | A1 |
| 20030185217 | Ganti et al. | Oct 2003 | A1 |
| 20040071089 | Bauer et al. | Apr 2004 | A1 |
| 20040085964 | Vaananen | May 2004 | A1 |
| 20040109408 | Mesh et al. | Jun 2004 | A1 |
| Number | Date | Country | |
|---|---|---|---|
| 20050271084 A1 | Dec 2005 | US |
