The present invention relates to methods of, software for and apparatus for enabling traffic engineering in carrier networks.
For many years now, telecommunications carriers have been deploying packet-switched networks in place of circuit-switched networks for reasons of efficiency and economy. Packet-switched networks such as Internet Protocol (IP) or Ethernet networks are intrinsically connectionless in nature and as a result suffer from Quality of Service (QoS) problems. Customers value services which are guaranteed in terms of bandwidth and QoS.
Carriers may use Multi-Protocol Label Switching (MPLS) over a layer 2 network to create connection-oriented label switched paths (or tunnels) across the intrinsically connectionless network, and thereby to provide guaranteed QoS and bandwidth services to customers. However. MPLS is a relatively unstable standard and carriers desire an alternative.
It is desired to use Ethernet switches in carrier's networks. Use of Ethernet switches in carrier's networks would have the advantages of interoperability (mappings between Ethernet and other frame/packet/cell data structures such as IP and ATM are well known) and economy (Ethernet switches are relatively inexpensive compared to IP routers, for example).
However, the behaviour of conventional switched Ethernet networks is incompatible with carriers' requirements for providing guaranteed services to customers. Carriers need networks to be meshed for load balancing and resiliency—ie there must be multiple paths across it—and the ability to perform traffic engineering—ie the ability of the network operator to control the provision of explicitly routed variable bandwidth connections (or tunnels) through which traffic may be directed.
In contrast, conventional Ethernet networks must be simply-connected—ie there must be one and only one path between each and every node of the network. As a consequence, conventional Ethernet networks do not have support for network-wide load balancing, suffer from resiliency problems and cannot support traffic engineering.
Spanning tree protocols are known which enable a physically meshed Ethernet network to be logically transformed into a simply-connected network by detecting physical loops and logically disabling connections to break up the loops. Spanning tree protocols are also known which are able to detect failure of a physical connection (thereby partitioning the fully-connected network) and automatically restore one or more previously-disabled physical connections so as to re-connect the network. This provides a degree of resiliency. However, carriers are capable of and so desire to plan their network traffic routes to achieve much higher resiliency, flexibility and efficiency than spanning tree can achieve. This routing can most easily be achieved by segregating the traffic into connections whose routes are determined as part of this planning process.
Virtual Bridged LANs (or VLANs) are described in the Institute of Electrical and Electronics Engineers (IEEE) standard 802.1 Q, 2003 Edition,
The Internet Engineering Task Force (IETF) has published an Internet Draft referred to as draft-kawakami-mpls-lsp-vlan-00:txt. This document describes the use of VLAN tags for label switching across Ethernet networks in a manner similar to use of MPLS labels for label switching over MPLS networks—VLAN tags are used as labels to mark traffic at an ingress point of a label switched path (LSP) as belonging to a Layer 2 tunnel: and VLAN-aware Ethernet switches in the network act as a VLAN label switched routers. Connections are formed using one or more LSPs. Intermediate nodes along the connection may optionally swap the inbound label to a different outbound label.
One problem with the method proposed in draft-kawakami-mpls-lsp-vlan-00.txt is that a maximum of 4094 unique VLAN tags are definable in 802.1 Q compliant equipment. This limits the flexibility in and increases the complexity of provisioning connections across the network. Another problem is that connections may not easily be re-routed once provisioned without creating transitory loops.
Another problem is that the Frame Check Sequence (FCS) in Ethernet frames is computed over both the payload and header portions of the frame. Thus, with the method proposed in draft-kawakami-mpls-lsp-vlan-00.txt, every time a VLAN tag (ie a label) is swapped at the ingress or egress point of a LSP, the FCS needs to be recomputed since the VLAN tag will have changed. This requires performing a computation function over the entire Ethernet frame. Moreover, during the interval from when the original FCS is removed and the new FCS added, the frame is vulnerable to corruption without the protection of any FCS.
Another problem with the ‘label-swapping’ approach proposed in draft-kawakami-mpls-lsp-vlan-00.txt is that it requires a “chain of correctness” in that forwarding relies on each local label-forwarded link on the LSP being correct, whereas conventional Ethernet which uses globally unique address information to perform forwarding. More importantly, from a practical perspective, ‘label-swapping’—behaviour represents a significant change from conventional Ethernet switch functionality, and current telecommunications standards.
The present invention relates to enabling traffic engineering in frame-based networks such as Ethernet networks. The term traffic engineering is used broadly in the present document to refer to functions for maintaining the quality of service of the customers' connections while permitting the owner to operate their network efficiently. Examples of this are ensuring that no link is over-loaded, load-balancing the connections evenly across the network, reequalizing the load on the network by re-routing some existing connections, establishing protection mechanisms, performing traffic restoration actions, and so on.
According to the present invention, connections are established in the carrier-network by configuring, in one or more network nodes, mappings for forwarding data frames such as Ethernet frames. The mappings are from a combination of a) a destination address corresponding to a destination node of a connection, such as a MAC address, and b) an identifier, such as a VLAN tag. The mappings are to selected output ports of the one or more nodes. By using the combination of destination address AND identifier, the mappings enable data frames belonging to different connections to be forwarded differentially (ie forwarded on different output ports) despite the different connections potentially having the same destination node. This enables flexibility in routing connections—eg the ability to perform traffic engineering. The reader should note that the term address is used in this document to denote any means of identifying a network node or an ingress or egress interface of a network node.
According to a first aspect of the present invention, there is provided a method of establishing connections in a frame-based network, the method comprising the step of configuring, in one or more nodes of the network, first mappings for use in forwarding data frames, the first mappings being from a combination of a first destination address corresponding to a first destination node of the network, and a first identifier, the first mappings being to a selected output port of, or to respective selected output ports of each of, the one or more nodes, thereby establishing at least part of a first connection through the one or more nodes to the first destination node.
Advantageously, the present invention enables connections to be established in a frame-based network in a highly flexible manner enabling network-wide traffic engineering. Furthermore, the specific problems inherent in the method proposed in draft-kawakami-mpls-lsp-vlan-00.txt (processing overhead and vulnerability of frames to corruption) are overcome since no label swapping is performed.
In one embodiment, the method of the present invention includes configuring, in at least one of the nodes, a second mapping for use in forwarding data frames, the second mapping being from a combination of: a second destination address corresponding to a second destination node of the network, and a second identifier, the second mapping being to a selected output port of the at least one node, thereby establishing at least part of a second connection through the at least one node to the second destination node, the selected output ports of the at least one node being different for the first and second mappings, thereby enabling, at the at least one node, differential forwarding of data frames associated with the first and second connections. Thus, advantageously, two connections may be established which converge in route at an intermediate node and then diverge again, for example.
In one embodiment, the first and second destination addresses and the first and second destination nodes are the same. Thus, for example, two connections may be established which converge at an intermediate node and then diverge, despite having the same destination node. This enables greater flexibility in setting up connections.
In one embodiment, the first and second identifiers are the same. Thus, for example, two connections may be established which converge at an intermediate node and then diverge, despite using the same identifier. Thus, limitations on the number of values identifiers can take do not significantly reduce flexibility in traffic engineering.
Preferably, the network is an Ethernet network and the one or more nodes are Ethernet switches. Preferably, the identifier is a VLAN tag. Advantageously, this enables traffic engineered carrier networks to be deployed using conventional and relatively inexpensive VLAN-aware Ethernet switches, albeit configured in an entirely novel and inventive manner. In one embodiment, the configuration is performed by a control plane of the network. Thus, carriers have direct control over the establishment of traffic engineering connections in the network. Preferably, the control plane is ASON/ASTN.
A frame-based communications network comprising one or more nodes arranged to perform the method of the first aspect of the present invention set out above is also provided.
Software arranged to perform the method of the first aspect of the present invention set out above is also provided.
According to a second aspect of the present invention, there is provided a connection controller for establishing connections in a frame-based network, the connection controller comprising: a signal generator capable of generating a first signal for configuring, in a transport node of the network, a first mapping for use in forwarding data frames, the first mapping being from a combination of: a first destination address corresponding to a first destination node of the network, and a first identifier, the first mapping being to a selected output port of the transport node, the first signal thereby establishing at least part of a first connection through the transport node to the first destination node.
According to a third aspect of the present invention, there is provided a method of establishing a connection in a frame-based network, the method comprising the steps of: configuring forwarding information in a plurality of nodes of the network the forwarding information enabling the nodes to forward data frames in dependence on a combination of a destination address and an identifier of the data frames.
According to a fourth aspect of the present invention, there is provided a method of data traffic engineering in a frame-based network, the method comprising the following steps: establishing a first and second connections in the network passing through a common switching node of the network, configuring the switching node to forward data frames differently in dependence on differences in either a destination address or an identifier of the data frames, thereby enabling data traffic engineering.
According to a fifth aspect of the present invention, there is provided a method of establishing connections in a frame-based network, the method comprising the step of: configuring, in each of a first plurality of nodes of the network, a first forwarding mapping from a first combination of a destination address and an identifier to a selected output port of a respective node of the first plurality of nodes.
According to a sixth aspect of the present invention, there is provided a connection controller for establishing connections in a frame-based network, the connection controller being arranged to configure a first forwarding mapping in a transport node, the first mapping being from a first combination of a destination address and an identifier to a first output port of the transport node.
According to a seventh aspect of the present invention, there is provided a method of forwarding data frames in a frame-based network, the method comprising the steps of: establishing a first connection in the network, the first connection being associated with a first combination of a destination address and an identifier, and forwarding data frames in the network in dependence on a combination of a destination address and an identifier of the data frames.
Further aspects of the present invention are set out in the appended claims. Further advantages of the present invention will be apparent from the following description.
In order to show how the invention may be carried into effect, embodiments of the invention are now described below by way of example only and with reference to the accompanying figures in which:
Embodiments of the present invention are described below by way of example only. These examples represent the best ways of putting the invention into practice that are currently known to the Applicant although they are not the only ways in which this could be achieved.
To support guaranteed QoS to customers, what is required is:
The present invention is primarily concerned with enabling requirements 1) and 2) above in frame-based networks such as Ethernet networks. Requirement 3) may be achieved using conventional mechanisms such as admission control at the ingress nodes of connections (trusted-edge policing).
Note that core network switches 24a, 24b, and 28 are fully-meshed—ie there is a direct communications link connecting each core network switch 24a, 24b, and 28 to each other. Edge network switches 22a, 22b and 26 are not fully-meshed but have at least one direct communications link to a core network switch 26. The reader will appreciate that the particular network arrangement described is exemplary. In general, carrier networks may be implemented with virtually any number of Ethernet switches which, according to the present invention, may be connected in a fully-meshed or partially-meshed manner.
Carrier edge switches 22a, 22b and 26 may be logically separated according to Nortel Network's Logical Provider Edge (LPE) architecture. Each carrier edge switch may thus be logically separated into a single Provider Edge—(PE) Core and one or more PE-Edge functions. The PE-Edge is the ingress/egress point at which customer traffic enters or leaves the provider network—ie carrier network 20. The PE-Core encapsulates incoming Ethernet traffic from the customer using Media Access Control (MAC) in MAC encapsulation and forwards the encapsulated traffic across the carrier network. Similarly the PE-Core decapsulates (strips) outgoing Ethernet traffic and forwards the stripped traffic on to the customer via the appropriate PEEdge. VLAN tags are used to provide customer separation at the PE-Core with each different customer site connected to each edge switch having a unique VLAN tag. Stacked VLAN (ie VLAN in VLAN encapsulation) may be used to protect any VLAN tags used by the customer traffic.
For example, customer switch 42a may send Ethernet traffic over communications link 46a to PE-Edge of edge switch 22a. PE-Core of edge switch 22a encapsulates each Ethernet frame in a further Ethernet frame using the MAC address of edge switch 22a as the source address and the MAC address of the appropriate egress point—in this case edge switch 22b—as the destination address. The encapsulated traffic is forwarded across a connection established over communications links 48 of carrier network 20 to edge switch 22b. Note that connections will typically be trunked in the sense that traffic from multiple customers will be routed through the same connection. At the PE-Core of edge switch 22b, the original frames are stripped out and sent over communications link-46b via PE-Edge of edge switch 22b to customer switch 42b.
The reader will appreciate that in alternative embodiments of the present invention the PE-Edge may also be physically separated from the PE-Core and may reside at customer premises whereas the PE-Core resides at a central office or Point of Presence (PoP) of the carrier. The reader will also appreciate that other edge switches 26 will have connections to customer sites and that customers may have be provided with connectivity between two or more geographically distant sites over carrier network 20.
It will now be described how carrier network 20 is arranged to establish connections through which to forward encapsulated Ethernet traffic. A connection may be defined as an entity configured in a network which provides transport of data from a source node to one or more sink nodes.
As described above, carrier network 20 must be at least partially-meshed—ie there must be multiple paths between at least some, and preferably all, nodes of the network. Thus, as will be explained below, Ethernet MAC address auto learning functionality should preferably be at least partially deactivated.
On start-up (or on re-start), conventional switched Ethernet networks behave like a “classic” Ethernet Local-Area Networks (LANs) in that every Ethernet frame is broadcast across the entire network. Thus, every switch, receiving an Ethernet frame on one port, broadcasts the frame out on every other port. The process repeats as the frame is received by other switches thus broadcasting the frame across the entire network.
MAC address auto-learning-functionality is provided to improve efficiency in switched Ethernet networks. Ethernet frames have source and destination MAC addresses corresponding to their source and destination Ethernet switches. When an Ethernet frame sent out by a source switch is received by an intermediate or destination Ethernet switch, the receiving switch observes the port on which the frame was received and the source address of the frame. It then builds up a forwarding table for use in future frame switching. The forwarding table maps destination address to output port and is built up using the source address of a received frame and the input port on which it was received. Over time, the network builds up forwarding state enabling efficient switching of Ethernet frames.
It can thus be seen that conventional switched Ethernet networks using auto-learning must be simply-connected—ie there must be one and only one path between each and every node of the network. If there were multiple paths between any two nodes, the input port on which a frame is received from a source node would not be a reliable indicator of the correct output port to forward future traffic destined for that node. Inconsistencies in forwarding tables on Ethernet switches would result in looping of frames. Moreover, if there exists any loop in an part of the network then any broadcast packet will be continuously duplicated in that loop and the duplicates forwarded all over the whole network, limited only by the link capacities concerned. This inevitably results in catastrophic failure of the network.
According to the present invention, instead of using auto learning to configure forwarding tables in Ethernet switches, forwarding tables are directly configured using a novel Ethernet control plane.
Control plane 30 comprises a number of connection controllers 32a, 32b, 34a; 34b, 36 and 38 corresponding to each of Ethernet switches 22a, 22b, 24a, 24b, 26 and 28 of carrier network 20 (not all connection controllers are labelled in
While CCs 32a, 32b, 34a, 34b, 36 and 38 are logically separate from Ethernet switches 22a, 22b, 24a, 24b, 26 and 28, the reader will understand that they may be implemented in the same physical nodes. Furthermore, NNI signalling may take place over the same communications links used for transporting user traffic.
The request to establish a connection is first sent to CC 32a. On receipt of the request, CC 32a checks whether the communications link between switches 22a and 24a has sufficient capacity to support the required bandwidth. If so, it forwards a connection setup request message 50 to CC 34a specifying the required bandwidth and explicit route. CC 34a then checks whether the communications link between switches 24a and 24b has sufficient capacity to support the required bandwidth. The process continues until the connection setup message request 50 reaches CC 32b. Along the route. CCs may optionally reserve bandwidth of their respective switches and communication links so as to avoid race conditions where competing connections are setup over the same resources.
When connection setup request message 50 reaches CC 32b, if there is sufficient bandwidth along the entire path to support the required connection, then CC 32b sends a connection setup response message 52 back to CC 34b, CC 34a and finally to CC 32a. As the connection setup response message 52 traverses the CCs, each CC sends CCI signalling 54 to its respective switch to configure the forwarding tables of each switch, thereby to establish the forwarding state required to setup the connection.
It will be appreciated that the mechanism for establishing connections across carrier network 20 described above is merely exemplary and other well known mechanisms may be used.
How forwarding tables of the Ethernet switches of carrier network 20 are used to support connections is a key aspect of the present invention and will now be described in detail.
Typically, there will be many thousands or tens of thousands of connections established across a carrier network at any time. These connections will share the physical resources of the carrier network—ie the switches and communications links. Thus, each switch will typically have a large number of connections established through it at any point in time. However, each switch must be able to forward data traffic according to the explicit route requirements of the specific connection through which that traffic is being sent. A likely scenario is that the carrier network will need to establish multiple connections from the same source nodes, multiple connections to the same destination nodes and multiple connections both from the same source nodes and to the same destination nodes. However, for traffic engineering, even the latter connections may need to be established through different routes across the network. Furthermore, these routes may need to converge and diverge again within the carrier network. To support such route flexibility in connections, what is required is that each switch be able to differentiate between data traffic travelling in different connections and forward accordingly.
However, conventional switched Ethernet is incapable of this. As described above, conventional Ethernet switches forward traffic based solely on a forwarding table (established through auto learning) mapping destination address to output port. As a result, a conventional Ethernet switch will not be able to differentiate between data traffic having the same destination address, although it may be associated with multiple different connections.
According to the present invention, VLAN tags are used in a novel manner to enable differentiation of connections established across a carrier network and thereby to enable differential forwarding. Ethernet switches of carrier network 20 are VLAN-aware and use the combination of destination address and VLAN tag to forward data traffic. This may be achieved by each Ethernet switch storing separate forwarding tables for each VLAN tag configured, the VLAN tag acting as an mapping (or indexing) to the forwarding tables and each forwarding table mapping destination address to output port as normal. Thus the group of forwarding tables provide a mapping from a combination of destination address and VLAN tag to output port.
Thus, as shown in
Note that this functionality is similar to that performed by the prior art VLAN-aware bridges described above with reference to
According to the present invention, VLAN tags and entries in forwarding tables corresponding to connections to be established across the carrier network are directly configured into the appropriate Ethernet switches using the connection setup process described above. Data traffic is associated with a particular connection on entry into the carrier network (more specifically at the ingress PE-Core) by giving the frames a selected VLAN tag as well as destination address (ie the MAC address of the egress PE-Core). Note that encapsulation will already have taken place and thus the raw Ethernet frames received from the customer will not be altered in this process.
In
In
From
So far, only the establishment of point-to-point (ie unicast) connections have been described. However, according to the present invention, point-to-multipoint or multipoint-to-multipoint connections may also be established across Ethernet networks as will now be briefly described. Conventional Ethernet switches are capable of multicast service. Typically this is achieved by configuring the forwarding table with more than one output port (but not necessarily all output ports) for a given multicast destination address.
According to the present invention, for relatively small scale multicast operation, a point-to-multipoint connection may be configured as described above but using a combination of VLAN tag and multicast address mapping to more than one output port (but not necessarily all output ports) of selected Ethernet switches. However, this approach is only suitable for relatively small seal multicast operation.
According to the present invention, where a carrier-network wishes to support a large number of point-to-multipoint or multipoint-to-multipoint connections, Resilient Packet Ring (RPR) are emulated over the Ethernet MAC addressed network using multiple unicast connections established as described above. The following description is given in the context of a virtual private network (VPN) service, i.e. where there is a limited community of interest for each data frame. Two modes of operation are envisaged: a sparse mode for many customers with few sites, and a dense mode for few customers with many sites. The detailed mechanisms are described in one of the Applicants' co-pending U.S. patent application Ser. No. 10/698,833 (Norte! Networks Reference 15877RO) entitled Virtual Private Networks Within A Packet Network Having A Mesh Topology which document is incorporated herein by reference. The dense and sparse modes of operation will now be briefly described with reference to
The traffic is forwarded around RPR 112 to each of the core switches 116a to d in one direction or the other, whichever is shortest for each respective core switch. Each core switch broadcasts the received frames over uni-directional connections 114a so that each of PE-Core switches 104a to d receives the traffic. Then, as with the sparse mode of broadcast operation described above, each PE-Core switch determines whether to process the frame for distribution to the customer via PE-Edge edge Ethernet switches 102 to which it is connected. This is done on the basis of broadcast destination addresses contained in the frames and involves decapsulating and replicating them as required to one or more of PE-Edge switches 102 for onward transmission to the customer sites. It has been described above how connections may be established over a meshed Ethernet carrier network through configuring forwarding tables in network nodes and how data may be forwarded over those connections. The reader will appreciate that connections may be removed by deleting the configuration data from every node over which the connection was established. It is important that all such configuration data is removed to avoid network failure or inefficiency. As described above, the default behaviour of Ethernet switches on receiving a frame addressed to an unknown destination (ie where there is no forwarding state configured for that destination address) is to broadcast the frame out on all output ports. In simply-connected networks this behaviour is appropriate. However, with a meshed topology, this behaviour can be catastrophic. Through partial removal of connections (in particular where configuration data is left at ingress points of a connection but deleted at points further along the connections towards or including the egress point), it remains possible that Ethernet frames for the PE may enter the network but arrive at a point where there is no configuration data for forwarding them, resulting in undesirable broadcast behaviour. Furthermore, partial removal of connections may leave forwarding loops configured by accident. One solution to the problem of partial removal of connections is to alter the behaviour of the Ethernet switches forming the carrier network so that instead of broadcasting unknown traffic, they discard packets and possibly issue an alarm, log or count the discarded packets. However, altering the basic behaviour of Ethernet switches may require a hardware modification. While possible, this is not preferable. However, conventional Ethernet switches generally provide a software configurable function called rate limitation. Preferably, at all or most switches of the carrier rate limitation is used to set a rate of zero, or a low rate if zero is not possible, for broadcast traffic including broadcast-on-unknown traffic.
Where this is not possible, other pre-emptive approaches to minimising the problems of partial removal of connections may be used. One approach is to use block lists. Conventional Ethernet switches provide a block list (typically of limited length) which may be used to specify certain destination MAC addresses such that received Ethernet frames addressed to these blocked address will be discarded without forwarding. By blocking, at all or most nodes of the network, the MAC addresses of many (but not all) MAC addresses of provider edge nodes it is possible to minimise the potential dangers of partial removal of connections without over restricting the carrier's flexibility in establishing connections across the network. Notably, it is necessary to block different MAC address at different nodes of the network. Typically, at a given node, the block list will include only the MAC address for provider edge nodes to which no connections are likely to be established through that node. This approach is not easily scaleable with large networks {the limited number of entries in block lists may be exhausted by large numbers of provider edge nodes). However, note that to prevent loops it is only necessary to block rogue frames at one node in any loop. Thus, it is possible to “spread” the blocked destination addresses more thinly across the network and still provide a degree of protection from loops thereby making more efficient use of the limited capacity of block lists.
While it is the use of VLAN tags in the present invention that enables flexibility in establishing connections across the network, the failure to remove VLAN state fully leaves the potential for looping of traffic. In particular, the problem will arise where a logical loop is left configured for any single given VLAN tag—ie the output ports of nodes defining a physical loop are left configured with membership of any single VLAN. Thus, another pre-emptive approach to minimising the problems of partial removal of connections is to allocate connections to or from neighbouring or nearby provider edge nodes using mutually exclusive VLAN tag pools. Thus, for example all connections to or from provider edge node PE1 will be guaranteed to have a different VLAN tag to those to or from neighbouring provider edge node PE2. In this way, VLAN loops including both PE1 and PE2 cannot accidentally be formed through the partial removal of connections since-by definition any state left configured in PE1 and PE2 will use different VLAN tags. This approach may be generalised by allocating connections to or from n adjacent provider edge nodes using n mutually exclusive VLAN tag pools. n is chosen to be sufficiently large to segregate use of VLAN tag pools as much as possible while providing sufficient flexibility in connection establishment to or from any particular provider edge node (remembering that there are only 4094 possible VLAN tags). With smaller carrier networks it may be possible for each provider edge node to use a different VLAN tag pool. However, with larger carrier networks it will be necessary to re-use VLAN tag pools at topologically distant provider edge nodes otherwise flexibility in connection establishment will be compromised though VLAN tag pools being too small.
It will be appreciated that combinations of the above approaches to minimising the problems of partial removal of connections may be employed.
While embodiments have been described above with reference to the use of VLAN tags for enabling flexibility in establishing and differential forwarding of data traffic associated with different connections, the reader will appreciate that other tags or identifiers may be used.
Also, while embodiments have been described above with reference Ethernet networks and Ethernet frames, to the reader will appreciate that the present invention applies in general to any frame-based, packet-based or cell-based switching network whether at OSI layer 2 or layer 3 network. And to data structures including frames, packets and cells. In the following claims, the term frame-based network, or cognate terms, shall denote any such switching network and the term frame, or cognate terms, shall denote any such data structure.
The reader will appreciate that the methods described above may be implemented in the form of hardware or software operating on conventional data processing hardware.
Number | Date | Country | |
---|---|---|---|
Parent | 10818685 | Apr 2004 | US |
Child | 13683668 | US |