In recent years, network service providers have been upgrading and managing networks based on Multi Protocol Label Switching (MPLS) technology. MPLS has been deployed in most backbone networks. MPLS provides capabilities such as Quality of Service (QoS), redundancy, Operations Administration and Maintenance (OAM), and Virtual Private Network (VPN). MPLS is typically used to provision and manage data streams at individual flow levels. Each flow is known as a Label Switched Path (LSP). Existing MPLS systems typically handle data traffic at the Layer-3 (IP) level and below.
Some MPLS networks use Pseudowires to map Open System Interconnections (OSI) Layer-1 or Layer-2 traffic flows into “virtual circuits.” A Pseudowire refers to the emulation of a Layer-1 or Layer-2 native service over a network. Examples of native services include Asynchronous Transfer Mode (ATM), Frame Relay, Ethernet Virtual Local Area Network (VLAN), Time Division Multiplexing (TDM), Synchronous Optical Network (SONET), Synchronous Digital Hierarchy (SDH), etc. In the control plane, the Pseudowires are maintained and managed using a simplified version of Label Distribution Protocol (LDP), the Target LDP. Each Pseudowire is associated with an MPLS label for packet forwarding and a control word for flow management.
Since existing MPLS networks only allow Layer-1 or Layer-2 connections to be mapped to Pseudowires in a one-to-one mapping, the system typically cannot guarantee the QoS for individual applications that generate application data in Layer-3 or above. QoS behavior in the application layer is sometimes different from the behavior in Layer-1 or Layer-2. For example, packet video streams can generally tolerate out-of-sequence delivery, and packet voice traffic can sometimes tolerate packet loss but is sensitive to packet delay. Existing Layer-1 and Layer-2 systems, however, typically do not address network-level QoS for these voice and video applications.
Some proposed IP-based models have been developed to address the QoS requirement associated with applications, but some issues remain. For example, the IntServ/RSVP model identifies connections by applications based on the IP addresses of the source and destination, the protocol type, and the protocol's source and destination port number (together known as the 5-tuple). Each connection is required to comply with a number of service parameters such as bandwidth consumption and delay budget. As a result, the intermediate nodes (such as the core routers) are required to store the identity of all the connections, perform deep packet inspection, and implement extensive QoS mechanisms to satisfy the service parameters for each flow. Network service providers tend to find this model limiting because it is not very scalable as the number of users grow.
The DiffSery model addresses the scalability problems associated with the IntServ/RSVP model. Instead of handling QoS on a per flow basis, the user applications are classified into a small number of uniformly defined traffic classes. Each data packet stores its traffic class information in its IP header. At each intermediate node, the packet receives appropriate QoS treatment according to its traffic class. Since DiffSery provides relative QoS, in order to guarantee QoS to a particular flow, the model typically requires the network bandwidth to be over-provisioned. Further, the model only incorporates a subset of available QoS technology, such as priority queuing and Random Early Discard (RED) in dealing with temporary traffic congestion. The model is often not applicable in networks where physical links cannot be sufficiently over-provisioned.
It would be useful to have a way to better manage application traffic over a carrier network without requiring changes to the intermediate nodes. It would also be desirable if QoS guarantee can be achieved at per-application flow level. Furthermore, the solution should to be scalable.
Various embodiments of the invention are disclosed in the following detailed description and the accompanying drawings.
The invention can be implemented in numerous ways, including as a process, an apparatus, a system, a composition of matter, a computer readable medium such as a computer readable storage medium or a computer network wherein program instructions are sent over optical or communication links. In this specification, these implementations, or any other form that the invention may take, may be referred to as techniques. A component such as a processor or a memory described as being configured to perform a task includes both a general component that is temporarily configured to perform the task at a given time or a specific component that is manufactured to perform the task. In general, the order of the steps of disclosed processes may be altered within the scope of the invention.
A detailed description of one or more embodiments of the invention is provided below along with accompanying figures that illustrate the principles of the invention. The invention is described in connection with such embodiments, but the invention is not limited to any embodiment. The scope of the invention is limited only by the claims and the invention encompasses numerous alternatives, modifications and equivalents. Numerous specific details are set forth in the following description in order to provide a thorough understanding of the invention. These details are provided for the purpose of example and the invention may be practiced according to the claims without some or all of these specific details. For the purpose of clarity, technical material that is known in the technical fields related to the invention has not been described in detail so that the invention is not unnecessarily obscured.
Transferring data over a network using Application Wires is disclosed. An Application Wire refers to the emulation of a virtual circuit or a transparent wire for transferring one or multiple application flows. An Application Wire maps one or more application flows into Pseudowires, and is at the same time aware of the application protocol and the protocol requirements associated with the application flows. As used herein, an application flow refers to a set of packets exchanged between two or more devices for accomplishing a specific function. Application flow data includes data associated with Layer-4 or above as defined by the OSI protocol stack. In some embodiments, an application flow includes data packets transmitted and received by an application, such as a Voice over IP (VoIP) session, instant messaging, Video-on-Demand (VoD), etc. The application may be configured to operate on various wired, wireless, or hybrid devices. The interface between the application and the underlying network is provided by protocols such as the Session Initialization Protocol (SIP) and the Real-time Transport Protocol (RTP). From the perspective of the application, transferring data over an Application Wire has the same effect as transferring data over a dedicated wire. As will be shown in more detail below, in some embodiments, Application Wires are formed by mapping application flows to Pseudowires.
Since the mapping of one data stream to one Pseudowire does not account for the bandwidth demand of individual applications generating traffic on the client devices, it is often difficult to guarantee the quality of service (QoS) for these individual applications. For example, assuming that Pseudowires A and B each support a data rate of 50 Mbps, and that the application executing on device 102 require 70 Mbps of bandwidth while the application on device 104 only requires 20 Mbps. Because of the one-to-one mapping, the bandwidth requirement of device 102 is not met by Pseudowire, even though there is excess capacity on the network overall.
Application Wires improve on the Pseudowire scheme described above.
At the edge of the core network, edge nodes 220 and 222 are configured to transfer the data streams between a core network (such as a backbone network of a service provider) and the client devices. As used herein, data traffic is said to be in the ingress direction if it is being transferred from client devices to the core network, and in the egress direction if it is being transferred from the core network to clients devices. A data connection that transfers data in the ingress direction is referred to as a network-bound connection. Depending on the direction of the data flow, an edge node may be referred to as an ingress node or an egress node.
As will be shown in more detail below, the edge nodes map the application flows into Pseudowires in the ingress direction. A number of mapping schemes are possible, such as a one-to-one mapping, a distributed mapping where an application flow is mapped to multiple Pseudowires, or an aggregated mapping where multiple flows are mapped to a single Pseudowire. In the example shown, application flows a and b are one-to-one mapped to a separate Pseudowire each, application flow d is distributed to Pseudowires d1 and d2, and application flows e and f are aggregated to a single Pseudowire (e+f). The mapping scheme for each application flow is selected based at least in part on the bandwidth and traffic requirement associated with the application. In the egress direction, packets transferred on the Pseudowires are reassembled to the corresponding application flows and sent to the appropriate destinations. Each Pseudowire may span a plurality of intermediate nodes such as core routers 230 and 232. Unlike data transfer schemes where deep packet inspection is required at each intermediate node, the intermediate nodes used in this example can be standard MPLS devices and no change is required to make them support the Application Wire scheme.
It is useful to inspect an example of a packet to understand how the application flow identification is done.
Returning to
If, however, the header information is found in the database, the packet is mapped to an application flow (410). Admission control is optionally performed on the flow (412). In various embodiments, admission control includes shaping traffic by changing the packets priority, applying a policy/rule, tagging, dropping the packet, etc. If the packet is not dropped by admission control, it is mapped to one or more Pseudowires configured to service the application flow (414). In some embodiments, a database of available Pseudowires is searched to find one or more suitable Pseudowires for carrying the application flow. The mapping is based on, among other things, IP routing or manual configuration.
As previously discussed, the mapping of application flow to Pseudowire may be one-to-one, N-to-one, or one-to-N. One-to-one mapping is the most straightforward. Sometimes multiple application flows are aggregated into a single flow (N-to-one). Aggregation is appropriate when, for example, the application flows are similar and have the same priority level. Sometimes, an application flow is distributed into multiple streams and transferred over the network via multiple Pseudowires (one-to-N). For example, a large flow exceeding a certain data rate threshold may be split into several Pseudowires to better utilize the available bandwidth. The division of the application flow into multiple streams is based at least in part on application-specific parameters. For example, a large RTP stream is sometimes split based on SSRC or payload frame type. In one example, an application flow involves a large RTP stream having a large amount of MPEG traffic over a network without any per flow QoS guarantee. To reduce the impact of dropped packets, the more important packets such as M-frames in the application flow are separated from the rest. The important packets are mapped to a Pseudowire with a higher priority level. The rest of the packets are mapped to one or more lower priority Pseudowires.
Returning to
Since the resulting packet is an MPLS formatted packet, it can be processed by any intermediate nodes on the network (e.g. network routers and switches as) a regular MPLS packet. So long as the intermediate node is a standard MPLS enabled device, no modification is required of the device for processing an Application Wire related packet.
Since it is possible for packets sent on different Pseudowires to arrive out of order, the packets are re-sequenced as appropriate (706). In some embodiments, the re-sequencing includes re-sequencing at the Pseudowire level. The sequence number field in the Pseudowire header is examined and used to sort the packets in the appropriate sequence. In some embodiments, the re-sequencing includes an application flow level re-sequencing. Application header and/or payload information is used to sort packets belonging to the same application flow in the appropriate order. For example, the SSRC and the sequence numbers in the RTP header, as well as the payload data are used in some embodiments to re-sequence an RTP flow. Once re-sequenced, the Pseudowire header of the packet is removed and the packet is forwarded to its destination (708).
In some embodiments, processes 300, 400 and 700 are carried out by an application flow engine (AFE).
When handling ingress data streams, the AFE identifies and maps application flows in the data streams to a plurality of Pseudowires. The application flows are denoted as F={fsub.1, f.sub.2, . . . f.sub.n} and the Pseudowires are denoted as W={w.sub.1, w.sub.2, . . . w.sub.m}. An application flow identifier 806 identifies new application flows in the data stream, and stores information associated with the application flows in a database labeled as an application flow table (AFT) 802. The AFT is also used to identify data packets that match application flows already stored in the AFT. Information stored in the AFT includes, among other things, flow identification information and service parameters. The flow identification information includes attributes used to identify the specific application flow and may vary depending on the application. For example, for an RTP-based application flow, the corresponding IP source and destination addresses, UDP protocol type, UDP source and destination port number, S SRC and CSRC are recorded in the AFT. Examples of the service parameters include various measured or assigned characteristics, such as the average and peak bandwidth of the flow, the burst size, the importance level of the flow (for example, emergency 911 traffic is assigned the highest importance and can preempt other flows at runtime), sub-flow information such as the bandwidth and importance levels associated with different sources, as well as other application dependent information such as whether to allow out-of-sequence packets in the flow. In some embodiments, at least a part of the AFT is populated ahead of time by the service provider. For example, the service parameters may be manually configured or populated using a configuration file when the system is initialized. Having a pre-populated AFT allows the service providers to offer different levels of services, and/or provide QoS guarantee based on subscription.
In
In some embodiments, the PWT maintains the network-bound Pseudowires, W={w.sub.1, w.sub.2, . . . w.sub.m}. For each Pseudowire, the following attributes are stored in one example: MPLS label for in packet encapsulation, QoS information indicating the level of QoS to be applied to the Pseudowire, Protection Path information identifying one or more backup Pseudowires used to protect this Pseudowire, OAM capability information used for error detection and loop-back, Multicast grouping information such as group ID used to transport multicast traffic over the MPLS/IP network.
When handling egress data streams, a process similar to 700 is carried out by the AFE. The PWT is used to look up the application flows that correspond to the packets received on various Pseudowires. The mapper re-sequences the packets, removes the Pseudowire headers, and forwards the packets to the destination.
The Application Wire techniques described above are also applicable for environments in which multimedia streams are multicasted to multiple sites in the network. To support multicast over Application Wires, a fully-meshed Pseudowire network for each multicast group is set up. Various mechanisms for supporting Pseudowire based multicast can be used, including Virtual Private LAN Service (VPLS) and IP LAN Service (IPLS).
On an egress network edge node, the Pseudowire headers of the packets are removed, the packets are reassembled and/or re-sequenced as necessary, and forwarded to the destination. Any IP or Layer-2 multicast scheme may be used to forward the packets to a destination beyond the network edge nodes.
An Application Wire based data transfer technique has been described. The technique gives service providers greater flexibility in providing services based on applications, without requiring changes to intermediate devices.
Although the foregoing embodiments have been described in some detail for purposes of clarity of understanding, the invention is not limited to the details provided. There are many alternative ways of implementing the invention. The disclosed embodiments are illustrative and not restrictive.
This application is a continuation of U.S. patent application Ser. No. 16/912,546, filed Jun. 25, 2020, which is a continuation of U.S. patent application Ser. No. 16/536,103, filed Aug. 8, 2019, which is a continuation of U.S. patent application Ser. No. 15/838,014, filed Dec. 11, 2017, which is a continuation of U.S. patent application Ser. No. 14/919,687, filed Oct. 21, 2015, which is a continuation of U.S. patent application Ser. No. 14/055,721, filed Oct. 16, 2013, which is a continuation of U.S. patent application Ser. No. 11/543,727, filed Oct. 5, 2006, which claims priority to U.S. Provisional Application No. 60/725,038, filed Oct. 7, 2005. The entireties of these related applications are incorporated herein by reference.
Number | Name | Date | Kind |
---|---|---|---|
5920705 | Lyon et al. | Jul 1999 | A |
6167051 | Nagami et al. | Dec 2000 | A |
6347088 | Katou et al. | Feb 2002 | B1 |
6430184 | Robins et al. | Aug 2002 | B1 |
6477166 | Sanzi et al. | Nov 2002 | B1 |
6546427 | Ehrlich et al. | Apr 2003 | B1 |
6574477 | Rathunde | Jun 2003 | B1 |
6621793 | Widegren et al. | Sep 2003 | B2 |
6665273 | Goguen et al. | Dec 2003 | B1 |
6680943 | Gibson et al. | Jan 2004 | B1 |
6751684 | Owen et al. | Jun 2004 | B2 |
6813271 | Cable | Nov 2004 | B1 |
6845389 | Sen et al. | Jan 2005 | B1 |
6917592 | Ramankutty et al. | Jul 2005 | B1 |
6985488 | Pan et al. | Jan 2006 | B2 |
7050396 | Cohen et al. | May 2006 | B1 |
7200104 | Saleh et al. | Apr 2007 | B2 |
7436782 | Ngo et al. | Oct 2008 | B2 |
7590070 | Asawa et al. | Sep 2009 | B1 |
7697528 | Parry et al. | Apr 2010 | B2 |
7830787 | Wijnands et al. | Nov 2010 | B1 |
9843509 | Pan et al. | Dec 2017 | B2 |
20010021175 | Haverinen | Sep 2001 | A1 |
20010023453 | Sundqvist | Sep 2001 | A1 |
20010043603 | Yu | Nov 2001 | A1 |
20020112072 | Jain | Aug 2002 | A1 |
20020141393 | Eriksson et al. | Oct 2002 | A1 |
20020146026 | Unitt et al. | Oct 2002 | A1 |
20030002482 | Kubler et al. | Jan 2003 | A1 |
20030039237 | Forslöw | Feb 2003 | A1 |
20030117950 | Huang | Jun 2003 | A1 |
20040057424 | Kokkonen | Mar 2004 | A1 |
20040105459 | Mannam | Jun 2004 | A1 |
20040114595 | Doukai | Jun 2004 | A1 |
20040133692 | Blanchet et al. | Jul 2004 | A1 |
20040156313 | Hofmeister et al. | Aug 2004 | A1 |
20040174865 | Oneill | Sep 2004 | A1 |
20040252717 | Solomon et al. | Dec 2004 | A1 |
20050018605 | Foote et al. | Jan 2005 | A1 |
20050044262 | Luo | Feb 2005 | A1 |
20050058060 | Caldwell et al. | Mar 2005 | A1 |
20050080890 | Yang et al. | Apr 2005 | A1 |
20050080911 | Stiers et al. | Apr 2005 | A1 |
20050093770 | De et al. | May 2005 | A1 |
20050125490 | Ramia | Jun 2005 | A1 |
20050220148 | Delregno et al. | Oct 2005 | A1 |
20050237927 | Kano et al. | Oct 2005 | A1 |
20050238049 | Delregno | Oct 2005 | A1 |
20050259586 | Hafid et al. | Nov 2005 | A1 |
20060002423 | Rembert et al. | Jan 2006 | A1 |
20060018252 | Sridhar et al. | Jan 2006 | A1 |
20060046658 | Cruz et al. | Mar 2006 | A1 |
20060047851 | Voit et al. | Mar 2006 | A1 |
20060090008 | Guichard et al. | Apr 2006 | A1 |
20060146832 | Rampal et al. | Jul 2006 | A1 |
20060215653 | LaVigne | Sep 2006 | A1 |
20060227767 | Johnson et al. | Oct 2006 | A1 |
20060233167 | McAllister et al. | Oct 2006 | A1 |
20070030851 | Sinicrope et al. | Feb 2007 | A1 |
20070038921 | Pekonen et al. | Feb 2007 | A1 |
20070053366 | Booth et al. | Mar 2007 | A1 |
20070070893 | Butenweg et al. | Mar 2007 | A1 |
20070127479 | Sinicrope et al. | Jun 2007 | A1 |
20070206607 | Chapman et al. | Sep 2007 | A1 |
20080031129 | Arseneault et al. | Feb 2008 | A1 |
20080144632 | Rabie et al. | Jun 2008 | A1 |
20080144641 | Le et al. | Jun 2008 | A1 |
20080186897 | Rune et al. | Aug 2008 | A1 |
Entry |
---|
Afferton, Thomas S, et al., “Ethernet Transport over Wide Area Networks”, Afferton, Thomas S. et al., Ethernet Transport over Wide Area Networks, Packet-Aware Transport for Metro Networks, IEEE Communications Magazine, pp. 120-127, Mar. 2004., Mar. 2004. |
Anderson L, et al., “LDP Specification”, Anderson, L. et al., LDP Specification, Network Working Group, Jan. 2001, Jan. 2001. |
Blake , et al., “An Architecture for Differentiated Services”, Blake et al. “An Architecture for Differentiated Services,” Network Working Group, Dec. 1998, Dec. 1998. |
Braden , et al., “Integrated Services in the Internet Architecture: an overview”, Braden et al. “Integrated Services in the Internet Architecture: an overview,” Network Working Group, Jun. 1994, Jun. 1994. |
Bryant , et al., “Pseudo wire Emulation Edge to-Edge (PWE3) Architecture”, Bryant et al. “Pseudo wire Emulation Edge4o-Edge (PWE3) Architecture,” Network Working Group, Mar. 2005, Mar. 2005. |
Martini Luca, et al., “Dynamic Placement of Multi Segment Pseudo Wires”, Martini, Luca et al., Dynamic Placement of Multi Segment Pseudo Wires, PWE3 Working Group, Jun. 2006., Jun. 2006. |
Martini Luca, et al., “Encapsulation Methods for Transport of Ethernet over MPLS Networks”, Martini, Luca et al., Encapsulation Methods for Transport of Ethernet over MPLS Networks, Network Working Group. Apr. 2006., Apr. 2006. |
Martini Luca, et al., “Encapsulation Methods for Transport of Frame Relay Over MPLS Networks”, Martini, Luca et al., Encapsulation Methods for Transport of Frame Relay Over MPLS Networks, Network Working Group, Feb. 2006., Feb. 2006. |
Martini, Luca , et al., “Internet Draft, Segmented Pseudo Wire”, Martini, Luca et al., Internet Draft, Segmented Pseudo Wire, Network Working Group, Jul. 2007, Jul. 2007. |
Martini Luca, et al., “Pseudowire Setup and Maintenance using LDP”, Martini, Luca et al., Pseudowire Setup and Maintenance using LDP. Network Working Group, Mar. 2005., Mar. 2005. |
Martini, Luca , et al., “Pseudowire Setup and Maintenance using the Label Distribution Protocol (LDP)”, Martini, L. et al., Pseudowire Setup and Maintenance using the Label Distribution Protocol (LDP), Network Working Group, Apr. 2006., Apr. 2006. |
McPherson , et al., “Pseudowire Emulation Edge to Edge (PWE3)”, McPherson et al., Pseudowire Emulation Edge to Edge (PWE3) Jun. 13, 2007, http://www.ietforg/html.charters/pwe3-carter.html, Jun. 2007. |
Metz Chris, et al., “Pseudowire Attachment Identifiers for Aggregation and VPN Autodiscovery”, Metz, Chris et al., Pseudowire Attachment Identifiers for Aggregation and VPN Autodiscovery, PWE3 Working Group, Feb. 25, 2006., Feb. 25, 2006. |
Newton, Harry , ““Newton's Telecom Dictionary”, 23rd Updated and Expanded Edition”, Harry Newton, “Newton's Telecom Dictionary”, 23rd Updated and Expanded Edition, p. 825,p. 239, Flatiron Publishing, New York, Mar. 2007., Mar. 2007, 825, 239. |
Pan, Ping , “Internet Draft, Dry-Martini: Supporting Pseudo-wires in Sub-IP Access Networks”, Pan, Ping, Internet Draft, Dry-Martini: Supporting Pseudo-wires in Sub-IP Access Networks, Network Working Group, Jul. 2005., Jul. 2005. |
Pan, P , et al., “Internet Draft, Pseudo Wire Protection”, Pan, P. et al., Internet Draft, Pseudo Wire Protection, Jul. 2006, Jul. 2006. |
Rosen, E , et al., “BGP-MPLS IP Virtual Private Networks (VPN)”, Rosen, E. et al., BGP-MPLS IP Virtual Private Networks (VPN), Network Working Group, Feb. 2006., Feb. 2006. |
Rosen, Eric , et al., “PWE3 Congestion Control Framework”, Rosen, Eric C. et al., Internet Draft, PWE3 Congestion Control Framework, Network Working Group, Mar. 2004. |
Shah, Himanshu , et al., “Internet Draft, ARP Mediation for IP Interworking of Layer 2 VPN”, Shah, Himanshu et al., Internet Draft, ARP Mediation for IP Interworking of Layer 2 VPN, L2VPN Working Group, Jul. 2007., Jul. 2007. |
Theimer, T , et al., “Requirements for OAM Functionality in MPLS”, Theimer, T. et al, “Requirements for OAM Functionality in MPLS”, Oct. 1999, Watersprings., Oct. 1999. |
Vasseur , et al., “Path Computation Element (pce)”, Vasseur, et al., Path Computation Element (pce), May 9, 2007, http://www.ietLorg/html.charters/pce.charter.html, May 2007. |
Number | Date | Country | |
---|---|---|---|
20220124032 A1 | Apr 2022 | US |
Number | Date | Country | |
---|---|---|---|
60725038 | Oct 2005 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 16912546 | Jun 2020 | US |
Child | 17406865 | US | |
Parent | 16536103 | Aug 2019 | US |
Child | 16912546 | US | |
Parent | 15838014 | Dec 2017 | US |
Child | 16536103 | US | |
Parent | 14919687 | Oct 2015 | US |
Child | 15838014 | US | |
Parent | 14055721 | Oct 2013 | US |
Child | 14919687 | US | |
Parent | 11543727 | Oct 2006 | US |
Child | 14055721 | US |