Patent Grant
9509582
This application is the national phase under 35 U.S.C. §371 of PCT International Application No. PCT/EP2012/050055 which has an International filing date of Jan. 3, 2012, which claims priority to European patent application number 11290011.3 filed Jan. 12, 2011; the entire contents of each of which are hereby encorporated by reference.
This invention relates generally to the technical field of networks delay/latency management.
Network-introduced delays are amongst the most important network performance metrics as they directly impact several wide area network applications ranging from real-time applications such as VoIP, interactive network gaming, to time-critical financial applications and localization systems.
Thus, monitoring the performance of data transmission delay within networks must involve a detailed understanding of how and where these delays are introduced.
Within traditional TDM technologies, network delay is predictable as per the deterministic transition time across TDM switches (e.g. in term of number of system clock transitions). However, with the tremendous increase in bandwidth demand, TDM is progressively replaced by Packet Switch Networks (PSNs) where packet jitters, also called as packet delay variations (random processes essentially induced by packet queuing), make packet resident time within the network node (called as network node resident time from now on) unpredictable.
Accordingly, PSN network operators need more than ever tools to monitor network delay or network latency in order to be able to take appropriate actions (e.g. network redesign/reconfiguration) aiming at respecting the Service Level Agreements (SLAs) and correcting SLA violations in term of network delay/latency.
To address these problems, Network operators, generally, rely on various end-to-end time-delay measurement tools such as
However, these tools return the whole end-to-end delay without any precision on the network node resident time (or latency). In other word, the returned delay value by these tools is considered as a single unitary component, already including the network node resident time without any precision thereon.
The network-introduced delay may be broadly divided into:
Accordingly, by returning the whole end-to-end delay in a single value without giving any details on its components, up-to-date end-to-end delay measurement tools do not allow the operator to figure out the network segment(s) or the network node(s) where corrective actions should be applied to solve the latency budget exceeding issue.
Yet another problem of the prior art is that existent network diagnostic tools do not permit to determine what fraction of the total transfer latency is due to network node resident time.
One object of the present invention is to address the above-noted and other problems with the related art.
Another object of the present invention is to pinpoint where dominant delays are introduced within a network path.
Another object of the present invention is to provide a fine-grained composition of the network-introduced delays.
Another object of the present invention is to propose a method that permits to determine the per-node latency.
Another object of the present invention is to provide a command which, by controlling the content of a probe message, provides a fine-grained picture of end-to-end delays that this packet undergo.
Another object of the present invention is to split the end-to-end delay into components distinguishing the nodes resident times along a path from a source to a destination within an IP network.
Another object of the present invention is to permit operators to make rapid and precise diagnostic of the SLA violation issue (quality of the committed service not respected) in term of network latency.
Another object of the present invention is to provide a diagnostic command that permit to accurately pinpoint the sources of important delays in Internet applications.
Another object of the present invention is to uncover dominant network hops introducing to most latency and being responsible for delay degradation.
The objects, advantages and other features of the present invention will become more apparent from the following disclosures and claims. The following non-restrictive description of preferred embodiments is given for the purpose of exemplification only with reference to the accompanying drawing in which
The present invention is directed to addressing the effects of one or more of the problems set forth above. The following presents a simplified summary of the invention in order to provide a basic understanding of some aspects of the invention. This summary is not an exhaustive overview of the invention. It is not intended to identify critical elements of the invention or to delineate the scope of the invention. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is discussed later.
The present invention relates to a method for measuring the resident time of a probe message in at least a network node comprised within a network path, said probe message provided with a Time-To-Live value, said method including the following steps:
In accordance with a broad aspect, the cited above method further comprises the following steps
In accordance with a broad aspect, the probe message is a modified Internet Control Message Protocol (ICMP) message.
In accordance with another broad aspect, the probe message is a modified Operation Administration Maintenance (OAM) message such as an MPLS-TP/MPLS OAM message or an Ethernet OAM message.
Advantageously, the computed resident time of the probe message within the network node is equal to its resident time within the main protocol layer/stack which is responsible for the probe message processing (i.e. coding/decoding). Thus, the use of probe messages at different protocol layers allows for analyzing the impact of different protocol layers on each node delay budget.
The present invention further relates to a network node comprising
The present invention further relates to a computer program product adapted to perform the method cited above.
While the invention is susceptible to various modification and alternative forms, specific embodiments thereof have been shown by way of example in the drawings. It should be understood, however, that the description herein of specific embodiments is not intended to limit the invention to the particular forms disclosed.
It may of course be appreciated that in the development of any such actual embodiments, implementation-specific decisions should be made to achieve the developer's specific goal, such as compliance with system-related and business-related constraints. It will be appreciated that such a development effort might be time consuming but may nevertheless be a routine understanding for those or ordinary skill in the art having the benefit of this disclosure.
A diagnostic command, designated below as traceroute_delay( ) is provided. This designation is given only for naming purpose, somewhat jointly referring to traditional traceroute( )command (e.g. RFC1393) and to delay measurement issue. Obviously, any other name may be given therefor.
By controlling probe message headers, traceroute_delay( )command collects, in addition to traversed node addresses, traversed node resident times and eventually traversed link propagation delays.
In one embodiment, the probe is a modified ICMP message. In fact, traceroute_delay( )command makes use of some fields of ICMP timestamp, and ICMP timestamp replay message (RFC 972), differently from what they are expected for (providing a new semantic to these existing fields).
With reference to
Therefore, the probe header, according to traceroute_delay( )command, comprises
It is noteworthy to mention that the use of modified ICMP Timestamp or Timestamp Reply message is only for the purpose of taking advantage of these already standardized but rather unused messages, leading to a rapid implementation and deployment.
Alternatively, the above-described probe format may be defined without any regards to ICMP Timestamp and/or Timestamp Reply message. Then, a probe header is conceived to include “Outbound Resident Time”, “Receive Timestamp”, “Return Resident Time” fields, and a protecting flag “L” associated to each one of “Outbound Resident Time” and “Return Resident Time” fields. These fields are programmed for the above described tasks.
The traceroute_delay( )command has the following syntax:
wherein
The format of “destination address” depends on the technology and the protocol used to transport the traceroute_delay( )command. For example, it can be
The input “QoS” is the value that will be set:
Indeed, the data packet delay often depends on its assigned “QoS” (at the originating/departing point), as per differentiated service treatment applied at each traversed node and on the related scheduling configured.
In regards with “mode” input, as per packet jitter magnitude differs in each communication direction (outbound direction and return direction), packet resident times are often different for each direction. Accordingly, the two modes, namely one-way mode and two-way mode, are provided. The “Identifier” field most significant bit allows for segregating between the one-way (value 0) and round-trip delay (value 1) measurement mode.
More generally, other input parameters may be also considered to define an invocation of the traceroute_delay( )command such as:
To introduce the behavior of traceroute_delay( )command, a brief reminder of the basic algorithm of the traditional traceroute command is depicted on
On ICMPv4,traditional traceroute( )command works by causing each node (from 2 to n) along the network path linking nodes (from 1 to n) to return an ICMP error message. Probing is done hop-by-hop, moving away from the source towards the destination i (i=2,. . . , n) in a series of round-trips 11-13. The Time-To-Live or hop count starts at one and is incremented after each round-trip 11-13 until the destination node n is reached, or until another stopping condition applies.
In fact, traditional traceroute( )sends its first group of packets with TTL=1 (Time-to-Live). The first router along the path (node 2) will therefore discard the packets (their TTL decremented to zero) and return the ICMP “TTL Exceed” error message (round-trip 11). Thus, the traceroute( )can register the first router (node 2) address. Packets can then be sent with TTL=2 (round-trip 12), and then TTL=3 and so on (round-trip 13), causing each router along the path to return an error, identifying it to the traceroute( )command (located at the source router or host). Eventually, either the final destination (node n) is reached, or the maximum TTL value (default is 30) is reached and traceroute( )ends. At final destination, a different error is returned.
Some implementations work by sending UDP datagram to some random high-numbered port where nothing is listening, some other implementations use ICMP Echo packets.
With reference now to
For the outbound direction, each traversed node i (i=2, n) realizes the following operations:
For the return direction, each node realizes the following operations:
An illustrative example, with TTL=2,of the above algorithm application is shown on
More generally, within each round-trip 11-13 wherein TTL=i (i=1,. . . , n-1), the resident time in the node i, in one-way and/or in two-way mode, is measured then stored within the probe message header. Accordingly, different unidirectional resident times (or unidirectional latencies) per nodes may be displayed, for the operator, in a summarizing table.
To that end, the network node i (i=2,. . . , n) comprises
It is to be noted that if a node receive a probe packet with a TTL strictly greater than one after being decremented by one, the probe packet is forwarded without resident time measurement step.
Another embodiment on MPLS-TP OAM makes use, in addition to the above described algorithm, the IETF document “Operating MPLS Transport Profile LSP in Loopback Mode”, March 2010.
Different OAM Loopbacks are to be performed by the sending/source Maintenance End Point (MEP) with increasing TTL from 1 until reaching the remote/destination MEP.
An MPLS-TP OAM-embedded traceroute_delay message is defined for this purpose. Its format (MPLS-TP traceroute_delay) is shown on
The last two field allows for the traceroute_delay( )command to compute an end-to-end one-way delay (i.e. “One-way Receive Timestamp”—“Originator Transmit Timestamp”). This supposes that the destination node clock is synchronized to the originator clock with an accuracy conformed to the measurement requirements.
The “Originator Transmit Timestamp” allows the originator for computing the round-trip delay at reception of the Return message. This imposes that the “Originator Transmit Timestamp” field of the “Outbound” message (this message is equivalent to the “Timestamp” message in the previous embodiment) is copied to the “Originator Transmit Timestamp” field of the “Return” message (this message is equivalent to the “Timestamp Reply” message in the previous embodiment).
It is to be noted that in the previous ICMP embodiment, the originator can still measure the round-trip delay even in the absence of the “Originator Transmit Timestamp”. It can, for instant, log for each message identifier (i.e. “Identifier” field) value the associated transmit timestamp locally (i.e. in its local context memory) and logs the receive timestamp of the “Timestamp Reply” message with the same identifier.
It is noteworthy to mention that successive round-trip times measured allow the traceroute_delay( ) command for computing the total link delay by subtracting the node resident times from the round-trip time (assuming link delays are symmetric).
The network node resident time is monitored at each protocol layer independently from the other layers. For example, in an IP/Ethernet network:
Thus, for a given network node, the average resident time as reported at the ICMP/IP layer is smaller than the one reported by the Ethernet OAM layer. For the latter (lowest protocol within the protocol stack), hardware timestamping can be implemented. This way allows for analyzing, within a given node, the protocol layer which impacts most the network node latency. The resident time measurement method at every layer within a node is implementation specific and is not in the scope of this invention.
It is to be noted that the “Timestamp Reply” message IP source address does not provides the traceroute_delay( ) with the IP address of the node where both the outbound and return resident times are measured but with the IP address of the next node on the outbound path. To gets the node IP address, the command should refer to the previous “Timestamp Reply” message.
traceroute_delay( ) may use different methods for sending probe messages, such as
It is noteworthy to mention that the command can be executed on-demand for diagnostic purpose, but can also be automatically executed at regular time intervals in a proactive manner in order to react rapidly before that the customer detects the issue.
traceroute_delay( ) command may be also included in operating systems, or encapsulated into network tools (such as NetTools).
It is to be noted that, as it concerns network node resident time measurements, there is really no need for synchronization of node clocks as the resident time is small, meaning below a few ms order. A traditional low-cost 100 ppm (part-per-million) accuracy clock (e.g. Ethernet interface clock) in free-run induces a measurement error of 1 μs (100×10−6×10×10−3) over 10 ms of resident time (and respectively a maximum measurement error below 10 μs for a typical cascade of 10 nodes).
Advantageously, the knowledge of the nodes resident times within a network path allows the identification of nodes that fail to offer acceptable delay bounds. Moreover, it permits conclusive and accurate assignment of introduced delays to either the network links, or nodes.
Advantageously, the above-described method permits to split the end-to-end time delay into two components: the transmission delays on network segments (or links) and nodes resident times. Accordingly, traceroute_delay( ) command provides a detailed view/apportionment of the end-to-end one-way (resp. two-way) delay allowing to point out network segment(s) or network node(s) to be reworked/re-engineered when the end-to-end one-way (resp. two-way) delay exceeds the SLA threshold.
| Number | Date | Country | Kind |
|---|---|---|---|
| 11290011 | Jan 2011 | EP | regional |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/EP2012/050055 | 1/3/2012 | WO | 00 | 10/23/2013 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2012/095335 | 7/19/2012 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 6868068 | Jain | Mar 2005 | B1 |
| 20040081101 | Bennett | Apr 2004 | A1 |
| 20050018647 | Lebrun | Jan 2005 | A1 |
| 20050111487 | Matta et al. | May 2005 | A1 |
| 20050243733 | Crawford | Nov 2005 | A1 |
| 20060159025 | Abdo | Jul 2006 | A1 |
| 20080019282 | Alaria | Jan 2008 | A1 |
| 20080259813 | Matta et al. | Oct 2008 | A1 |
| 20090161569 | Corlett | Jun 2009 | A1 |
| 20100177643 | Matta et al. | Jul 2010 | A1 |
| 20140241189 | Larsen | Aug 2014 | A1 |
| Number | Date | Country |
|---|---|---|
| WO-02095609 | Nov 2002 | WO |
| Entry |
|---|
| International Search Report PCT/ISA/210 for PCT/EP2012/050055 dated Apr. 12, 2012. |
| Written Opinion PCT/ISA/237 for PCT/EP2012/050055 dated Apr. 12, 2012. |
| Number | Date | Country | |
|---|---|---|---|
| 20140043992 A1 | Feb 2014 | US |
