The subject matter described herein relates to measuring performance of network nodes in time sensitive networks. More particularly, the subject matter described herein relates to methods, systems, and computer readable media for measuring schedule update time for a time aware shaper implementation.
Ethernet has become the de-facto data networking technology. It has been tested over time and has gained dominance in data communication. Today, Ethernet is being extended to industrial networks where robots, machines, and sensors communicate among themselves to take some action collectively. To be suitable for industrial networks, Ethernet needs to overcome some of its weakness. One set of improvements in Ethernet to make Ethernet suitable for industrial networks is referred to as Time Sensitive Networking (TSN). Out of the various TSN standards that help Ethernet overcome its drawbacks, IEEE 802.1Qbv—Enhancements for Scheduled Traffic (also known as Time Aware Shaper) has become important. IEEE 802.1Qbv provides scheduling across bridges for time critical application traffic. This ensures predictable and low latency end-to-end paths for industrial and in-vehicle applications that require precise communication.
For example, the speed at which an 802.1Qbv implementation can adapt to a configuration change and accurately measuring the time required to implement and 802.1Qbv configuration change are important. Accordingly, there is a need for methods, systems, and computer readable media for measuring schedule update time for a time aware shaper implementation.
A method for measuring a schedule update time of a time aware shaper device under test (DUT) includes providing an emulated time aware shaper centralized network configuration (CNC) node, an emulated time aware shaper talker, and an emulated time aware shaper listener. The method further includes configuring the time aware shaper DUT with a first configuration that blocks traffic from at least one gate of the DUT. The method further includes transmitting traffic from the emulated time aware shaper talker to the emulated time aware shaper listener via the DUT and confirming the blocking of the traffic from the at least one gate of the DUT. The method further includes transmitting, from the emulated CNC node to the DUT, a second configuration, where the second configuration opens the at least one gate of the DUT that was closed by the first configuration to allow traffic from the at least one gate. The method further includes recording a time T1 of transmission of the second configuration to the DUT. The method further includes detecting traffic from the at least one gate opened by the second configuration at the emulated time aware shaper listener. The method further includes recording a time T2 of receipt of the traffic at the emulated time aware shaper listener. The method further includes calculating a response time of the DUT to the second configuration based on T1 and T2.
According to another aspect of the subject matter described herein, the emulated time aware shaper CNC node comprises an emulated 802.1Qbv CNC node.
According to yet another aspect of the subject matter described herein, configuring the DUT with the first configuration includes transmitting the first configuration from the emulated time aware shaper CNC node to the DUT. The method of claim 1 wherein transmitting the second configuration to the DUT includes transmitting the second configuration via a Netconf, Yang, or representational state transfer (REST) interface.
According to yet another aspect of the subject matter described herein, the method for measuring the schedule update time of the time aware shaper DUT includes, prior to transmitting the second configuration, confirming the blocking of traffic from the at least one gate of the DUT.
According to yet another aspect of the subject matter described herein, the first configuration comprises a first gate control list and the second configuration comprises a second gate control list having at least one open gate that is closed in the first gate control list.
According to yet another aspect of the subject matter described herein, the response time comprise a time for the DUT to respond to a dynamic configuration change.
According to yet another aspect of the subject matter described herein, calculating the response time includes calculating response time using the following equation: response time=T2−T1−(PdelayT1+PdelayT2), where PdelayT1 is a wire delay between the emulated talker and the DUT and PdelayT2 is a wire delay between the DUT and the emulated listener.
According to yet another aspect of the subject matter described herein, the method for measuring the schedule update time of the time aware shaper DUT comprises repeatedly transmitting configuration changes to the DUT and measuring the response time of the DUT to each configuration change.
According to yet another aspect of the subject matter described herein, the emulated CNC node, the emulated talker, and the emulated listener are components of a network equipment test device and connect to the DUT through different ports of the network equipment test device.
According to yet another aspect of the subject matter described herein, the time aware shaper DUT comprises a time aware shaper bridge.
According to yet another aspect of the subject matter described herein, a system for measuring a schedule update time of a time aware shaper device under test (DUT) is provided. The system includes an emulated time aware shaper centralized network configuration (CNC) node. The system further includes an emulated time aware shaper talker, the system further includes an emulated time aware shaper listener. The system further includes a test controller. The emulated time aware shaper CNC node configures the DUT with a first configuration that blocks traffic from at least one gate of the DUT, the emulated time aware shaper talker sends traffic to the emulated time aware shaper listener via the DUT. The emulated CNC node confirms blocking of the traffic from the at least one gate of the DUT and transmits a second configuration to the DUT. The second configuration opens the at least one gate of the DUT that was closed by the first configuration to allow traffic from the at least one gate. The emulated CNC node records a time T1 of transmission of the second configuration to the DUT. The emulated time aware shaper listener detects traffic from the at least one gate opened by the second configuration and records a time T2 of receipt of the traffic at the emulated time aware shaper listener. The test controller calculates a response time of the DUT to the second configuration based on T1 and T2.
According to yet another aspect of the subject matter described herein, the emulated time aware shaper CNC node transmits the first configuration to the DUT.
According to yet another aspect of the subject matter described herein, the emulated time aware shaper CNC node transmits the second configuration to the DUT via a Netconf, Yang, or representational state transfer (REST) interface.
According to yet another aspect of the subject matter described herein, the test controller is configured to calculate the response time using the following equation: response time=T2−T1−(PdelayT1+PdelayT2), where PdelayT1 is a wire delay between the emulated talker and the DUT and PdelayT2 is a wire delay between the DUT and the emulated listener.
According to yet another aspect of the subject matter described herein, the emulated time aware shaper CNC node repeatedly transmits configuration changes to the DUT and the test controller measures the response time of the DUT to each configuration change.
According to yet another aspect of the subject matter described herein, the system comprises a network equipment test device, wherein the emulated CNC node, the emulated talker, and the emulated listener are components of the network equipment test device and connect to the DUT through different ports of the network equipment test device.
According to another aspect of the subject matter described herein, a non-transitory computer readable medium having stored thereon executable instructions that when executed by a processor of a computer control the computer to perform steps. The steps include providing an emulated time aware shaper centralized network configuration (CNC) node, an emulated time aware shaper talker, and an emulated time aware shaper listener. The steps further include configuring a time aware shaper device under test (DUT) with a first configuration that blocks traffic from at least one gate of the DUT. The steps further include transmitting traffic from the emulated time aware shaper talker to the emulated time aware shaper listener via the DUT. The steps further include confirming the blocking of the traffic from the at least one gate of the DUT. The steps further include transmitting, from the emulated CNC node to the DUT, second configuration, where the second configuration opens at least one gate of the DUT that was closed by the first configuration to allow traffic from the at least one gate. The steps further include recording a time T1 of transmission of the second configuration to the DUT. The steps further include detecting traffic from the at least one gate opened by the second configuration at the emulated time aware shaper listener. The steps further include recording a time T2 of receipt of the traffic at the emulated time aware shaper listener. The steps further include calculating a response time of the DUT to the second configuration based on T1 and T2.
The subject matter described herein can be implemented in software in combination with hardware and/or firmware. For example, the subject matter described herein can be implemented in software executed by a processor. In one exemplary implementation, the subject matter described herein can be implemented using a non-transitory computer readable medium having stored thereon computer executable instructions that when executed by the processor of a computer control the computer to perform steps. Exemplary computer readable media suitable for implementing the subject matter described herein include non-transitory computer-readable media, such as disk memory devices, chip memory devices, programmable logic devices, and application specific integrated circuits. In addition, a computer readable medium that implements the subject matter described herein may be located on a single device or computing platform or may be distributed across multiple devices or computing platforms.
Conventional Ethernet is best effort service, i.e. a packet originated from an ingress node reaches its destination in best effort manner. In “best effort manner” packet may reach the destination, there is no predictability of how much time the packet will take to reach the destination, or it may not reach at all. The reason for this unpredictability is that the outgoing queue of a switch might be full already. So additional incoming packets may get queued or may get dropped or may get delayed.
To overcome the bottleneck of conventional Ethernet, IEEE 802.1Qbv or time aware shaper is used to correctly allow frames to egress from a device which are intended for transmission at a specific point of time.
IEEE 802.1Qbv describes time gated queues at egress port of a device which open/close at specific time intervals to ensure transmission of intended frames and disallowing frames which are not slated to be transmitted.
Each gate maps uniquely to one of the queues present at the port under transmission for the device (egress port). For the sake of simplicity this document assumes that there is one to one mapping between the time-gates and the queues. Also, each queue can be assigned a unique priority similar to a VLAN priority which is used to determine which TSN streams are designated for a specific queue.
As stated above, transmission gates 120, 122, 124, 126, and 128 can be in any one of the following two states:
As described above, 802.1Qbv uses time gated queues so that predictability of traffic delivery can be assured in a time sensitive network. In an industrial Ethernet based network, deploying TSN standards is often complex. One needs to constantly monitor the dynamic changes that are happening in the network and provision the TSN aware switches accordingly so that the switches can adapt to the dynamic changes accurately. Different consortiums together have come up with an architecture (IEEE P802.1Qcc configuration models) which helps solve this dynamic configuration changes in the network more efficiently.
In summary, the architecture illustrated in
The fully centralized network model is being adopted by both automotive and industrial segments for deploying the network. Although the fully centralized network model brings in lot of flexibility in terms of dynamic configuration updates happening within a network still there are areas which needs to be thoroughly monitored for ensuring robustness of the network.
A TSN bridge needs to adhere to the time aware shaper (IEEE 802.1Qbv) implementation stringently in order to ensure that its able to maintain the timeliness of the time critical traffic within the network. In case of a fully centralized network, the dynamic schedule updates are generated by the CNC based on the network requirements as well as network load at a point of time. A TSN bridge upon receiving such dynamic configuration changes needs to update its currently running gate control list definitions and attach to the updated gate control list quickly. How fast a TSN switch adapts to the new change ensures how quickly the new changes happening in a network are followed. Any delay incurred by a bridge while adopting the new configuration update can introduce a significant anomaly in the network.
In the scenario illustrated in
One problem lies with the accuracy of bridge to update its schedule based on the dynamic update, if the bridge fails to update itself within the threshold of a predefined cycle time, the bridge can adversely affect the time sensitivity of the entire network. If the bridge is too late in updating itself, the bridge will not allow the new traffic to flow through and thereby cause urgent as well as sensitive sensor information to get delayed. The subject matter described herein provides a methodology which can be leveraged for testing and quantifying how accurate a time aware bridge can update itself to the dynamic changes taking place inside of a time sensitive network.
In the previous sections we have seen how industrial Ethernet is moving into fully centralized networking model. We have noticed the need for a time aware device to be compliant to the dynamic changes taking place in a network within a threshold parameter. In this section we would introduce the methodology for measuring the accuracy of device for remaining compliant to the update.
Test stations identified as test station 1 (TS1) 302, and test station 2 (TS2) 304. The Test Stations are connected to DUT 208 at ports identified as DUT.TS1 and DUT.TS2. TS1 302 is responsible for transmitting frames to the device under test, and TS2 304 is responsible for receiving frames from DUT.TS2. The test scenario assumes that both test stations TS1 302 and TS2 304 along with DUT 208 are synchronized with help of 802.1AS (gPTP) emulation. There is another test port connected to the management interface of DUT 208, which emulates a CNC. This test port is responsible for exchanges messages over a Netconf/Restconf channel which will configure DUT 208. All the three test ports are part of the same test system which ensures pre-existing time synchronization amongst all three of them.
In the first test step (as illustrated in
After the initialization stage illustrated in
A test controller 612 controls the overall operation of network equipment test device 600. For example, test controller 612 may allow the user to configure emulated CNC node to send the dynamic configuration updates to DUT 208, to configure emulated talker 302 to transmit traffic to emulated listener 304 via DUT 208, and to configure emulated listener 304 to listen and report the time of receipt of traffic from emulated talker 302. Test controller 612 may also calculate the response time of DUT 208 to dynamic configuration changes.
In step 702, the process includes configuring the DUT with a first configuration that blocks traffic from at least one gate of the DUT. Configuring the DUT may include transmitting the first configuration from the emulated CNC node to the DUT. For example, emulated CNC node 300 may transmit configuration commands to DUT 208 to instruct DUT 208 to block traffic on one of its egress gates (see
In step 704, the process includes sending traffic from the emulated talker to the emulated listener via the DUT. For example, network equipment test device 600 may configure emulated talker 302 to send traffic to listener 304 via DUT 208 and start sending the traffic. The traffic may be any suitable test traffic, such as application traffic that would be found in a time sensitive network. One example of such traffic would be emulated traffic that would be transmitted between industrial machines or automobiles for real time control.
In step 705, the process includes confirming the blocking of traffic by at least one gate of the DUT. For example, emulated listener 304 may report that application traffic from the gate of DUT 208 that was blocked by the first configuration data is not being received.
In step 706, the process includes transmitting, from the emulated CNC to the DUT, a second configuration, where the second configuration allows traffic from the at least one gate of the DUT that was closed by the first configuration to allow traffic from the at least one gate. For example, emulated CNC node 300 may transmit an updated configuration to DUT 208 that provides a new gate control list to open one or more gates 208 to allow previously blocked traffic from emulated talker 302 to emulated listener 304.
In step 708, the process includes recording the time T1 of transmission of the second configuration to the DUT. For example, emulated CNC node 300 may time stamp the transmission of the second configuration to DUT 208 and record the time stamp in memory 604.
In step 710, the process includes detecting traffic from the gate opened by the second configuration at the emulated listener. For example, emulated listener 304 may detect when the first packet from the previously closed gate is received from DUT 208.
In step 712, the process includes recording the time T2 of receipt of the previously blocked traffic at the emulated listener. For example, emulated listener 304 may generate a time stamp corresponding to the time of receipt of the first packet that was previously blocked from DUT 208 and record the time stamp in memory 604.
In step 714, the process includes calculating a response time of the DUT to the configuration change based on the time stamps T1 and T2. For example, test controller 612 may calculate the response time based on the difference in T1 and T2 and subtracting the wire delay using the equations described above.
In step 716, it is determined whether all gates of the DUT have been tested. If all gates of the DUT have been tested, the process ends. If all gates have not been tested, test controller 612 begins the test process for the next gate in step 718, and control returns to step 702 where steps 702-714 are repeated for the next gate.
It will be understood that various details of the subject matter described herein may be changed without departing from the scope of the subject matter described herein. Furthermore, the foregoing description is for the purpose of illustration only, and not for the purpose of limitation, as the subject matter described herein is defined by the claims as set forth hereinafter.
Number | Name | Date | Kind |
---|---|---|---|
7272750 | Sirbu | Sep 2007 | B2 |
9288777 | Hollabaugh et al. | Mar 2016 | B2 |
10425321 | Joseph et al. | Sep 2019 | B2 |
10623297 | Regev | Apr 2020 | B2 |
20020065929 | Kamentsky et al. | May 2002 | A1 |
20130138800 | Gelter | May 2013 | A1 |
20140269337 | Gintis | Sep 2014 | A1 |
20140304505 | Dawson | Oct 2014 | A1 |
20140321285 | Chew et al. | Oct 2014 | A1 |
20150103828 | Chandhoke et al. | Apr 2015 | A1 |
20150103832 | Chandhoke et al. | Apr 2015 | A1 |
20150103836 | Chandhoke et al. | Apr 2015 | A1 |
20150103848 | Chandhoke et al. | Apr 2015 | A1 |
20150245306 | Boehlke | Aug 2015 | A1 |
20150365255 | Gunther | Dec 2015 | A1 |
20170331748 | Mangin | Nov 2017 | A1 |
20180070373 | Muench | Mar 2018 | A1 |
20180160424 | Cavalcanti et al. | Jun 2018 | A1 |
20180184438 | Cavalcanti et al. | Jun 2018 | A1 |
20180191642 | Biederman et al. | Jul 2018 | A1 |
20180227067 | Hu et al. | Aug 2018 | A1 |
20180237039 | Mong et al. | Aug 2018 | A1 |
20180302330 | Bush | Oct 2018 | A1 |
20180302331 | Bush | Oct 2018 | A1 |
20180309655 | Joseph et al. | Oct 2018 | A1 |
20180309656 | Regev | Oct 2018 | A1 |
20180316592 | Ellegaard | Nov 2018 | A1 |
20190386924 | Srinivasan et al. | Dec 2019 | A1 |
20200067792 | Aktas et al. | Feb 2020 | A1 |
20200120029 | Sankaran et al. | Apr 2020 | A1 |
20200304429 | Marce | Sep 2020 | A1 |
20200326971 | Yang | Oct 2020 | A1 |
20200366588 | Bergeron | Nov 2020 | A1 |
20200366608 | Pan et al. | Nov 2020 | A1 |
20210072373 | Schoenberg | Mar 2021 | A1 |
20210112002 | Pan et al. | Apr 2021 | A1 |
Number | Date | Country |
---|---|---|
101631080 | Jan 2010 | CN |
101854268 | Oct 2010 | CN |
108737003 | Nov 2018 | CN |
10 2018 109 689 | Oct 2018 | DE |
3396910 | Oct 2018 | EP |
2017082779 | May 2017 | WO |
Entry |
---|
Jiang, Junhui, Li, Yuting, Hong, Seung Ho, Xu, Aidong, Wei, Min, “A Simulation Model for Time-sensitive Networking (TSN) with Experimental Validation” Oct. 17, 2019, https://ieeexplore.ieee.org/document/8869206 (Year: 2019). |
Hackel, Timo, Meyer, Philipp, Korf, Franz, Schmidt, Thomas C., “SDN4CoRE: A Simulation Model for Software-Defined Networking for Communication over Real-Time Ethernet”, Aug. 26, 2019, https://arxiv.org/abs/1908.09649 (Year: 2019). |
Heise, Peter, Geyer, Fabien, Obermaisser, Roman, “TSimNet: An industrial Time Sensitive Networking simulation framework based on OMNeT++” Dec. 22, 2016, https://ieeexplore.ieee.org/document/7792488 (Year: 2016). |
Oliver, Ramon Serna, Craciunnas, Silviu S., Steiner Wilfried, “IEEE 802.1Qbv Gate Control List Synthesis using Array Theory Encoding” Aug. 9, 2018, https://ieeexplore.ieee.org/document/8430062 (Year: 2018). |
Nasrallah, Ahmed, Balasubramanian, Vekatraman, Thyagaturu, Akhilesh, Reisslein, Martin, ElBakoury, Hesham, “Reconfiguration Algorithms for High Precision Communications in Time Sensitive Networks: Time-Aware Shaper Configuration with IEEE”, Jun. 27, 2019, https://ieeexplore.ieee.org/document/9024705 (Year: 2019). |
Ansah, Frimpong, Abid, Mohamed Amine, de Meer, Hermann, “Schedulability Analysis and GCL Computation for Time-Sensitive Networks” Jan. 30, 2020, https://ieeexplore.ieee.org/document/8971965 (Year: 2020). |
Cisco, “Time-Sensitive Networking: A Technical Introduction White Paper” 2017, https://www.cisco.com/c/dam/en/us/solutions/collateral/industry-solutions/white-paper-c11-738950.pdf (Year: 2020). |
Notice of Allowance and Fee(s) Due for U.S. Appl. No. 15/647,207 (dated Jan. 15, 2020). |
Advisory Action and AFCP 2.0 Decision for U.S. Appl. No. 15/647,207 (dated Nov. 27, 2019). |
Final Office Action for U.S. Appl. No. 15/647,207 (dated Sep. 12, 2019). |
Non-Final Office Action for U.S. Appl. No. 15/647,207 (dated May 15, 2019). |
Notice of Allowance and Fee(s) Due for U.S. Appl. No. 15/498,452 (dated May 10, 2019). |
Alhady et al., “Time-aware Traffic Shaper using Time-based Packet Scheduling in Intel I210,” International Journal of Research and Engineering, vol. 5, No. 9, pp. 494-499 (Sep.-Oct. 2018). |
Non-Final Office Action for U.S. Appl. No. 15/498,452 (dated Nov. 5, 2018). |
“Time-Sensitive Networking,” Wikipedia, the free encyclopedia, https://en.wikipedia.org/wiki/Time-Sensitive_Networking, pp. 1-7 (Aug. 29, 2017). |
Teener, “IEEE 802 Time-Sensitive Networking: Extending Beyond AVB,” Accessed on wayback machine, https://web.archive.org/web/20140724130142/http://standards.ieee.org/events/automotive/08_Teener_TSN.pdf, pp. 1-30 (Jul. 24, 2017). |
“IEEE Standard for Local and metropolitan area networks—Bridges and Bridged Networks—Amendment 29: Cyclic Queuing and Forwarding,” IEEE Computer Society, IEEE Std 802.1QchTM—2017, pp. 1-28 (May 18, 2017). |
“IEEE Standard for Local and metropolitan area networks—Bridges and Bridged Networks—Amendment 26: Frame Preemption,” IEEE Computer Society, IEEE Std 802.1QbuTM—2016, pp. 1-51 (Jun. 30, 2016). |
“IEEE Standard for Local and metropolitan area networks—Bridges and Bridged Networks—Amendment 25: Enhancements for Scheduled Traffic,” IEEE Computer Society, IEEE Std 802.1QbvTM—2015, pp. 1-55 (Dec. 5, 2015). |
Wisniewski et al., “Seamless Schedule Switching in Time Triggered Ethernet based Communication Systems,” ResearchGate, pp. 1-11 (Nov. 2015). |
“IEEE Standard for Local and metropolitan area networks—Bridges and Bridged Networks—Amendment 24: Path Control and Reservation,” IEEE Computer Society, IEEE Std 802.1QcaTM—2015, pp. 1-105 (Sep. 3, 2015). |
Thangamuthu et al., “Analysis of Ethernet—Switch Traffic Shapers for In-Vehicle Networking Applications,” Design, Automation & Test in Europe Conference & Exhibition, pp. 1-6 (2015). |
Thangamuthu, S., Analysis of Automotive Traffic Shapers in Ethernet In-Vehicular Networks, Eindhoven University of Technology, pp. 1-89 (2014). |
Boiger, “Deterministic Ethernet—IEEE 802.1 standards for real-time process control, industrial automation, and vehicular networks,” IEEE 802 Tutorial, pp. 1-72 (Nov. 12, 2012). |
Boiger, “Time Aware Shaper,” Deggendorf University of Applied Sciences, IEEE 802.1 Pleanry, pp. 1-12 (Sep. 2012). |
Boiger, Christain, “Time Aware Shaper,” IEEE 802 Plenary, Deggendorf University of Applied Sciences, pp. 1-9 (May 2012). |
Spirent, “Precision Time Protocol (PTP) IEEE 1588,” YouTube “alantalkstech”, https://www.youtube.com/watch?v=yw-gd01aOYg, pp. 1-11 (Dec. 7, 2011). |
“IEEE Standard for Local and metropolitan area networks—Timing and Synchronization for Time-Sensitive Applications in Bridged Local Area Networks,” IEEE Computer Society, IEEE Std 802.1ASTM—2011, pp. 1-274 (Mar. 30, 2011). |
Eidson, “IEEE-1588 Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems—A Tutorial,” Agilent Technologies, pp. 1-94 (Oct. 2005). |
Notice of Publication for Great Britain Application No. GB2100371.0 (dated Sep. 20, 2021). |
First Office Action for Great Britain Application No. GB2100371.0 (dated Aug. 6, 2021). |
Notice of Allowance and Fee(s) Due for Chinese Patent Application No. 201810373215.5 (dated Aug. 18, 2021). |
Non-Final Office Action for U.S. Appl. No. 17/001,614 (dated Apr. 23, 2021). |
First Office Action for Chinese Patent Application No. 201810373217.5 (dated Feb. 2, 2021). |
Advisory Action for U.S. Appl. No. 16/258,485 (dated Nov. 19, 2020). |
Parry et al., “A Network Forensics Tool for Precise Data Packet Capture and Replay in Cyber-Physical Systems,” ACSW '16 Multiconference, pp. 1-10 (Feb. 2-5, 2016). |
Number | Date | Country | |
---|---|---|---|
20210226843 A1 | Jul 2021 | US |