The Precision Time Protocol (PTP) is a network protocol used to exchange timing information among nodes (network devices such as switches, routers, host computers, machinery, etc.) in the network to achieve microsecond to nanosecond accuracy across the network. Nodes that participate in PTP are referred to as “clocks” and can have one or more ports that are enabled for PTP. Each PTP-enabled port on a clock can serve as master port and/or a slave port. PTP defines several kinds of messages, including messages that establish a timing hierarchy among the clocks and their ports and messages that perform synchronization between clocks. PTP includes a type of message called a management message that supports the PTP network by managing the passing of PTP information between clocks, setting different PTP configurations in the network, and generating events.
With respect to the discussion to follow and in particular to the drawings, it is stressed that the particulars shown represent examples for purposes of illustrative discussion and are presented in the cause of providing a description of principles and conceptual aspects of the present disclosure. In this regard, no attempt is made to show implementation details beyond what is needed for a fundamental understanding of the present disclosure. The discussion to follow, in conjunction with the drawings, makes apparent to those of skill in the art how embodiments in accordance with the present disclosure may be practiced. Similar or same reference numbers may be used to identify or otherwise refer to similar or same elements in the various drawings and supporting descriptions. In the accompanying drawings:
A PTP network can configure its clock and ports into a timing tree by passing Announce message among the ports, causing port states to change to a MASTER, SLAVE, or PASSIVE state in accordance with a Best Master Clock Algorithm (BMCA). Nodes in the PTP network pass management messages to access PTP information and set configurations. PTP restricts the forwarding of management messages on ports that are in certain port states so as to prevent cycling of management messages in the network. The restriction is effective when the port states are in steady state.
However, changes in the network such as link failures or switch failures can temporarily cause the timing tree to be unstable, resulting in ports changing their port state in order to reform the timing tree. At this time, the timing tree may no longer be a tree and timing loops may temporarily arise due to the formation of redundant links as port states attempt to reconverge. During this time, management messages may cycle in a loop continuously creating more and more management messages, eventually causing a PTP network outage.
Embodiments in accordance with the present disclosure employ loop detection table that allows for detecting loop backed messages, which are management messages that had been forwarded by a clock that have looped back to that clock. When the clock receives a message that has looped back, the clock can drop the message thus breaking the loop and preventing PTP network outage.
In the following description, for purposes of explanation, numerous examples and specific details are set forth in order to provide a thorough understanding of the present disclosure. It will be evident, however, to one skilled in the art that the present disclosure as expressed in the claims may include some or all of the features in these examples, alone or in combination with other features described below, and may further include modifications and equivalents of the features and concepts described herein.
System 100 can be configured with one or more logical networks 106. In some embodiments, for example, the logical networks can be virtual local area networks (VLANs). Merely to illustrate,
Network devices 104 and computing devices 108 can be configured to synchronize their respective clocks. In some embodiments, for example, clock synchronization in accordance with the present disclosure can be based on the Precision Time Protocol (PTP) standard defined by the IEEE 1588-2008 specification, which is incorporated herein by reference for all purposes and referred to herein variously as the PTP standard, PTP protocols, and so on. Accordingly, and merely to provide a vehicle for disclosing aspects of the present disclosure, embodiments in accordance with the present disclosure will be explained in the context of PTP processing and PTP terminology as defined in the PTP standard. Persons of ordinary skill, however, will understand that the present disclosure can be adapted to other clock synchronization methodologies. In keeping with PTP terminology, devices 104, 108 can also be referred to the following descriptions as “clocks.”
A device 104, 108 can be referred to as an “ordinary clock” when the device has only one port (interface) that is configured (enabled) for processing in accordance with the PTP standard; e.g., performing clock synchronization operations. For instance, computing devices 108 communicate over a single port and so can be referred to as ordinary clocks. A device can be referred to as an “boundary clock” when the device has multiple ports that are PTP-enabled. For example, network devices 104 such as switches and routers that are configured for PTP can be referred to as boundary clocks.
The PTP standard specifies how the clocks in system 100 synchronize with each other. As explained below, the clocks are organized into a master/slave synchronization hierarchy (timing tree) with a grandmaster clock at the top of the hierarchy that determines the reference time for the entire system. The PTP standard defines various types of messages. For example, the Announce message is used to establish the master/slave timing tree. Synchronization messages (e.g., Sync, Delay_Req, Follow_Up, and Delay_Resp) are used to communicate timing information needed to synchronize the clocks. Management messages are used to query and update clock data (data sets) maintained by the clocks. Management messages are also used to customize a PTP system and for initialization and fault management.
Referring to
The PTP standard defines a hop counter (boundaryHops) in the management message that is decremented each time the message is forwarded. Upon reaching zero, the management message is dropped, thus limiting the number of hops a message can take. However, a network fault can result in many different loops involving many different ports in the network, resulting in the looping of many management messages through the network. The continued injection of management messages into the network can perpetuate the condition despite the hop counters; older messages will get dropped but not before new management messages are introduced. The resulting storm of management messages looping through the network can load the processing at each node to the point of impeding attempts at convergence to a new timing tree, resulting in PTP network outage.
The discussion will now turn to embodiments for addressing the formation of loops in a timing tree in accordance with the present disclosure.
Referring to
At operation 402, the clock can receive a management message (“message”) on one of its ports for processing, and in particular to determine whether or not to forward the received message. The message format shown in
At operation 404, the clock can determine whether or not it has previously received a message from the same original sender of the currently received message. In some embodiments, for example, the clock can maintain a loop detection table to keep track of senders of previously received messages. Referring for a moment to
Entry not Found in Loop Detection Table
If an entry is not found in the loop detection table 502, this can indicate that the sender of the currently received message has not sent any previous messages. Accordingly, at operation 406, the clock can create a new entry in the loop detection table 502. In some embodiments, for example, the sourcePortIdentity information 304 contained in the received message can be copied into the sender ID field 504 of the new entry. The absolute value of the difference between the startingBoundaryHops information 308 and the BoundaryHops information 310 in the received message can be stored in the hop difference field 506 of the new entry. The aging field 508 is discussed below. This newly created entry 510 will represent the currently received management message as a previously received message that was sent by the sender of the currently received message. Processing can continue at operation 414 to forward the received message.
Entry Found in Loop Detection Table
If an entry is found in the loop detection table 502, this can indicate that the sender of the currently received message has sent a previous message. Accordingly, at operation 408, in response to a determination that an entry 510 in the loop detection table 502 has been found, the clock can assess whether or not the received message is a new management message from the sender or a looped message from a management message previously sent by the same sender. In some embodiments, for example, the assessment can include using the entry 510 found in the loop detection table 502 at operation 404 to determine whether or not the boundary hop difference in the received message is the same as or smaller than before. As noted above, the entry 510 represents a message previously sent by the sender of the currently received message. In some embodiments, the clock can compute the hop difference between startingBoundaryHops 308 and BoundaryHops 310 contained in the received message. If this computed hop difference is less than or equal to the hop difference field 506 in the entry 510, then processing can continue at operation 410. If the computed hop difference is greater than the hop difference field 506 in the entry 510, then processing can continue at operation 412.
At block 410, in response to a determination that the computed hop difference of the received message is less than or equal to the hop difference field 506, the clock can reset the hop difference field 506 with the computed hop difference. For example, the computed hop difference can be equal to what is already stored in the hop difference field 506 if the received message is simply a new management message from the sender. The computed hop difference can be smaller than before if the sender sends a new management message after one or more boundary clocks between the sender and the receiver have been removed. Examples of these situations are illustrated below. In either case, the received message can be deemed to be a newly sent management message and not a looped back message. Processing can continue at operation 414 to forward the received message.
At operation 412, in response to a determination that the computed hop difference of the received message is greater than the hop difference field 506, the clock can drop the received message. For instance, the computed hop difference can be greater than before if the sender sends a new management message after one or more boundary clocks between the sender and the receiver have been added, in which case the received message is not a looped message. On the other hand, the computed hop difference can also be greater than before in the case of a looped message. Examples of these situations are illustrated below. Determining whether the message is looped or not can be involve additional processing by the clock that may impact the clock's performance. Accordingly, in some embodiments, the clock can drop the received message, irrespective of whether the computed hop difference being greater than the hop difference field 506 is due to a newly sent message or a looped back message. Processing of the received message can be deemed complete.
At operation 414, in response to a determination that the computed hop difference is less than or equal to the hop difference field 506, the clock can forward the received message. In accordance with some embodiments, the clock can decrement the BoundaryHops data field 310 in the received message by one to reduce the remaining number of retransmissions permitted for the message. The clock can forward the receive message, with the updated BoundaryHops data field 310 down the timing tree 200 (
Returning to the loop detection table 502 shown in
Entries 510 are aged out when the interval between management messages exceeds the predetermined aging time. The aging process provides a self-correcting mechanism for dropping non-looped management messages, which can happen as explained above. Recall from operation 412 that, in some embodiments, a new management message sent after a boundary clock has been inserted between the sender and the receiver will be dropped; this message is not a looped back message and should be forwarded instead of being dropped. The aging process can correct for this situation. Over time, the table entry will be aged out and subsequent management messages from the sender will resume being forwarded. The setting of the predetermined aging time will affect how long it takes before the clock can start forwarding management messages from the sender.
Bus subsystem 604 can provide a mechanism for letting the various components and subsystems of computer system 600 communicate with each other as intended. Although bus subsystem 604 is shown schematically as a single bus, alternative embodiments of the bus subsystem can utilize multiple busses.
Network interface subsystem 616 can serve as an interface for communicating data between computer system 600 and other computer systems or networks. Embodiments of network interface subsystem 616 can include, e.g., an Ethernet card, a Wi-Fi and/or cellular adapter, a modem (telephone, satellite, cable, ISDN, etc.), digital subscriber line (DSL) units, and/or the like.
User interface input devices 612 can include a keyboard, pointing devices (e.g., mouse, trackball, touchpad, etc.), a touch-screen incorporated into a display, audio input devices (e.g., voice recognition systems, microphones, etc.) and other types of input devices. In general, use of the term “input device” is intended to include all possible types of devices and mechanisms for inputting information into computer system 600.
User interface output devices 614 can include a display subsystem, a printer, or non-visual displays such as audio output devices, etc. The display subsystem can be, e.g., a flat-panel device such as a liquid crystal display (LCD) or organic light-emitting diode (OLED) display. In general, use of the term “output device” is intended to include all possible types of devices and mechanisms for outputting information from computer system 600.
Memory subsystem 606 includes memory subsystem 608 and file/disk storage subsystem 610 represent non-transitory computer-readable storage media that can store program code and/or data, which when executed by processor 602, can cause processor 602 to perform operations in accordance with embodiments of the present disclosure.
Memory subsystem 608 includes a number of memories including main random access memory (RAM) 618 for storage of instructions and data during program execution and read-only memory (ROM) 620 in which fixed instructions are stored. File storage subsystem 610 can provide persistent (i.e., non-volatile) storage for program and data files, and can include a magnetic or solid-state hard disk drive, an optical drive along with associated removable media (e.g., CD-ROM, DVD, Blu-Ray, etc.), a removable flash memory-based drive or card, and/or other types of storage media known in the art.
It should be appreciated that computer system 600 is illustrative and many other configurations having more or fewer components than system 600 are possible.
The discussion will now turn to various examples to illustrate aspects of the present disclosure discussed in the processing shown in
Time 1 At time 1, the GM makes an initial transmission of a management message M to BC1. The figure shows information contained in the message; for example, the sender of the message is GM which is identified as 0xabc (sender ID). The sender ID in the received message is preserved when the message is retransmitted from one clock to the next, thus preserving the identity of the original sender of the management message, in this case GM 702. The starting hops value is set to 30, although any suitable value can be used; this value remains constant during retransmissions. The boundary hops value is initially set to 30 and is decremented with each retransmission.
Time 2 BC1 maintains a loop detection table 712 (LDT1), and for discussion purposes suppose there is no entry in LDT1 that identifies GM as the sender. Accordingly, at time 2, when BC1 receives message M, BC1 can create/add an entry 712a for the received message and store 0xabc in the sender ID field of entry 712a to identify GM as the sender of the message. BC1 can compute the difference between the starting hops value and the boundary hops value, i.e., 30−30=0, and store that value in the hop diff field of the created entry 712a (reference operation 406 in
Time 3 At time 3, BC1 can retransmit message M to BC2 (reference operations 406, 414 in
Time 4 BC2, likewise, maintains its own loop detection table 714 (LDT2), and likewise we will suppose for discussion purposes there is no entry in LDT2 that identifies GM as the sender. Accordingly, at time 4, when BC2 receives message M, BC2 can create/add an entry 714a for the received message and store 0xabc in the sender ID field of entry 714a to identify GM as the original sender of the message. BC2 can compute the difference between the starting hops value and the boundary hops value, i.e., 30−29=1, and store that value in the hop diff field of the created entry 714a.
Time 5 At time 5, BC1 can retransmit message M to one or more downstream clocks (not shown) in the PTP network. Retransmitting message M includes decrementing the boundary hops value by one.
Time 6 At time 6, the presumed configuration failure in the PTP network loops message M back to BC2. The message will contain a boundary hops value that is less than (or equal) to 28. The boundary hops value can remain at 28 if the message M was looped back to BC2 by a transparent clock or by a device that was not PTP-enabled. The boundary hops value will be less than 28 if message M was retransmitted by one or more boundary clocks before being looped back to BC2.
Time 7 At time 7, BC2 will determine that an entry 714a in loop detection table LDT2 already exists for the original sender of the received message at time 6, namely 0xabc. BC2 will compute the hop difference in the received message (reference operation 408 in
Although not shown in
Hop Difference
It is noted that the hop difference for a management message computed in the foregoing embodiments can be viewed as the number of hops the message had taken since the message was originally sent. Referring to
Accordingly, in some embodiments, the hop difference can refer to the number of hops that a received message had taken since the message was originally sent by the time it reached a clock, where the original sending of the message does not count as a hop. In some embodiments, the operations described in connection with
In accordance with some embodiments, a method can include receiving a message comprising a sender identifier (ID), a starting hop value, and a remaining hop value. The received message can be forwarded when the sender ID in the received message and the sender ID in a previously received message identify the same sender and a difference computed from the starting hop value and the remaining hop value in the received message is less than or equal to a difference computed from the starting hop value and the remaining hop value in the previously received message. The received message can be dropped when the sender ID in the received message and the sender ID in the previously received message identify the same sender and the difference between the starting hop value and the remaining hop value in the received message is greater than the difference between the starting hop value and the remaining hop value in the previously received message.
In some embodiments, the method can further include forwarding the received message when the sender of the received message has not sent any previously received message.
In some embodiments, the method further includes using a data table of senders of previously received messages to determine whether to forward or drop the received message, the data table comprising sender IDs in previously received messages and differences computed from starting hop values and the remaining hop values in the previously received messages. The method can further include creating an entry in the data table when the sender ID in the received message does not occur in the data table and storing the sender ID in the received message and the difference between the starting hop value and the remaining hop value in the received message in the created entry. The method can further include deleting a given entry from the data table when an age of the given entry exceeds a predetermined age.
In some embodiments, forwarding the received message can include decrementing the remaining hop value in the received message.
In some embodiments, the received message can be a management message in accordance with Precision Time Protocol.
In accordance with some embodiments, an apparatus can include one or more computer processors and a computer-readable storage medium comprising instructions for controlling the one or more computer processors to receive a message comprising a sender identifier (ID) and hop information indicative of a number of hops taken by the received message; forward the received message when the sender ID in the received message and the sender ID in a previously received message identify the same sender and the number of hops taken by the received message is less than or equal to a number of hops taken by the previously received message; and drop the received message when the sender ID in the received message and the sender ID in a previously received message identify the same sender and the number of hops taken by the received message is greater than the number of hops taken by the previously received message.
In some embodiments, the received message can further comprise a starting hop value and the remaining hop value, wherein the number of hops taken by the received message is a difference between the starting hop value and the remaining hop value in the received message, wherein the number of hops taken by the previous message is a difference between a starting hop value and a remaining hop value in the previous message.
In some embodiments, the computer-readable storage medium can further comprise instructions for controlling the one or more computer processors to forward the received message when the sender of the received message has not sent any previously received message.
In some embodiments, the computer-readable storage medium can further comprise instructions for controlling the one or more computer processors to use a data table of senders of previously received messages to determine whether to forward or drop the received message, the data table comprising sender IDs in previously received messages and information indicative of how many hops the previously received messages had taken. In some embodiments, the computer-readable storage medium can further comprise instructions for controlling the one or more computer processors to create an entry in the data table when the sender ID in the received message does is not found in the data table and storing the sender ID in the received message and the difference between the starting hop value and the remaining hop value in the received message in the created entry. In some embodiments, the computer-readable storage medium can further comprise instructions for controlling the one or more computer processors to delete a given entry from the data table when an age of the given entry exceeds a predetermined age.
In some embodiments, forwarding the received message can include decrementing the remaining hop value in the received message.
In accordance with some embodiments, an apparatus can include one or more computer processors and a computer-readable storage medium comprising instructions for controlling the one or more computer processors to receive a message comprising information that identifies a sender of the received message; forward the received message to a downstream apparatus when the received message is not a previously received message sent by the sender that has looped back to the apparatus; and drop the received message when the received message is a previously received message sent by the sender that has looped back to the apparatus.
In some embodiments, the received message can further comprise a starting hop value and a remaining hop value, wherein the previously received message is deemed not to have looped back to the apparatus when a difference computed from the starting hop value and the remaining hop value in the received message is less than or equal to a difference computed from the starting hop value and the remaining hop value in the previously sent message.
In some embodiments, wherein the received message can further comprise a starting hop value and a remaining hop value, wherein the previously received message is deemed to have looped back to the apparatus when a difference computed from the starting hop value and the remaining hop value in the received message is greater than a difference computed from the starting hop value and the remaining hop value in the previously sent message.
In some embodiments, the computer-readable storage medium can further comprise instructions for controlling the one or more computer processors to use a data table of senders of previously received messages to determine whether to forward or drop the received message, the data table comprising sender identifiers (IDs) that identify senders of previously received messages and information indicative of how many hops the previously received messages had taken. In some embodiments, the computer-readable storage medium can further comprise instructions for controlling the one or more computer processors to create an entry in the data table when a sender ID in the received message is not found in the data table and storing the sender ID in the received message and the difference between the starting hop value and the remaining hop value in the received message in the created entry. In some embodiments, the computer-readable storage medium can further comprise instructions for controlling the one or more computer processors to delete a given entry from the data table when an age of the given entry exceeds a predetermined age.
The above description illustrates various embodiments of the present disclosure along with examples of how aspects of the particular embodiments may be implemented. The above examples should not be deemed to be the only embodiments, and are presented to illustrate the flexibility and advantages of the particular embodiments as defined by the following claims. Based on the above disclosure and the following claims, other arrangements, embodiments, implementations and equivalents may be employed without departing from the scope of the present disclosure as defined by the claims.
Number | Name | Date | Kind |
---|---|---|---|
9155062 | Traore | Oct 2015 | B2 |
9819541 | Garg | Nov 2017 | B2 |
10432336 | Rabinovich | Oct 2019 | B2 |
20130227172 | Zheng | Aug 2013 | A1 |
20150318941 | Zheng | Nov 2015 | A1 |
20160149692 | Kim | May 2016 | A1 |
Entry |
---|
Garner, Specialization of IEEE 1588 Best Master Clock Algorithm to 802.1AS, SAMSUNG Electronics, IEEE 802.1 AVB 2006/IEEE 1588, 37 pages, Jun. 6, 2006. |
Broman et al., Endlessly Circulating Messages in IEEE 1588-2008 Systems, downloadable at http://www.eecs.berkeley.edu/Pubs/TechRpts/2014/EECS-2014-62.html, 9 pages, May 13, 2014. |
Silicon Labs, AN 1202: ITU-T Precision Time Protocol Grandmaster, Boundary Clock, and Slave Clock Mode, downloadable at silabs.com, pp. 1-21, Mar. 2019. |
IEEE Std 1588™-2019, IEEE Standard fora Precision Clock Synchronization Protocol for Networked Measurement and Control Systems, IEEE, 499 pages, Nov. 7, 2019. |
Number | Date | Country | |
---|---|---|---|
20210152267 A1 | May 2021 | US |