Patent Application
20240195730
The present disclosure relates to Time Sensitive Networking (TSN) and Deterministic Networking (DetNet) and, more specifically, to frame or packet replication and elimination in a TSN or DetNet network.
Time-Sensitive Networking (TSN) is currently being developed at the Institute for Electronics and Electrical Engineering (IEEE) as a new technology that enhances IEEE 802.1 and IEEE 802.3 Ethernet standards to an entirely new level of determinism. It can be seen as an evolution of Ethernet to guarantee low end-to-end latency, low jitter, and low packet loss.
The TSN Task Group (TG) within the IEEE 802.1 Working Group (WG) deals with deterministic services through IEEE 802 networks. The TSN TG specifies the tools of the TSN toolbox, as well as the use of the tools for a particular purpose. TSN TG is chartered to provide deterministic services through IEEE 802 networks with:
In order to achieve extremely low packet loss, the TSN TG specified Frame Replication and Elimination for Reliability (FRER) (802.1CB), which is targeted to avoid frame loss due to equipment failure. It is practically a per-frame 1+1 (or 1+n) redundancy function. There is no failure detection/switchover incorporated. FRER sends frames on two (or more) maximally disjoint network paths, then combines the streams and deletes extra frames.
Note that the same functions are defined for Deterministic Networking (DetNet) networks as Packet Replication and Elimination Functions (PREFs) in order to simplify implementation and allow use of the same concept in both Layer2 (TSN) and Layer3 (DetNet) networks, such as described in Internet Engineering Task Force (IETF) RFC8655 (October 2019). In the description provided herein, the focus is on FRER although the embodiments can be applied to PREF.
Some aspects of the present disclosure are based on IEEE 802.1CB, so various terminology and variable names described in IEEE 802.1CB are used here where appropriate, denoted as “VariableName”. In contrast, new variables, functions, and parameters follow IEEE 802.1CB naming convention and are denoted as “NewEntityName”. Note as per IEEE 802.1CB: “ . . . this standard defines Frame Replication and Elimination for Reliability (FRER), which divides a Stream into one or more linked Member Streams, thus making the original Stream a Compound Stream. It replicates the packets of the Stream, splitting the copies into the multiple Member Streams, and then rejoins those Member Streams at one or more other points, eliminates the replicates, and delivers the reconstituted Stream from those points.”
An Elimination function evaluates the “sequence_number” sub-parameter of a packet of one or more Member Streams passed up from the lower layers, in order to discard duplicated packets. The “SequenceHistory” variable maintains a history of the “sequence_number” sub-parameters of recently received packets. During duplicate elimination, “sequence_number” is checked against a history window (defined by “frerSeqRcvyHistoryLength”). Packets being outside the history window are discarded as invalid. Under normal operation, received packets are within the history window and only duplicates are dropped.
IEEE 802.1CB defines a timeout mechanism for the Elimination function in order to cope with some networking scenarios that results in unnecessarily dropped frames (e.g., if Elimination function somehow gets out of step with its corresponding Sequence generation function; if a Sequence generation function is reset; etc.). If a timeout occurs, the history is reset, and it is allowed to accept the next packet by the recovery algorithm, no matter what the value of its “sequence_number” sub-parameter (see “TakeAny” in Section 7.4.3.2.6 in IEEE 802.1CB).
Some embodiments of the present disclosure are directed to a method for packet or frame replication and elimination in a network. The method includes obtaining by a gate controller downstream information indicating which application instance in an application cluster is downstream active needing to send a stream of packets or frames in a downstream direction through the network. The method further includes, based on the downstream information, controlling by the gate controller a gate within a gate cluster that is associated with the downstream active application instance to be open state which operates to forward the stream of packets or frames in the downstream direction from the downstream active application instance to an associated packet or frame replication entity for replication and transmission via at least two disjoint paths through the network.
Some other embodiments of the present disclosure are directed to a network for packet or frame replication and elimination. The network includes a gate controller adapted to obtain downstream information indicating which application instance in an application cluster is downstream active needing to send a stream of packets or frames in a downstream direction through the network. The gate controller is further adapted to, based on the downstream information, control a gate within a gate cluster that is associated with the downstream active application instance to be open state which operates to forward the stream of packets or frames in the downstream direction from the downstream active application instance to an associated packet or frame replication entity for replication and transmission via at least two disjoint paths through the network.
Potential advantages that may be provided by one or more of these embodiments and further embodiments disclosed herein can include any one or more of the following:
Other networks and methods according to embodiments will be or become apparent to one with skill in the art upon review of the following drawings and detailed description. It is intended that all such networks and methods be included within this description, be within the scope of the present disclosure, and be protected by the accompanying claims. Moreover, it is intended that all embodiments disclosed herein can be implemented separately or combined in any way and/or combination.
The accompanying drawing figures incorporated in and forming a part of this specification illustrate several aspects of the disclosure, and together with the description serve to explain the principles of the disclosure.
Some of the embodiments contemplated herein will now be described more fully with reference to the accompanying drawings. Other embodiments, however, are contained within the scope of the subject matter disclosed herein, the disclosed subject matter should not be construed as limited to only the embodiments set forth herein; rather, these embodiments are provided by way of example to convey the scope of the subject matter to those skilled in the art.
The embodiments set forth below represent information to enable those skilled in the art to practice the embodiments and illustrate the best mode of practicing the embodiments. Upon reading the following description in light of the accompanying drawing figures, those skilled in the art will understand the concepts of the disclosure and will recognize applications of these concepts not particularly addressed herein. It should be understood that these concepts and applications fall within the scope of the disclosure.
Generally, all terms used herein are to be interpreted according to their ordinary meaning in the relevant technical field, unless a different meaning is clearly given and/or is implied from the context in which it is used. All references to a/an/the element, apparatus, component, means, step, etc. are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, step, etc., unless explicitly stated otherwise. The steps of any methods disclosed herein do not have to be performed in the exact order disclosed, unless a step is explicitly described as following or preceding another step and/or where it is implicit that a step must follow or precede another step. Any feature of any of the embodiments disclosed herein may be applied to any other embodiment, wherever appropriate. Likewise, any advantage of any of the embodiments may apply to any other embodiments, and vice versa. Other objectives, features, and advantages of the enclosed embodiments will be apparent from the following description.
TSN Node: As used herein, a Time Sensitive Networking (TSN) node is any network node in a TSN network. Examples of a TSN node include a TSN endpoint and a TSN bridge.
There currently exist certain challenge(s) with respect to TSN and Deterministic Networking (DetNet). The mechanisms available for reset of seamless redundancy functions include too many unnecessary packet drops and are not cloud ready. As the ultimate goal of seamless redundancy is to avoid packet loss as much as possible. The unnecessary packet drops due to the operations of a seamless redundancy mechanism should be minimized. Furthermore, moving seamless redundancy (e.g., Institute for Electronics and Electrical Engineering (IEEE)802.1CB) components to a cloud environment creates challenges regarding availability, so seamless redundancy functions are reset much more frequently than in current industrial hardware environments. Therefore, seamless detection and adaptation to scenarios caused by reset of seamless redundancy (e.g., IEEE 802.1CB) functions are essential.
In addition, the above-described history window and timeout mechanism require good design of related parameters. However, these are not trivial tasks as contradicting requirements must be fulfilled. During the history window design, one may intend to select a small window size, for example, in order to protect the Elimination node resources or to protect against bogus packets (security). In contrast, large window size values are more tolerant to network failures and errors. It may be a hard task to find an optimum window size. Similarly, designing a too low timeout parameter value may cause frequent (and unnecessary) reset of the Elimination function. On the other hand, a too large timeout parameter value slows down the recovery after failure scenarios and causes unwanted networking transient. Furthermore, using Frame Replication and Elimination for Reliability (FRER) for bursty (non-Constant Bit Rate (CBR)) streams makes the above design more challenging or even impossible to find a right balance.
In PCT Patent Application International Publication No. WO2021005397A1 (hereinafter referred to as “the '397 Application”), an explicit notification solution is described that is based on a new flag included in the R-TAG, namely the “SeqResetFlag”. This flag is set by the Replication function when the sequence generation function was reset, so such events can be recognized easily by the Elimination functions. With this solution, the focus is on the scenario where frames might be dropped unnecessarily (with a high probability) when the sequence generation function was reset. This solution provides a much better solution than IEEE 802.1CB-2017 when the sequence generation function was reset, but it cannot provide fully lossless recovery in all cases. It may result in some dropped packets after the reset event.
Certain aspects of the present disclosure and their embodiments may provide solutions to the aforementioned or other challenges. Systems and methods are described herein that provide seamless recovery for sequence number based seamless redundancy mechanisms after a sequence number generation reset event.
Embodiments of the solution proposed herein target solving the scenario where frames are dropped unnecessarily by IEEE 802.1CB-2017 functions due to the reset of the sequence generation function.
Aspects of the proposed solution include:
Note that the existing cyclic sequence number space of IEEE 802.1CB-2017 is referred to in this document as the “original sequence number space”.
Explicit notification of the reset event is based on a flag included in the R-TAG, which is referred to herein as the “SeqResetFlag”. The usage of the new linear initial sequence number space (which is referred to herein as “InitSeqNumSpace”) is noted via a new flag included in the R-TAG, which is referred to herein as the “InitSeqFlag”. Sequence values of the new number space (“InitSeqNumSpace”) are also included in the R-TAG.
While the embodiments described herein focus on FRER of TSN, the solutions proposed herein are also applicable to PREF of DetNet or other seamless redundancy mechanisms based on sequence numbering or equivalent functionality (e.g., provided by timestamps).
Embodiments of the solution described herein enable cloudification of seamless redundancy by providing seamless reset for sequence number generation (Sequence generating function) via improvements of, e.g., the Replication and Elimination function of IEEE 802.1CB-2017. In some embodiments, the following aspects are introduced (1) a new flag for explicit indication of the Sequence generating function reset and (2) the use of a new linear initial sequence number space to the existing cyclic sequence number space of, e.g., IEEE 802.1CB-2017. Again, while the description provided herein focuses on FRER as defined in IEEE 802.1CB, the solution described herein is applicable to FRER of TSN, PREF of DetNet, or other seamless redundancy mechanisms based on sequence numbering or equivalent functionality (e.g., provided by timestamps).
There are, proposed herein, various embodiments which address one or more of the issues disclosed herein. Certain embodiments may provide one or more of the following technical advantage(s). Embodiments of the proposed solution described herein enable cloudification of seamless redundancy by providing seamless reset for sequence number generation (Sequence generating function) via improvements of the Replication and Elimination function of IEEE 802.1CB-2017. These improvements ensure much faster and seamless adaptation to network failure scenarios and protect again unnecessary packet drops when sequence generation is reset.
The main motivation for these improvements is to allow cloudification, i.e., “cloud native” implementation of FRER/Packet Replication and Elimination Function (PREF) functions. Today's trend is to move applications towards virtualized environments. This trend has reached applications used in industrial environments as well, see, e.g., edge computing, fog computing. Using FRER/PREF in a cloud-based scenario, where Talkers/Listeners (Sources/Destinations) are moved to the Cloud, creates many additional challenges for the FRER/PREF functions. FRERs/PREFs are TSN/DetNet functions that belong to the endpoints, so FRER/PREF must work inside the Cloud. For example, a FRER/PREF is usually an instance in a controller-cluster (ctrl-cluster) serving an industrial application in a cloud, as illustrated in
Typical Cloud actions like running multiple Virtual Machines (VMs)/containers/instances, creating a VM/container/instance, moving a VM/container/instance, resetting a function, removing a VM/container/instance, etc. require FRER functions to adapt to the changing environment seamlessly. Changes in this environment are much more frequent than in current industrial network scenarios or network deployments without cloud.
High availability systems require the elimination of any single point of failure. Therefore, FRER/PREF functions (i.e., Sequence generation) must be improved to be able to support various Cloud specific redundancy solutions.
Using FRER of IEEE 802.1CB as an example, the impact of the reset of the sequence number generation function depends on the actual value of the sequence_number used for the last packet sent before the reset (noted here as SNL) and the value of the sequence_number used for the first packet sent after the reset (noted here as SNR). According to IEEE 802.1CB-2017, SNR is always 0 and SNL is a value in the range of {0; . . . ; GenSeqSpace−1}.
The following ranges can be defined to analyze the impact of the reset for (1) the solution described in IEEE 802.1CB-2017, (2) the solution described in the '397 Application, and (3) the solution described herein.
The assumption for the analysis is that (i) there are no other events in the network, only the sequence generation function is reset and (ii) the “frerSeqRcvyHistoryLength” (d) takes value of 100 for the probability calculations.
For the evaluation of the solution described in IEEE 802.1CB-2017, the following range is important:
As per IEEE 802.1CB-2017, packets with a sequence_number out of the history window (HSW) are dropped and packets within the history window are evaluated against the “SequenceHistory” to decide whether or not they are duplicates. Therefore, IEEE 802.1CB-2017 operates as follows for each of the ranges A-E:
That is, there is high probability of packet drop (99.8%) in most cases until timeout triggers the accept of the next packet. Note that packet drop means packet loss for the application whose operation may be damaged by the lost packets.
For the evaluation of the solution described in the '397 Application, the following additional range is important:
RIR is the range where the Eliminator function ignores the received “SeqResetFlag” and checks the received packet against the HSW. Note that range “D” is also part of RIR, as duplicates over the slower redundancy paths may have a sequence_number below SNL and can be as low as SNL−d. Therefore, the solution described in the '397 Application behaves as follows for each of the ranges:
There is a low probability of packet drop (0.3%) in most cases. When packet drop does happen, it is limited to a maximum of “2d” number of packets after the reset. Some TSN applications may tolerate this when reset occurs, some may not.
The solution described in various disclosed embodiments has the following characteristics for each of the ranges:
There is no packet drop. Packets sent after the reset are immediately considered valid. TSN applications are not impacted at all, and the implementation impact on FRER nodes is moderate.
Systems and methods are disclosed herein for improving the elimination function in a TSN network using FRER in accordance with IEEE 802.1CB (or likewise in a DetNet network using PREFs. Note that the discussion herein uses IEEE 802.1CB terminology and variable names where appropriate, denoted as “VariableName”. New variables, functions, and parameters follow IEEE 802.1CB naming conventions and are denoted as “NewEntityName”.
From now on, the description focuses on embodiments of the proposed solution for IEEE 802.11 FRER and as such uses terms and notations as described in 802.1CB-2017 FRER. However, the solution proposed herein is equally applicable to other seamless redundancy mechanisms (e.g., PREF for DetNet) based on sequence numbering or equivalent functionality (e.g., provides by timestamps).
As illustrated, the TX node 302 includes a FRER function 308 that operates to provide FRER in accordance with, in this example, IEEE 802.1CB. The FRER 308 includes a Replication function 310 and an Elimination function 312 (illustrated as optional in the sense that it is not used for transmission of the Stream to the RX node 304). In a similar manner, the RX node 304 includes a FRER function 314 that operates to provide FRER in accordance with, in this example, IEEE 802.1CB. The FRER 314 includes a Replication function 316 (illustrated as optional in the sense that it is not used for reception of the Stream from the TX node 302) and an Elimination function 322.
As illustrated, the Replication function 310 includes a Sequence Generation function 320. The Sequence Generation function 320 operates to generate sequence numbers for the packets in the packet Stream. The Elimination function 318 includes a Sequence Recovery function 322. The Sequence Recovery function 322 operates on packets passed up the protocol stack towards the higher layer functions and uses the sequence number sub-parameter to decide which packets to pass and which to discard.
In some embodiments, the TX node 302 is part of a Cloud implementation (e.g., part of a Ctrl-cluster as described above with respect to
A Sequencing function of the FRERs 308 and 314 provides the “sequence_number” sub-parameter for FRER functions. In particular, the Sequencing function has two kinds of component functions: (1) a Sequence Generation function (e.g., the Sequence Generation function 320) that operate on packets passed down the protocol stack towards the physical layer and generates a value for the sequence_number sub-parameter and (2) a Sequence Recovery function (e.g., the Sequence Recovery function 322) that operates on packets passed up the protocol stack towards the higher layer functions and uses the sequence_number sub-parameter of the received packets to decide which packets to pass and which to discard.
Embodiments of the solution described herein use the flag introduced by the '379 Application to the R-TAG, namely the “SeqResetFlag”. This flag is used as follows:
Embodiments of the solution described herein introduce a new additional sequence number space, which is referred to herein as “InitSeqNumSpace”, which is used after initialization or reset of the Sequence Generation function 320. The newly introduced InitSeqNumSpace is illustrated by the bolded linear sequence number space in
Embodiments of the solution described herein introduce a new flag in the R-TAG, which is referred to as the “InitSeqFlag”. This flag is used by the Replication function 310 at the TX node 302 as follows:
The “InitSeqNumSpace” is used as follows:
Embodiments of the solution herein define a new procedure for the Elimination function 318 at the RX node 304 as well. It uses the new set of variables to handle packets containing sequence_number from the above described “InitSeqNumSpace”. The size of the history window is same for both sequence number spaces. The Elimination function 318 operates as follows upon receiving a packet from one of the Member Streams of the Stream transmitted by TX node 302:
Note that the timeout mechanism for the sequence recovery—to accept the next packet, no matter what the value of its sequence_number subparameter (see “TakeAny” variable in 802.1CB-2017 for the original sequence number space)—is not changed and applied for both sequence number spaces.
An implementation may use two variables to enable/disable the use of the reset flag (“UsingResetFlag”=I/O (enable/disable)) and the new initial sequence number space (“UsingInitSpace”=I/O (enable/disable)).
Whether proceeding from step 610 or step 612, the Replication function 310 determines whether use of “InitSeqNumSpace” is enabled (step 614). If not (614, NO), the process proceeds to step 632, which is described below. Otherwise (step 614, YES), the Replication function 310 determines whether “UseInitSeqNum” is enabled (e.g., set to “True”) (step 616). If so (step 616, YES), the Replication function 310 enables “InitSeqFlag” (e.g., sets it to “1”) (step 618), adds a sequence number (seq_num) equal to “InitGenSeqNum” to the packet (step 620), increments “InitGenSeqNum” (step 622), and determines whether the “InitSeqNumSpace” is exhausted (step 624). If not (step 624, NO), the process proceeds to step 628, which is described below. Otherwise (step 624, YES), the Replication function 310 sets “GenSeqNum” equal to “0” and sets “UseInitSeqNum” to False (step 626). Then, whether proceeding from the “NO” branch of step 624 or step 626, the Replication function 310 adds an R-Tag to the packet (step 628) and the procedure proceeds to step 604 where the packet is ready to send.
Returning to step 616, if “UseInitSeqNum” is not set to “TRUE”, the Replication function 310 disables the “initSeqFlag” (e.g., sets it to “0”) (step 630), adds a sequence number (seq_num) equal to “GenSeqNum” to the packet (step 632), and increments “GenSeqSum” (step 634). The procedure then proceeds to step 628 where an R-Tag is added to the packet and then the packet is ready to be sent.
In one embodiment, each of the first plurality of packets further comprises an explicit indication that the linear sequence number space is being used.
In one embodiment, the network is a TSN network. Further, in one embodiment, the method further comprises resetting the sequence generation function 310, wherein resetting the sequence generation function 310 comprises resetting a sequence number history, a history window (e.g., resetting “RecovSeqNum”, which is the midpoint of the history window), or both the sequence number history and the history window. In one embodiment, the steps of determining (700) that the sequence generation function 310 at the TX node 302 has been reset, transmitting (702) the first plurality of packets in the Stream of packets, determining (704) that the end of the linear sequence number space has been reached or that use of the linear sequence number space has been otherwise disabled, and transmitting (706) the second plurality of packet in the Stream of packets are performed by the FRER function 308 of the TX node 302 and, more specifically, by the Replication function 310 of the TX node 302.
In another embodiment, the network is a DetNet network. Further, in one embodiment, the steps of determining (700) that the sequence generation function 310 at the TX node 302 has been reset, transmitting (702) the first plurality of packets in the Stream of packets, determining (704) that the end of the linear sequence number space has been reached or that use of the linear sequence number space has been otherwise disabled, and transmitting (706) the second plurality of packet in the Stream of packets are performed by a PRER of the TX node 302.
Returning to step 902, if “UsingResetFlag” is enabled (step 902, YES), the Elimination function 318 determines whether use of “InitSeqNumSpace” is enabled (step 920). If not (step 920, NO), the Elimination function 318 determines whether “SeqResetFlag” is enabled and the sequence number of the received packet is out of the RIR (step 922). If not (step 922, NO), the procedure proceeds to step 904. Otherwise (step 922, YES), the procedure proceeds to step 912.
Whether proceeding from the YES branch of step 906, the NO branch of step 910, or the YES branch of step 922, the Elimination function 318 then updates the “RecovSeqNum” (step 912), updates the “SequenceHistory” (step 914), clears “TakeAny” (step 916), and accepts the packet (step 918).
Returning to step 920, if use of “InitSeqNumSpace” is enabled (step 920, YES), the Elimination function 318 determines whether “InitSeqFlag” is enabled (step 924). If not (step 924, NO), the process proceeds to step 922. Otherwise (step 924, YES), the Elimination function 318 determines whether “SeqResetFlag” is enabled and the sequence number of the received packet is out of iRIR (step 926). If so (step 926, YES), the procedure proceeds to step 932, which is described below. Otherwise (step 926, NO), the Elimination function 318 determines whether the sequence number of the packet is in iHSW (step 928). If so (step 928, YES), the Elimination function 318 determines whether the packet is already in history (step 930). If no (step 930), the procedure proceeds to step 932. Otherwise (step 932, NO), the Elimination function 318 drops the packet (step 944). Returning to step 928, if the sequence number of the packet is not in iHSW (step 928, NO), the Elimination function 318 determines whether “InitTakeAny” is enabled (step 942). If not (step 942, NO), the Elimination function 318 discards the packet (step 944). Otherwise (step 942, YES), the process proceeds to step 932.
Whether proceeding from the YES branch of step 926, the NO branch of step 930, or the YES branch of step 942, the Elimination function 318 updates the “InitRecovSeqNum” (step 932), updates “InitSequenceHistory” (step 934), and clears “InitTakeAny” (step 936). The Elimination function 318 determines whether the sequence number is in STAR (i.e., the range {“GenSeqSpace”−d; . . . ; “GenSeqSpace”−2× d}) (step 938). If so (step 938, YES), the Elimination function 318 sets “TakeAny” to TRUE (step 940) and the packet is accepted (step 918). Otherwise (step 938, NO), the Elimination function 318 accepts the packet (step 918).
One possible option to encode the “SeqResetFlag”, the “InitSeqFlag”, and the “new sequence number belonging to the linear sequence number space” in the R-TAG are described in the following:
As used herein, a “virtualized” network node is an implementation of the network node 1100 in which at least a portion of the functionality of the network node 1100 is implemented as a virtual component(s) (e.g., via a virtual machine(s) executing on a physical processing node(s) in a network(s)). As illustrated, in this example, the network node 1100 includes one or more processing nodes 1200 coupled to or included as part of a network(s) 1202. Each processing node 1200 includes one or more processors 1204 (e.g., CPUs, ASICs, FPGAS, and/or the like), memory 1206, and a network interface 1208. In this example, functions 1210 of the network node 1100 described herein (e.g., one or more functions of the TX node 302 or one or more functions of the RX node 304, as described herein) are implemented at one of the processing nodes 1200 or distributed across two or more of the processing nodes 1200 in any desired manner. In some particular embodiments, some or all of the functions 1210 of the network node 1100 described herein are implemented as virtual components executed by one or more virtual machines implemented in a virtual environment(s) hosted by the processing node(s) 1200.
In some embodiments, a computer program including instructions which, when executed by at least one processor, causes the at least one processor to carry out the functionality of the network node 1100 or a node (e.g., a processing node 1200) implementing one or more of the functions 1210 of the network node 1100 in a virtual environment according to any of the embodiments described herein is provided. In some embodiments, a carrier comprising the aforementioned computer program product is provided. The carrier is one of an electronic signal, an optical signal, a radio signal, or a computer readable storage medium (e.g., a non-transitory computer readable medium such as memory).
Any appropriate steps, methods, features, functions, or benefits disclosed herein may be performed through one or more functional units or modules of one or more virtual apparatuses. Each virtual apparatus may comprise a number of these functional units. These functional units may be implemented via processing circuitry, which may include one or more microprocessor or microcontrollers, as well as other digital hardware, which may include Digital Signal Processor (DSPs), special-purpose digital logic, and the like. The processing circuitry may be configured to execute program code stored in memory, which may include one or several types of memory such as Read Only Memory (ROM), Random Access Memory (RAM), cache memory, flash memory devices, optical storage devices, etc. Program code stored in memory includes program instructions for executing one or more telecommunications and/or data communications protocols as well as instructions for carrying out one or more of the techniques described herein. In some implementations, the processing circuitry may be used to cause the respective functional unit to perform corresponding functions according one or more embodiments of the present disclosure.
While processes in the figures may show a particular order of operations performed by certain embodiments of the present disclosure, it should be understood that such order is exemplary (e.g., alternative embodiments may perform the operations in a different order, combine certain operations, overlap certain operations, etc.).
Other embodiments of the present disclosure are directed to providing gate controlled FRER and/or PRER for implementation in a cloud environment. These embodiments are now described below.
Moving industrial applications to a cloud environment is a challenging task. Redundancy solutions developed for cloud environment mainly focus on broadband services and need to be adapted for industrial scenarios having much stronger requirements. In the transport network between the cloud and the industrial endpoint, special functions (FRER) were developed by IEEE 802.1 to meet the reliability requirements of the industrial applications. Cloud specific reliability solutions must be compatible with the FRER-domain. Some embodiments of the present disclosure are directed to a system and operations for enabling adaptation of cloud reliability and FRER functions of the transport domain.
In the cloud environment it is quite usual that single point of failure is avoided by implementing multiple instances of an application. When combined with a transport domain using FRER, the FRER domain is used also within the Cloud environment, such as shown in
The mechanisms available for reset of redundancy functions include too many unnecessary packet drops and are not cloud ready. As the ultimate goal of seamless redundancy is to avoid packet loss as much as possible, the unnecessary packet drops due to the operations of a redundancy mechanism should be minimized. Furthermore, moving seamless redundancy (e.g., IEEE 802.1CB) components to a cloud environment creates challenges regarding availability, as the reset of functions is much more frequent than in current industrial hardware environments. Therefore, seamless detection and adaptation to scenarios caused by reset of redundancy (e.g., IEEE 802.1CB) functions are essential.
Some embodiments of the present disclosure are directed to utilizing additional functions to enable effective FRER functionalities in a cloud environment and allow the combination/integration of FRER functionalities with cloud specific redundancy functions.
In some embodiments, gate functionality is provided between the application instances and the FRER entities, and which forwards/blocks packets towards or from FRER entities based on gate control information (open or close). Timeout functionality in FRER entities is provided to reset the Sequence generation function, when no packets were received for a given time period. It is noted that reset of Sequence generation function can be achieved by other operations as well. These new functions allow seamless interaction between cloud specific redundancy solutions and the connected packet network using FRER.
A Cloud Management (“Cloud Mngmnt”) (orchestrator) 1430 controls the operation and maintenance of application instances in the Application cluster 1410. An application instance may not be aware of other application instances.
An Application Cluster Controller (ACC) 1412 (also “App cls-ctrl.”) selects the downstream active application, that provides the application's output packet flow, i.e., the input for the FRER entity 1420. The ACC 1412 informs about its selection (provides notification of the selection) to the Gate controller 1402 (e.g., indicating which application instance is downstream active at a given time or during a time-period).
The Gate controller 1402 sets the gate open or close states according to the received information from the ACC 1412. In some or all embodiments, there is only a single gate operated in open state for forwarding packets or frames in a stream from a downstream active application instance, as a given TSN Stream, towards the FRER domain 1420 at any given time, therefore, only one FRER entity receives packets or frames from the application cluster 1410. The Gate controller 1402 may trigger the open FRER in the FRER domain 1420 operating in the downstream direction to reset its sequence number generation function, which can also be called “sequence generation function”, (e.g., via the frerSeqRcvyReset managed object (802.1 CB-2017, Section 10.4.1.4)).
GATE-x, which can be any of the illustrated Gate-1 through Gate-M, forwards or blocks packet or frame forwarding depending upon the open or closed state, respectively. GATE-x can be implemented as a standalone function (not integrated with FRER) or integrated in FRER Sequencing function (802.1CB-2017, Section 7.4)
FRER entities (i.e., FRER-1 to FRER-M) in the FRER domain 1420 provide the replication and/or elimination function to create and/or merge the TSN member Streams. The FRER entities 1420 can be implemented according to one or more of the embodiments described above. As will be described in further detail below, a FRER entity may support a new TimeOutBasedSequenceGenerationReset function. The FRER entity resets its Sequence generation function if no input frames were received for a given (threshold) time interval by the FRER entity for a given TSN Stream. Using TimeOutBasedSequenceGenerationReset function is optional and depends on the system design.
Potential advantages that may be provided by one or more of the embodiments disclosed herein can include any one or more of the following:
The term “downstream” direction refers to forwarding a TSN Stream from the application instance (the “downstream active application instance”) towards the actor(s). The term “upstream” direction refers to forwarding a TSN Stream forwarded from the actor towards the application instance(s) (the “upstream active application instance(s)”.
Forwarding of TSN streams in the downstream direction from a downstream active application instance towards one or more actors is now described in the context of
Referring to
Corresponding operations by the gate controller 1402 are now described in the context of the flowchart of
The packet or frame replication entity 1420a replicates the packets or frames in the stream received from the gate associated with the downstream active application instance to create Time-Sensitive Networking (TSN) streams for transmission via the at least two disjoint paths through the network.
In a further embodiment, the control operation 1710 to change any of the gates between different ones of the open and closed states based on the downstream information is performed in a time synchronized manner, i.e., so that the gates which are controlled to switch states concurrently perform the state switching.
In some embodiments, the control operation 1710 to control the gate within the gate cluster 1400 that is associated with the downstream active application instance to be open state, operates to allow only a single one of the gates in the gate cluster 1400 to be in the open state at a time which operates to forward the stream of packets or frames in the downstream direction to the associated packet or frame replication entity 1420a for replication and transmission via the at least two disjoint paths through the network.
Output packets of the downstream active application instance (e.g., Ctrl-1) are forwarded by the associated one of the Gates (e.g., Gate-1) to the associated packet or frame replication entity (e.g., FER-1) entity 1420a, which provides a defined level of packet or frame replication for communication through the network in the downstream direction. There is a Gate-x positioned before each F
ER entity. In at least some embodiments, each Gate is associated with only a single F
ER entity in a one-to-one association. A single Gate-x may be fed packets from multiple application instances executing in the application cluster 1410. The ACC 1412 informs the Gate controller 1402 about its decision regarding which application instance output should be forwarded towards the F
ER entity 1420a. The Gate-controller 1402 drives the Gates responsive to the received information. In some or all embodiments, in the downstream direction there is only a single one of the Gates that is open (allowing packets to flow through to an associated (connected) one of the F
ER entities) for a given TSN Stream at any given time, all the other Gates are closed. It is implementation dependent how to achieve that a change between the open and the closed Gate states happens at the same time for all affected Gates. In one embodiment, the Gates are controlled in a time synchronized manner, e.g., based on a common synchronization clock.
The FER entity 1420a operates as the entry point to the F
ER domain 1420. The F
ER entity 1420a adds the sequence number information to the packet(s) received from the corresponding open Gate. For example, F
ER-1 adds sequence number information to packet(s) received from open state Gate-1. When there is a change regarding which Gate is open, e.g., Gate-1 is closed and Gate-M is opened, the Sequence generation function of the F
ER entity 1420a that serves the newly opened Gate (e.g., F
ER-M serving Gate-M) must be reset. This reset can be triggered by the Gate controller 1402 (e.g., via the frerSeqRcvyReset managed object (802.1 CB-2017, Section 10.4.1.4)). Alternatively, reset can be done in the F
ER entity 1420a (e.g., F
ER-M) by supporting a new functionality (TimeOutBasedSequenceGenerationReset). The F
ER entity 1420a (e.g., F
ER-M) resets the Sequence generation function if no input frames were received for a given time by the F
ER entity 1420a (e.g., F
ER-M) for a given TSN Stream. Reset of Sequence generation function should or must happen before changing the corresponding gate state from closed to open (e.g., the Sequence generation function is reset before Gate-M is changed from closed to open state).
Accordingly, with the reference to the embodiments of
The gate controller 1402 and the gate cluster 1400 may be part of the FRER domain 1420.
Forwarding of a TSN stream in an upstream direction from actors towards application instances is now described in the context of
Referring to
The first case where all Gates are always open, can be implemented by the Gate controller 1402 responsive to when the information from the ACC 1412 indicates that the packet or frame sent by the actor must reach all the application instances, i.e., where all application instances are upstream active needing to receive the packet or frame in the upstream direction from the actor via the network. For example, when sensor data is a needed input for each of the application instances of the application cluster 1410 in order to calculate the control for the actor(s), i.e., all of the application instances are “upstream active”, the gate-controller 1402 controls all the Gates (e.g., Gate-1 to Gate-M) to be open and forward the TSN Stream to each of the application instances (e.g., Ctrl-1 to Ctrl-N) associated with the Gates.
The second case where only some of the Gates are open, can be implemented by the Gate controller 1402 when the information from the ACC 1412 indicates that some of the application instances are “upstream inactive” (also referred to as “standby”) during which those application instances do not require input. In this scenario the architecture works as described below.
The Cloud Management 1430 creates the necessary (desired) application instances. Each application instance may operate as a listener for the upstream TSN Stream, i.e., to receive packets of the upstream TSN Stream. Redundancy of application instances is controlled by the ACC 1412. The ACC 1412 selects which of the application instances are upstream active and should receive packets from the packet or frame elimination entities 1420b (e.g., selected ones of the FRR-1 through FR
R-M) of the FR
R domain 1420 (
R-1 to one or more associated application instances, Gate-2 selectively forwards (when in “open state”) or blocks (when in “closed state”) packets from FR
R-2 to one or more associated upstream active application instances, and so-on with Gate-M selectively forwarding (when in “open state”) or blocking (when in “closed state”) packets from FRER-M to one or more associated upstream active application instances, etc. A Gate-x may feed multiple application instances, e.g., feeding one or more application instances that have been programmatically defined to be associated with Gate-x. The ACC 1412 informs the Gate controller 1402 about a decision provided by ACC 1412 regarding which application instance(s) is(are) upstream active and need input from the FRER entities 1420b of the FRER domain 1420 (
Corresponding operations by the gate controller 1402 are now described in the context of the flowchart of
The packet or frame elimination (FRR) entities 1420b act as the exit point of the FRER domain 1420 (
R) entity 1420b removes the sequence number information from the packet and sends it to the related one of the Gates. For example, FR
R-1 removes the sequence number information from a packet and sends it to the related Gate-1 which, when in the open state, forwards the packet to an associated one or more upstream active application instances. Similarly, FR
R-2 removes the sequence number information from a packet and sends it to the related Gate-2 which, when in the open state, forwards the packet to an associated one or more active application instances. In contrast, FR
R-M removes the sequence number information from a packet and sends it to the related Gate-M which, when in the closed state, blocks (prevents) forwarding of the packet to an associated one or more inactive application instances.
With further reference to
In another embodiment, the gate controller 1402 operates to obtain 1800 upstream information indicating which one or more application instances in the application cluster 1410 are upstream active needing to receive a stream of packets or frames in an upstream direction from the network. Based on the upstream information, the gate controller 1402 controls 1810 one or more gates within the gate cluster (1400) that are associated with the upstream active one or more application instances to be open state. The one or more packet or frame elimination entities 1420b operate to remove sequence number information from a packet or frame received in the stream by the one or more packet or frame elimination entities 1420b to generate a modified packet or frame. The one or more packet or frame elimination entities 1420b further operate to forward the modified packet or frame to the one or more gates within the gate cluster 1400 that are associated with the upstream active one or more application instances. While the one or more gates associated with the upstream active one or more application instances are open state, the open state gates operate to forward the modified packet or frame in the upstream direction to the upstream active one or more application instances. In contrast, while the one or more gates associated with the upstream active one or more application instances are closed state, the closed state gates operate to block the modified packet or frame from being forwarded to the upstream active one or more application instances.
A listing of example embodiments of the present disclosure follows:
Embodiment 1. A method for packet or frame replication and elimination in a network, the method comprising:
Embodiment 2. The method of Embodiment 1, further comprising:
Embodiment 3. The method of any of Embodiments 1 to 2, wherein the controlling (1710) to change any of the gates between different ones of the open and closed states based on the downstream information is performed in a time synchronized manner.
Embodiment 4. The method of any of Embodiments 1 to 3, wherein the controlling (1710) by the gate controller (1402) of the gate within the gate cluster (1400) that is associated with the downstream active application instance to be open state, operates to allow only a single one of the gates in the gate cluster (1400) to be in the open state at a time which operates to forward the stream of packets or frames in the downstream direction to the associated packet or frame replication entity (1420a) for replication and transmission via the at least two disjoint paths through the network.
Embodiment 5. The method of any of Embodiments 1 to 4 wherein the packet or frame replication entity (1420a) replicates the packets or frames in the stream for transmission via the at least two disjoint paths through a Time-Sensitive Networking, TSN, network.
Embodiment 6. The method of any of Embodiments 1 to 5 further comprising triggering by the gate controller (1402) the packet or frame replication entity (1420a) to reset a sequence number generation function (320) based on the downstream information.
Embodiment 7. The method of Embodiment 6 further comprising:
Embodiment 8. The method of Embodiment 7 wherein:
Embodiment 9. The method of any of Embodiments 6 to 8 wherein:
Embodiment 10. The method of any of Embodiments 6 to 9 wherein the triggering by the gate controller (1402) the packet or frame replication entity (1420a) to reset the sequence number generation function (320), comprises:
Embodiment 11. The method of any of Embodiments 1 to 10 wherein:
Embodiment 12. The method of any of Embodiments 1 to 11 further comprising:
Embodiment 13. The method of any of Embodiments 1 to 12 further comprising:
Embodiment 14. The method of any of Embodiments 1 to 13 further comprising:
Embodiment 15. A network for packet or frame replication and elimination, the network comprising:
Embodiment 16. The network of Embodiment 15, wherein the gate controller (1402) is further adapted to:
Embodiment 17. The network of any of Embodiments 15 to 16, wherein the gate controller (1402) is further adapted to change any of the gates between different ones of the open and closed states based on the downstream information in a time synchronized manner.
Embodiment 18. The network of any of Embodiments 15 to 17, wherein the gate controller (1402) is further adapted to allow only a single one of the gates in the gate cluster (1400) to be in the open state at a time for operating to forward a stream of packets or frames in the downstream direction to the associated packet or frame replication entity (1420a) for replication and transmission via at least two disjoint paths through the network.
Embodiment 19. The network of any of Embodiments 15 to 18 wherein the packet or frame replication entity (1420a) is adapted to replicate the packets or frames in the stream for transmission via the at least two disjoint paths through a Time-Sensitive Networking, TSN, network.
Embodiment 20. The network of any of Embodiments 15 to 19 wherein the gate controller (1402) is further adapted to trigger the packet or frame replication entity (1420a) to reset a sequence number generation function (320) based on the downstream information.
Embodiment 21. The network of Embodiment 20 wherein the gate controller (1402) is further adapted to:
Embodiment 22. The network of Embodiment 21 wherein the gate controller (1402) is further adapted to:
Embodiment 23. The network of any of Embodiments 19 to 22 wherein the gate controller (1402) is further adapted to:
Embodiment 24. The network of any of Embodiments 19 to 23 wherein the gate controller (1402) is further adapted to trigger the packet or frame replication entity (1420a) to reset the sequence number generation function (320), based on:
Embodiment 25. The network of any of Embodiments 15 to 24 wherein:
Embodiment 26. The network of any of Embodiments 15 to 25 wherein the packet or frame replication entity (1420a) is adapted to:
Embodiment 27. The network of any of Embodiments 15 to 26 wherein the gate controller (1402) is further adapted to:
Embodiment 28. The network of any of Embodiments 15 to 27 wherein:
Embodiment 29. A computer program product comprising a non-transitory computer readable medium storing instructions executable by a network node to perform the method of any of Embodiments 1 to 14.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/IB2022/051255 | 2/11/2022 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 63179818 | Apr 2021 | US |
