Patent Application
20250141782
The present disclosure relates to network communication.
Training, as used herein, refers to any method by which a receiver's performance is improved by modifying the transmitter's signaling characteristics.
Ethernet has a specified link training method for non-segmented links, such as backplane and copper cables, such as that defined in clause 136.8.11 (“PMD control function”) of IEEE 802.3cd (IEEE 802.3cd-2018). This method cannot be used for links that comprise multiple segments, such as when retimers are present in the link. Therefore, training of multiple segments within an Ethernet link is not supported by the standard. Specific examples are the Ethernet original Physical Media Dependent (PMD) sublayer control function for a backplane physical layer (PHY), specified in clause 72.6.10 of IEEE 802; subsequent similar specifications for higher data rates, clauses 92.7.12 and 136.8.11; management-based “transmitter equalization feedback” specified for Chip Attachment Unit Interface (CAUI-4) chip-to-chip, in 83D.5; and Common Management Interface Specification (CMIS) Link Training, CMIS-LT.
Techniques are provided to extend training to an end-to-end method, where a ready-to-send (RTS) variable is added to training frames to indicate that, on a current interface, the Physical Medium Attachment (PMA) sublayer, with all its lanes, has valid data and appropriate clock timing and is ready to send data to the output when training is completed. These techniques are applicable to a PMA that is an endpoint (and thus has only a single network interface involved) as well as to PMAs that have two network interfaces connected to segment partners. Moreover, these techniques may be configured to work with legacy devices that do not support segmented link training.
In one form, a method is provided for a device that is part of a multi-segment network link but has only one network interface in the link (i.e., it is an end point) on the link, the method involving: on a network interface facing a segment partner, receiving incoming training frames from the segment partner and transmitting outgoing training frames to the segment partner; and including a ready-to-send indication in an outgoing training frame to be sent to the segment partner, wherein the ready-to-send indication indicates that the device has, on all lanes of the network interface, valid data and is ready to send data on the network interface to the segment partner.
In another form, a method is provided for a device that is part of a multi-segment network link but has two interfaces in the link (i.e., it is mid-point) on the link. The method involves: on a first network interface facing a first segment partner, receiving incoming training frames from the first segment partner and transmitting outgoing training frames to the first segment partner; on a second network interface facing a second segment partner, receiving incoming training frames from the second segment partner and transmitting outgoing training frames to the second segment partner; first including a first ready-to-send indication, when the first ready-to-send indication is included in an incoming training frame received from the first segment partner, in an outgoing training frame to be sent to the second segment partner; and second including a second ready-to-send indication, when the second ready-to-send indication is included in an incoming training frame received from the second segment partner, in an outgoing training frame to be sent to the first segment partner.
In accordance with still another aspect, a method is provided that is performed by a device that is a mid-point of a multi-segment network link, and one of its segment partners connected to one of its two network interfaces, is not enabled to perform the segment-based training techniques presented herein. The method involves: disabling a first transmitter on a first network interface of the device for a period of time; disabling a second transmitter on a second network interface of the device; on a first network interface facing a first segment partner, receiving incoming training frames from the first segment partner; after the period of time, enabling the first transmitter on the first network interface and transmitting outgoing training frames to the first segment partner; when a first ready-to-send indication is included in an incoming training frame received from the first segment partner, enabling the second transmitter and transmitting a binary pattern from a second network interface to a second segment partner; and when a receiver ready indication is determined for the second segment partner, including a second ready-to-send indication in an outgoing training frame transmitted to the first segment partner.
Arrangements for devices configured to perform these methods are also presented herein.
Link bring-up in systems that include multiple segments is usually implemented without training, and is heavily dependent on management software that is implemented separately from the serializer-deserializer (SerDes), which can be difficult to integrate and debug. In addition, these algorithms are proprietary and may have compatibility challenges with other products. This may result in sub-optimal bit error rate (BER) and prolonged development times.
Again, training refers to enabling a receiver to tell the transmitter, on the other side, to adjust the transmit signal so that the receiver receives it appropriately. If the signal is too strong, the receiver may ask the transmitter to attenuate the signal or it may ask the transmitter to adjust the equalization of the signal. This training happens before data is sent to the receiver to adjust the signal characteristics for the signal sent by the transmitter, for the receiver's benefit. The signal affected during training may be an electrical signal or an optical signal. During training, training frames are sent between the two link partners. This can take some time, e.g., several seconds. At some point, each of the receivers can indicate training is done, such as by sending a link ready signal to the link partner when it is done training and ready. When the status on both sides of the link is ready, then the link can be brought and data can start being sent.
When the link is not end-to-end link that is guaranteed to have data from both ends, link training as it currently stands cannot be used. For example, there may be no received data on the optical link to transmit toward the host, or there is no device on the other side of the optical link sending data.
The Physical Media Dependent (PMD) sublayer control function is suitable for a single end-to-end electrical channel. The PMD control function was originally coupled with an auto-negotiation state diagram. When training is completed, the PMD switches to DATA mode—and sends its input (originally assumed to be the co-packaged Physical Coding Sublayer (PCS) and Physical Medium Attachment (PMA) sublayer (PCS+PMA output) in the transmit direction to the media. The auto-negotiation (AN) time for link training is limited by a link_fail_inhibit_timer (originally 500 ms, in IEEE 802.3ck PMDs: 12 seconds). Auto-negotiation is defined in Clause 73 of IEEE 802.3, and is a communication protocol that enables exchanging information about data rates available on each side and thus finding the highest common data rate. After the data rate is established, training is performed at this data rate. AN can be disabled if both sides are pre-configured to one data rate. Training can be used in this case too. Clause 136 link training can be used without AN, but is similarly constrained in time, and also assumes that data is immediately available.
An Attachment Unit Interface-Chip-to-Chip (AUI-C2C) could be present between the PMD and the PCS. The PMD resides within the “chip” (e.g., retimer), so the retimer performs the PMD link training with the partner. The AUI (possibly on both sides) needs to be fully active beforehand, otherwise when switching to DATA mode the PMD has nothing to transmit. Thus, any AUI training should already be completed and achieve an acceptable AUI BER. This dependency also means that autonomous link training by the PMD can be done only with known signaling rates (making AN useless).
In-band “PMD control” cannot be used for AUIs. If clause 136 in-band training was used, the problem is the same—the adjacent segments need to be active beforehand. There is no alternative training protocol specified for the AUIs.
AUI-Chip-to-Module (C2M) links can be optimized using module management, e.g., CMIS. Training can use locally generated test patterns such as the PRBS31Q test pattern. This pattern is defined as optional in the Ethernet standard, but ubiquitously implemented. The training-related variables defined in clause 136 could theoretically be exchanged between PMAs using management registers (see IEEE 802.3ck, 120F.3.1.4).
However, this has not been widely adopted. The Optical Internetworking Forum (OIF) CMIS-LT project is in progress but it is unclear if it will follow the same direction.
Moreover, this approach has several drawbacks. Management and SerDes are separated in many aspects, making it difficult to integrate. Using management to exchange requests and set the transmitter on both sides (as proposed for CMIS-LT) creates proprietary, long and complex out-of-band management workflows to be implemented in high-layer software. Centralized management in large systems creates bottlenecks, and is difficult to debug. This can have a substantial impact on software development time. An out-of-band (OOB) solution should ideally be a backup plan, not used as the first choice.
In summary, using retimers and AUIs within physical layers is not fully covered by existing standards. Although there are implementations that include retimers, they are relatively rare and not straightforward. Similar problems are expected if in-band “link training” is to be performed on AUI-C2M in optical PHYs. Out-of-band training is not a sufficient solution. The issues of the Clause 136 in-band link training are: auto switching to DATA when training is completed, and lack of indication of availability of data to be sent.
Reference is now made to
The ASIC 112 may exchange data with a media access control (MAC) layer at 800 Gbps (e.g., 800GMII), and may use an AUI-C2C interface across multiple lanes for communication with the retimer 114. Moreover, the retimer 114 may communicate with the module 116 via an AUI-C2M interface across multiple lanes. The module 116 may communicate with the module 122 via a PMD to PMD interface, such as using the 400GBASE-DR4 standard. The devices in Host B may communication between each other, across multiple lanes, using similar technology used in Host A.
In current and future generations, not all modules are “known” to the host and different modules (from different vendors) may use different parameters for signals. Each of the segments may benefit from training, but there is no single management entity that can orchestrate all the segments. The segments within a host need to complete training before sending data to a link partner, across the physical media. Ideally, each segment should train autonomously and training can occur in any order. If there is a standardized procedure for link training in a multi-segment environment such as that shown in
At 200 Gb/s per lane, and more than 30 dB insertion loss, training on AUIs (including chip-to-module (C2M)) may be just as important as between CR/KR PMDs, where, in Ethernet nomenclature, CR denotes Ethernet over copper cable assemblies with a reach of a few meters and KR denotes Ethernet over backplanes (printed circuit boards and connectors). The module-to-module link creation involves both sides to transmit a valid signal. A module that does not detect a signal may (and typically does) squelch its output. AUI training involves a valid signal and this can be handled using a test pattern generated by the module until a signal is detected.
The link partner might not be active while the AUI training is performed. This can happen on both sides (e.g., medium is disconnected). Bringing up a link involves sequencing of training completion, signal detect, and transmitter disable/enable. Again, all that can be done using centralized management, but that creates unnecessary burden on system software. A decentralized solution may be preferable.
Each PMA sublayer with a physical interface (AUI or PMD) has a handshake logic function on that interface. PMAs with two interfaces, such as retimers, have separate handshake functions, one for each interface. The handshake logic function manages how a state, called Ready to Send (RTS), is propagated segment-by-segment as each segment is trained, and then across the link to the link partner and its segments.
Reference is made to
RR indicates that the PMA receiver can receive data (including successful completion of training if required). To extend the training to an end-to-end method, the RTS variable is added to indicate that, on the current interface, the PMA (with all its lanes) has valid data and appropriate clock timing and is ready to send data to the output when handshake (training) is completed.
The handshake logic 210 includes two AND gates 212 and 214. AND gate 212 receives as input Remote Receiver Ready (RR) obtained from a segment partner and Local RR of the PMA 200. The AND gate 212 generates an output across all lanes of the PMA 200 that is supplied as input to AND gate 214. The intent of the expression “all lanes” as used herein is that when there are multiple lanes, the output of the AND gate 212 indicates that all lanes have both local RR and remote RR. The AND gate 214 also receives as input a !reset signal generated internally by the PMA 200 and generates as output a Local Ready to Send (RTS) value. Thus, in a PMA 200 with a single interface, the Local RTS is set to 1 when Local RR and Remote RR are 1 on all lanes of the interface.
Thus, each handshake logic 230 has per-interface local and remote versions of each of RR and RTS. RR is per lane and RTS is common for all lanes. The Remote RTS is exchanged as part of the training frames that are sent between each segment.
Reference is now made to
Using RR, a single segment can be brought up, with the state diagram of
As a result, when a segment is up, the RTS is propagated by the PMAs in both directions, and this is depicted in
If handshake signaling is available on an interface, variables are communicated to the segment partner using the handshake protocol. Handshake signaling continues until RTS and all RR are true in both local and remote. Then (after a delay), the PMA switches to its normal functionality in both directions. There is no specified timeout in waiting for RR/RTS. A management process can access devices and restart link-up on a specific segment if desired.
If handshake signaling is not available on an interface (disabled, or not defined for the interface type), then:
The output on that interface is enabled or disabled to indicate Local_RTS to the partner.
In the link up state flow used today, without the solution presented herein, a Receiver Ready on the Local and Remote was enough to jump to the Link_Ready state 370. The Segment_Ready state 360 did not exist. Segment_Ready state 360 prevents data from being sent until all segments of an end-to-end link are in a RTS true state. Segment_Ready state 360 means that the current segment being trained has completed training. The receivers on both sides of the segment are satisfied with the transmitter configuration on the other side, but there is not necessarily any data to be sent. A segment could continue transmitting training frames until there is some data to be sent. A device knows that there is data to be sent and to stop sending training frames when it receives an additional bit that is exchanged between the two sides of the segment, the aforementioned RTS value.
The handshake function is mandatory for all PMAs, but a handshake signaling protocol (that enables segment training) may not always be available. This may depend on medium or interface type, and details of handshake signaling may be different. Even if available, handshake protocol functionality may be disabled and replaced by squelch and energy detection functions (but similar to bypassing training, it has to be done on both ends of a segment). Devices that use handshake protocol on one interface but not on the other may use a locally generated pattern for the non-handshake interface (instead of forwarding the incoming handshake protocol), as described below.
The proposed method can be implemented in multiple types of handshake signaling. It can work with “legacy” segments that use squelching instead (but on such segments training will not be available). As an example, the handshake signaling function can be based on clause 136 PMD management, with local pattern generation and detection logic. Local clock generation ability is assumed; retimers use recovered clock for transmission if available, or local clock otherwise.
Reference is now made to
Similarly, for Interface B, there is a transmit function 410-B and receive function 412-B, where the transmit function 410-B is coupled to an output 414-B to send signals or data to an adjacent segment and the receive function 412-B is coupled to an input 416-B to receive signals or data from an adjacent segment. For Interface B, there is a switch 420-B, a TF generator 430-B, and a TF decoder 432-B. The switch 420-B is controlled to select either the output of the TF generator 430-B or the output of the receive function 412-A. The TF decoder 432-B is coupled to the output of the receive function 412-B to decode a received training frame.
A training logic block 440 is coupled to the TF generator 430-A and TF decoder 432-A, and is coupled to the TF generator 430-B and TF decoder 432-B. The training logic block 440 generates a signal select control 442-A for switch 420-A and a signal select control 442-B for switch 420-B. The training logic block 440 also generates a squelch control signal 444-A that is coupled to the output 414-A and a squelch control signal 444-B that is coupled to the output 414-B. The training logic block 440 controls the TF generator 430-A to send training frames out on Interface A, and similarly controls the TF generator 430-B to send training frames out on Interface B. The training logic block 440 incorporates the handshake logic 230 of
In the state diagram of
The TF generators, TF decoders and training logic shown in
Reference is now made to
At step 458, a wait for partner state is entered where the training logic block 440 sets a receiver frame lock parameter=1 (true) in the outgoing training frame, and waits for an incoming frame with receiver frame lock=1. When a received frame indicates receiver lock=1, then at step 460, a train segment state is entered during which both sides exchange training frames (on Interface A) until both sides indicate receiver ready (RR). When both sides indicate receiver ready, then at step 462, the RTS value in incoming training frames is relayed to Interface B in outgoing training frames, and the RTS value from Interface B incoming training frames is obtained and sent in outgoing training frames on Interface A. When it is determined that RTS is true on sent and received training frames on Interface A, then at 464, a switch is made to data mode where, after a hold off period of time, the training logic block 440 uses the signal select control 442-A to switch from sending training frames out via the transmit function 410-A to sending data out from Interface A that is received by receive function 412-B from Interface B. Likewise, the training logic block 440 uses the signal select control 442-B to switch from sending training frames out via the transmit function 410-B to sending data out from Interface B that is received by receive function 412-A from Interface A.
Thus,
As explained herein, the first ready-to-send indication indicates that the first segment partner has, on all lanes, valid data and is ready to send data to the device on the first network interface, and the second ready-to-send indication indicates that the second segment partner has, on all lanes, valid data and is ready to send data to the device on the second network interface.
This method may further include: determining when an incoming training frame received from the first segment partner includes an incoming receiver ready indication; and including an outgoing receiver ready indication in an outgoing training frame transmitted to the first segment partner, wherein the steps of first including and the second including are performed when the incoming receiver ready indication is detected in an incoming training frame and when the outgoing receiver ready indication is included in an outgoing training frame.
As explained above, the step of first including may involve including the first ready-to-send indication in an outgoing training frame to be sent to the second segment partner when the device has a receiver ready indication and a received incoming training frame from the first segment partner has a receiver ready indication of the first segment partner on all lanes and a received incoming training frame from the second segment partner has a receiver ready indication of the second segment partner on all lanes.
Likewise, the step of second including may involve including the second ready-to-send indication in an outgoing training frame to the first segment partner when the device has a receiver ready indication and a received incoming training frame from the first segment partner has a receiver ready indication of the first segment partner on all lanes and a received incoming training frame from the second segment partner has a receiver ready indication of the second segment partner on all lanes.
Further, the method may involve, when both the first ready-to-send indication and the second ready-to-send indication are received, after a period of time, switching the first network interface from transmitting outgoing training frames to transmitting data that is received from the second network interface, and switching the second network interface from transmitting outgoing training frames to transmitting data that is received from the first network interface.
In the scenario where the device that is part of the multi-segment network link is an end point (and thus has a single network interface connected to a segment partner), a different method is performed, as depicted in
This method may further involve: determining when an incoming training frame received from the segment partner includes a receiver ready indication, wherein the step of including involves including the ready-to-send indication when the device has a receiver ready indication and a received incoming training frame from the segment partner has a receiver ready indication on all lanes.
This method may further involve, when both the ready-to-send indication is included and a received incoming training frame from the segment partner has a receiver ready indication, after a period of time, switching the network interface from transmitting outgoing training frames to transmitting data.
A device or apparatus that is configured with the components of
Turning to
For Interface B, there is a transmit function 510-B and a receive function 512-B, an output 514-B to send outgoing frames or data, an input 516-B to receive incoming frames or data, a switch 520-B, a pseudorandom binary sequence (PRBS) generator 530-B (or some other binary pattern), and a signal detector block 532-B. The switch 520-B is controlled to select either the output of the PRBS generator 530-B or the output of the receive function 512-A.
A training logic block 540 is coupled to the TF generator 530-A and TF decoder 532-A, and is coupled to the PRBS generator 530-B and signal detect block 532-B. The training logic block 540 generates a signal select control 542-A for switch 520-A and a signal select control 542-B for switch 520-B. The training logic block 540 also generates a squelch control signal 544-A that is coupled to the output 514-A and a squelch control signal 544-B that is coupled to the output 514-B. The training logic block 440 controls the TF generator 530-A to send training frames out on Interface A, and similarly controls the PRBS generator 530-B to send a PRBS pattern out on Interface B. The training logic block 540 incorporates the handshake logic 230 of
If a valid signal is detected from the input 516-B on Interface B, this means that the receive function 512-B is ready, and it also means that the link partner is sending something that is data because there is no training from the partner connected to Interface B, so there is nothing else that link partner could be sending. It is transmitting data and waiting to receive data. When the signal detect block 532-B detects this, this means both Receiver Ready and Ready-to-Send are true. The training logic block 540 processes this accordingly.
If Interface A provides an RTS true, but there is no signal from Interface B, the training logic block 540 will not bring up the link, and instead it will control the PRBS generator 530-B to send a PRBS pattern out to the link partner since the link partner does not understand training frames. It is possible to replace the PRBS generator 530-B with a training frame generator. The link partner does not understand training frames, so it will just ignore the frames anyhow. The PRBS pattern is just a place holder sequence that notifies the link partner that there is something connected locally to Interface B that is ready to send.
The TF generator, TF decoder, signal detector, pattern generator and training logic shown in
Reference is now made to
At step 558, a wait for partner state is entered where the training logic block 540 sets a receiver frame lock parameter=1 (true) in the outgoing training frame, and waits for an incoming training frame with receiver frame lock=1 on Interface A. When a received frame indicates receiver lock=1, then at step 560, a train segment state is entered during which both sides exchange training frames (on Interface A) until both sides indicate receiver ready (RR). When both sides indicate receiver ready, then at step 562, the training logic block 540 unsquelches the output of Interface B and controls the PRBS generator 530-B to send a PRBS pattern or other binary pattern (not necessarily a PRBS pattern) out on Interface B. The training logic block 540 also activates the receiver function 512-B on Interface B. If receiver ready state is discerned on Interface B (by the signal detect block 532-B detecting an incoming signal), then the training logic block 540 causes a RTS=1 to be sent out on Interface A.
When the training logic block 540 determines that RTS is true on both Interfaces (sent and received), then at step 564, the training logic block 540 switches to data mode. First, a hold off time is initiated, and when that expires, the training logic block 540 switches Interface A from sending out training frames to sending out data that is received on Interface B, and switches Interface B from sending a PRBS pattern to sending data received on Interface A.
Thus,
In this method, the first ready-to-send indication indicates that the first segment partner has, on all lanes, valid data and is ready to send data to the device on the first network interface, wherein the second ready-to-send indication indicates that the second segment partner has, on all lanes, valid data, and is ready to send data to the device on the second network interface.
This method may further involve: when the first ready-to-send indication is received and the second ready-to-send indication is included in an outgoing training frame to the first segment partner, after a period of time, switching the first network interface from transmitting outgoing training frames to transmitting data that is received from the second network interface, and switching the second network interface from transmitting the binary pattern to transmitting data that is received from the first network interface.
Moreover, the binary pattern may be a pseudorandom binary sequence.
A device or apparatus that is configured with the components of
Reference is now made to
The signal detector and logic shown in
Operation of the arrangement of
Reference is now made to
Thus, in the methods presented above, the ready-to-send indication may be a single bit that is included in a bit of a status field of a training frame, where a first state of the single bit indicates that no data is available and training is to continue and a second state of the single bit indicates to switch to data mode when training is complete.
Turning now to
In
In
In
In
In
In
In
In
In
In
In
In
In
In
Referring now to
In at least one embodiment, the device 1000 may be any apparatus that may include one or more processor(s) 1002, one or more memory element(s) 1004, storage 1006, a bus 1008, one or more network processor unit(s) 1010 interconnected with one or more network input/output (I/O) interface(s) 1012, one or more I/O interface(s) 1014, and control logic 1020. In various embodiments, instructions associated with logic for device 1000 can overlap in any manner and are not limited to the specific allocation of instructions and/or operations described herein. Many of the components described herein may be implemented as part of the network processor unit(s) 1010, in digital logic gates, such as part of one or more integrated circuits.
In at least one embodiment, processor(s) 1002 is/are at least one hardware processor configured to execute various tasks, operations and/or functions for device 1000 as described herein according to software and/or instructions configured for device 1000. Processor(s) 1002 (e.g., a hardware processor) can execute any type of instructions associated with data to achieve the operations detailed herein. In one example, processor(s) 1002 can transform an element or an article (e.g., data, information) from one state or thing to another state or thing. Any of potential processing elements, microprocessors, digital signal processor, baseband signal processor, modem, PHY, controllers, systems, managers, logic, and/or machines described herein can be construed as being encompassed within the broad term ‘processor’.
In at least one embodiment, memory element(s) 1004 and/or storage 1006 is/are configured to store data, information, software, and/or instructions associated with device 1000, and/or logic configured for memory element(s) 1004 and/or storage 1006. For example, any logic described herein (e.g., control logic 1020) can, in various embodiments, be stored for device 1000 using any combination of memory element(s) 1004 and/or storage 1006. Note that in some embodiments, storage 1006 can be consolidated with memory element(s) 1004 (or vice versa), or can overlap/exist in any other suitable manner. The control logic 1020 may include executable instructions that enable the processor(s) 1002 to perform one or more portions of the methods presented herein, so that the processor may serve as a “controller” for an apparatus that includes the components and capabilities described above in connection with
In one form, the operations described herein may be embodied in an apparatus that includes a memory; and one or more integrated circuits configured with digital logic, or a processor device configured with instructions, to perform operations including: on a network interface facing a segment partner, receiving incoming training frames from the segment partner and transmitting outgoing training frames to the segment partner; and including a ready-to-send indication in an outgoing training frame to be sent to the segment partner, wherein the ready-to-send indication indicates that the device has, on all lanes of the network interface, valid data and is ready to send data on the network interface to the segment partner.
Similarly, the operations described herein may be embodied in an apparatus that includes a memory; and one or more integrated circuits configured with digital logic, or a processor device configured with instructions, to perform operations including: on a first network interface facing a first segment partner, receiving incoming training frames from the first segment partner and transmitting outgoing training frames to the first segment partner; on a second network interface facing a second segment partner, receiving incoming training frames from the second segment partner and transmitting outgoing training frames to the second segment partner; first including a first ready-to-send indication, when the first ready-to-send indication is included in an incoming training frame received from the first segment partner, in an outgoing training frame to be sent to the second segment partner; and second including a second ready-to-send indication, when the second ready-to-send indication is included in an incoming training frame received from the second segment partner, in an outgoing training frame to be sent to the first segment partner.
Further still, the operations described herein may be embodied in an apparatus that includes a memory; and one or more integrated circuits configured with digital logic, or a processor device configured with instructions, to perform operations including: disabling a first transmitter on a first network interface of the device for a period of time; disabling a second transmitter on a second network interface of the device; on a first network interface facing a first segment partner, receiving incoming training frames from the first segment partner; after the period of time, enabling the first transmitter on the first network interface and transmitting outgoing training frames to the first segment partner; when a first ready-to-send indication is included in an incoming training frame received from the first segment partner, enabling the second transmitter and transmitting a binary pattern from a second network interface to a second segment partner; and when a receiver ready indication is determined for the second segment partner, including a second ready-to-send indication in an outgoing training frame transmitted to the first segment partner.
In at least one embodiment, bus 1008 can be configured as an interface that enables one or more elements of device 1000 to communicate in order to exchange information and/or data. Bus 1008 can be implemented with any architecture designed for passing control, data and/or information between processors, memory elements/storage, peripheral devices, and/or any other hardware and/or software components that may be configured for device 1000. In at least one embodiment, bus 1008 may be implemented as a fast kernel-hosted interconnect, potentially using shared memory between processes (e.g., logic), which can enable efficient communication paths between the processes.
In various embodiments, network processor unit(s) 1010 may enable communication between device 1000 and other systems, entities, etc., via network I/O interface(s) 1012 (wired and/or wireless, e.g., ports or interfaces) to facilitate operations discussed for various embodiments described herein. In various embodiments, network processor unit(s) 1010 can be configured to perform the network communication techniques presented herein as a combination of hardware and/or software, such as one or more Ethernet driver(s) and/or controller(s) or interface cards, Fibre Channel (e.g., optical) driver(s) and/or controller(s), wireless receivers/transmitters/transceivers, baseband processor(s)/modem(s), and/or other similar network interface driver(s) and/or controller(s) now known or hereafter developed to enable communications between device 1000 and other systems, entities, etc. to facilitate operations for various embodiments described herein. In various embodiments, network I/O interface(s) 1012 can be configured as one or more Ethernet port(s), Fibre Channel ports, any other I/O port(s), and/or antenna(s)/antenna array(s) now known or hereafter developed. Thus, the network processor unit(s) 1010 and/or network I/O interface(s) 1012 may include suitable interfaces for receiving, transmitting, and/or otherwise communicating data and/or information in a network environment.
I/O interface(s) 1014 allow for input and output of data and/or information with other entities that may be connected to device 1000. For example, I/O interface(s) 1014 may provide a connection to external devices such as a keyboard, keypad, a touch screen, and/or any other suitable input and/or output device now known or hereafter developed. In some instances, external devices can also include portable computer readable (non-transitory) storage media such as database systems, thumb drives, portable optical or magnetic disks, and memory cards. In still some instances, external devices can be a mechanism to display data to a user, such as, for example, a computer monitor, a display screen, or the like.
In various embodiments, control logic 1020 can include instructions that, when executed, cause processor(s) 1002 to perform operations, which can include, but not be limited to, providing overall control operations of computing device; interacting with other entities, systems, etc. described herein; maintaining and/or interacting with stored data, information, parameters, etc. (e.g., memory element(s), storage, data structures, databases, tables, etc.); combinations thereof; and/or the like to facilitate various operations for embodiments described herein.
The programs described herein (e.g., control logic 1020) may be identified based upon application(s) for which they are implemented in a specific embodiment. However, it should be appreciated that any particular program nomenclature herein is used merely for convenience; thus, embodiments herein should not be limited to use(s) solely described in any specific application(s) identified and/or implied by such nomenclature.
In various embodiments, any entity or apparatus as described herein may store data/information in any suitable volatile and/or non-volatile memory item (e.g., magnetic hard disk drive, solid state hard drive, semiconductor storage device, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM), application specific integrated circuit (ASIC), etc.), software, logic (fixed logic, hardware logic, programmable logic, analog logic, digital logic), hardware, and/or in any other suitable component, device, element, and/or object as may be appropriate. Any of the memory items discussed herein should be construed as being encompassed within the broad term ‘memory element’. Data/information being tracked and/or sent to one or more entities as discussed herein could be provided in any database, table, register, list, cache, storage, and/or storage structure: all of which can be referenced at any suitable timeframe. Any such storage options may also be included within the broad term ‘memory element’ as used herein.
Note that in certain example implementations, operations as set forth herein may be implemented by logic encoded in one or more tangible media that is capable of storing instructions and/or digital information and may be inclusive of non-transitory tangible media and/or non-transitory computer readable storage media (e.g., embedded logic provided in: an ASIC, digital signal processing (DSP) instructions, software [potentially inclusive of object code and source code], etc.) for execution by one or more processor(s), and/or other similar machine, etc. Generally, memory element(s) 1004 and/or storage 1006 can store data, software, code, instructions (e.g., processor instructions), logic, parameters, combinations thereof, and/or the like used for operations described herein. This includes memory element(s) 1004 and/or storage 1006 being able to store data, software, code, instructions (e.g., processor instructions), logic, parameters, combinations thereof, or the like that are executed to carry out operations in accordance with teachings of the present disclosure.
In some instances, software of the present embodiments may be available via a non-transitory computer useable medium (e.g., magnetic or optical mediums, magneto-optic mediums, CD-ROM, DVD, memory devices, etc.) of a stationary or portable program product apparatus, downloadable file(s), file wrapper(s), object(s), package(s), container(s), and/or the like. In some instances, non-transitory computer readable storage media may also be removable. For example, a removable hard drive may be used for memory/storage in some implementations. Other examples may include optical and magnetic disks, thumb drives, and smart cards that can be inserted and/or otherwise connected to a computing device for transfer onto another computer readable storage medium.
In some aspects, the techniques described herein relate to a method performed by a device that is part of a multi-segment network link, the method including: on a network interface facing a segment partner, receiving incoming training frames from the segment partner and transmitting outgoing training frames to the segment partner; and including a ready-to-send indication in an outgoing training frame to be sent to the segment partner, wherein the ready-to-send indication indicates that the device has, on all lanes of the network interface, valid data and is ready to send data on the network interface to the segment partner.
In some aspects, the techniques described herein relate to a method, further including: determining when an incoming training frame received from the segment partner includes a receiver ready indication, wherein including includes including the ready-to-send indication when the device has a receiver ready indication and a received incoming training frame from the segment partner has a receiver ready indication on all lanes.
In some aspects, the techniques described herein relate to a method, wherein the ready-to-send indication is a single bit that is included in a bit of a status field of a training frame, a first state of the single bit indicating that no data is available and training is to continue and a second state of the single bit indicating to switch to data mode when training is complete.
In some aspects, the techniques described herein relate to a method, further including, when both the ready-to-send indication is included and a received incoming training frame from the segment partner has a receiver ready indication, after a period of time, switching the network interface from transmitting outgoing training frames to transmitting data.
In some aspects, the techniques described herein relate to a method performed by a device that is part of a multi-segment network link, the method including: on a first network interface facing a first segment partner, receiving incoming training frames from the first segment partner and transmitting outgoing training frames to the first segment partner; on a second network interface facing a second segment partner, receiving incoming training frames from the second segment partner and transmitting outgoing training frames to the second segment partner; first including a first ready-to-send indication, when the first ready-to-send indication is included in an incoming training frame received from the first segment partner, in an outgoing training frame to be sent to the second segment partner; and second including a second ready-to-send indication, when the second ready-to-send indication is included in an incoming training frame received from the second segment partner, in an outgoing training frame to be sent to the first segment partner.
In some aspects, the techniques described herein relate to a method, wherein the first ready-to-send indication indicates that the first segment partner has, on all lanes, valid data and is ready to send data to the device on the first network interface, and the second ready-to-send indication indicates that the second segment partner has, on all lanes, valid data and is ready to send data to the device on the second network interface.
In some aspects, the techniques described herein relate to a method, further including: determining when an incoming training frame received from the first segment partner includes an incoming receiver ready indication; and including an outgoing receiver ready indication in an outgoing training frame transmitted to the first segment partner, wherein the first including and the second including are performed when the incoming receiver ready indication is detected in an incoming training frame and when the outgoing receiver ready indication is included in an outgoing training frame.
In some aspects, the techniques described herein relate to a method, wherein the first including includes including the first ready-to-send indication in an outgoing training frame to be sent to the second segment partner when the device has a receiver ready indication and a received incoming training frame from the first segment partner has a receiver ready indication of the first segment partner on all lanes and a received incoming training frame from the second segment partner has a receiver ready indication of the second segment partner on all lanes.
In some aspects, the techniques described herein relate to a method, wherein the second including includes including the second ready-to-send indication in an outgoing training frame to the first segment partner when the device has a receiver ready indication and a received incoming training frame from the first segment partner has a receiver ready indication of the first segment partner on all lanes and a received incoming training frame from the second segment partner has a receiver ready indication of the second segment partner on all lanes.
In some aspects, the techniques described herein relate to a method, wherein the first ready-to-send indication is a single bit that is included in a bit of a status field of a training frame, a first state of the single bit indicating that no data is available and training is to continue and a second state of the single bit indicating to switch to data mode when training is complete.
In some aspects, the techniques described herein relate to a method, further including: when both the first ready-to-send indication and the second ready-to-send indication are received, after a period of time, switching the first network interface from transmitting outgoing training frames to transmitting data that is received from the second network interface, and switching the second network interface from transmitting outgoing training frames to transmitting data that is received from the first network interface.
In some aspects, the techniques described herein relate to a method performed by a device that is part of a multi-segment network link, the method including: disabling a first transmitter on a first network interface of the device for a period of time; disabling a second transmitter on a second network interface of the device; on a first network interface facing a first segment partner, receiving incoming training frames from the first segment partner; after the period of time, enabling the first transmitter on the first network interface and transmitting outgoing training frames to the first segment partner; when a first ready-to-send indication is included in an incoming training frame received from the first segment partner, enabling the second transmitter and transmitting a binary pattern from a second network interface to a second segment partner; and when a receiver ready indication is determined for the second segment partner, including a second ready-to-send indication in an outgoing training frame transmitted to the first segment partner.
In some aspects, the techniques described herein relate to a method, wherein the first ready-to-send indication indicates that the first segment partner has, on all lanes, valid data and is ready to send data to the device on the first network interface, wherein the second ready-to-send indication indicates that the second segment partner has, on all lanes, valid data, and is ready to send data to the device on the second network interface.
In some aspects, the techniques described herein relate to a method, wherein the first ready-to-send indication is a single bit that is included in a bit of a status field of a training frame, a first state of the single bit indicating that no data is available and training is to continue and a second state of the single bit indicating to switch to data mode when training is complete.
In some aspects, the techniques described herein relate to a method, further including: when the first ready-to-send indication is received and the second ready-to-send indication is included in an outgoing training frame to the first segment partner, after a period of time, switching the first network interface from transmitting outgoing training frames to transmitting data that is received from the second network interface, and switching the second network interface from transmitting the binary pattern to transmitting data that is received from the first network interface.
In some aspects, the techniques described herein relate to a method, wherein the binary pattern is a pseudorandom binary sequence.
In some aspects, the techniques described herein relate to an apparatus including: a first network interface; a first transmitter and a first receiver associated with the first network interface; a second network interface; a second transmitter and a second receiver associated with the second network interface; a first training frame generator and a first training frame decoder associated with the first network interface; a second training frame generator and a second training frame decoder associated with the second network interface; and a controller coupled to the first training frame generator, the first training frame decoder, the second training frame generator and the second training frame decoder, wherein the controller is configured to: cause the first training frame generator to transmit outgoing training frames to a first segment partner via the first network interface and obtain training frame content obtained by the first training frame decoder from incoming training frames received from the first segment partner; cause the second training frame generator to transmit outgoing training frames to a second segment partner via the second network interface and obtain training frame content obtained by the second training frame decoder from incoming training frames received from the second segment partner; when a first ready-to-send indication is determined to be included in an incoming training frame received from the first segment partner, cause the second training frame generator to include the first ready-to-send indication in an outgoing training frame to be sent to the second segment partner; and when a second ready-to-send indication is determined to be included in an incoming training frame received from the second segment partner, cause the first training frame generator to include the second ready-to-send indication in an outgoing training frame to be sent to the first segment partner.
In some aspects, the techniques described herein relate to an apparatus, wherein the first ready-to-send indication indicates that the first segment partner has, on all lanes, valid data and is ready to send data to the apparatus on the first network interface, and the second ready-to-send indication indicates that the second segment partner has, on all lanes, valid data and is ready to send data to the apparatus on the second network interface.
In some aspects, the techniques described herein relate to an apparatus, wherein the controller is further configured to: determine when an incoming training frame received from the first segment partner includes an incoming receiver ready indication; and cause the first training frame generator to include an outgoing receiver ready indication in an outgoing training frame transmitted to the first segment partner, wherein the controller controls the first training frame generator to include the first ready-to-send indication and controls the second training frame generator to include the second ready-to-send indication when the incoming receiver ready indication is detected in an incoming training frame and when the outgoing receiver ready indication is included in an outgoing training frame.
In some aspects, the techniques described herein relate to an apparatus, wherein the controller causes the first training frame generator to include the first ready-to-send indication in an outgoing training frame to be sent to the second segment partner when the apparatus has a receiver ready indication and a received incoming training frame from the first segment partner has a receiver ready indication of the first segment partner on all lanes and a received incoming training frame from the second segment partner has a receiver ready indication of the second segment partner on all lanes.
In some aspects, the techniques described herein relate to an apparatus, wherein the controller causes the second training frame generator to include the second ready-to-send indication in an outgoing training frame to the first segment partner when the apparatus has a receiver ready indication and a received incoming training frame from the first segment partner has a receiver ready indication of the first segment partner on all lanes and a received incoming training frame from the second segment partner has a receiver ready indication of the second segment partner on all lanes.
In some aspects, the techniques described herein relate to an apparatus, wherein the first ready-to-send indication is a single bit that is included in a bit of a status field of a training frame, a first state of the single bit indicating that no data is available and training is to continue and a second state of the single bit indicating to switch to data mode when training is complete.
In some aspects, the techniques described herein relate to an apparatus, wherein the controller is configured to: when both the first ready-to-send indication and the second ready-to-send indication are received, after a period of time, switch the first network interface from transmitting outgoing training frames to transmitting data that is received from the second network interface, and switch the second network interface from transmitting outgoing training frames to transmitting data that is received from the first network interface.
In some aspects, the techniques described herein relate to an apparatus including: a first network interface; a first transmitter and a first receiver associated with the first network interface; a second network interface; a second transmitter and a second receiver associated with the second network interface; a training frame generator and a training frame decoder associated with the first network interface; a binary pattern generator and a signal detector associated with the second network interface; and a controller coupled to the training frame generator, the training frame decoder, the binary pattern generator and the signal detector, wherein the controller is configured to: disable the first transmitter on the first network interface for a period of time; disable the second transmitter on the second network interface; cause the training frame generator to obtain training frame content obtained by the training frame decoder from incoming training frames received from a first segment partner; after the period of time, enable the first transmitter on the first network interface and cause the training frame generator to transmit outgoing training frames to the first segment partner; when a first ready-to-send indication is determined to be included in an incoming training frame received from the first segment partner, enable the second transmitter and cause the binary pattern generator to generate a binary pattern to be transmitted via the second network interface to a second segment partner; and cause the training frame generator to include a second ready-to-send indication in an outgoing training frame to be transmitted to the first segment partner when a receiver ready indication is determined for the second segment partner based on an output from the signal detector.
In some aspects, the techniques described herein relate to an apparatus, wherein the first ready-to-send indication indicates that the first segment partner has, on all lanes, valid data and is ready to send data to the apparatus on the first network interface, wherein the second ready-to-send indication indicates that the second segment partner has, on all lanes, valid data, and is ready to send data to the apparatus on the second network interface.
In some aspects, the techniques described herein relate to an apparatus, wherein the first ready-to-send indication is a single bit that is included in a bit of a status field of a training frame, a first state of the single bit indicating that no data is available and training is to continue and a second state of the single bit indicating to switch to data mode when training is complete.
In some aspects, the techniques described herein relate to an apparatus, wherein when both the first ready-to-send indication is received and the second ready-to-send indication are is included in an outgoing training frame to the first segment partner, after a period of time, the controller switches the first network interface from transmitting outgoing training frames to transmitting data that is received from the second network interface, and switches the second network interface from transmitting the binary pattern to transmitting data that is received from the first network interface.
In some aspects, the techniques described herein relate to an apparatus, wherein the binary pattern is a pseudorandom binary sequence.
Further, in some aspects, one or more non-transitory computer readable storage media may be provided, that is encoded with instructions, that when executed by one or more processors, cause the one or processors to perform operations of the methods presented herein.
Embodiments described herein may include one or more networks, which can represent a series of points and/or network elements of interconnected communication paths for receiving and/or transmitting messages (e.g., packets of information) that propagate through the one or more networks. These network elements offer communicative interfaces that facilitate communications between the network elements. A network can include any number of hardware and/or software elements coupled to (and in communication with) each other through a communication medium. Such networks can include, but are not limited to, any local area network (LAN), virtual LAN (VLAN), wide area network (WAN) (e.g., the Internet), software defined WAN (SD-WAN), wireless local area (WLA) access network, wireless wide area (WWA) access network, metropolitan area network (MAN), Intranet, Extranet, virtual private network (VPN), Low Power Network (LPN), Low Power Wide Area Network (LPWAN), Machine to Machine (M2M) network, Internet of Things (IoT) network, Ethernet network/switching system, any other appropriate architecture and/or system that facilitates communications in a network environment, and/or any suitable combination thereof.
Networks through which communications propagate can use any suitable technologies for communications including wireless communications (e.g., 4G/5G/nG, IEEE 802.11 (e.g., Wi-Fi®/Wi-Fi6®), IEEE 802.16 (e.g., Worldwide Interoperability for Microwave Access (WiMAX)), Radio-Frequency Identification (RFID), Near Field Communication (NFC), Bluetooth™ mm.wave, Ultra-Wideband (UWB), etc.), and/or wired communications (e.g., T1 lines, T3 lines, digital subscriber lines (DSL), Ethernet, Fibre Channel, etc.). Generally, any suitable means of communications may be used such as electric, sound, light, infrared, and/or radio to facilitate communications through one or more networks in accordance with embodiments herein. Communications, interactions, operations, etc. as discussed for various embodiments described herein may be performed among entities that may directly or indirectly connected utilizing any algorithms, communication protocols, interfaces, etc. (proprietary and/or non-proprietary) that allow for the exchange of data and/or information.
In various example implementations, any entity or apparatus for various embodiments described herein can encompass network elements (which can include virtualized network elements, functions, etc.) such as, for example, network appliances, forwarders, routers, servers, switches, gateways, bridges, loadbalancers, firewalls, processors, modules, radio receivers/transmitters, or any other suitable device, component, element, or object operable to exchange information that facilitates or otherwise helps to facilitate various operations in a network environment as described for various embodiments herein. Note that with the examples provided herein, interaction may be described in terms of one, two, three, or four entities. However, this has been done for purposes of clarity, simplicity and example only. The examples provided should not limit the scope or inhibit the broad teachings of systems, networks, etc. described herein as potentially applied to a myriad of other architectures.
Communications in a network environment can be referred to herein as ‘messages’, ‘messaging’, ‘signaling’, ‘data’, ‘content’, ‘objects’, ‘requests’, ‘queries’, ‘responses’, ‘replies’, etc. which may be inclusive of packets. As referred to herein and in the claims, the term ‘packet’ may be used in a generic sense to include packets, frames, segments, datagrams, and/or any other generic units that may be used to transmit communications in a network environment. Generally, a packet is a formatted unit of data that can contain control or routing information (e.g., source and destination address, source and destination port, etc.) and data, which is also sometimes referred to as a ‘payload’, ‘data payload’, and variations thereof. In some embodiments, control or routing information, management information, or the like can be included in packet fields, such as within header(s) and/or trailer(s) of packets. Internet Protocol (IP) addresses discussed herein and in the claims can include any IP version 4 (IPv4) and/or IP version 6 (IPv6) addresses.
To the extent that embodiments presented herein relate to the storage of data, the embodiments may employ any number of any conventional or other databases, data stores or storage structures (e.g., files, databases, data structures, data or other repositories, etc.) to store information.
Note that in this Specification, references to various features (e.g., elements, structures, nodes, modules, components, engines, logic, steps, operations, functions, characteristics, etc.) included in ‘one embodiment’, ‘example embodiment’, ‘an embodiment’, ‘another embodiment’, ‘certain embodiments’, ‘some embodiments’, ‘various embodiments’, ‘other embodiments’, ‘alternative embodiment’, and the like are intended to mean that any such features are included in one or more embodiments of the present disclosure, but may or may not necessarily be combined in the same embodiments. Note also that a module, engine, client, controller, function, logic or the like as used herein in this Specification, can be inclusive of an executable file comprising instructions that can be understood and processed on a server, computer, processor, machine, compute node, combinations thereof, or the like and may further include library modules loaded during execution, object files, system files, hardware logic, software logic, or any other executable modules.
It is also noted that the operations and steps described with reference to the preceding figures illustrate only some of the possible scenarios that may be executed by one or more entities discussed herein. Some of these operations may be deleted or removed where appropriate, or these steps may be modified or changed considerably without departing from the scope of the presented concepts. In addition, the timing and sequence of these operations may be altered considerably and still achieve the results taught in this disclosure. The preceding operational flows have been offered for purposes of example and discussion. Substantial flexibility is provided by the embodiments in that any suitable arrangements, chronologies, configurations, and timing mechanisms may be provided without departing from the teachings of the discussed concepts.
As used herein, unless expressly stated to the contrary, use of the phrase ‘at least one of,’ one or more of, ‘and/or’, variations thereof, or the like are open-ended expressions that are both conjunctive and disjunctive in operation for any and all possible combination of the associated listed items. For example, each of the expressions ‘at least one of X, Y and Z’, ‘at least one of X, Y or Z’, ‘one or more of X, Y and Z’, ‘one or more of X, Y or Z’ and ‘X, Y and/or Z’ can mean any of the following: 1) X, but not Y and not Z; 2) Y, but not X and not Z; 3) Z, but not X and not Y; 4) X and Y, but not Z; 5) X and Z, but not Y; 6) Y and Z, but not X; or 7) X, Y, and Z.
Each example embodiment disclosed herein has been included to present one or more different features. However, all disclosed example embodiments are designed to work together as part of a single larger system or method. This disclosure explicitly envisions compound embodiments that combine multiple previously-discussed features in different example embodiments into a single system or method.
Additionally, unless expressly stated to the contrary, the terms ‘first’, ‘second’, ‘third’, etc., are intended to distinguish the particular nouns they modify (e.g., element, condition, node, module, activity, operation, etc.). Unless expressly stated to the contrary, the use of these terms is not intended to indicate any type of order, rank, importance, temporal sequence, or hierarchy of the modified noun. For example, ‘first X’ and ‘second X’ are intended to designate two ‘X’ elements that are not necessarily limited by any order, rank, importance, temporal sequence, or hierarchy of the two elements. Further as referred to herein, ‘at least one of’ and ‘one or more of’ can be represented using the ‘(s)’ nomenclature (e.g., one or more element(s)).
One or more advantages described herein are not meant to suggest that any one of the embodiments described herein necessarily provides all of the described advantages or that all the embodiments of the present disclosure necessarily provide any one of the described advantages. Numerous other changes, substitutions, variations, alterations, and/or modifications may be ascertained to one skilled in the art and it is intended that the present disclosure encompass all such changes, substitutions, variations, alterations, and/or modifications as falling within the scope of the appended claims.
This application claims priority to U.S. Provisional Application No. 63/594,154, filed Oct. 30, 2023, the entirety of which is incorporated by reference.
| Number | Date | Country | |
|---|---|---|---|
| 63594154 | Oct 2023 | US |
