Patent Grant
12170619
Many network control loops, such as congestion control, traffic engineering and network operations, make decisions based on the congestion experienced by application flows. However, the signals currently used to determine congestion are often implicitly derived from end-to-end signals, approximated over larger timescales than desired, or obtained out-of-band from the network, which leads to poor congestion control and the attendant drawbacks of suboptimal application performance and inefficient network usage. At the same time, applications are becoming more demanding, and their appetite for fast network performance is rising. For example, many newer artificial intelligence/machine learning (AI/ML) applications require fast network transfers to avoid idling expensive tensor processing units (TPUs) and graphics processing units (GPUs). Similarly, storage disaggregation requires fast network transfers to make a remote storage device appear local.
It has been recognized that without explicit information from networks, end-to-end congestion control algorithms (CCAs) have come to rely on heuristics that can either undershoot or overshoot the bottleneck bandwidth, which can lead to slower flow completion times (FCTs), increased round-trip times (RTTs), and/or packet losses. It has been further recognized that there continue to be blind spots for current CCAs regarding opportunities to increase flow rates. For example, current CCAs are deficient as to determining the appropriate starting rate for a flow and the rate at which flow can increase without experiencing congestion.
In view of the desire for improved CCAs, the presently disclosed technology is provided.
In accordance with the presently disclosed technology, a network node is configured to enable congestion control of network traffic using congestion signaling across a variety of hosts executing congestion control algorithms relying on provided congestion signals. The configured node, e.g., a network interface card (NIC) of a data center or host machine, can facilitate congestion control to reduce network bottlenecks, while allowing the network to communicate at line rate. The technology provides compare-and-replace support for congestion signal values that are reported in the same packets as that of a running application, so as to maintain the size of the packets through which the values are reported and thereby minimize the burden that congestion control signaling places on the network. In addition, the technology provides access to a variety of different congestion signals that may be used as input by implemented CCAs, to manage packets on a per-connection basis, or even a per-packet basis. Apropos, a node can make decisions on congestion control using selected signals which provide information related to, for example, a minimum available path bandwidth and load, and max hop delay.
In one aspect, the presently disclosed technology provides a congestion control system including one or more processors of a first node, the one or more processors configured to control adding a congestion signal tag header to each of one or more transmission packets prior to transmission of the transmission packets by the first node to a second node, the congestion signal tag header specifying one or more congestion signal types and, for each of the congestion signal types, specifying a congestion signal value by providing an initial congestion signal value for the congestion signal value; receiving one or more return packets generated by the second node in response to receipt of the transmission packets, the return packets including a congestion signal reflection header having one or more return congestion signal values, and the return congestion signal values corresponding respectively to the congestion signal types; determining whether transmission rate control is necessary based on the return congestion signal values; and when transmission rate control is necessary, controlling a transmission rate for transmission of packets from the first node to the second node.
In another aspect, the present disclosure provides a congestion control method including adding a congestion signal tag header to each of one or more transmission packets prior to transmission of the transmission packets by the first node to a second node, the congestion signal tag header specifying one or more congestion signal types and, for each of the congestion signal types, specifying a congestion signal value by providing an initial congestion signal value for the congestion signal value; receiving one or more return packets generated by the second node in response to receipt of the transmission packets, the return packets including a congestion signal reflection header having one or more return congestion signal values, and the return congestion signal values corresponding respectively to the congestion signal types; determining whether transmission rate control is necessary based on the return congestion signal values; and when transmission rate control is necessary, controlling a transmission rate for transmission of packets from the first node to the second node.
The accompanying drawings are not intended to be drawn to scale. Also, for purposes of clarity not every component may be labeled in every drawing. In the drawings:
The currently disclosed technology concerns network congestion control through network congestion signals provided to network nodes at line rate. To this end, congestion signals of the technology are compare-and-replace style signals that are provided to nodes in the same packets as that of a running application. For example, congestion signals of the technology may be provided to a NIC hardware (H/W) transport within a server. In this manner, the NIC can enable support for congestion signaling on every packet, in hardware. Moreover, the congestion signal types for a connection between the NIC and another node may be determined by a rate update engine (RUE), and the NIC may receive and transmit the congestion signals from H/W to the RUE. The RUE may be implemented in hardware, software, or a combination of the two. The RUE may be programmable.
Using the presently disclosed technology, communication between network nodes can ramp up quickly to maximally use all network bandwidth and to complete close to the ideal time. For instance, the presently disclosed technology can provide for improved execution of transfer of data, for example in support of various workloads, including machine learning (ML) accelerated tasks. Further, a NIC as configured herein can enable multipath flows to choose paths with the most available bandwidth. The NIC can also offload valuable information for traffic engineering and/or debugging purposes, such as operational bottlenecks within a network.
Traffic Engineering (TE) also benefits from congestion control as described herein. In accordance with the presently disclosed technology, a NIC can identify congested points and flows experiencing congestion right away, which in turn can lead to more efficient and timely provisioning for bursty traffic. By contrast, inferring the congested flows through an offline process via superimposition of network traffic stats, topological information and routing information, has been a much longer process.
Aspects of the presently disclosed technology can also assist with debugging network level performance of datacenter applications. For instance, large scale applications including ML training workloads open thousands of connections at the transport layer, and upon a network slowdown identifying the bottleneck hops without joining many data sources across switches and hosts has been extremely difficult. However, with the present technology, because a node can receive path bottleneck characteristics, network choke points can be promptly identified which, in turn, leads to better bandwidth provisioning, timely repair processes, etc.
Technical advantages of the presently disclosed technology include explicit congestion signaling within a fleet of interconnected devices. The technology also allows for signaling of minimum available path bandwidth (or “bottleneck bandwidth”) and load, which can be used for telemetry and congestion control purposes. In addition, the presently disclosed technology also provides for compare-and-replace support for congestion signaling between nodes at line rate, as well as reduced overhead for implementation relative to prior congestion control signaling techniques.
In some examples, programmable rate update engines can be adapted to receiving and processing diverse signal types. Support for congestion control (CC) and telemetry/debugging may be implemented in hardware, obviating the need to specify support in software. A rate update engine implemented as part of a NIC may be configured to support different signals specified by a congestion signal packet tag, on a per-connection basis, or a per-packet basis. In other words, the NIC may handle congestion control according to different parameters provided by tagged packets handled across multiple different connections. The congestion control is programmable and allows for efficiently communicating signals from a hardware transport to programmable software (S/W). The rate update engine can apply different signal types to outgoing packets.
In accordance with the presently disclosed technology, a NIC can be configured on a per-connection basis to support diverse types of congestion signals. The NIC can include an option to not provide congestion signals to outgoing packets, allowing the NIC to perform sampling within RTT time scales or to not insert congestion signals when the recipient device does not support the same type of congestion signaling.
A NIC according to the presently disclosed technology can support an array of entries corresponding to different congestion signal types. The NIC can dynamically choose indexes of the array of entries to enable congestion signaling of a particular type, and specify an initial value for the congestion signal. Once the NIC indicates an entry which is not equivalent to turning off congestion signaling, then the NIC would add in corresponding headers on every outgoing packet. Along the same lines, the NIC can disable congestion signaling by choosing the appropriate index.
Congestion signaling according to the presently disclosed technology may be used in a NIC/RUE on a per-connection basis for purposes of congestion control, telemetry, multipathing, load balancing and debuggability. The NIC/RUE can also make congestion signaling visible to an application stack, perhaps in some summarized way. Similarly, the NIC/RUE can summarize and make congestion signals visible to software-defined networking (SDN) systems, such as a traffic engineering controller.
Example Systems
Examples of systems and methods are described herein. It should be understood that the words “example.” “exemplary” and “illustrative” are used herein to mean “serving as an example, instance, or illustration.” Any embodiment or feature described herein as being an “example,” “exemplary” or “illustration” is not necessarily to be construed as preferred or advantageous over other embodiments or features. In the following description, reference is made to the accompanying figures, which form a part thereof. In the figures, similar symbols typically identify similar components, unless context dictates otherwise. Other embodiments may be utilized, and other changes may be made, without departing from the spirit or scope of the subject matter presented herein.
The example embodiments described herein are not meant to be limiting. It will be readily understood that the aspects of the present disclosure, as generally described herein, and illustrated in the figures, can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations, all of which are explicitly contemplated herein.
The memory 114, 124 can store information accessible by the processor 112, 122, including data that can be retrieved, manipulated or stored by the processor, instructions that can be executed by the processor, or a combination thereof. The memory 114, 124 may be a type of non-transitory computer readable medium capable of storing information accessible by the processor 110, 120 such as a hard-drive, solid state drive, tape drive, optical storage, memory card, ROM, RAM, DVD, CD-ROM, write-capable, or read-only memories. In some examples, the memory 114, 124 is a type of physically stacked memory, or a type of high bandwidth memory (HBM).
Although
The communication device 116, 126 for each node of the communication network may facilitate communication between the node and other remote devices that are in communication with the node. The remote devices may include other nodes of the communication network 100, one or more user devices in communication with the node, or any combination thereof. The communication device 116, 126 may be capable of transmitting data to and from other computers such as modems (e.g., dial-up, cable or fiber optic) and wireless interfaces. For example, each node may receive communications via the network connection 130, such as through the Internet, World Wide Web, intranets, virtual private networks, wide area networks, local networks, private networks using communication protocols proprietary to one or more companies, Ethernet, WiFi (e.g., 702.71, 702.71b, g, n, or other such standards), and RPC, HTTP, and various combinations of the foregoing.
The network connection 130 between the nodes 110120 of the communication network 100 may support multiple layers of communication protocols implemented by the respective processors 112, 122 of the nodes. For example, each node may support communication using an open systems interconnection (OSI) model. Further, congestion control signaling for an OSI connection between the nodes 110 and 120 may be implemented in packets of one or more running applications by tagging packets at various layers of the OSI connection between nodes 110 and 120.
As can be seen from
The return packets received from the second node over the incoming stream 202 and stored at the data cache 210 may be analyzed to determine whether the rate of packet transmission to the second node should be updated. The analysis may be divided into two distinct stages. At a first stage, a determination is made as to which of the return congestion signal values should be selected for use in determining whether to initiate a rate update event. This first stage may be implemented using one or more computing devices 230 included in hardware of the first node and communicatively coupled to the main data path 201. At a second stage, a determination is made, based on the selected return congestion signal values, as to whether transmission rate control is necessary. When transmission rate control is necessary, the second stage controls a transmission rate for transmission of packets from the first node to the second node.
The second stage may be implemented using a rate update engine 240 that is separate from the main data path 201. For example, data from the incoming stream 201 may be stored separately from the data cache 210, and the rate update engine 240 may analyze the separately stored data in order to determine the updated parameters for data packet transmission. Separating the congestion control process between these two stages helps to offload processing from the main data path, and thus improves performance over the main data path. Other advantages of offloading the congestion control process include increased flexibility in defining the congestion control algorithm, and in at least some examples increased flexibility in controlling whether rate update events are initiated. These and other aspects of the two-stage congestion control process are described in greater detail herein.
It should be noted that in some embodiments both the first stage and second stage of return packet analysis may be performed by the RUE 240. That is, the RUE may be configured to determine which of the return congestion signal values should be selected for use in determining whether to initiate a rate update event, determine whether transmission rate control is necessary, and control the transmission rate for transmission of packets from the first node to the second node. In such embodiments the one or more computing devices 230 are not needed.
Having described the congestion control system 200 of
Two components in the packet header are used to achieve end-to-end congestion signaling and network congestion control. One component is the congestion signal tag header. The other component is the congestion signal reflection header. In some embodiments the congestion signal tag header is a header for a first layer protocol of an OSI model of a network, and the congestion signal reflection header is a header for a second layer protocol of the OSI model, wherein the first layer protocol is different from the second layer protocol. For instance, the first layer protocol may be lower than the second layer protocol. In a more specific example, the first layer protocol is an OSI level 2 protocol in which end hosts and transit devices participate, and the second layer protocol is an OSI level 4 protocol in which only end hosts participate. In other examples, the congestion signal tag header can be implemented in different ways consistent with a corresponding communication model used for communicating congestion signals between network devices.
To illustrate incorporation of the congestion signal tag header and the congestion signal reflection header into OSI packets in one example,
As can be seen from
The congestion signal tag header 305 may include one or more congestion signal types and one or more respective congestion signal values. The congestion signal types each indicate a type of signal being carried in the congestion signal tag and may each be, for example, three or four bits long. The congestion signal values each capture the value of the signal specified by the corresponding congestion signal type and may each be, for example, five or twenty bits long. The specific length of congestion signal types and their corresponding values can vary, for example, based on constraints or design requirements for the corresponding communication model for which the congestion signals are implemented.
In the example of
Regarding congestion signal types, a sender host at a first node (e.g., node A 110 of
Regarding congestion signal values, a sender host at a first node (e.g., node A 110 of
Regarding the congestion signals in general, the signals may be aggregation functions of individual per-hop or per-port signals across the path of a packet. The typical definition of such signals with max/min aggregations captures the notion of a path bottleneck for different definitions of bottleneck. However, structurally, the format supports arbitrary aggregation functions including max, min, count and sum, allowing future use cases to leverage the structure for new signals. For purposes of brevity of description, this disclosure will illustrate the congestion signals by describing three types of congestion signals in more detail, a minimum available bandwidth congestion signal (min(ABW)), a maximum link utilization congestion signal (max(U/C) or min(ABW/C)), and a maximum per-hop delay congestion signal (max(PD)).
The min(ABW) captures the minimum absolute available bandwidth, in bits per second (bps), across all the ports in the packet path. The available bandwidth (ABW) is defined per egress port on each device. Further, the ABW can be computed using one of many algorithm variants, each having implications on HW or SW implementation complexity, timescales of computation, and accuracy of the signal.
The min(ABW/C) captures the link utilization bottleneck along the path of the packet, with ABW/C capturing the fraction or percentage of available bandwidth on a given link relative to the link's capacity. The min(ABW/C) is most relevant in paths with heterogeneous link speeds, where it distinguishes itself from min(ABW). The min(ABW/C) is equivalent to max(U/C), where U=utilization of a given egress port in bps, C=capacity of a given egress port in bps, and ABW=available bandwidth of a given egress port in bps. Therefore, max(U/C)=max (1−ABW/C)=1−min(ABW/C).
The max(PD) captures the maximum per-hop delay experienced by a packet among all the hops in the packet path. Per-hop delay (PD) is the time spent by the packet in the device pipeline. The PD may include link layer delays or may include only the delays observed in the forwarding pipeline. Unlike ABW and ABW/C which are per-port signals, PD is a per-packet signal. Device implementations may track ingress and egress timestamps explicitly for each packet and perform a diff in the final stages of the pipeline. Precise definitions of these stages depend on the architecture of the device. For example, some devices could leverage existing timestamping support from tail timestamping capabilities for this purpose.
Turning now to the congestion signal reflection header, it is noted that the congestion signal reflection header enables consumption of congestion signal values at the point where the signals are needed for telemetry or control. This mechanism is particularly relevant for sender-driven/source-based telemetry and control. For receiver-driven transports and controllers, the congestion signal reflection header may not be necessary as the signals defined in the congestion signal tag header are available at the receiver without reflection. Moreover, the location of the congestion signal reflection header and the choice of which packets carry the header are transport-specific. As an example, the congestion signal reflection header can be carried on transmission control protocol (TCP) acknowledge (ACK) packets from the receiver back to the sender. In any event, the congestion signal reflection header may include all of the congestion signals specified in the corresponding congestion signal header or, in the interest of optimizing header size, include only a subset of the congestion signals specified in the corresponding congestion signal header.
Having described congestion control signaling according to embodiments, overall operation of embodiments will be described in additional detail.
In embodiments, congestion signaling operation may begin when a sender host constructs a congestion signal tagged packet for a flow of interest and sends out the packet with the congestion signal tag header fields initialized. The sender host (e.g., software running on a NIC of a first node) determines these initial values for the packet, including the one or more congestion signal types and the one or more initial congestion signal values. As the packet traverses through the network from the sender host to a receiver host (e.g., software running on a NIC of a second node), each of the first node, the second node, and the transit device(s) between the first node and second node may or may not perform a compare-and-replace on the congestion signal tag header to update the congestion signal values. That is, the congestion signal values may be updated by any one of (i) only the first node, (ii) only the second node, (iii) one or more of the transit devices, and (iv) any combination of (i), (ii), and (iii). As the packet traverses through the network, the congestion signal tag header accumulates the desired aggregation of the requested signals.
In some embodiments, a sender host and a receiver host exchange congestion related information through congestion signal types supported only by the sender host and the receiver host. In these embodiments, even without the network supporting certain signal types, any transit devices between the sender host and receiver host can act pass-through while the sender host and receiver host exchange end-point specific congestion information. For example, a NIC of a sender node and a NIC of a receiver node may exchange congestion signals regarding the packets per second (PPS) bandwidth available at the receiver node, and/or the unoccupied buffer space/credits at the receiver node. In another example, a NIC of a receiver node may use congestion signals to indicate to a NIC of a sender node the number of buffer slots remaining at the receiver node, the receiver node's desired rate of transmission for sender-to-receiver packets, or some other value.
In any event, when the congestion signal tagged packet reaches a receiver host, the data fields in the congestion signal tag header are extracted and delivered to a transport layer at the receiver host. The receive host, e.g., software running on a NIC of a second node, stores the congestion signal types and congestion signal values of the packet to be reflected, or a summary of these types and values across packets, and reflects these types and values in a congestion signal reflection header on packets traversing the reverse path from the receiver host to the sender host. The congestion signal reflection header is unmodified as the packet travels from the receiver host to the sender host. The sender host extracts the congestion signal types and values from the congestion signal reflection header of the incoming packet, and hands the types and values to the transport layer of the sender host for use in applications at the sender host. As a result, the sender host learns the desired signal for a flow within approximately one round-trip time.
If a sender host desires to obtain multiple signals for the same flow, it may choose congestion signal types on a per-packet basis. For example, the sender host may choose congestion signal types in a round-robin fashion across the flow's packets, and internally keep track of all of the requested congestion signal types as part of the flow's state variables. This approach allows the sender host transport to use all supported congestion signal types for use cases such as congestion control, load balancing, and/or multipathing.
Referring now to
Embodiments of the present technology include, but are not restricted to, the following.
Unless otherwise stated, the foregoing alternative examples are not mutually exclusive, but may be implemented in various combinations to achieve unique advantages. As these and other variations and combinations of the features discussed above can be utilized without departing from the subject matter defined by the claims, the foregoing description should be taken by way of illustration rather than by way of limitation of the subject matter defined by the claims.
| Number | Name | Date | Kind |
|---|---|---|---|
| 8811183 | Anand | Aug 2014 | B1 |
| 9444754 | Noureddine | Sep 2016 | B1 |
| 20040052212 | Baillargeon | Mar 2004 | A1 |
| 20120051216 | Zhang | Mar 2012 | A1 |
| 20140153387 | Wu | Jun 2014 | A1 |
| 20140223053 | Yoshida | Aug 2014 | A1 |
| 20150295827 | Zhu | Oct 2015 | A1 |
| 20230362099 | Tilmans | Nov 2023 | A1 |
| Number | Date | Country |
|---|---|---|
| 108965121 | Dec 2018 | CN |
| 3442180 | Nov 2020 | EP |
| Entry |
|---|
| US 11,711,311 B2, 07/2023, Wang et al. (withdrawn) |
