Patent Application
20070226375
The subject matter of this application is related to the subject matter in a co-pending non-provisional application by the same inventors as the instant application and filed on the same day as the instant application entitled, “Method and Apparatus for Dynamically Changing the TCP Behavior of a Network Connection,” having serial number TO BE ASSIGNED, and filing date TO BE ASSIGNED (Attorney Docket No. SUN06-0663).
1. Field of the Invention
The present invention generally relates to computer networks. More specifically, the present invention relates to a plug-in architecture for a network stack in an operating system.
2. Related Art
The transmission control protocol (TCP) is part of the core Internet protocol which is used to transfer data between computing devices. The goal of TCP is to transfer data from an application on a computing device through a shared network resource to a second device as quickly, efficiently, and reliably as possible, despite potential contention and congestion.
While the basic operation of TCP has not changed dramatically since the initial publication of the standard in 1981, the protocol has been forced to evolve in response to changing network conditions such as new link types (e.g., wireless networks) and higher bandwidth wired networks. Substantial ongoing research on congestion control and avoidance has resulted in numerous TCP congestion control techniques, such as Reno, New Reno, Vegas, HS-TCP, Fast TCP, S-TCP, and Bic-TCP. However, such congestion control techniques add substantial complexity to TCP and the network stack. Furthermore, end-to-end links can traverse numerous networks with diverse characteristics, and no single congestion control approach encompasses the wide range of modern networks.
Hence, what is needed are architectures and methods that facilitate congestion control for TCP without the limitations of existing approaches.
One embodiment of the present invention provides a plug-in architecture for a network stack in an operating system. The network stack includes a set of functions configured to modify a set of parameters that are likely to change based on the network environment. The architecture includes a plug-in framework within the network stack that allows the set of functions to be dynamically changed in order to change the TCP behavior of the network stack to suit the network environment.
In a variation on this embodiment, the parameters include:
In a variation on this embodiment, changing the set of functions changes the transmit and receive characteristics of the network stack, thereby changing the congestion-control technique for the network stack.
In a further variation, the set of functions are triggered by events that include:
In a further variation, triggering an event prompts the set of functions to update the set of parameters.
In a variation on this embodiment, the network stack maintains a set of generic state information, and the set of functions maintains a set of state separate from the set of generic state information. The set of functions can access the set of generic state information.
In a variation on this embodiment, the set of functions is implemented as a dynamically loadable kernel module.
In a variation on this embodiment, changing the set of functions allows the network stack to dynamically change TCP behavior and thereby transmit efficiently across diverse and changing network environments.
The following description is presented to enable any person skilled in the art to make and use the invention, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the present invention. Thus, the present invention is not limited to the embodiments shown, but is to be accorded the widest scope consistent with the claims.
The data structures and code described in this detailed description are typically stored on a computer-readable storage medium, which may be any device or medium that can store code and/or data for use by a computer system. This includes, but is not limited to, magnetic and optical storage devices such as disk drives, magnetic tape, CDs (compact discs), DVDs (digital versatile discs or digital video discs), or any device capable of storing data usable by a computer system.
TCP Congestion Control
The TCP layer comprises an important part of the network stack 108. The core of the TCP protocol is based on a set of parameters that together determine a set of data packets, a timeframe in which they will be transmitted from the sender side, and how acknowledgements will be generated on the receiving side. The sending side constantly recalculates the set of parameters based on feedback from, for instance, acknowledgement packets and local timers, in order to decide which data to send or resend, and when. Important parameters include:
TCP strives to maximize the utilization of the available network bandwidth in a “fair” manner (i.e. friendly to other TCP traffic), while avoiding, or otherwise quickly recovering from, network congestion. Achieving this goal is difficult given the wide diversity of modem networking technologies. The effectiveness of congestion control in artificial and production environments is: often sorely tested by factors such as the distance between sender and receiver, window sizes, the number of streams, network configuration, load, varying drop rates, link reliability, etc. While many different TCP techniques have been proposed over the years, including but not limited to Reno, New Reno, Vegas, HS-TCP, S-TCP, Bic-TCP, Cubic, Fast-TCP, and TCP-Westwood, no technique has been found that performs best across all instances.
Traditionally, the congestion-control technique is hard-wired in the TCP implementation, and can only be changed by compiling a second operating system kernel with a new technique, shutting down the system, and replacing the current operating system kernel. Since no single, definitive solution exists nor seems to be forthcoming, a traditional, network-stack architecture with one hard-wired TCP congestion-control technique will not provide a production solution nor keep up with future advances in TCP research and the possible proliferation of TCP techniques.
The present invention extends TCP using a plug-in architecture for the network stack of an operating system.
A Plug-In Architecture for TCP Congestion Control
The present invention extends existing network stacks (including stacks deployed in kernel space, user space, and/or in TCP offload engines) to allow core functions of the TCP congestion control system to be changed easily and dynamically. While many portions of the TCP implementation contribute to TCP dynamics, only a subset of the implementation is likely to still evolve. One such area still seeing significant changes is transmission-side congestion avoidance.
In one embodiment of the present invention, a subset of the TCP transmit functionality becomes a swappable plug-in, while the standardized and unchanging portion of the TCP layer remains hard-wired. The system enters the swappable portion whenever an event is encountered that triggers a recomputation of congestion parameters, for instance cwnd, ssthresh, and RTT. Such triggers for the TCP sender side include:
The plug-in architecture allows the system to switch between different congestion avoidance techniques. Each technique uses a different approach, and may therefore maintain a different set of internal state. For instance, a delay-based technique such as Fast-TCP may track average queuing delay as well as minimum and biased RTTs, while TCP-Westwood gleans data from successive acknowledgement packets to compute an eligible rate estimate (ERE). Alternatively, High-Speed TCP (HS-TCP), a loss-based technique, keeps an internal table of congestion window sizes (i.e. a table for “a (cwnd)” and “b(cwnd)”). These internal parameters are typically not visible outside the plug-in, but can be used by the plug-in to adjust key parameters that control TCP behavior. The system can effectively give full control of TCP behavior to the plug-in by only allowing control parameters to be changed in the plugged-in functions.
In general, given the changing nature (e.g. increasing bandwidth, distances, topology variations, production requirements, etc) of production and experimental networks, allowing TCP behavior to be easily replaced provides significant advantages over the previous hard-wired approach, which provides only limited capability. Allowing the TCP behavior to be easily modified, either manually or dynamically, provides an opportunity to tune network performance of production networks as well as provide a flexible way to explore, implement, and test new congestion control techniques.
In one embodiment of the present invention, the plug-in functionality is implemented using a dynamically-loaded kernel module that can be loaded or unloaded both at system boot-time as well as when the system is active.
Per-Connection TCP Congestion Control
While a plug-in architecture for TCP allows TCP behavior to be changed at the system level, each network connection may encounter different conditions based on the destination or other factors, so a more ideal solution allows multiple techniques to be applied simultaneously on the computer system.
One embodiment of the present invention provides network resource- and bandwidth-control by extending the plug-in architecture to allow different TCP behaviors to be plugged-in on a per-connection basis. The system maintains a vector of function pointers that point to the chosen TCP technique for each connection. Depending on system policy, the appropriate technique for a connection may be chosen at a very fine granularity, and vary dynamically, based on:
Fine-grained per-connection control of TCP behavior enables additional possibilities not available with a traditional hard-wired TCP layer. Traditionally, quality-of-service (QoS) and bandwidth control occur outside of the transport layer, for instance at the IP layer or in the network. While this approach is less intrusive to the network stack, it also has many limitations, e.g. providing end-to-end QoS in the network typically requires the configuration and cooperation of all of the switches and routers the traffic flows through, which is often infeasible. A plug-in function for a connection can provide a level of QoS and bandwidth control directly inside the TCP layer, thereby taking advantage of knowledge that is difficult to obtain from outside of the transport layer. For instance, in a traditional system, an attempt to throttle-down transmission might be interpreted as a sign of congestion and/or time-out, and prompt undesired re-transmission. The traditional approach of performing resource control and bandwidth management outside of the transport layer at a fine granularity also incurs heavy processing overhead in parsing headers and maintaining state on a per-flow basis. In the present invention, such capabilities can be added to the TCP behavior using a plug-in and handled appropriately.
The plug-in approach also enables employing an aggressive, special-purpose technique in a controlled network environment. For instance, a server in a data center with a well-controlled traffic pattern or well-tuned queuing model might deploy a non-compliant congestion control technique that allows packets to be sent without slow-start or any bandwidth throttling. This technique could be useful, for example, to eliminate the overhead of congestion control for connections that transfer data between two servers on a dedicated network link, or to expedite connections that exchange cluster membership heartbeat messages within the data center. Previously, such service variation either was not possible, or would require multiple servers.
Finally, per-connection tuning can also be used to deploy and test experimental TCP behaviors on a limited set of TCP connections on a production server without exposing other, normal operations on the server to the riskier new behavior.
In summary, the present invention extends TCP behavior using a plug-in architecture. This architecture allows TCP behavior to be tuned on a per-connection basis, thereby enabling the core functions of the TCP congestion control system to adapt to changing network conditions and improving the speed and efficiency of data transfers.
The foregoing descriptions of embodiments of the present invention have been presented only for purposes of illustration and description. They are not intended to be exhaustive or to limit the present invention to the forms disclosed. Accordingly, many modifications and variations will be apparent to practitioners skilled in the art. Additionally, the above disclosure is not intended to limit the present invention. The scope of the present invention is defined by the appended claims.
