Full offload of stateful connections, with partial connection offload

Abstract

A method of operating a system including a host and intelligent network interface circuitry extends a passive open operation to accomplish full offload of a stateful connection between a host and a peer. In response to observing a synchronization signal from the peer, an offload target creates a connection control block in the offload target in anticipation of a corresponding stateful connection being offloaded from the host. The observed synchronization signal is allowed to be provided to the host. In response to observing a synchronization acknowledgment signal from the host and destined for the peer, the offload target performs establishment processing for the stateful connection. Based on an acknowledgment of the synchronization acknowledgment signal from the peer, acknowledgement of the synchronization acknowledgement signal is processed and a corresponding acknowledgment signal is provided to the host. In response to receiving an offload event indication from the host, the stateful connection is operated between the host and the peer.

Description

BACKGROUND

Protocol offload processing is known. For example, an interface device (sometimes known as intelligent network interface circuitry or an “offload target”) may be provided to operate in coordination with a host, for protocol offload processing with a peer device across a network. For example, the protocol offload processing may be according to the Transmission Control Protocol (TCP) whereas communication across the network may be via high-speed Ethernet, such as 10 Gbps Ethernet. FIG. 1 illustrates an architecture of a system including a host, network interface circuitry including an offload target, and a peer configured for communication with the host according to a stateful protocol such as TCP/IP.

Furthermore, it is known that the protocol offload processing may be, for example, “partial offload” or “full offload.” For example, with “full offload,” the offload target handles all the phases of a connection. That is, in addition to handling the data transfer phase of a connection, the offload target is additionally at least responsible for establishing a connection, including handling retransmissions, if necessary, during connection establishment.

“Partial offload” has various flavors. Examples include the following:

• • offload target handles only the data transfer phase of a connection-oriented protocol, which for example might include processing to reorder the received packets when they are received out of order.
  • offload target handles only the data transfer phase and the connection teardown phase of a connection-oriented protocol.
  • offload target handles the “mainstream” part of the data transfer phase, deferring exceptions, such as out of order processing, to the host system.

Partial offload solutions have been promulgated such as, for example, the “TCP Chimney” feature of recently released Vista operating system from Microsoft. Like conventional partial offload solutions, the TCP Chimney feature appears to be designed to retain full control over connection acceptance while, in addition, simplifying management of listening servers.

A listening server, for example, is used when a peer requests a connection to a local host, such as a TCP/IP connection at a specified IP address and port number. A listening server typically has an LIP, LP, FIP, and FP attribute, such that a listening server on the local host is listening for connect requests to local IP address LIP, at local port number LP, from a foreign IP address FIP, and from foreign port FP at the FIP. The FIP and FP fields for the listening server many times have “don't care values,” such that connect requests are accepted from any FP on any FIP. A listening server may, in addition, have a protocol attribute such as TCP, or SCTP (Stream Control Transfer Protocol), as the SCTP protocol and other connection oriented protocols use a similar listening server mechanism to receive connect requests.

The connect request is received by the listening server that then forwards the request to a connection manager that either accepts all connect requests, or that uses various criteria such as ACL (Access Control Lists) to either accept or reject a connect request.

FIG. 2 illustrates an example of an “active open” operation to open a TCP/IP connection between the host and the peer, according to the TCP Chimney architecture. At 201, the host stack sends an active open request to the peer via a SYN packet. At 202, the peer responds with a SYN/ACK. The host stack sends an ACK in response to the SYN/ACK. At this point, the connection has been established on the host stack.

At 204, the host stack initiates offload of the connection to the offload target. At 205, a data packet arrives from the peer after the host stack initiates the connection offload but before the connection has been set up on the offload target. The data packet is therefore sent to the host. At 206, the host stack receives the data packet. Detecting that the offload is in progress, the host stack retains the data packet.

At 207, the offloading is complete. At 208, the host stack forwards the data packet back to the offload target for processing. At 209, the offload target processes the packet and provides an acknowledgment back to the peer.

We now discuss how a control block may be configured in accordance with an example of an active open operation. An example of such a control block is shown immediately below:

CONST_STATE
Flags                   = 0x0
RemotePort              = 0xF3FB
LocalPort               = 0x46B4
SndWindScale            = 0x0
RcvWindScale            = 0x0
RemoteMss               = 1460:0
CACHED_STATE
Flags                   = 0x2
InitialRcvWnd           = 0
RcvIndicationSize       = 0
KaProbeCount            = 0
KaTimeout               = 0x0
KaInterval              = 0x0
MaxRT                   = 0x0
FlowLabel               = 0x0
TtlOrHopLimit           = 0x80
TosOrTrafficClass       = 0x0
UserPriority            = 0x0
DELEGATED_STATE
State                   = TcpConnectionSynSent
Flags                   = 0x0
RcvNxt                  = 0x0
RcvWnd                  = 0x0
SndUna                  = 0x0
SndNxt                  = 0x0
SndMax                  = 0x0
SndWnd                  = 17520
MaxSndWnd               = 0x0
SendWL1                 = 0x0
CWnd                    = 0xB68
SsThresh                = 0xFFFFFFFF
SRtt                    = 0x0
RttVar                  = 0x1E
TsRecent                = 0x0
TsRecentAge             = 0x0
TsTime                  = 0x0
TotalRT                 = 0x0
DupAckCount             = 0x0
SndWndProbeCount        = 0x0
KeepAlive.ProbeCount    = 0x0
KeepAlive.TimeoutDelta  = 0x0
Retransmit.Count        = 0x0
Retransmit.TimeoutDelta = 0xFFFFFFFF
SendDataHead            = 0x0
SendDataTail            = 0x0
SendBacklogSize         = 0x0
Buffered Data           = 0x0
ReceiveBacklogSize      = 0x0

In accordance with one example of an active open operation, a “constant state” portion of a connection control block is provided in a manner that it may otherwise be provided in a configuration that does not include an offload target. A “cached state” portion of the connection control block may be partly determined by the offload target. The delegated state portion may be partially filled in by the offload target when the connection establishment succeeds. It is noted that, prior to connection establishment succeeding, all send and receive sequence numbers are undefined and the TCP state is SYN_SENT. The offload target interprets such a control block as being for an offloaded active open.

In accordance with other conventional implementations, an active open operation is achieved in a full offload manner. That is, in essence, the host requests the offload target to establish and operate a connection. FIG. 3 illustrates an example of an active open operation, for a TCP/IP connection, being achieved in a full offload manner. At 301, the host stack provides an “initiate offload” request to the offload target. The “initiate offload” request may be more generally referred to an offload event indication. At 302, the offload target provides a SYN packet to the remote peer. It is noted that the offload target handles any SYN retransmission issues, if any. At 303, the peer responds with a SYN/ACK packet. At this point, the connection is established and the connection control block is populated on the offload target. At 304, the offload target provides an “initiate complete” indication back to the host stack.

At 305, the offload target provides an ACK packet to the peer, in response to the SYN/ACK packet provided from the peer. At 306 and 307, the offload target receives a data packet (306) and acknowledges the data packet (307).

FIG. 4 illustrates an example of a partial offload “passive open” operation to open a TCP/IP connection between the host and the peer, according to the TCP Chimney architecture. At 401, the host stack receives a passive open request from the peer via a SYN packet. At 402, the host stack responds with a SYN/ACK. At 403, the peer sends an ACK in response to the SYN/ACK. At this point, the connection has been established on the host stack.

At 404, a data packet arrives from the peer. Meanwhile, at 405, the host receives the ACK and initiates a connection offload to the offload target. At 406, the host stack receives the data packet before the connection has been set up on the offload target. Detecting that the offload operation is in progress, the host stack retains the data packet. At 407, the host stack receives a message from the offload target that the offload operation is complete.

At 408, the host stack forwards the data packet to the offload target for processing. At 409, the offload target processes the data packet and sends an ACK to the peer. At 410, the peer receives the ACK corresponding to receipt of the data packet by the host.

We now discuss how a control block may be configured in accordance with an example of a partial offload passive open operation such as described relative to FIG. 4. An example of such a control block is shown immediately below:

CONST_STATE
Flags                   = 0x0
RemotePort              = 0xF3FB
LocalPort               = 0x46B4
SndWindScale            = 0x0
RcvWindScale            = 0x0
RemoteMss               = 1460:0
CACHED_STATE
Flags                   = 0x2
InitialRcvWnd           = 0
RcvIndicationSize       = 0
KaProbeCount            = 0
KaTimeout               = 0x0
KaInterval              = 0x0
MaxRT                   = 0x0
FlowLabel               = 0x0
TtlOrHopLimit           = 0x80
TosOrTrafficClass       = 0x0
UserPriority            = 0x0
DELEGATED_STATE
State                   = TcpConnectionSynRcvd
Flags                   = 0x0
RcvNxt                  = 0xA1CD
RcvWnd                  = 0xFFFF
SndUna                  = 0x0
SndNxt                  = 0x0
SndMax                  = 0x0
SndWnd                  = 17520
MaxSndWnd               = 0xFFFF
SendWL1                 = 0x0
CWnd                    = 0xB68
SsThresh                = 0xFFFFFFFF
SRtt                    = 0x0
RttVar                  = 0x1E
TsRecent                = 0x0
TsRecentAge             = 0x0
TsTime                  = 0x0
TotalRT                 = 0x0
DupAckCount             = 0x0
SndWndProbeCount        = 0x0
KeepAlive.ProbeCount    = 0x0
KeepAlive.TimeoutDelta  = 0x0
Retransmit.Count        = 0x0
Retransmit.TimeoutDelta = 0xFFFFFFFF
SendDataHead            = 0x0
SendDataTail            = 0x0
SendBacklogSize         = 0x0
Buffered Data           = 0x0
ReceiveBacklogSize      = 0x0

In accordance with one example of a passive open operation, a “constant state” portion of a connection control block is provided in a manner that it may otherwise be provided in a configuration that does not include an offload target. In addition, a “cached state” portion of the connection control block may also be provided in a manner that may otherwise be provided in a configuration that does not include an offload target.

The “delegated state” portion of the connection control block may be partially filled in by the offload target when the connection establishment succeeds. It is noted that, prior to connection establishment succeeding, all send and receive sequence numbers are undefined and the TCP state is SYN_RCVD. The offload target interprets such a control block as being for an offloaded passive open.

FIG. 5 illustrates an example of a full offload “passive open” operation to open a connection (in this case, a TCP/IP connection) between the host and the peer. At 501, the peer issues a passive open via a SYN packet. The offload target initiates a connection control block in the SYN_RCVD state and, also, provides the SYN packet to the host stack. At 502, the host stack provides an initiate offload indication to the offload target. Alternately, the indication may be an indication that the connection is not to be established.

At 503, based on the initiate offload indication, the offload target provides a SYN/ACK to the peer. Furthermore, at 503, the offload target deals with connection establishment issues such as SYN/ACK retransmission issues. At 504, the peer issues an ACK packet for the SYN/ACK packet. At 505, the peer sends a data packet. At 506, the offload target provides an “initiate complete” indication to the host stack. It is noted that the initiate complete indication may be temporally before or after the offload target receives the data packet (505). In any case, the offload target handles the data packet and, at 507, provides a corresponding ACK packet back to the peer.

SUMMARY

A method of operating a system including a host and intelligent network interface circuitry extends a passive open operation to accomplish full offload of a stateful connection between a host and a peer. In response to observing a synchronization signal from the peer, an offload target creates a connection control block in the offload target in anticipation of a corresponding stateful connection being offloaded from the host. The observed synchronization signal is allowed to be provided to the host;

In response to observing a synchronization acknowledgment signal from the host and destined for the peer, the offload target performs establishment processing for the stateful connection. Based on an acknowledgment of the synchronization acknowledgment signal from the peer, acknowledgement of the synchronization acknowledgement signal is processed and a corresponding acknowledgment signal is provided to the host.

In response to receiving an offload event indication from the host, the stateful connection is operated between the host and the peer.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an architecture of a system including a host and an offload target, configured to communicate with a peer via a stateful connection.

FIG. 2 illustrates an example of an “active open” operation to open a TCP/IP connection between the host and the peer, according to the TCP Chimney architecture.

FIG. 3 illustrates an example of an active open operation, for a TCP/IP connection, being achieved in a full offload manner.

FIG. 4 illustrates an example of a partial offload “passive open” operation to open a TCP/IP connection between the host and the peer, according to the TCP Chimney architecture.

FIG. 5 illustrates an example of a full offload “passive open” operation to open a connection (in this case, a TCP/IP connection) between the host and the peer.

FIG. 6 illustrates an example of a method to utilize a full offload capability of an offload target to “shadow” a host that operates according to a partial offload model.

DETAILED DESCRIPTION

In accordance with an aspect, a method is provided to utilize a full offload capability of an offload target to “shadow” a host that operates according to a partial offload model. The offload target accomplishes a passive open operation including “shadowing” connection establishment with the remote peer. The host retains control over connection acceptance. In addition, connection “handoff” may be simplified. In some examples, no change to the application program interface (API) for TCP offload may be required.

FIG. 6 illustrates one such example, for a TCP/IP connection. At 601, the peer issues a passive open via a SYN packet. At 602, the offload target observes the SYN and creates a connection control block at the offload target. This is in anticipation by the offload target of the connection being offloaded. In addition, a SYN packet is provided to the host.

At 603, the host begins connection establishment processing, responding to the provided SYN packet with a SYN/ACK packet and, for example, creating a connection control block at the offload target. At 604, the offload target detects the SYN/ACK packet from the host and performs connection establishment processing for the connection, including causing a SYN/ACK packet to be transmitted to the peer. The offload target may retransmit the SYN/ACK if necessary. At 605, the peer issues an ACK for the SYN/ACK and, at 606, the peer sends a data packet.

At 607, the host provides an “initiate offload” event to the offload target and the offload target provides an “initiate complete” event in response. In this case, a device driver in the host, for the offload target, knows that the offload target has been shadowing the connection setup process. The initiate offload event signals the offload target that it can now be in control of the connection and that the offload target can deliver to the host any data that may have been held in the offload target, including providing ACK packets to the peer as appropriate (608).

On the other hand, if the “initiate offload” event is not received by the device driver, the offload target operates under the assumption that the connection is to remain under control of the host. In this case, the data packets received by the offload target from the peer are provided to the host without maintaining the connection state in the offload target.

In the FIG. 6 example, at 602, the offload target may fetch an offloaded listening server process, if any the SYN packet received from the peer matches the LIP,LP, FIP,FP criteria of the listening server. Listening server processes (in the context of TCP/IP connections being handled by a host stack, in a non-offloaded manner) are conventionally used in some operating systems such as Linux as a coarse-grained mechanism to filter connect requests to the host stack. In particular, as is well-known, the TCP (Transmission Control Protocol) uses point to point connections to connect two communicating endpoints, with the endpoints often referred to as the local peer, and the foreign peer. The connection is typically identified by the IP (Internet Protocol) addresses of the local peer (LIP), and foreign peer (FIP), and the port number of the connection on the local peer (LP), and the port number on the foreign peer (FP). A listening server is utilized for the foreign peer to request a connect to the local peer at a specified IP address and port number. A listening server therefore has an LIP, LP, FIP, and FP attribute, such that a listening server on the local peer is listening for connect requests to local IP address LIP, at local port number LP, from a foreign IP address FIP, and from foreign port FP at the FIP. Typically the FIP and FP for the listening server have “don't care” values, i.e. accept connect requests from any FP on any FIP. A listening server in addition has a protocol attribute such as TCP, or SCTP (Stream Control Transfer Protocol), as the SCTP protocol and other connection oriented protocols use a similar listening server mechanism to receive connect requests. The connect request is received by the listening server that than forwards the request to a connection manager that either accepts all connect requests, or that uses various criteria such as ACL (Access Control Lists) to either accept or reject a connect request.

In the context of a system including an offload target, the listening server process concept has been extended to include an “offload policy” portion. Turning back to the FIG. 6 example, if the offload policy in the listening server matching the received SYN packet is set to offload, then the offload target creates the connection control block in anticipation of the connection being offloaded. If no matching listening server is found, or if one is found and the offload policy is set to “don't offload,” then no connection control block is created.

The many features and advantages of the invention are apparent from the written description and, thus, it is intended by the appended claims to cover all such features and advantages of the invention. Further, since numerous modifications and changes will readily occur to those skilled in the art, it is not desired to limit the invention to the exact construction and operation as illustrated and described. Hence, all suitable modifications and equivalents may be resorted to as falling within the scope of the invention.

Claims

1. A method of operating an offload target in a system including a host and the offload target, to extend a passive open operation to accomplish full offload of a stateful connection between the host and a peer via a network, the method comprising: in response to observing a synchronization signal from the peer, processing an offload listening server corresponding to the observed synchronization signal and, in accordance with a result of processing the offload listening server, creating a connection control block in anticipation of a corresponding stateful connection being offloaded from the host;allowing the observed synchronization signal to be provided to the host;in response to observing a synchronization acknowledgment signal from the host and destined for the peer and, performing establishment processing for the stateful connection;receiving an acknowledgment of the synchronization acknowledgment signal from the peer, processing the acknowledgement of the synchronization acknowledgement signal and providing a corresponding acknowledgment signal to the host; andin response to receiving an offload event indication from the host, operating the stateful connection between the host and the peer,the method further comprising, in response to not receiving a offload event indication from the host, not operating the stateful connection between the host and the peer, causing data packets received from the peer to be provided to the host without maintaining a connection state in the offload target.

2. The method of claim 1, wherein: observing a synchronization signal includes observing a TCP packet including a set SYN flag.

3. The method of claim 1, wherein: observing a synchronization acknowledgement signal from the host includes observing a TCP packet from the host including set SYN and ACK flags.

4. The method of claim 1, wherein: the stateful connection is a connection according to a TCP/IP protocol.

5. A method of operating an offload target that interfaces a host to a network for communication with a peer via a stateful connection; comprising: observing signals to establish a connection between the host and the peer;processing an offload listening server corresponding to the observed connection establishment signals and, in accordance with a result of processing the offload listening server, establishing a shadow connection between the host and peer, usable for the offload target to offload data transfer phase processing of the connection, from the host; andcarrying out, in the offload target, data transfer phase processing of the connection,the method further comprising, in response to not receiving an offload event indication from the host, not carrying out, in the offload target, data transfer phase processing of the connection, causing data packets received from the peer to be provided to the host without maintaining the shadow connection in the offload target.

6. A system including a host and an offload target, configured to extend a passive open operation to accomplish full offload of a stateful connection between the host and a peer via a network, wherein the offload target is configured to: in response to observing a synchronization signal from the peer, process an offload listening server corresponding to the observed synchronization signal and, in accordance with a result of processing the offload listening server, create a connection control block in anticipation of a corresponding stateful connection being offloaded from the host;allow the observed synchronization signal to be provided to the host;in response to observing a synchronization acknowledgment signal from the host and destined for the peer and, performing establishment processing for the stateful connection;receive an acknowledgment of the synchronization acknowledgment signal from the peer, and process the acknowledgement of the synchronization acknowledgement signal and provide a corresponding acknowledgment signal to the host; andin response to receiving an offload event indication from the host, operate the stateful connection between the host and the peer,wherein the offload target is further configured to, in response to not receiving an offload event indication from the host, not operate the stateful connection between the host and the peer, cause data packets received from the peer to be provided to the host without maintaining a connection state in the intelligent network interface circuitry.

7. The system of claim 6, wherein: observing a synchronization signal includes observing a TCP packet including a set SYN flag.

8. The system of claim 6, wherein: observing a synchronization acknowledgement signal from the host includes observing a TCP packet from the host including set SYN and ACK flags.

9. The system of claim 6, wherein: the stateful connection is a connection according to a TCP/IP protocol.

10. An offload target that interfaces a host to a network for communication with a peer via a stateful connection; the offload target configured to: observe signals to establish a connection between the host and the peer;process an offload listening server corresponding to the observed connection establishment signals and, in accordance with a result of processing the offload listening server, establish a shadow connection between the host and peer, usable for the offload target to offload data transfer phase processing of the connection, from the host; andcarry out, in the offload target, data transfer phase processing of the connection,the offload target further configured to, in response to not receiving an offload event indication from the host, not carry out, in the offload target, data transfer phase processing of the connection, and cause data packets received from the peer to be provided to the host without maintaining the shadow connection in the offload target.

11. The system of claim 10, wherein: the offload target is further configured to carry out shutdown phase processing of the connection.