The present invention relates to a method and apparatus to implement ingress packet filtering (including optional re-writing) in network interface circuitry associated with a host computer. The filtering is based on matching explicit properties of ingress packets (one or more packet header fields) with associated filtering rules. This is in contrast to matching implicit properties of the ingress packets, such as properties of a connection with which an ingress packet is associated. The filtering rules have associated with them actions that may include, for example, accept, accept with re-write, and reject of a particular packet. The universe of filtering rules may be complete such that that every ingress packet that is not for connection that is offloaded to the network interface circuitry is processed according to some filtering rule.
Filtering of data communicated over a network to a host is known. Typically, such filtering occurs either as part of processing by the host computer (e.g., the netfilter capability of Linux) or on equipment somewhat disassociated with the host computer (such as a switch or a firewall appliance).
Transfer of data is facilitated between at least one peer application and a host, via a network and network interface circuitry associated with the host. That is, data destined for the host is provided from the peer to the network interface circuitry via the network. The network interface circuitry has the capability to offload the processing of data provided according to particular protocols. In addition, based on characteristics of the data, a filtering rule associated with those characteristics may be applied to the data prior to providing the data to the host. When there are a plurality of filter rules associated with characteristics of the data, in some examples, it is automatically determined which one of the plurality of filter rules associated with characteristics of the data to apply to the data.
The data provided to the network interface circuitry may be in packet form. Filtering action according to a filter rule may include passing a packet to the host with or without modifications to the packet contents, rejecting the packet (i.e., dropping both header portion(s) and packet payload). In the case of a packet provided according to an offloaded protocol, the filtering action may include processing the packet according to protocol processing rules for the offloaded protocol.
According to some conventional systems, a protocol processing stack is offloaded from a host to intelligent network interface circuitry (such as a Network Interface Card, or “NIC”). The intelligent network interface circuitry is a data link layer interface between the host computer and the network. For example, communication across the network may be Ethernet protocol at the data link layer and the intelligent network interface circuitry has one or more MAC addresses associated with it. In one example, the intelligent network interface circuitry is configured to apply protocol processing to Transmission Control Protocol (TCP) packets having a particular Internet Protocol (IP) destination address, and being destined for a particular TCP port number. Thus, the conventional offload intelligent network interface circuitry, in some sense, “filters” packets arriving to the intelligent network interface circuitry from the network. That is, packets are associated with particular connections based on protocol type and packet header characteristics (e.g., for TCP/IP offload, matching the 4-tuple in the header). However, the filtering action association is with a particular connection. Such processing is implicitly a function of the connection state as indicated, for example, in a TCB (Transmission Control Block) maintained by the intelligent offload circuitry.
Particularly when the intelligent interface circuitry is configured to process packets of connections offloaded from the host computer, it is advantageous to perform filtering in the intelligent interface circuitry on packets that are not of the offloaded connections. That is, since the packets headers are already being evaluated to determine if the packets are of offloaded connections, little (if any) additional processing is required in the intelligent interface circuitry to match the packets with filtering actions.
In the example discussed below, the intelligent network interface circuitry is capable of filtering (including rewriting) data (e.g., in packets) arriving from a peer via a network, and nominally destined to the host with which the intelligent network interface circuitry is associated, according to a flexible set of filtering rules. In some examples, the intelligent network interface circuitry may further have capability to offload protocol processing for some protocols. For example, the intelligent network interface circuitry may have the capability to offload protocol processing from the host for the TCP protocol.
As one example, as a result of the filtering rules, and an additional capability to dynamically update the filtering rules, the intelligent network interface circuitry can, in conjunction with control software running on the host, efficiently implement a high-capability firewall. For example, the firewall can be configured to defend against denial of service attacks, enforce tight port access controls, drop packets with a particular protocol type, drop packets arriving from a particular IP address, etc.
Furthermore, filtering rules may be rewrite rules for packets to be accepted. In this case, the intelligent network interface circuitry can, for example, efficiently implement a load balancer such that incoming packets are distributed across a pool of host server computers. In another example, a Network Address Translation (NAT) device can be efficiently implemented.
At block 54, the filter rule is obtained based on the determined location. There may be one or more nested headers in an input data packet. Where the packet is received according to an Ethernet protocol, for example, a layer-2 (L2) packet such as an ARP packet only has an Ethernet header, but a packet such as a layer-3 (L3) ICMP packet has an Ethernet header and, immediately following the Ethernet header, an IP header. As another example, for a non-fragmented layer-4 (L4) TCP packet, a TCP header immediately follows an L3 IP header, that immediately follows an L2 Ethernet header. The lookup device 52 can, in principle, process all of the L2, L3, and L4 header fields. In some examples, in the interest of efficiency of operation, only a subset of header fields, less than all the header fields, is used.
Broadly speaking, in accordance with one aspect, the filtering rules apply an accept/reject action to each received packet. For an offloaded protocol, such as TCP, a data packet that is a connect (SYN) request may result in a lookup of a listening server and, if such a listening server is not available, will further result in lookup of a filter rule. More specifically, the listening server indicates that connect requests to the destination IP address and TCP port number contained in the TCP/IP header of the connect (SYN) request should be offloaded to the network interface circuitry.
The connect request will typically still be processed by the host in a request/response manner. An offloaded protocol packet that is not a connect request will result in lookup of a location of a control block (“CB,” i.e., a control block that represents the connection state) for a corresponding offloaded connection. If such a control block does not exist, then an attempt will be made to find the location of a control block for a filtering rule. This way, it remains optional to offload connections for protocols that can be offloaded. That is, in the case where a connection is offloaded to the network interface circuitry, the connection state and protocol processing rules are in control of the processing, even if filter rules corresponding to the packet characteristics exist. On the other hand, if the connection is not offloaded, a data packet for that same connection is filtered and either rejected or passed to the host computer.
For a packet that is accepted according to a filtering rule, the control block corresponding to the filtering rule may optionally specify one or more actions to modify the packet before providing it to the host computer. In one example, the action(s) may specify rewriting the destination IP address (LIP) and the destination port number (LP) which, in addition, results in recomputation of the IP header checksum, and of the UDP and TCP payload checksums for UDP and TCP packets, respectively.
The filtering and rewrite rules are dynamic to support cases such as a firewall temporarily opening access for certain source IP addresses (FIP) to a certain local port (LP) or a range of port numbers. To support this capability, in one example, the update of the filtering and rewrite rules can be accomplished atomically. In other words, multiple filter-rule database writes and CB writes can be executed without other particular operations, such as filter-rule reads, being interleaved between the writes. In this way, it is guaranteed that the filter rules and/or CB's are only read when they are in a consistent state.
We now describe a processing pipeline of a specific example of network interface circuitry. While the architecture in the described example is a flow processor architecture, other architectures (perhaps not even processors) may be employed. Furthermore, while filter rule matching in the described example is accomplished using a Ternary Content Addressable Memory (TCAM), other architecture (for example, hash functions or search trees) may be employed.
Turning now to
It is noted that the arbiter 102 is a feature of the particular flow processor architecture of the
When the arbiter 102 operates to allow an ingress Ethernet packet through (at input 104b), the protocol header of the ingress packet is provided to the protocol processing block 107.
The protocol processing block 107 includes a lookup block 108. A packet is identified by the header or headers contained by the packet. As an example, an Ethernet packet includes at least an L2 Ethernet header, and when the Ethernet packet encapsulates IP packets, the packet also includes an L3 IP header. When the IP header encapsulates an L4 TCP (or UDP) protocol, the packet also contains a TCP (UDP) header. For a TCP packet, a 4-tuple including a source and destination IP address and a source and destination port numbers are said to uniquely identify a point-to-point connection that uses the protocol. For TCP packets, the filtering rules minimally consider the 4-tuple information and, in addition, they can contain information about the VLAN to which the packet belongs, the NIC (or, more generally, network interface circuitry) port on which the packet arrived, and the Ethernet address (SA) to which the packet is destined.
The lookup block 108 operates to match the protocol header to an internal identification (“tid,” used by the interface device and the host) corresponding to a particular CB. In the
The TCAM provides the first matching entry (in one example, the entry with the lowest tid value), when there are multiple matching entries. As discussed below, by the organization of the TCAM, then, a “priority” of offloading versus filtering and/or among multiple filtering rules can be established. Furthermore, the TCAM is able to match the different filtering rules in parallel and perform filter rule lookup in a pipelined fashion, to provide lookup results every cycle after the pipeline startup delay.
In the
This last property (i.e., producing the index of the first entries that matches) is used to prioritize the entries within the TCAM as shown in
With this ordering, the tuple of an incoming TCP packet presented to the TCAM 110 will produce a tid pointing to a CB for an active connection or, possibly, a tid pointing to an SCB, if the TCP packet is a connect request (i.e., has the SYN flag set in the TCP header). Finally, a tuple that does not result in a hit in either the active or server regions may produce a hit in the filter region or, possibly, miss completely in the lookup, in which case the TCAM provides a no matching entry indication. In one example, a packet whose tuple does not hit any entry in the TCAM is passed to the host, i.e., has a default filtering action of accept. This behavior may be modified by including an entry in the last entry of the TCAM with none of the mask bits set, such that all packets whose tuple does not match any previous entry (i.e., any entry with a lower index value) will match this entry. The action corresponding to this entry can be set to reject or to any other desired default action.
Turning now away from the TCAM 110 specifically, the lookup block 108 provides the tid, received from the TCAM 110, to connection manager circuitry 112. (Furthermore, the packet headers are also provided down the pipeline.) The connection manager circuitry 112 manages the CB connection state and attributes. In the
In particular, for offloaded connections, the connection manager provides the corresponding tid to the CB 114, and the CB 114 provides the current connection state and attributes for the offloaded connection (i.e., the connection to which the tid corresponds) back to the connection manager 112. Based on the current connection state and on attributes provided from the CB 114, the connection manager 112 determines that the packet corresponds to an offloaded connection, how to appropriately modify the connection state for that offloaded connection and provides, to the payload command manager 116, an indication of the modification to the connection state. Based on the indication of the modification, the payload command manager 116 issues one or more appropriate payload commands (PCMD) to the payload manager block 118 to cause data to be forwarded to the host or to be stored in a receive buffer, and create receive modulation events, as appropriate, to schedule delivery of the packet payload to the host.
For a CB corresponding to a filtering rule, the filtering action is stored as a field in the CB. If the action is to reject the packet, the connection manager 112 issues a PCMD to the payload manager 118 to flush the payload. When the filtering action is to pass the packet through to the host, an appropriate PCMD(s) is issued to cause the payload manger 118 to send the payload to the host and the headers are passed to the block 120 to form the header portion of the host packet. It is at this point, also, that editing of the destination IP address (LIP) and destination port number (LP) may occur: the action stored within the CB 114 indicates if the LIP and/or LP stored in the CB are to be written into the header in place of the address and port number heretofore stored there. The checksum is regenerated as part of the form packet 120 operation.
The 4-tuple configuration is used for filtering TCP packets. In one example, only the reject rule is supported such that a TCB entry is created for each TCP 4-tuple for which corresponding packets are to be rejected by the NIC. In another example with the 4-tuple configuration, a rule is created for each possible TCP 4-tuple and each TCB entry contains the pass/reject action to perform on packets having that 4-tuple.
The connection manager 112 writes the modified connection state and attributes back into the TCB 114. The read, modify and write of the connection state and attributes is done in an atomic operation.
The connection manager 112 provides an appropriate packet header for data transmission to a form packet block 120. For example, if the packet has been protocol-processed by the intelligent network interface circuitry (because, for example, the TCP 4-tuple in the incoming packet corresponds to a TCB entry being handled by the intelligent network interface circuitry), the packet header provided by the connection manager 112 may include a “command” that provides processing in the host with information about the packet and how it has been protocol processed by the intelligent network interface circuitry.
As another example, if the packet has not been protocol-processed by the intelligent network interface circuitry, the connection manager 112 provides the header that originally entered the arbiter 102, except that the appropriate filtering rule has been applied. Meanwhile, the payload manager block 118 provides the corresponding payload to the form packet block 120 (as discussed above, based on payload commands from the payload command manager 116). The form packet block 120 combines the packet header and corresponding payload into a packet to provide to the host. In the case where the appropriate filtering rule indicates that the packet is to be dropped, then the connection manager 112 flushes the header and, also, the payload command manger 116 provides a command to the payload manager block 118 to flush the corresponding payload.
It can thus be seen that packet filtering can be provided in circuitry closely associated with a host computer, but in a manner that does not greatly impact the processing resources of the host computer. Furthermore, in the case where intelligent network interface circuitry would already be providing a protocol offload function, implementation of packet filtering may be applied as an extension of processing used to match a particular packet with the state of the connection to which the packet belongs.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4445116 | Grow | Apr 1984 | A |
| 4533996 | Hartung et al. | Aug 1985 | A |
| 5497476 | Oldfield et al. | Mar 1996 | A |
| 5778189 | Kimura et al. | Jul 1998 | A |
| 6087581 | Emmer et al. | Jul 2000 | A |
| 6226680 | Boucher et al. | May 2001 | B1 |
| 6240094 | Schneider | May 2001 | B1 |
| 6247060 | Boucher et al. | Jun 2001 | B1 |
| 6334153 | Boucher et al. | Dec 2001 | B2 |
| 6389479 | Boucher et al. | May 2002 | B1 |
| 6393487 | Boucher et al. | May 2002 | B2 |
| 6397316 | Fesas, Jr. | May 2002 | B2 |
| 6401177 | Koike | Jun 2002 | B1 |
| 6427171 | Craft et al. | Jul 2002 | B1 |
| 6427173 | Boucher et al. | Jul 2002 | B1 |
| 6434620 | Boucher et al. | Aug 2002 | B1 |
| 6470415 | Starr et al. | Oct 2002 | B1 |
| 6510164 | Ramaswamy et al. | Jan 2003 | B1 |
| 6591302 | Boucher et al. | Jul 2003 | B2 |
| 6594268 | Aukia et al. | Jul 2003 | B1 |
| 6625671 | Collette et al. | Sep 2003 | B1 |
| 6658480 | Boucher et al. | Dec 2003 | B2 |
| 6681244 | Cross et al. | Jan 2004 | B1 |
| 6687758 | Craft et al. | Feb 2004 | B2 |
| 6697868 | Craft et al. | Feb 2004 | B2 |
| 6701372 | Yano et al. | Mar 2004 | B2 |
| 6708223 | Wang et al. | Mar 2004 | B1 |
| 6708232 | Obara | Mar 2004 | B2 |
| 6717946 | Hariguchi et al. | Apr 2004 | B1 |
| 6751665 | Philbrick et al. | Jun 2004 | B2 |
| 6757245 | Kuusinen et al. | Jun 2004 | B1 |
| 6757746 | Boucher et al. | Jun 2004 | B2 |
| 6792502 | Pandya et al. | Sep 2004 | B1 |
| 6798743 | Ma et al. | Sep 2004 | B1 |
| 6807581 | Starr et al. | Oct 2004 | B1 |
| 6813652 | Stadler et al. | Nov 2004 | B2 |
| 6862648 | Yatziv | Mar 2005 | B2 |
| 6925055 | Erimli et al. | Aug 2005 | B1 |
| 6941386 | Craft et al. | Sep 2005 | B2 |
| 7031267 | Krumel | Apr 2006 | B2 |
| 7093099 | Bodas et al. | Aug 2006 | B2 |
| 7114096 | Freimuth et al. | Sep 2006 | B2 |
| 7133902 | Saha et al. | Nov 2006 | B2 |
| 7133914 | Holbrook | Nov 2006 | B1 |
| 7191318 | Tripathy et al. | Mar 2007 | B2 |
| 7239642 | Chinn et al. | Jul 2007 | B1 |
| 7254637 | Pinkerton et al. | Aug 2007 | B2 |
| 7260631 | Johnson et al. | Aug 2007 | B1 |
| 7284047 | Barham et al. | Oct 2007 | B2 |
| 7313623 | Elzur et al. | Dec 2007 | B2 |
| 7376147 | Seto et al. | May 2008 | B2 |
| 7408906 | Griswold et al. | Aug 2008 | B2 |
| 7453892 | Buskirk et al. | Nov 2008 | B2 |
| 7474670 | Nowshadi | Jan 2009 | B2 |
| 7493427 | Freimuth et al. | Feb 2009 | B2 |
| 7583596 | Frink | Sep 2009 | B1 |
| 20010010046 | Muyres et al. | Jul 2001 | A1 |
| 20010021949 | Blightman et al. | Sep 2001 | A1 |
| 20010036196 | Blightman et al. | Nov 2001 | A1 |
| 20010037406 | Philbrick et al. | Nov 2001 | A1 |
| 20020039366 | Sano | Apr 2002 | A1 |
| 20020087732 | Boucher et al. | Jul 2002 | A1 |
| 20020091844 | Craft et al. | Jul 2002 | A1 |
| 20020095519 | Philbrick et al. | Jul 2002 | A1 |
| 20020156927 | Boucher et al. | Oct 2002 | A1 |
| 20020161919 | Boucher et al. | Oct 2002 | A1 |
| 20030018516 | Ayala et al. | Jan 2003 | A1 |
| 20030035436 | Denecheau et al. | Feb 2003 | A1 |
| 20030140124 | Burns | Jul 2003 | A1 |
| 20030200284 | Philbrick et al. | Oct 2003 | A1 |
| 20030204631 | Pinkerton et al. | Oct 2003 | A1 |
| 20040003094 | See | Jan 2004 | A1 |
| 20040003126 | Boucher et al. | Jan 2004 | A1 |
| 20040028069 | Tindal et al. | Feb 2004 | A1 |
| 20040030745 | Boucher et al. | Feb 2004 | A1 |
| 20040042487 | Ossman | Mar 2004 | A1 |
| 20040054813 | Boucher et al. | Mar 2004 | A1 |
| 20040062245 | Sharp et al. | Apr 2004 | A1 |
| 20040062246 | Boucher et al. | Apr 2004 | A1 |
| 20040064578 | Boucher et al. | Apr 2004 | A1 |
| 20040064589 | Boucher et al. | Apr 2004 | A1 |
| 20040064590 | Starr et al. | Apr 2004 | A1 |
| 20040073703 | Boucher et al. | Apr 2004 | A1 |
| 20040078480 | Boucher et al. | Apr 2004 | A1 |
| 20040088262 | Boucher et al. | May 2004 | A1 |
| 20040100952 | Boucher et al. | May 2004 | A1 |
| 20040111535 | Boucher et al. | Jun 2004 | A1 |
| 20040117509 | Craft et al. | Jun 2004 | A1 |
| 20040158640 | Philbrick et al. | Aug 2004 | A1 |
| 20040158793 | Blightman et al. | Aug 2004 | A1 |
| 20040165592 | Chen et al. | Aug 2004 | A1 |
| 20040190533 | Modi et al. | Sep 2004 | A1 |
| 20040199808 | Freimuth et al. | Oct 2004 | A1 |
| 20040213235 | Marshall et al. | Oct 2004 | A1 |
| 20040240435 | Craft et al. | Dec 2004 | A1 |
| 20050071490 | Craft et al. | Mar 2005 | A1 |
| 20050083935 | Kounavis et al. | Apr 2005 | A1 |
| 20050120037 | Maruyama et al. | Jun 2005 | A1 |
| 20050122986 | Starr et al. | Jun 2005 | A1 |
| 20050125195 | Brendel | Jun 2005 | A1 |
| 20050135378 | Rabie et al. | Jun 2005 | A1 |
| 20050135396 | McDaniel et al. | Jun 2005 | A1 |
| 20050135412 | Fan | Jun 2005 | A1 |
| 20050147126 | Qiu et al. | Jul 2005 | A1 |
| 20050190787 | Kuik et al. | Sep 2005 | A1 |
| 20050216597 | Shah et al. | Sep 2005 | A1 |
| 20050259644 | Huitema et al. | Nov 2005 | A1 |
| 20050259678 | Gaur | Nov 2005 | A1 |
| 20050289246 | Easton et al. | Dec 2005 | A1 |
| 20060031524 | Freimuth | Feb 2006 | A1 |
| 20060039413 | Nakajima et al. | Feb 2006 | A1 |
| 20060075119 | Hussain | Apr 2006 | A1 |
| 20060080733 | Khosmood et al. | Apr 2006 | A1 |
| 20060133267 | Alex et al. | Jun 2006 | A1 |
| 20060168649 | Venkat et al. | Jul 2006 | A1 |
| 20060206300 | Garg et al. | Sep 2006 | A1 |
| 20060209693 | Davari et al. | Sep 2006 | A1 |
| 20060221946 | Shalev et al. | Oct 2006 | A1 |
| 20060281451 | Zur | Dec 2006 | A1 |
| 20070064737 | Williams | Mar 2007 | A1 |
| 20070070901 | Aloni et al. | Mar 2007 | A1 |
| 20070110436 | Bennett | May 2007 | A1 |
| 20070201474 | Isobe | Aug 2007 | A1 |
| 20080002731 | Tripathi et al. | Jan 2008 | A1 |
| 20080016511 | Hyder et al. | Jan 2008 | A1 |
| 20080043750 | Keels et al. | Feb 2008 | A1 |
| 20080232386 | Gorti et al. | Sep 2008 | A1 |