Patent Application
20090190486
The present invention generally relates to networks. More specifically, the present invention relates to forwarding packets while metering outgoing packet traffic at network devices.
In networks, it is desirable to measure packet traffic coming into and going out of a network device. Networking devices, such as switches and routers, examine packets to determine how to forward the packets. The forwarding process requires packet processing, such as developing forwarding rules, security policies, as well as Quality of Service (QoS) policies, such as bandwidth metering.
The packet processing mentioned above is typically expensive, particularly if the packets are processed while coming into (ingress) or leaving from (egress) a network device. As known in the art, some network devices are able to measure incoming packet traffic, some network devices are able to measure outgoing packet traffic, and other network devices are able to measure both. However, these network devices leave much to be desired, either in terms of limited finctionality or in terms of undue operating/processing costs. Typically, network devices that are able to monitor both input and output packet traffic are unduly expensive, particularly in terms of hardware and process time. For example, tracking both ingress and egress packet traffic requires processing the packets multiple times, which is cost prohibitive. As such, to manage operating costs, network operators are often forced to use network devices that only process ingress traffic. However, in several applications it is highly desirable to meter egress packet traffic. That is, some egress processing is particularly desirable but cannot be otherwise accomplished on the ingress side of a network device.
By way of example, egress rate metering cannot effectively be performed on the ingress side of a network device, especially in a multi-chip implementation (e.g., a chassis switch). Specifically, egress rate metering cannot be performed on the ingress side of a device because each ingress chip is unaware of the processes performed by every other ingress chip. In similar fashion, in a purely egress-based solution, each egress chip would be unaware of the processes performed by every other egress chip. Accordingly, implementing a shared egress meter (e.g., metering the total bandwidth of a trunk which spans multiple chips) doesn't work in either environment.
To date, attempts to obviate the limitations described above generally include network devices that either perform packet-processing only during packet ingress, perform packet processing only during egress, or perform packet-processing during both ingress and egress. Network devices that perform packet-processing only during ingress are capable of mimicking some egress-type processing functionality by replicating logic on ingress. However, this approach leaves much to be desired as some functions, such as output metering with multiple chips, cannot be accomplished by this type of ingress processing alone. Further, network devices that perform egress packet processing only, e.g., hubs and shared-backplane switches, are generally unfavored. This is largely a result of the requirement that for such implementations, every ingress packet must be seen by every egress device in order to be processed. Accordingly, the required interconnect bandwidth quickly becomes cost prohibitive. Network devices that perform packet-processing during both ingress and egress are typically implemented as high-end devices that require separate, dedicated chips for each of ingress processing and egress processing and are likewise excessively expensive. Finally, it should be appreciated that attempts have been made that involve metering each port of each network device. However, these solutions typically lack packet processing functionality and cannot handle multi-chip output architectures.
Embodiments of the present invention provide systems and methods for metering egress packet traffic while avoiding the requirement of processing packets multiple times. According to one embodiment, a first network device sends across filter and meter indices from the ingress side to the egress side to use in determining which ports the packets will be sent from. This is further accomplished by employing filters and attaching egress meters to obtain a true count of the egress packets. As a result, the requirement of processing the packets again on the egress side is obviated. Rather, meters placed on the egress side, that are indexed by the ingress-generated pointers, count the egress packet traffic.
According to another embodiment, accurate egress packet metering is accomplished on a chassis-wide basis. Moreover, metering can be accomplished in architectures having multiple ports, where those ports may span multiple blades.
The foregoing has outlined rather broadly the features and technical advantages of the present invention in order that the detailed description of the invention that follows may be better understood. Additional features and advantages of the invention will be described hereinafter which form the subject of the claims of the invention. It should be appreciated by those skilled in the art that the conception and specific embodiment disclosed may be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the present invention. It should also be realized by those skilled in the art that such equivalent constructions do not depart from the spirit and scope of the invention as set forth in the appended claims. The novel features which are believed to be characteristic of the invention, both as to its organization and method of operation, together with fluther objects and advantages will be better understood from the following description when considered in connection with the accompanying figures. It is to be expressly understood, however, that each of the figures is provided for the purpose of illustration and description only and is not intended as a definition of the limits of the present invention.
For a more complete understanding of the present invention, reference is now made to the following descriptions taken in conjunction with the accompanying drawing, in which:
Embodiments of the present invention provide systems and methods for forwarding packets in a network while metering outgoing, i.e., egress, packet traffic on network devices. The systems and methods disclosed herein expand upon the systems and methods disclosed in U.S. Pat. No. 7,103,045, the disclosure of which is incorporated by the reference in its entirety. Further, preferred embodiments provide systems and methods of forwarding packets and metering output traffic where the packets are not required to be processed multiple times. Also, according to disclosed embodiments, a network operator is able to implement true chassis-wide output-based meters, where multiple inputs may go to multiple outputs, but the overall bandwidth is correctly metered. Finally, a network operator is able to attach these meters to individual user's unicast traffic, as well as to individual multicast streams.
It should be appreciated that, in the following discussion, use of the term “index” is meant to be interchangeable with the term “indices.” While an index will generally be used to indicate the singular, it should be understood that the concepts discussed herein generally extend to multiple indices as well. Accordingly, singular terms (e.g., “index” or “meter”) and plural terms (e.g., “indices” and “meters”) to are generally interchangeable with one another in the following discussion.
Embodiments disclosed herein employ ingress packet processing logic and additionally create a meter index or multiple meter indices. The Egress meter is computed during packet ingress and is sent to each potential destination chip in a request message. According to a preferred embodiment, once the filter logic has determined that some port on the chip may receive the packet, the meter indices are used to access a bandwidth meter associated with this destination. The meter indices may be computed in any number of ways, including: destination MAC address lookup, destination IP address lookup, BMP (Best Match Prefix: IP subnet) lookup, or ACL (Access Control List) matches (including flows). Also, it should be appreciated that one index may be computed in a first manner while additional indices may be computed in the same or a different manner, in accordance with those described above. The meter may use a single rate (i.e. peak) if the only action on over-bandwidth is to drop, or, may be multi-rate, if other actions, such as ECN (Explicit Congestion Notification) marking, are possible. Potential destination chips receive a respective meter index in a request and maintain their meters to be in sync with meters on other destination chips. Further, the destination chips act on (e.g. drop or mark) packets when the combined bandwidth from all inputs exceeds a determined threshold.
According to embodiments described herein, metering can be implemented in a number of ways. The implementation will depend upon the placement of the meters relative to the port filters and buffer space considerations. In any event, the specific configured implementation will affect how a network operator perceives the packet traffic metering.
According to a first embodiment, the meter logic is applied before any port filters. In this case, a single meter may span multiple indices of the filter (e.g. one meter for multiple VLANs). In such case, the meters are unique across the entire chassis. However, it may be desirable to apply some filters before metering, e.g., those related to security—to prevent, for example, DoS attacks by depleting the meter with dropped packets, or to virtualize a meter by using the same index for different purposes on different egress blades. In such case, the meter cannot span multiple indices in that filter (e.g. VLAN). Instead, “virtual” meters are employed so that network device ports belonging to distinct values of the filter in question do not span blades (that is, if unique sets of VLANs, for example, are constrained to only certain blades).
In another embodiment, the meter is applied before buffer considerations. As a result, a network operator perceives the meter as applying to the amount of traffic destined to a network device, regardless of output congestion. If the meter is applied after buffer considerations, then the network operator perceives the meter as counting bandwidth as actually transmitted. This allows a meter to “play catch-up” in congested traffic conditions, after the congestion has passed.
Referring now to
The process by which packets are transferred from fabric driver 140, though fabric 121, to fabric receiver 160 is a request-reply process. That is, request 150, which is characterized by a number of filter indices that specify packet forwarding criteria, is sent by fabric driver 140 to fabric Receiver 160. The filter indices in request 150 serve to determine which of ports 180 receive packets based on criteria, such as VLAN membership, etc. In response, reply 190 is sent from fabric receiver 160 to fabric driver 140. Fabric receiver 160 comprises Tables 170 and logic within each table 170 is responsible for determining which of ports 180 will receive the requested packets.
As will be discussed in detail, some of tables 170 may have logic for determining which ports will receive the packets. However, at least one of tables 170 is dedicated to computing outgoing, or egress, metering values. That is, at least one of tables 170 will have logic, e.g., counters, for keeping track of the number of packets or bytes sent from the node. Such a configuration is advantageous because it ensures that, in an architecture having multiple nodes, e.g., an architecture having multiple fabric Receivers 160, those multiple fabric receivers 160 will be kept in sync. Accordingly, each of the ports that make up a corresponding multi-destination-chip trunk would be, for example, counted as one, at least in terms of the output bandwidth that is utilized.
Referring to
Referring to
At step 310, fabric driver 140 creates a buffer request 150 having a number of filters for specifying forwarding criteria for a packet or packets. As a part of step 310, fabric driver 140 may examine an incoming packet to determine the filter indices. The filters may be based on criteria, such as, for example, VLAN membership, packet source port, trunk balancing, multicasting, unicasting, and protocols supported.
At step 315, fabric driver 140 determines which fabric receivers 160, if any, are to receive buffer request 150 by mapping the filters to masks in one or more tables 171. For example, fabric driver 140 indexes at least one table 171 with the filter indices and then performs a logical AND of the results. The result may indicate that no fabric receiver is to receive buffer request 150 because, for example, no port 180 is to receive the packet, based on the filter indices. In this case, the process 300 may end.
At step 320, fabric driver 140 sends buffer request 150 to one or more fabric receivers 160 through a fabric 121. Fabric driver 140 may send multiple buffer requests 150 to fabric receivers 160, wherein they forward packets to ports 180 coupled to those fabric receivers 160. However, embodiments may provide for a single buffer request command 150 being forwarded on to multiple fabric receivers 160 by, for example, the fabric 121. Fabric driver 140 may be referred to as a first logical unit and fabric receiver 160 as a second logical unit. For clarity, steps 320-390 will be discussed will respect to the processing done at a single fabric receiver 160. However, these steps may be undertaken by multiple fabric receivers 160, essentially simultaneously, in the event buffer request commands 150 are sent to multiple fabric receivers 160.
At step 322, fabric receiver 160 uses the meter index or meter indices to account for packet bandwidth. The accounting process begins once fabric receiver 160 receives request 150. This ensures that the meter is counted properly, regardles of which filters are employed.
At step 324, it is determined whether or not a meter is over its available bandwidth. That is, it is determined whether or not the packet traffic is exceeding the allowed bandwidth for the meter. If the meter's bandwidth is not exceeded, process 300 continues to step 330. If the meter's bandwidth is exceeded, process 300 proceeds to step 340.
At step 330, fabric receiver 160 determines which ports 180, if any, are to receive the packet(s) by mapping the filters in buffer request 150 to port masks 210 stored in tables 170. This step may comprise indexing tables 170 with the filter indices to determine bit masks 210 and combining the bit masks 210 to determine the ports 180 that are to receive the packet(s). For example, port masks 210 may be combined by performing a logical AND between the bit masks 210. The fabric receivers 160 may perform this step in parallel and independent of each other in the case that multiple fabric receivers are processing buffer request commands 150.
At step 340, if it has been determined that the bandwidth of a meter has been exceeded, or that no ports are to receive packets, fabric receiver 160 formulates a buffer reply 190 and sends it through fabric 121 to fabric driver 140. In this case, reply 190 contains instructions to fabric driver 140 to drop the packet. Upon receiving this reply, fabric driver 140 will drop the packet(s) and process 300 may end.
Alternatively, if fabric receiver 160 determines that at least one port 180 is to receive the packet, then fabric receiver 160 determines, at step 350, if there is sufficient space in the outbound memory 185 for the port(s) 180 to receive the packet at this time. The decision may be based on whether all ports 180 coupled to a given fabric receiver 160 are ready or whether at least one port 180 is ready.
If there is not sufficient space in the outbound memory 185, fabric receiver 160 formulates a buffer reply 190 that specifies that fabric driver 140 should wait a period of time and retry buffer request 150, in 360. This insufficient memory reply may be sent even if some ports 180 have sufficient room for the packet. For example, if other ports 180 do not have room and an “all-or-nothing” flag was set in buffer request 150, then fabric receiver 160 may formulate a buffer reply 190 for wait and retry. After step 360, the process 300 may repeat by fabric driver 140 waiting a specified period of time and re-sending buffer request 150. When the request is retried, it will be specially marked so step 322 will be skipped—the bandwidth has already been accounted for by the previous attempt to resolve the request.
If there is sufficient memory in outbound memory 185, fabric receiver 160 reserves the space and keeps track of a reservation number and the space reserved in outbound memory 185 for this buffer request 150, at step 370.
Continuing on with the case of sufficient space, fabric receiver 160 formulates and sends a buffer reply 190 that specifies that fabric driver 140 should send the packet, at step 380. If buffer request 150 specifies that not all ports 180 need to be ready to receive, the buffer reply 190 may indicate that at least one port 180 has enough memory associated with it. On the other hand, if the buffer request specified “all-or-nothing”, then this buffer reply 190 may indicate that all ports 180 to receive the packet have sufficient associated memory.
At step 385, fabric driver 140 sends the packet to the fabric receiver(s) 160. Fabric driver 140 may also send associated information that specifies the reservation number.
At step 390, fabric receiver 160 receives the packet and associated information that specifies the reservation number. Using the reservation number and the information it saved at step 370, fabric receiver 160 may transfer the packet into the outbound memory 185.
Following from the discussion above, the difference between
It should be appreciated that the embodiments disclosed in each of
Referring to
As shown, the first quarter of buffer request 150 command has a logical_port field 414 which may be for logical port number and may allow certain traffic to be stopped, for example, if the source equals the destination. The exemplary commands and fields maybe of any suitable length. The second quarter of buffer request command 150 contains the Vlan_I field 421, which may be for specifying a VLAN filter index (i.e., the index values shown in
SrcPort_I field 422 may be used to specify a source port index, defining the source port from which the packet originated. Mesh_I field 423 may be used to specify a mesh index and may be used as a part of a load balancing forwarding process. TrkBal_I field 424 may be used to specify a trunk balancing index and may be used, for example, to load balance between multiple links connecting two nodes. Meter1_I field 425 may be used to specify a first outbound meter index, as described in this application. Dst_chip field 426 may be for specifying the destination fabric receiver 160 for this buffer request 150. Fabric 121 may use this field when routing the buffer request 150.
Referring now to the third and fourth quarters of request buffer 150 of
Length field 428 may be for indicating the number of bytes necessary for the packet. Src_chip field 429 may be for identifying the chip (e.g., fabric driver 140) from which the request is originating and may be used to route the buffer reply 190. Src_tag field 430 may specify information that is echoed back in buffer reply 190 to associate the buffer reply 190 with the buffer request 150.
Referring now to the fourth quarter of the buffer request command 150, Mcast_I field 431 may be for specifying a multicast index, which is used by fabric receiver 160 to index a table 170 of bit masks 210 to determined which ports 180 are to receive the multicast. Dpath field 432 may be for a filter index and may specify data path information that may be related to protocols, for example. TrkUcastFld_I field 433 may be for a TrunklUnicast/Flood filter index and may be used in specifying the destination logical port(s) for unicast packet(s). Proto_I field 434 may be for a protocol filter index and may be for identifying protocols, for example. Meter2_I field 435 may be reserved for a future meter index. Dst_chip field 436 may be identical to the dst_chip information from the second quarter of buffer request command 150.
Although the present invention and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the invention as defined by the appended claims. Moreover, the scope of the present application is not intended to be limited to the particular embodiments of the process, machine, manufacture, composition of matter, means, methods and steps described in the specification. As one of ordinary skill in the art will readily appreciate from the disclosure of the present invention, processes, machines, manufacture, compositions of matter, means, methods, or steps, presently existing or later to be developed that perform substantially the same function or achieve substantially the same result as the corresponding embodiments described herein may be utilized according to the present invention. Accordingly, the appended claims are intended to include within their scope such processes, machines, manufacture, compositions of matter, means, methods, or steps.
