Patent Application
20120096220
The present disclosure relates to improved techniques for compressing packet classifiers.
Packet classification is the core mechanism that enables many networking devices, such as routers and firewalls, to perform services such as packet filtering, virtual private networks (VPNs), network address translation (NAT), quality of service (QoS), load balancing, traffic accounting and monitoring, differentiated services (Diffserv), etc. The essential problem is to compare each packet with a list of predefined rules, which we call a packet classifier, and find the first (i.e., highest priority) rule that the packet matches. Table 1 below shows an example packet classifier of three rules. The format of these rules is based upon the format used in access control lists (ACLs), such as those found on Cisco routers. In this disclosure, the terms packet classifiers, ACLs rule lists, and lookup tables are used interchangeably.
Hardware-based packet classification using Ternary Content Addressable Memories (TCAMs) is now the de facto industry standard. TCAM-based packet classification is widely used because Internet routers need to classify every packet on the wire. Although software based packet classification has been extensively studied, these techniques cannot match the wire speed performance of TCAM-based packet classification systems.
As a traditional random access memory chip receives an address and returns the content of the memory at that address, a TCAM chip works in a reverse manner. That is, it receives content and returns the address of the first entry where the content lies in the TCAM in constant time (i.e., a few clock cycles). Exploiting this hardware feature, TCAM-based packet classifiers store a rule in each entry as an array of 0's, 1's, or *'s (don't-care values). A packet header (i.e., a search key) matches an entry if and only if their corresponding 0's and 1's match. Given a search key to a TCAM, the hardware circuits compare the key with all its occupied entries in parallel and return the index (or the content, depending on the chip architecture and configuration,) of the first matching entry.
Although TCAM-based packet classification is currently the de facto standard in industry, TCAMs do have several limitations. First, TCAM chips have limited capacity. The largest available TCAM chip has a capacity of 36 megabits (Mb). Smaller TCAM chips are the most popular due to the other limitations of TCAM chips stated below. Second, TCAMs require packet classification rules to be in ternary format. This leads to the well-known range expansion problem, i.e., converting packet classification rules to ternary format results in a much larger number of TCAM rules, which exacerbates the problem of limited capacity TCAMs. In a typical packet classification rule, the three fields of source and destination IP addresses and protocol type are specified as prefixes (e.g., 1011****) where all the *s are at the end of the ternary string, so the fields can be directly stored in a TCAM. However, the remaining two fields of source and destination port numbers are specified in ranges (i.e., integer intervals such as [1, 65534]), which need to be converted to one or more prefixes before being stored in a TCAM. This can lead to a significant increase in the number of TCAM entries needed to encode a rule. For example, 30 prefixes are needed to represent the single range [1, 65534], so 30×30=900 TCAM entries are required to represent the single rule r1 in Table 1. Third, TCAM chips consume lots of power. The power consumption of a TCAM chip is about 1.85 Watts per Mb. This is roughly 30 times larger than a comparably sized SRAM chip. TCAMs consume lots of power because every memory access searches the entire active memory in parallel. That is, a TCAM is not just memory, but memory and a (very fast) parallel search system. Fourth, TCAMs generate lots of heat due to their high power consumption. Fifth, a TCAM chip occupies a large footprint on a line card. A TCAM chip occupies 6 times (or more) board space than an equivalent capacity SRAM chip. For networking devices such as routers, area efficiency of the circuit board is a critical issue. Finally, TCAMs are expensive, costing hundreds of dollars even in large quantities. TCAM chips often cost more than network processors. The high price of TCAMs is mainly due to their large die area, not their market size. Power consumption, heat generation, board space, and cost lead to system designers using smaller TCAM chips than the largest available. For example, TCAM components are often restricted to at most 10% a of an entire board's power budget, so a 36 Mb TCAM may not be deployable on all routers due to power consumption reasons.
While TCAM-based packet classification is the current industry standard, the above limitations imply that existing TCAM-based solutions may not be able to scale up to meet the future packet classification needs of the rapidly growing Internet. Specifically, packet classifiers are growing rapidly in size and width due to several causes. First, the deployment of new Internet services and the rise of new security threats lead to larger and more complex packet classification rule sets. While traditional packet classification rules mostly examine the five standard header fields, new classification applications begin to examine addition fields such as classifier-id, protocol flags, ToS (type of service), switch-port numbers, security tags, etc. Second, with the increasing adoption of IPv6, the number of bits required to represent source and destination IP address will grow from 64 to 256. The size and width growth of packet classifiers puts more demand on TCAM capacity, power consumption, and heat dissipation.
To address the above TCAM limitations and ensure the scalability of TCAM-based packet classification, the TCAM-based classifier compression problem has been studied; that is, given a packet classifier, efficiently generate a semantically equivalent packet classifier that requires fewer TCAM entries. Note that two packet classifiers are (semantically) equivalent if and only if they have the same decision for every packet. TCAM-based classifier compression helps to address the limited capacity of deployed TCAMs because reducing the number of TCAM entries effectively increases the fixed capacity of a chip. Reducing the number of rules in a TCAM directly reduces power consumption and heat generation because the energy consumed by a TCAM grows linearly with the number of ternary rules it stores. Finally, TCAM-based classifier compression lets us use smaller TCAMs, which results in less power consumption, less heat generation, less board space, and lower hardware cost.
Several prior TCAM-based classifier compression schemes have been developed. While these techniques vary in effectiveness, they all suffer from one fundamental limitation: they only produce prefix classifiers, which means they all miss some opportunities for compression. Therefore, it is desirable to provide a new TCAM-based classifier compression scheme that is not limited to producing prefix classifiers.
This section provides background information related to the present disclosure which is not necessarily prior art.
An improved technique is provided for compressing a packet classifier for a computer network system. A set of packet classification rules is first partitioned into one or more partitions. For each partition, columns of bits in each of the ternary strings of a given partition are reordered, the ternary strings within each partition are consolidated into one or more replacement strings and then the columns of bits of the replacement strings are rearranged back to the starting order. The rearranged replacement strings from each of the partitions are appended together to form a compressed packet classifier which may be instantiated in a content-addressable memory device.
In one aspect of the disclosure, the columns of bits are arranged to form ternary strings having a prefix format.
In another aspect of the disclosure, the ternary strings are consolidated by merging ternary strings that differ by one bit together.
This section provides a general summary of the disclosure, and is not a comprehensive disclosure of its full scope or all of its features. Further areas of applicability will become apparent from the description provided herein. The description and specific examples in this summary are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure.
The drawings described herein are for illustrative purposes only of selected embodiments and not all possible implementations, and are not intended to limit the scope of the present disclosure. Corresponding reference numerals indicate corresponding parts throughout the several views of the drawings.
Bit weaving applies two new techniques, bit swapping and bit merging, to first identify and then merge such rules together. Bit swapping first cuts a rule list at 12 into a series of partitions. Within each partition, a single permutation is applied at 13 to each rule's predicate to produce a reordered rule predicate, which forms a single prefix where all *'s are at the end of the rule predicate. This single prefix format allows the use of existing dynamic programming techniques to find a minimal TCAM table for each partition in polynomial time. Bit merging then searches each partition and merges together rules at 15 that differ by a single bit. Upon completing bit merging, all ternary strings are reverted back at 16 to their original bit permutation and appended together at 17 to form a compressed packet classifier. Lastly, the packet classifier is instantiated at 18 in a content-addressable memory device.
An example of the bit weaving approach is further illustrated in
Next the concepts of fields, packets, and packet classifiers are formally defined below. A field Fi is a variable of finite length (i.e., of a finite number of bits). The domain of field Fi of w bits, denoted D(Fi,), is [0, 2w−1]. A packet over the d fields F1, . . . , Fd is a d-tuple (p1, . . . , pd) where each pi (1≦i≦d) is an element of D(Fi,). Packet classifiers usually check the following five fields: source IP address, destination IP address, source port number, destination port number, and protocol type. The lengths of these packet fields are 32, 32, 16, 16, and 8, respectively. It is understood that the bit weaving approach is applicable to classifiers having different fields and/or field with differing lengths. Σ is used to denote the set of all packets over fields Fi, . . . , Fd. It follows that Σ is a finite set and |Σ|=ID(F1)I x . . . x ID(Fd)I, where |Σ| denotes the number of elements in set Σ and ID(Fi) I denotes the number of elements in set D(Fi).
A rule has the form (predicate)→(decision). A (predicate) defines a set of packets over the fields F1 through Fd, and is specified as F1, εS1 . . .
FdεSd where each Si, is a subset of D(Fi) and is specified as either a ternary string, a prefix or a nonnegative integer interval. A ternary string {0, 1, *}k denotes the set of integers whose binary representation matches the ternary string. For purposes of matching, the * is treated as a wild card. For example, the string *0 denotes the set {0, 2}. A prefix {0, 1}k{*}w-k with k, leading 0 s or 1 s for a packet field of length w denotes the integer interval [{0, 1}k{0}w-k, {0, 1}k{1}w-k] and is a special case of a ternary string. For example, prefix 01** denotes the interval [0100,0111]. In other words, a prefix rule is a rule in which each field is specified as a prefix bit string, where * all appear at the end of the string in the least significant bit positions. A rule F1, εS1
. . .
FdεSd→(decision) is a prefix rule if and only if each Si, is a prefix, and a rule is a ternary rule if and only if each Si is a ternary string. Every prefix rule is a ternary rule, but not vice versa.
A packet matches a rule if and only if the packet matches the predicate of the rule. A packet (p1, . . . , pd) matches a predicate F1, εS1 . . .
Fd εSd if and only if the condition p1, εS1
. . .
pd εSd holds. DS is used to denote the set of possible values that (decision) can be. Typical elements of DS include accept, discard, accept with logging, and discard with logging.
A sequence of rules (r1, . . . , rn;) is complete if and only if for any packet p, there is at least one rule in the sequence that p matches. To ensure that a sequence of rules is complete and thus a packet classifier, the predicate of the last rule is usually specified as F1εD (F1) . . . Fdε
D(Fd). A packet classifier C is a sequence of rules that is complete. The size of C, denoted |C|, is the number of rules in C. A packet classifier C is a prefix packet classifier if and only if every rule in C is a prefix rule, and a packet classifier C is a ternary packet classier if and only if every rule in C is a ternary rule. Every prefix classifier is a ternary classifier, but not vice versa.
Two rules in a packet classifier may overlap; that is, a single packet may match both rules. Furthermore, two rules in a packet classifier may conflict; that is, the two rules not only overlap but also have different decisions. Packet classifiers typically resolve such conflicts by employing a first-match resolution strategy where the decision for a packet p is the decision of the first (i.e., highest priority) rule that p matches in C. The decision that packet classifier C makes for packet p is denoted C(p).
A packet classifier C can be thought of as defining a many-to-one mapping function from Σ to DS. Two packet classifiers C1 and C2 are equivalent, denoted C1≡C2, if and only if they define the same mapping function from Σ to DS that is, for any packet p εΣ, C1 (p)=C2 (p). For any classifier C, denote the set of equivalent classifiers as {C}. A rule is redundant in a classifier if and only if removing the rule does not change the semantics of the classifier.
In a typical packet classifier rule, the fields of source IP, destination IP, and protocol type are specified in prefix format, which can be directly stored in TCAMs; however, the remaining two fields of source port and destination port are specified as ranges (i.e., non-negative integer intervals), which are typically converted to prefixes before being stored in TCAMs. This leads to range expansion, the process of converting a non-prefix rule to prefix rules. In range expansion, each field of a rule is first expanded separately. The goal is to find a minimum set of prefixes such that the union of the prefixes corresponds to the range. For example, if one 3-bit field of a rule is the range [1, 6], a corresponding minimum set of prefixes would be 001, 01*, 10*, 110. The worst-case range expansion of a w-bit range results in a set containing 2w −2 prefixes. The next step is to compute the cross product of the set of prefixes for each field, resulting in a potentially large number of prefix rules.
Given these formal definitions, the bit swapping technique is described in more detail. A bit-swap β of a length m ternary string t, is a permutation of the m ternary bits; that is, β rearranges the order of the ternary bits t. The resulting permuted ternary string is denoted β(t). For example, if β is permutation 312 and string t is 0*1, then β(t)=10*. For any length m string, there are m! different permutations and thus m! different bit-swaps. A bit-swap β of a ternary string t is a prefix bit-swap if the permuted string β(t) is in prefix format. Let P(t) denote the set of prefix bit-swaps for ternary string t: specifically, the bit-swaps that move the * bits of t to the end of the string.
A bit-swap β can be applied to a list l of ternary strings (t1, . . . , tn) where l is typically a list of consecutive rules in a packet classifier. The resulting list of permuted strings is denoted as β(l). Bit-swap β is a prefix bit-swap for l if β is a prefix bit-swap for every string ti in list l for 1≦i.≦n. Let P(l) denote the set of prefix bit-swaps for list l. It follows that
Prefix bit-swaps are useful for compressing classifiers for at least two reasons. First, there are known minimization algorithm that can be used to minimize prefix rule lists. These algorithms depend on the structure of prefix rules to run in polynomial time, so converting rules lists into prefix rules is a preferred step. Second, prefix format facilitates the second key idea of bit weaving, bit merging. Specifically, special properties of prefix format rules to identify candidate rules that be merged together without changing the semantics of the classifier. After bit merging, the classifier is reverted to its original bit order, which typically results in a non-prefix format classifier.
Unfortunately, given a list l of ternary strings, it is possible that P(l)=(/), which means that no bit-swap is a prefix bit-swap for every string in l. For example, the list 0*,*0
does not have a valid prefix bit-swap. The necessary and sufficient conditions for P(l)=(/) are presented after defining certain notation.
Given ternary strings x and y, let x[i] and y[i] denote the ith ternary bit in x and y respectively. Define the relation x y to be shorthand for {i|x[i]=*}
{j|y[j]=*}. For example, 0*1
1** but 0**
0*0. Two ternary strings t1 and t2 form a cross pattern if and only if (t1
t2)
(t2
t1). In such cases, ti crosses t2. First, observe that bit swaps have no effect on whether or not two strings cross each other. That is, two ternary strings, t1 and t2, and a bit-swap β, t1
t2 if and only if β(ti)
β(t2), and t1
t2 if and only if β(t1)
β(t2).
Accordingly, given a list l=t1, . . . , tn
of n ternary strings, P(l)≠(/) if and only if no two ternary strings ti and tj, (l≦i<j≦n) cross each other (referred to as first theorem). The implication is that it is given that there exists a prefix bit-swap βεP(l). Suppose that string ti crosses string tj, then β(ti) crosses β(tj) according to the observation above. This implies that one of the two ternary strings β(ti) and β(tj) has a * before a 0 or I and henceforth is not in prefix format. Thus, β is not in P(l), which is a contra-diction.
Conversely, it is given that no two ternary strings cross each other, it follows that we can impose a total order on the ternary strings in l using the relation . Note, there may be more than one total order if ti
tj and tj
t1 for some values of i and j. Lets reorder the ternary strings in l according to this total order; that is, t′1
t′2
. . .
t′n-1
t′n. Any bit swap that puts the * bit positions of t′1 last, preceded by the * bit positions of t′2, . . . , preceded by the* bit positions of t′n, finally preceded by all the remaining bit positions will be a prefix bit-swap for l. Thus, the result follows.
The first theorem above gives us a simple algorithm for detecting whether a prefix bit-swap exists for a list of ternary strings. If a prefix bit-swap exists, the proof of the theorem yields a simple and elegant algorithm for constructing a prefix bit-swap. That is, simply sort ternary bit positions in increasing order by the number of ternary strings that have a * in that bit position. Psuedo code for this algorithm is set forth below.
(i, j) | 1 ≦ i ≦ n and j is the number of *'s in M[1... n, ]
;
Before formally presenting the bit swapping algorithm, the concepts of bit matrix and decision array for a possibly incomplete rule list (i.e., there may exist at least one packet that none of the n, rules matches) are defined. Any list of n rules defines a bit matrix M[l . . . n,1 . . . b] and a decision array D[I . . . n], where for any 1≦i≦n and 1≦j≦b, M[i, j] is the j-th bit in the predicate of the i-th rule and D[i] is the decision of the i-th rule. Conversely, a bit matrix M[1 . . . n, I . . . b] and a decision array D[1 . . . n] also uniquely defines a rule list. Given a bit matrix M[0 . . . n−1, 0 . . . b−1] and a decision array D[I . . . n] defined by a rule list, the bit swapping algorithm swaps the columns in M such that for any two columns i and j in the resulting bit matrix M′ where i<j, the number of *s in the i-th column is less than or equal to the number of *s in the j-th column.
Prior to bit swapping, the rule set is partitioned into one or more partitions in the manner set forth below. Given a classifier C, if P(C)=, cut C into partitions where each partition has no cross patterns and thus has a prefix bit-swap. Classifier C is treated as a list of ternary strings by ignoring the decision of each rule.
Given an n-rule classifier C=r1, . . . , rn
, a partition P on C is a list of consecutive rules
ri, . . . , rj
in C for some i and j such that 1≦i≦j≦n. A partitioning, P1, . . . , Pk, of C is a series of k partitions on C such that the concatenation of P1, . . . , Pk, is C. A partitioning is cross-free if and only if each partition has no cross patterns. Given a classifier C, a cross-free partitioning with k partitions is minimal if and only if any partitioning of C with k−1 partitions is not cross-free.
To maximize bit weaving's effectiveness, an algorithm is developed that finds a minimal cross-free partitioning for a given classifier. At any time, there is one active partition. In an exemplary embodiment, the initial active partition is started with the last rule of the classifier to facilitate efficient incremental updates. New rules are more likely to be added to the front of a classifier than at the end. Each rule in the classifier is then considered in reverse order.
Specifically, the next rule in the list is retrieved and checked against each rule in the active partition. When the next rule does not “cross” any rule in the active partition, the rule is added to the active partition. The next rule in the list is retrieved and the checking process is repeated. When a rule “crosses” any rule in the active partition, the currently active partition is complete and a new active partition is created using the current rule. The next rule in the list is then retrieved and the checking process continues until each rule in the list has been added to a partition. Psuedo code for this exemplary algorithm is set forth below.
r1, ... , rn
each rule has b bits.
Other types of partitioning algorithms are also contemplated within the broader aspects of this disclosure.
The core operation in the above partitioning algorithm is to check whether two ternary strings cross each other. This check requires computing the negation of x y
y
x. This computation can be performed in constant time using bitmap representations of sets. An exemplary computation is as follows. For any ternary string t of length m, define the bit mask of t, denoted M(t), to be a binary string of length m where the i-th bit (0≦i<m) M(t,)[i]=0 if t[i]=* and M(t)[i]=1 otherwise. For any two binary strings a and b, use a && b to denote the resulting binary string of the bitwise logical AND of a and b. For any two ternary string t1, and t2, t1, does not cross t2 if and only if M(t1)&& M(t2) is equal to M(t1) or M(t2) (referred to as the second theorem). For example, given two ternary strings t1=01*0 and t2=101*, whose bit masks are M(t1)=1101 M(t1)=1110, we have M(t1)&& M(t2)=1100. Therefore, t1=01 *0 crosses t2=101* because M(t1)&& M(t2)≠M(t1) and M(t1)&& M(t2)≠M(t2). Other techniques for checking whether two ternary strings cross are also contemplated.
Before bit merging, the ternary strings in each partition may be minimized or reduced using a partial prefix list minimization algorithm. Each partition may be viewed as a list of 1-dimensional prefix rules. If a list of 1-dimensional prefix rules is complete (i.e., any packet has a matching rule in the list), there are known algorithms that can to produce an equivalent minimal prefix rule list. However, the rule list in a given partition is often incomplete; that is, there exist packets that do not match any rule in the partition. There are two reasons why packets will not be classified by any rule in the partition. First, if any of the first k−1 partitions are complete, all later partitions can be deleted without changing the classifier semantics given the first match semantics of TCAMs. Thus, it can be assumed each of the first k−1 partitions do not assign a decision to some packets. Second, for any of the final k−1 partitions, many packets are classified by earlier partitions and thus will not be classified by the given partition even though these packets may match rules in the given partition.
To solve this problem of minimizing incomplete or partial rule lists, the weighted 1-dimensional prefix list minimization algorithm is adapted. Further details regarding the weighted 1-dimensional prefix list minimization algorithm may be found in U.S. patent application Ser. No. 12/578,824 entitled “Systematic Approach Towards Minimizing Packet Classifiers” which is incorporated by reference herein.
Given a 1-dimensional packet classifier f of n prefix rules r, r2, . . . , rn
, where {Decision (r1),Decision, (r2), . . . , Decision (rn)={(d1, d2, . . . , dz} and each decision di; is associated with a cost Cost(d1) (for 1≦i≦z), the cost of packet classifier f is defined as follows: Cost(f)=Σi=1n, Cost (Decision (r1)). The problem of weighted one-dimensional TCAM minimization is stated as follows: given a one-dimensional packet classifier f1 where each decision is associated with a cost, find a prefix packet classifier f2ε{f1} such that for any prefix packet classifier f ε{f1}, the condition Cost(f2)≦Cost(f) holds. To minimize a partial 1-dimensional prefix rule list L over field F, let {d1, d2, . . . , d2} be the set of all the decisions of the rules in L. Let
Next, the bit merging process is further explained. When two ternary strings t,1 and t2 differ only in one bit, i.e., their hamming distance is one, and the two strings are said to be ternary adjacent. The ternary string produced by replacing the one differing bit by a * in t1 (or t2) is called the ternary cover of t1 and t2. For example, 0** is the ternary cover for 00* and 01*. This process of replacing two ternary adjacent strings by their cover is called bit merging or just merging. For example, merge 00* and 01* to form their cover 0**.
Two rules can be bit merge (or just merge) as follows. For any rule r, use P(r) to denote the predicate of r. Two rules ri and rj are ternary adjacent if their predicates P(ri) and P(rj) are ternary adjacent. The merger of ternary adjacent rules ri and rj is a rule whose predicate is the ternary cover of P(ri) and P(rj) and whose decision is the decision of rule ri. Any two rules in a classifier can be merged into one rule without changing the classier se-mantics if and only if they satisfy the following three conditions: (1) they can be moved to be adjacent without changing the semantics of the classifier; (2) they are ternary adjacent; (3) they have the same decision in order to preserve the semantics of the classifier (referred to the third theorem). The basic idea of bit merging is to repeatedly find two rules in the same bit-swapped partition that can be merged based on the three conditions above. Merging rules from different bit swapped partitions is not considered because any two bits from the same column in the two bit-swapped rules may correspond to different columns in the original rules.
To address the first condition above, need to quickly determine what rules in a bit-swapped partition can be moved together without changing the semantics of the partition (or classifier). For any 1-dimensional minimum prefix classifier C, let C′ denote the prefix classifier formed by sorting all the rules in C in decreasing order of prefix length. For any two rules ri and rj (i<j) in a prefix classifier r1, . . . , rn
that has no upward redundant rules P(ri) ∩ P(ri)≠ if and only if P(ri) ⊂ P(rj) (referred to as a first lemma). Note that a rule r is upward redundant if and only if there are no packets whose first matching rule is r. Clearly, upward redundant rules can be removed from a classifier with no change in semantics.
For any one-dimensional minimum prefix packet classifier C, we have C≡C8 (referred to as the fourth theorem) Consider any two rules ri, rj (i<j) in C. If the prefixes of ri and rj do not overlap (i.e., P(ri) ∩ P(rj)=(/), changing the relative order between ri, and rj does not change the semantics of C. If the prefixes of ri and rj do overlap (i.e., P(ri) ∩ P(rj)=(/), then according to the first lemma, P(ri) ⊂ P(rj). This means that P(ri) is strictly longer than P(rj). This implies that ri is also listed before ri, in C*. Thus, the result follows.
Given a minimum sized prefix bit-swapped partition, first sort the rules in decreasing order of their prefix length. Second, further partition or group the rules into subgroups based on their prefix length (these subgroups are also referred to herein as prefix chunks). According to the fourth theorem, the order of the rules in each prefix chunk is irrelevant, so the rules can be reordered without changing the semantics of the partition.
To address the second condition in the third theorem, there is a need to quickly determine what rules are ternary adjacent. Based on the fourth theorem, the search space can be significantly reduced by searching for mergeable rules only among the rules which have the same bit mask and decision.
Given a list of rules where the rules have the same decision and no rule's predicate is a proper subset of another rule's predicate, if two rules are mergeable, then the hit masks of their predicates are the same (referred to as the fifth theorem). Suppose in such a list there are two rules ri, and rj that are mergeable and have different bit masks. Because they are mergeable, P(ri) and P(rj) differ in only one bit. Because the bit masks are different, one predicate must have a * and the other predicate must have a 0 or 1 in that bit column. Without loss of generality, let ri be the rule whose predicate has a *. Because the two rules have the same decision, P(ri) ⊂ P(rj), which is a contradiction.
Given this backdrop, an exemplary bit merging algorithm (BMA) works as follows. BMA takes as input a minimum, possibly incomplete prefix classifier C that corresponds to a cross-free partition generated by bit swapping. BMA first creates classifier C8 by sorting the rules of C in decreasing order of their prefix length and partitions C8 into prefix chunks. Second, for each prefix chunk, BMA groups all the rules with the same bit mask and decision together, eliminates duplicate rules, and searches within each group for mergeable rules. The second step repeats until no group contains rules that can be merged. Let C′ denote the output of the algorithm.
Psuedo code for this exemplary bit merging algorithm is set forth below.
r1, ... , rn
where each rule
This bit merging algorithm is further illustrated in
During each execution of the second step, BMA never introduces two rules ri and rj such that P(ri) ⊂ P(rj) where both rl and rj have the same decision. Consider any prefix chunk in C8. Let k be the length of the prefix of this prefix chunk. Consider any rule r in C′ that was, formed from this prefix chunk. The k-th bit of r must be 0 or 1, not *. The output of BMA, C′, contains no pair of mergeable rules.
Within each prefix chunk, after applying BMA, there are no pairs of mergeable rules for two reasons. First, in each run of the second step of the algorithm, all mergeable rules are merged. Second, repeatedly applying the second step of the algorithm guarantees that there are no mergeable rules in the end.
Next, it is shown that any two rules from different prefix chunks cannot be merged. Let ri and rj be two rules from two different prefix chunks in C′ with the same decision. Suppose ri is from the prefix chunk of length ki and rj is from the prefix chunk of length kj where ki>kj. By Lemma 5.3, the ki-th bit of ri's predicate must be 0 or 1. Because ki>kj, the ki-th bit of rj's predicate must be *. Thus, if ri and rj are mergeable, then −ri and rj should only differ in the ki-th bit of their predicates, which means P(ri) ⊂ P(rj).
Continuing the example in
Redundancy removal procedure may be applied as a preprocessing and/or a post-processing step. Redundancy removal is applied at the beginning because redundant rules may introduce more cross patterns. Redundancy removal is applied at the end because our incomplete 1-dimensional prefix list minimization algorithm may introduce redundant rules across different partitions. An exemplary redundancy removal procedure is further described in an article entitled “Complete redundancy detection in firewalls” by Alex X. Liu and Mohamed G. Gouda, In proc. 19th Annual IFIP Conf. on Data and Applications Security, LNCS 3654, August 2005. Other redundancy removal procedures may be applied.
Classifier rules periodically need to be updated when net-working services change. Sometimes classifiers are updated manually by network administrators, in which case the timing is not a concern and rerunning the fast bit weaving algorithm will suffice. Sometimes classifiers are updated automatically in an incremental fashion; in these cases, fast updates may be critically important.
The bit weaving approach set forth herein supports efficient incremental classifier changes by confining change impact to one cross-free partition. An incremental classifier change is typically one of the three actions: inserting a new rule, deleting an existing rule, or modifying a rule. Given a change, first locate the cross-free partition where the change occurs by consulting a precomputed list of all the rules in each partition. Then, rerun the bit weaving algorithm on the affected partition. There may be need to further divide the partition into cross-free partitions if the change introduces a cross pattern. Note that deleting a rule never introduces cross patterns.
Bit weaving, like redundancy removal, never returns a classifier that is larger than its input. Thus, bit weaving, like redundancy removal, can be composed with other classifier minimization schemes. Since bit weaving is an efficient algorithm, it can be applied as a post processing step with little performance penalty. As bit weaving uses techniques that are significantly different than other compression techniques, it can often provide additional compression.
Bit weaving can also enhance other compression techniques. Specifically, multiple techniques rely on generating single field TCAM tables. These approaches generate minimal prefix tables, but minimal prefix tables can be further compressed by applying bit merging. Therefore, every such technique can be enhanced with bit merging (or more generally bit weaving).
For example, TCAM Razor compresses multiple field classifiers by converting a classifier into multiple single field classifiers, finding the minimal prefix classifiers for these classifiers, and then constructing a new prefix field classifier from the prefix lists. A natural enhancement is to use bit merging to convert the minimal prefix rule lists into smaller non-prefix rule lists. Experiments have shown that bit weaving enhanced TCAM Razor yields significantly better compression results than TCAM Razor alone.
Range encoding techniques can also be enhanced by bit merging. Range encoding techniques require lookup tables to encode fields of incoming packets. When such tables are stored in TCAM, they are stored as single field classifiers. Bit merging offers a low cost method to further compress these lookup tables. Results show that bit merging significantly compresses the lookup tables formed by the topological transformation technique.
The most computationally expensive stage of bit weaving is bit merging. With the application of the binomial theorem, we arrive at a worst case time complexity of O(bxn25) where b is the number of bits within a rule predicate, and n is the number of rules in the input. Therefore, bit weaving is the first polynomial-time algorithm with a worst-case time complexity that is independent of the number of fields in that classifier. This complexity analysis excludes redundancy removal because redundancy removal is an optional pre/post-processing step. The space complexity of bit weaving is dominated by finding the minimum prefix list.
The bit weaving approach set forth above has many significant benefits. First, it is the first TCAM compression method that can create non-prefix classifiers. Previous compression methods created only prefix classifiers. This restriction to prefix format may miss important compression opportunities. Second, it is the first efficient compression method with a polynomial worst-case running time with respect to the number of fields in each rule. Third, it is orthogonal to other techniques, which means that it can be run as a pre/post-processing routine in combination with other compression techniques. Fourth, it supports fast incremental updates to classifiers.
The foregoing description of the embodiments has been provided for purposes of illustration and description. It is not intended to be exhaustive or to limit the disclosure. Individual elements or features of a particular embodiment are generally not limited to that particular embodiment, but, where applicable, are interchangeable and can be used in a selected embodiment, even if not specifically shown or described. The same may also be varied in many ways. Such variations are not to be regarded as a departure from the disclosure, and all such modifications are intended to be included within the scope of the disclosure.
This application claims the benefit of U.S. Provisional Application No. 61/293,360 filed on Jan. 8, 2010. The entire disclosure of the above application is incorporated herein by reference.
| Number | Date | Country | |
|---|---|---|---|
| 61293360 | Jan 2010 | US |
