Hashing is a well-known method in the computer industry, by which large number values (called Keys and denoted by “K”) are compressed (Hashed) to smaller number values (called hashed numbers and denoted by “h”), in order to make it practical to use it as index for lookup tables. Using the key “K” directly as a lookup table index, without Hashing, requires a very large memory of size 2^n (2 to the power n, where n is the size of K in number of bits), which may have very little data scattered in it. Hashing will basically optimize the memory size requirements when large number values are to be used for indexing.
A well-known problem in Hashing is called “Collision”. Collision happens when two or more Keys (K) are hashed to the same small Hashed value (h). Collision is possible because the mapping from the Keys to Hashed values is not a 1:1 relationship, rather, it is an N:1 relationship, in which many (N) Keys can map the same Hashed value (h).
There are also numerous Hashing functions that are used to convert Keys (K) to Hashed values (h). The ideal Hashing functions are those that are more random and have less correlation between Keys (K) and Hashed values (h). There are also numerous methods in the industry to resolve hashing collisions, which include, but are not limited to:
This invention uses unique Hashing functions and Hashing collision mechanisms that are novel. The main purpose of this invention is, for example, for Ethernet MAC address lookup and double VLAN tag lookups, but it can be used for other lookup purposes, as well.
This invention consists of unique hashing functions, as well as a novel hashing collision resolution method. The collision resolution method consists of populating the hash lookup tables, while doing the collision resolution.
The invention uses multiple hashing functions (example 3), to create multiple hashing tables. Starting for the first hash function, we do not do the lookup (i.e. do not have to read, to speed up the process), because a flag tells us if we have to go to the next hash function.
For the first function overflow, we try the overflow flag, and if that flag is zero, then we go to the next hash function, and repeat this process again and again.
Each hashing table entry consists of multiple index values (example 3). In order to select the candidate index, a matching of a 2-bit extract of the Key is done against the stored extracts corresponding to each of those indexes. The candidate index goes through a final check, by doing a reverse lookup from Index to Key on a table that has stored the key.
Given 3 distinct keys, it is always possible to choose 2 bit positions that result in 3 distinct extracts. So, in each entry in H0_TBL, H1_TBL etc., the selected two-bit positions are stored, as are the resulting extracts, one for each candidate index. This allows the correct candidate index, and thus, the correct candidate record, to be selected, without having to actually inspect the keys in all 3 records.
This invention comprises of two sets of functions/methods:
Hashing function: There are 6 hashing functions used in this invention. The hash Key (K) is of length “N”. When a hash key has a shorter length than “N”, then it is extended to length N by appending enough zeros to it. Two actual implementations are described below:
Implementation 1 (Double VLAN tag lookup in Ethernet): In this implementation, the hash key contains 37 non-zero bits that are zero, extended to 54 bits, when used in the hash function calculations. The bits in the key are:
The first sets of hash functions for this invention are:
In the above expressions, bit extraction from an expression is shown with [left: right]. E.g. if exp1 is 5 then exp1 [2:1] is 2. The symbol “^” above represents XOR function.
The symbol “RoL” represents the Rotate-Left (RoL) function. A Rotate-Left (RoL) function does a bit-wise rotate left for a number of bits. The result is thereby a value with the same number of bits. E.g. if (0x1234 RoL 4)=0x2341, and (0x1234 RoL 12)=0x4123.
Implementation 2 (L2 MAC address lookup in Ethernet): In this implementation the hash key contains 61 non-zero bits that are zero extended to 72 bits, when used in the hash function calculations. The bits in the Key are:
The hash functions are:
Hashing collision resolution method: Any hashing would naturally map many keys to the same hashed number. This is called collision. This invention uses a unique method for collision resolution, which consists of combining 3 methods, namely Chaining, Overflowing and re-hashing.
The collision resolution method consists of populating the hash lookup tables, while doing the collision resolution.
The invention uses multiple hashing functions (example 3), to create multiple hashing tables. Starting for the first hash function, we do not do the lookup (i.e. do not have to read, to speed up the process), because a flag tells us if we have to go to the next hash function.
For the first function overflow, we try the overflow flag, and if that flag is zero, then we go to the next hash function, and repeat this process as needed. The second stage is to do a lookup of the keys, based on the state built in the 1st stage.
Each hashing table entry consists of multiple index values (example 3). In order to select the candidate index, a matching of a 2-bit extract of the Key is done against the stored extracts corresponding to each of those indexes. The candidate index goes through a final check, by doing a reverse lookup from Index to Key on a table that has stored the key.
The purpose of the hash lookup is to find a unique index for a specific key. The mapping between key and index is configured in the main hash table called H0_TBL [ ] using the main hashing function called H0 (key). Each entry in the hash tables can hold three index values (corresponding to 3 key values that hash to the same hash number), and additional information is provided in the entry in order to pick one of the index values based on the key. This part of the algorithm is known as “chaining” method. When more than 3 keys map to the same hash value, a second hash table called H1_TB[ ] is used that has similar structure to the main hash table, in which there are also 3 index entries for each hash value, but use a different hashing function called H1(key). Note that the 3 corresponding index entries from previous table are remapped to 3 new entries in the new table. Similarly when more than 3 keys map to the same hash value in the 2nd table, a 3rd hash table called H2_TB[ ] is used that has similar structure to the main hash table, in which there are also 3 index entries for each hash value, but use a different hashing function called H2(key). This part of the algorithm is a combination of “overflowing” and “re-hashing” methods. This hierarchy can continue to as many tables as desired. However, statistical calculations show that with 3 hashing tables, the possibility of collision can be reduced to a negligible value.
An example to illustrate the populating the hashing tables are illustrated in
The hash tables H0_TBL [ ], H1_TBL [ ] and H2_TBL [ ] hold the actual Key to Index mappings data, while the select tables, H0_SEL_TBL [ ] and H1_SEL_TBL [ ], indicate if the corresponding hash table is to be used, or if the relation is to be found in the next hash table. There is no select table for hash table H2_TBL [ ], since H2_TBL[ ] is the last table. The hash selection table has one bit entry for every row of its corresponding hash table. A value of “1” indicates that the corresponding row is valid and a value of “0” indicates that the lookup must be done on the next hash table.
Seven key-to-index relations are inserted, starting with no relations in the tables. As illustrated in
The example shows that congestion in a hash table is handled by redistributing the entries in that table to the next hash table. Due to using different hash functions, the new distribution is likely to resolve the congestion. If, however, congestion in hash table H2_TBL [ ] occurs, then there is no resolution since there is no next level. In this case, it is not possible to insert the new relation. To limit the probability of such critical congestions, there are more hash table entries than the maximum number of key to index relations. The result is that the risk of critical congestion is very low.
Assuming there are N hash tables and N−1 select tables as explained above, in order to do a lookup of a certain key, first the key is hashed using first hash function and the hash value is used to do a lookup on the first hash select table. If the corresponding select bit is “1”, the candidate index should be searched in the first hash table, otherwise, the index is not in the first hash table and a lookup must be done on the second hash select and hash tables, and this process continues until a candidate index is found.
Each entry in the hash tables can hold three index values, and additional information is provided in the entry in order to pick one of the index values, based on the key.
The hash tables, H0_TBL [ ], H1_TBL [ ] and H2_TBL [ ] each contain three index values per entry. The Key is used to differentiate the index values, and select only one of the indices. The format of a hash table entry is shown in
As can be seen from
In
Hn_TBL [ ].extractGen must be selected so that the key extract values are different for valid index values. Such a value of Hn_TBL [ ].extractGen can always be selected, since all three key values for the index values are different.
Note that the symbol “&” denotes concatenation.
If a match is found, it is called the “candidate index”, since a key for which no index is associated can also return a candidate index. The candidate index is therefore finally qualified, to ensure that the key for the candidate index matches the original key. The final qualification of candidate index is done by a reverse lookup from Index to Key is of a table that holds the Keys related to each Index. The reverse lookup ensures that the correct index is found.
Any variations of the above teaching are also intended to be covered by this patent application.
Number | Name | Date | Kind |
---|---|---|---|
6889225 | Cheng et al. | May 2005 | B2 |
6917934 | Cheng et al. | Jul 2005 | B2 |
7107258 | Cheng et al. | Sep 2006 | B2 |
7822927 | Scheer | Oct 2010 | B1 |
20060143168 | Rossmann | Jun 2006 | A1 |
Number | Date | Country | |
---|---|---|---|
20090292721 A1 | Nov 2009 | US |