File systems include directories, where the directories provide a means for organizing the contents of the file system. The directory includes a number of directory entries. Each directory entry is as a <name, value> pair, where name corresponds to string, typically 1 to 255 characters in length and the value corresponds to a fixed-size number (e.g., 32 bits). No additional metadata is stored with (or in) the directory entry.
When the directory entries are stored on disk, they are stored in accordance with an on-disk format. Typically, the on-disk format corresponds to storing each directory entry as a <name, value> pair in the order in which the directory entry was added to the directory. Said another way, the directory entries are laid out on disk in a sequential manner, based on the order in which the directory entry was added to the directory. Further, each new directory entry added to the directory is stored after the last directory entry that was stored in the directory. Thus, once a directory entry is laid on disk, its location does not change.
File systems typically include functionality to perform various actions on a directory. These actions typically include: inserting a new directory entry, removing a directory entry, looking up a specific directory entry, and listing all the entries in a directory. With respect to listing all directory entries in the directory, the file system (or a process associated with the file system) starts at the location of the first directory entry on disk and proceeds to sequentially read the directory entries from disk. If the listing operation is halted (or pauses) prior to completing the listing of all entries in the directory, then the file system (or the process associated with the file system) tracks the last physical location on disk, from which a directory entry was obtained. Once the listing operation resumes, the listing operation obtains the last physical location and starts listing directory entries from this location.
In general, in one aspect, the invention relates to a disk, which includes plurality of files, a directory associated with the plurality of files comprising a plurality of directory entries, wherein each of the plurality of directory entries is associated one of the plurality of files, wherein each of the plurality of directory entries is associated with a collision differentiator (CD), wherein a hash value calculated for each of the plurality of directory entries is used to determine the CD associated with each of the plurality of directory entries.
In general, in one aspect the invention relates to method for inserting a new directory entry into a directory, which includes obtaining a calculated hash value for the new directory entry, determining whether the calculated hash value is equal to a hash value associated with any of a plurality of directory entries currently stored in the directory, if the calculated hash value is equal to a hash value associated with any of the plurality of directory entries currently stored in the directory, determining a lowest unused collision differentiator (LCD) associated with any of the plurality of directory entries associated with the hash value equal to the calculated hash value, associating the new directory entry with a new CD, wherein the new CD is set to LCD, and if the calculated hash value is not equal to a hash value associated with any of the plurality of directory entries currently stored in the directory: associating the new directory entry with the new CD, wherein the new CD is set to zero, and storing the new directory entry and the new CD in the directory using the hash value and the new CD.
In general, in one aspect, the invention relates to a method for listing directory entries in a directory, that includes retrieving a first potion of directory entries, wherein the first portion of directory entries are retrieved in <hash value, collision differentiator (CD)> order, determining a cookie associated with last directory entry retrieved in the first portion of directory entries, wherein the cookie comprises a <hash value, CD> pair for the last directory entry, and retrieving a second portion of directory entries starting at a directory entry after a directory entry referenced by the cookie.
Other aspects of the invention will be apparent from the following description and the appended claims.
Specific embodiments of the invention will now be described in detail with reference to the accompanying figures. Like elements in the various figures are denoted by like reference numerals for consistency.
In the following detailed description of one or more embodiments of the invention, numerous specific details are set forth in order to provide a more thorough understanding of the invention. However, it will be apparent to one of ordinary skill in the art that the invention may be practiced without these specific details. In other instances, well-known features have not been described in detail to avoid obscuring the invention.
In general, embodiments of the invention relate to a method and system for organizing a file system directory on disk. More specifically, embodiments of the invention relate to a method and apparatus for using extensible hashing techniques to organize directory entries on disk and to retrieve directory entries from the disk.
In one embodiment of the invention, the hash value (114, 116, 118, 120, 122, 124) associated with a given DE (e.g., DE 1, DE 2, DE 3, DE 4, DE 5, DE 6) may be obtained using any hashing algorithm (e.g., Message Digest (MD)-5). Further, the entire DE (i.e., the name and the value) or a portion of the DE may be used as input to the hashing algorithm.
In one embodiment of the invention, the CD corresponds to an n-bit value. The CD is associated with DE at the time the DE is inserted into the directory. The calculation of the CD for a given entry is discussed in
In one embodiment of the invention, each hash bucket (140, 142, 144) includes a pointer to a head of a linked list, where the linked list includes all the DEs that have the same hash value. In addition, the DEs in the linked list are organized such that DEs, starting at the head, are organized in ascending CD order. For example, hash bucket 1 (140) points to a linked list that includes three entries: a first entry containing DE 1 (146), a second entry containing DE 2 (148), and a third entry containing DE 3 (150). Further, DE 1 is associated with CD=0, DE 2 is associated with CD=1, and DE 3 is associated with CD=8. In one embodiment of the invention, each hash bucket (140, 142, 144) may include a pointer to an array as opposed to a linked list.
Further, those skilled in the art will appreciate that not all has buckets (140, 142, 144) may be associated with an entry. For example, hash bucket 2 (142) is not associated with any entries. Further, each hash bucket (140, 142, 144) may include a variable number of entries. For example, hash bucket 1 (140) includes three entries (146, 148, 150), while hash bucket N (144) only includes one entry (152).
If the calculated hash value is equal to a hash value for a directory entry currently stored in the directory, then the lowest unused CD (LCD) associated with all of directory entries having the same hash value is determined (ST106).
For example, if the directory includes three directory entries having hash values equal with CD's equal to 0, 1, and 99, then the LCD would be 2. Alternatively, if the CD's were 0, 1, and 2, then LCD would be 3. Note that the use of LCD allows the directory to support arbitrary remove of entries and reuse of CD's associated with a given hash value.
Returning to
Alternatively, if the calculated hash value is not equal to a hash value for any directory entry currently stored in the directory, then a CD=0 is associated with the directory entry to be inserted into the directory (i.e., the directory entry received in ST100) (ST108). Once the CD for the directory entry to be inserted into the directory (i.e., the directory entry received in ST100) is determined (see ST104, ST106, ST108, ST112), the directory entry along with the associated CD is stored in the directory at a location determined using the hash value and the CD (ST110). In one embodiment of the invention, the hash value is also stored with the directory entry and CD.
A determination is then made about whether the listing of all directory entries is complete (ST128). If all directory entries have been listed (i.e., the cookie corresponds to the last directory entry in <hash value, CD> order), then the process ends. Alternatively, if there are remaining directory entries to retrieve, then another portion of directory entries is retrieved starting at the directory entry following the directory entry corresponding to the cookie (ST130). The process then proceeds to ST126. Those skilled in the art will appreciate that the number of directory entries retrieved in each portion may vary from implementation to implementation.
The invention may be implemented on virtually any type of computer regardless of the platform being used. For example, as shown in
Further, those skilled in the art will appreciate that one or more elements of the aforementioned computer (100) may be located at a remote location and connected to the other elements over a network. Further, the invention may be implemented on a distributed system having a plurality of nodes, where each portion of the invention may be located on a different node within the distributed system. In one embodiment of the invention, the node corresponds to a computer system. Alternatively, the node may correspond to a processor with associated physical memory.
While the invention has been described with respect to a limited number of embodiments, those skilled in the art, having benefit of this disclosure, will appreciate that other embodiments can be devised which do not depart from the scope of the invention as disclosed herein. Accordingly, the scope of the invention should be limited only by the attached claims.
This application claims benefit of U.S. Provisional Application Ser. No. “Extensible Hashing for File System Directories” (Application Ser. No. 60/733,850) filed on Nov. 4, 2005 in the names of Matthew A. Ahrens and Mark J. Maybee and is hereby incorporated by reference.
Number | Date | Country | |
---|---|---|---|
60733850 | Nov 2005 | US |