Patent Application
20100146213
The present disclosure relates to data cache technologies, and more particularly to a data cache processing method, system and a data cache apparatus.
This section provides background information related to the present disclosure which is not necessarily prior art.
In applications of computer and Internet, in order to increase access speeds of users and decrease burdens of back-end servers, a cache technology is generally used in the front-end of a slow system or apparatus such as a database and a disk. In the cache technology, an apparatus with a rapid access speed, e.g. a memory, is used for storing data which the user often accesses. Because the access speed of the memory is much higher than that of the disk, the burden of the back-end apparatus can be decreased and user requests can be responded in time.
The cache may store various types of data, e.g. attribute data and picture data of a user, various types of files which the user needs to store, etc.
Each node stores a key, data and a pointer pointing to a next node, and is a main operating cell for caching. When the length of a node chain corresponding to a certain Hash value is not enough, an additional node chain composed of multiple nodes is set for backup, and a head pointer of the additional node chain is stored in an additional head. The additional node chain is organized as the node chain.
When one record is inserted, data to be written into a cache and a key corresponding to the data are obtained, a Hash value is determined according to the key by using a Hash algorithm, a node chain corresponding to the Hash value is traversed sequentially to search for a record corresponding to the key; if a record corresponding to the key exists, the record is updated; if a record corresponding to the key does not exist, the data is inserted into the last node of the node chain. If nodes in the node chain have been used up, the key and the data are stored in an additional node chain to which a head pointer of the additional node chain points.
When one record is read, a Hash value corresponding to the record is determined according to a key of the record by using the Hash algorithm, a node chain corresponding to the Hash value is traversed sequentially to search for a record corresponding to the key; if a record corresponding to the key does not exist, an additional node chain is searched; if a record corresponding to the key exists, data corresponding to the record are returned.
When one record is deleted, a Hash value corresponding to the record is determined according to a key of the record by using the Hash algorithm, a node chain corresponding to the Hash value is traversed sequentially to search for a record corresponding to the key; if a record corresponding to the key does not exist, an additional node chain is searched, and the key and data corresponding to the record are deleted after the record corresponding to the key is searched out.
In the conventional cache technology, since one block of data must be stored in one node, a data space in the node must be larger than the length of data to be stored. In this way, it is needed to learn the size of data to be cached before using a cache, so as to avoid that larger data can not be cached. In addition, since sizes of data in practical applications generally have large differences and each block of data needs to occupy one node, the memory space is often wasted; and the smaller the data is, the larger the wasted memory space is. Further, record searching efficiency is low; if a record is not searched out after a single node chain is searched, the additional node chain needs to be searched, and thus the consumed time is much more if the additional node chain is long.
This section provides a general summary of the disclosure, and is not a comprehensive disclosure of its full scope or all of its features.
Embodiments of the present invention provide a data cache processing method to solve a problem that memory space is wasted and record searching efficiency is low when data is cached by using a conventional cache structure.
The embodiments of the present invention are implemented as follows: a data cache processing method includes:
Embodiments of the present invention also provide a data cache processing system, including:
Embodiments of the present invention further provide a data cache apparatus, including:
In the embodiments of the present invention, nodes of a cache, memory chunks corresponding to the nodes, a key of data stored in the node, the length of data in the node and a pointer pointing to a corresponding memory chunk are configured, data are stored in the memory chunks, and various data cache processing operations are performed according to the node and the memory chunks corresponding to the node. The embodiments of the present invention have little requirements for the size of data and good universality, do not need to learn the size and distribution of stored single data, which increases the universality of cache, decreases the waste of memory space, and increases the usability of memory.
Further areas of applicability will become apparent from the description provided herein. The description and specific examples in this summary are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure.
The drawings described herein are for illustrative purposes only of selected embodiments and not all possible implementations, and are not intended to limit the scope of the present disclosure.
Corresponding reference numerals indicate corresponding parts throughout the several views of the drawings.
Example embodiments will now be described more fully with reference to the accompanying drawings.
Reference throughout this specification to “one embodiment,” “an embodiment,” “specific embodiment,” or the like in the singular or plural means that one or more particular features, structures, or characteristics described in connection with an embodiment is included in at least one embodiment of the present disclosure. Thus, the appearances of the phrases “in one embodiment” or “in an embodiment,” “in a specific embodiment,” or the like in the singular or plural in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
In order to make the object, technical schemes and merits of the present invention clearer, the present invention is described hereinafter in detail with reference to accompanying drawings and embodiments. It should be understand that the embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
In the embodiments of the present invention, nodes, memory chunks corresponding to the nodes are configured in a cache. The node stores a key of data, length of the data and a pointer pointing to the memory chunk. In the node, the length of the data is used to represent the size of data practically stored through the node. Data is stored in the memory chunks, and various data cache processing operations, e.g. inserting a record, reading a record or deleting a record, are performed according to the nodes and the memory chunks corresponding to the nodes.
The node region includes a head structure, a Hash bucket and at least one node. The head structure mainly stores the following information:
The Hash bucket mainly stores a node chain head pointer corresponding to each Hash value. According to a key corresponding to data, a Hash value corresponding to key is determined by using a Hash algorithm, the location of the Hash value at the Hash bucket is obtained, a node chain head pointer corresponding to the Hash value is searched for, so as to search out a whole node chain corresponding to the Hash value.
The node stores the following information:
In the embodiments of the present invention, node configurations, e.g. node inserting or deleting can be performed flexibly for a node chain according to the node chain former pointer and the node chain later pointer. For example, when a node is deleted, a node chain later pointer of a previous node of this node and a node chain former pointer of a next node of this node are adjusted according to the node chain former pointer and the node chain later pointer of this node, so as to make the node chain from which the node is deleted continuous.
In addition, in the embodiments of the present invention, operations of the cache, e.g. the LRU operation can be implemented by using the node using state chain head pointer, the node using state chain tail pointer, the node using state chain former pointer, the node using state chain later pointer, and the last visiting time and visiting times of the node; the LRU data in the node are removed from the memory, and memory chunks and node corresponding to the LRU data are reclaimed, so as to save the memory space.
In the embodiments of the present invention, the using state of the node is recorded, and the LRU operation is performed according to the last visiting time and visiting times of the node, so as to replace the node. When a node is visited, a node using state chain later pointer of a previous node of this node points to a next node of this node, a node using state chain former pointer of a next node of this node points to a previous node of this node, so as to make the previous node of this node connect with the next node of this node; and then the node using state chain later pointer of this node points to a node to which the node using state chain head pointer points, and the node using state chain head pointer points to this node, so that this node is inserted in the head of the node using state chain. When another node is visited, similar processing is performed, and the node using state chain tail pointer points to a LUR node. When the LRU operation is performed, data in the memory chunk corresponding to the node to which the node using state chain tail pointer points are deleted, and the memory chunk corresponding to the node is reclaimed.
The memory chunk region mainly stores a chain structure of memory chunks and data of records, and includes a head structure and at least one memory chunk.
The head structure mainly stores the following information:
The memory chunk includes a data region and a memory chunk later pointer, respectively adapted to practically store the data of the records and a next memory chunk pointer. If one memory chunk is not enough to store the data of one record, multiple memory chunks can be connected, and the data are stored in a data region corresponding to each memory chunk.
In Step S301, data to be written in a cache and a key corresponding to the data are obtained, and a Hash value is obtained according to the key by using a Hash algorithm.
In Step S302, a node chain head pointer corresponding to the Hash value is obtained according to the location of the Hash value at the Hash bucket.
In Step S303, a node chain in the Hash bucket is traversed according to the node chain head pointer, and it is determined whether the key is searched out; if the key is searched out, Step S304 is performed; otherwise, Step S308 is performed.
In Step 304, it is determined whether idle memory chunks can accommodate the data to be written into the cache after reclaiming memory chunks which store a record corresponding to the key; if the idle memory chunks can accommodate the data to be written in the cache, Step S305 is performed; otherwise, the procedure terminates.
In Step S305, data of the record corresponding to the key are deleted, and the memory chunks from which the data are deleted are reclaimed.
In Step S306, needed memory chunks are reallocated according to the length of data in the node.
In Step S307, the data are written in the allocated memory chunks in turn after the data are chunked, to form a memory chunk chain for storing the data, and a memory chunk chain head pointer of the node points to the head of the memory chunk chain.
In Step S308, it is determined whether idle memory chunks can accommodate the data to be written in the cache; if the idle memory chunks can accommodate the data to be written in the cache, Step S309 is performed; otherwise, the procedure terminates.
In Step S309, a node is taken out from an idle node chain.
In Step S310, memory chunks are allocated according to the length of the data to be stored and the size of a memory chunk, the allocated memory chunks are taken out from an idle memory chunk chain, and Step S307 is performed, i.e. the data are written in the allocated memory chunks in turn after the data are chunked, to form the memory chunk chain for storing the data, and the memory chunk chain head pointer of the node points to the head of the memory chunk chain.
In the embodiments of the present invention, when a record is inserted, if the quantity of data exceeds the quantity of data which one memory chunk can store, the data need to be chunked and stored in multiple memory chunks. Suppose that N memory chunks are needed, each of the former n−1 data chunks stores data the quantity of which equals to the capacity of the memory chunk, and the last memory chunk stores the remained data, the remained data may be smaller than the capacity of the memory chunk. The procedure of reading one record is opposite, the data in the memory chunks are read in turn, and a whole data block is recovered.
In Step S401, a key corresponding to data to be read is obtained, and a Hash value corresponding to the key is obtained according to the key by using a Hash algorithm.
In Step S402, a node chain head pointer corresponding to the Hash value is searched for according to the location of the Hash value at the Hash bucket.
In Step S403, a node chain in the Hash bucket is traversed according to the node chain head pointer, and it is determined whether the key is searched out; if the key is searched out, Step S404 is performed; otherwise, the procedure terminates.
In Step S404, a memory chunk chain head pointer corresponding to the node is searched for.
In Step S405, data in memory chunks are read in turn from the memory chunk chain to which the memory chunk chain head pointer points, a whole data block is recovered and the data are returned to the user.
In Step S501, a key corresponding to data to be deleted from a cache is obtained, and a Hash value corresponding to the key is obtained according to the key by using a Hash algorithm.
In Step S502, a node chain head pointer corresponding to the Hash value is searched for according to the location of the Hash value at the Hash bucket.
In Step S503, a node chain in the Hash bucket is traversed according to the node chain head pointer, and it is determined whether the key is searched out; if the key is searched out, Step S504 is performed; otherwise, the procedure terminates.
In Step S504, a memory chunk chain head pointer corresponding to the node is searched.
In Step S505, data stored in a memory chunk chain corresponding to the memory chunk chain head pointer are deleted, and the memory chunks are reclaimed to the idle memory chunk chain.
In Step S506, the memory chunk chain head pointer of the node points to the idle node chain, so as to reclaim the node to the idle node chain.
A cache configuring module 61 is adapted to configure a node and a memory chunk corresponding to the node in a cache 63. The node stores a key of data, the length of the data and a pointer pointing to the memory chunk. The memory chunk corresponding to the node stores data written in the cache 63. As mentioned in the foregoing, the node includes the key of the data, the length of the data, a memory chunk chain head pointer corresponding to the node, a node chain former pointer, a node chain later pointer and so on.
When the cache 63 is configured, a node region configuring module 611 is adapted to configure information stored in a node region, and the node region includes a head structure, a Hash bucket and at least one node. The head structure of the node region, the Hash bucket and the information stored in the node are as mentioned in the foregoing, and will not be described. A memory chunk region configuring module 612 is adapted to configure information stored in the memory chunk region. The memory chunk region includes a head structure and at least one memory chunk. The head structure of the memory chunk region and the information stored in the memory chunk are as mentioned in the foregoing, and will not be described.
A cache processing operation module 62 is adapted to perform cache processing for data according to the configured node and memory chunk corresponding to the node.
When a record is inserted, a record inserting module 621 is adapted to search a node chain according to a key corresponding to data to be written into the cache 63; when the key exists in the node chain, delete data in a memory chunk corresponding to the key, reclaim the memory chunk from which the data are deleted, allocate a memory chunk according to the size of data of the record, and write the data of the record into the allocated memory chunk in turn after chunking the data; when the key does not exist in the node chain, allocate one idle node and a memory chunk corresponding to the length of the data, and write the data into the allocated memory chunks in turn.
When a record is read, a record reading module 622 is adapted to search a node chain according to a key corresponding to the data to be read from the cache 63; when the key exists in the node chain, read data in a memory chunk corresponding to the key in turn, and recover a whole data block.
When a record is deleted, a record deleting module 623 is adapted to search a node chain according to a key corresponding to the data to be deleted from the cache 63; when the key exists in the node chain, delete data in a memory chunk corresponding to the key, and reclaim the memory chunk from which the data are deleted and the node corresponding to the key.
As an embodiment of the present invention, a LRU processing module 624 is adapted to perform a LRU operation for data in the cache 63 according to a last visiting time and visiting times of a record, remove LRU data from the memory, and reclaim memory chunks and node, to save the memory space.
The embodiments of the present invention have little requirements for the size of the data and have good generality, and do not need to learn the size and distribution of stored single data, which increases universality of the cache, effectively decreases waste of the memory space, and increases usability of memory. Simultaneously, data searching efficiency is high and the LRU operation is supported.
The foregoing descriptions are only preferred embodiments of the present invention and are not for use in limiting the protection scope thereof. Any modification, equivalent replacement and improvement made under the spirit and principle of the present invention should be included in the protection scope thereof.
The foregoing description of the embodiments has been provided for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention. Individual elements or features of a particular embodiment are generally not limited to that particular embodiment, but, where applicable, are interchangeable and can be used in a selected embodiment, even if not specifically shown or described. The same may also be varied in many ways. Such variations are not to be regarded as a departure from the invention, and all such modifications are intended to be included within the scope of the invention.
| Number | Date | Country | Kind |
|---|---|---|---|
| 200710077039.3 | Sep 2007 | CN | national |
This application is a continuation of International Application No. PCT/CN2008/072302, filed Sep. 9, 2008. This application claims the benefit and priority of Chinese Application No. 200710077039.3, filed Sep. 11, 2007. The entire disclosures of each of the above applications are incorporated herein by reference.
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/CN2008/072302 | Sep 2008 | US |
| Child | 12707735 | US |
