Patent Application
20160077980
Embodiments are related to caching key-value data item pairs in memory, such as may be carried out, for example, relative to Memcached operation.
As introduced above, Memcached is a general-purpose distributed memory caching system. It is often used to speed up dynamic database-driven websites by caching data and objects in RAM to reduce the number of times an external data source (such as a database or API) must be read. When storing large values, say a database result set in the form of an array, care must be exercised not to exceed the value size limit (default limit is 1 MB). To store larger values, the maximum size may be increased or the stored value may be compressed. Conventionally, key-value compression is carried out on the client side (by the Memcached clients 203) at the cost of additional CPU cycles. Enabling compression on the Memcached clients 203 makes efficient use of the in-memory storage, and reduces the amount of data sent over the network to the database servers and to the key-value stores.
While many Memcached clients provide compression services, no compression conventionally happens in the Memcached servers themselves. As alluded to above, client-side compression means that less data is sent over the network. Significantly, client-side compression means that individual compressed cached items are generally of a smaller size than their uncompressed counterparts. Smaller objects will likely be stored more efficiently as they are more likely to fall into smaller-sized memory locations.
Thereafter, and without a delay attributable to compressing the received item, the key-value store 510 may report to the key-value store client 505 that the item has been safely stored in the key-value pair store 510. Note also that the database server 508 need not wait until the item is sent to the key-value store 510 before acknowledging back to the key-value store client 505. For example, if the database server 508 stores the item to be key-value stored in non-volatile memory within the database server 508, it may report back to the key-value store client 505 as soon as the item is stored in a non-volatile memory or otherwise safe, and need not wait until the item has been sent to the key-value store 510. Thereafter at (5) (which may be long after the uncompressed value is stored in the key-value store 510), the item may be compressed by the key-value store 510 (a server, in one implementation), as shown at (5). The resultant compressed value of the item may then be stored in the key-value store 510, as shown at (6) in a slab storing similarly-sized values as the just-compressed item value, as shown at (6). This slab may be a different slab than the slab in which the uncompressed value was previously stored at (4). Finally, the memory location storing the uncompressed value may be de-allocated. This may be done, for example, by dropping the uncompressed value stored at (4) into an appropriate slab for later re-use, as the uncompressed value need not be further accessed (as being redundant relative to the compressed value thereof stored at (6)). The de-allocated memory location may then be re-used for later writes.
The compressed item will be stored at (6) in a slab of other items having a size that is the same or comparable to the size of the just-compressed item. On the other hand, if the compression at (5) does not significantly reduce the size of the item, the compressed item may be stored in the same slab as the uncompressed version thereof, or another slab storing similarly-sized items.
As shown at B66, the just-compressed item may be stored in the key-value store 510. To do so, a slab storing items that are similarly sized as the just-compressed item may be selected. The just-compressed item may then be stored in the selected slab, as shown at B67. The memory in the key-value store(s) 510 that was previously allocated for storage of the uncompressed item in Block B63 may now safely be de-allocated (stored in another slab) and made available for later writes, as shown at B68. For example, the uncompressed item may be moved to the tail end of the LRU, which will be dropped from the linked list when another item is subsequently stored in that LRU.
According to one embodiment, when new item values are sent by key-value store clients, they are stored in an uncompressed format, and a flag may be set to identify them these new item values as being uncompressed. A pointer may be added to a hardware queue, such that the uncompressed items may be compressed in the order in which they were received (for example). The uncompressed items may then be compressed by hardware as rapidly as possible (but significantly slower, in one embodiment, than line rate). As the storing of uncompressed items is carried out by hardware without delay, there is no significant latency penalty on write/store requests from key-value store clients. Some items may constitute uncompressible data. In that case, the uncompressed flag remains set and no attempt to decompress the item need be carried out upon a retrieval request from a key-value store client.
If, however, the item is compressible and if the size of the compressed item is less than the uncompressed size thereof, the following may be carried out, according to one embodiment. If the size of the compressed item is indeed less than the uncompressed size thereof, the hardware may pull the correct size buffer off of the LRU tail and copy the compressed item into this buffer. As the hardware knows the size of the items in each LRU, it is able to pull a buffer from an LRU that stores items that are similarly-sized as the compressed item. The hardware may then respond to the software with this result and may move the compressed item to the head of the selected LRU. The software may then replace the original uncompressed item in the hash chain, thereby making that buffer available for later-occurring stores. Upon a retrieval request (read request), the hardware may read the compressed/uncompressed flag and, if the compressed flag indicates that the stored item value is compressed, may decompress the item on the fly before providing the item to the requesting key-value store client.
According to one embodiment, all pointers may be managed by hardware, with the software converting the pointers to virtual space. For example, at least some of the functionality disclosed herein and shown in
According to one embodiment, the hardware may be configured to handle situations in which access and modifications to items stored in the key-value store(s) 510 may cause potential race conditions and manage those exceptions. For example, such a race condition may occur when an adjacent item (e.g., the next or previous item in the LRU) may be independently in the process of being fetched by another request. This would modify the pointer from the item sought to be modified to the next or previous item in the chain. This may occur because the value of the next or previous item in the chain is unknown to the hardware until the memory read has returned the value of the item. In the intervening time, there is the possibility either the next or previous item in the chain is being similarly accessed. Therefore, the modification of next and previous LRU pointers must be carried out in a coherent manner to prevent the linked list of items within a slab from becoming corrupted.
Therefore, on an access, the next item in the LRU chain, the previous item in the LRU chain, the next item from the head and/or the previous item from the tail of the chain must be prevented from being modified. To do this without a locking mechanism, one embodiment implements a “Do Not Access List” of items that must not be accessed until the current access in the pipeline of such access requests has completed. By placing the requisite items on the Do Not Access List, race conditions are avoided. A Do Not Access List is unlikely to be successful in a software implementation, due to locking problems. For example, consider three memory accesses A, B and C. In this scenario, memory access A came first in time, then B and then C. Suppose that memory access A is accessing item X, but that is unknown to the hardware until the value of item X is actually returned. Suppose further that later-occurring (with respect to A) memory access B is also accessing item X. Memory access B may have already started and may have blocked further access to item X, which would prevent earlier-occurring memory access A from accessing item X. As memory access A must take precedence over later-occurring memory access B, which has locked access to that same item, a wrong-order memory contention has occurred. In such and similar situations, all pipelined memory reads are flushed and restarted from an appropriate point. Continuing with the example being developed herewith, a right-order contention situation may arise in the context, for example, memory access A reading item X first, and temporarily blocks access to at least X, the next-adjacent item and the previous-adjacent item. Thereafter, memory access B also accesses item X, the next adjacent item or the previous item and sees that these items are already blocked (i.e., placed on the Do Not Access List). In that instance, there is no need to flush the pipelined reads, as memory access B may simply wait until the requested item is no longer blocked (i.e., is no longer on the Do Not Access List). That is, memory access B may simply wait until memory access A has completed, updated X as needed and removed X from the Do Not Access List. Unlike a locking mechanism that ensures that collisions never happen, placing items on a Do Not Access List does not ensure that collisions do not happen. Instead, the Do Not Access List, according to one embodiment, still allows collisions to occur but provides a streamlined method of recovering from such a collision. Such cases in which pipelined reads are flushed are rare and implementation should not impose undue performance issues.
Portions of the detailed description herein describe processes and symbolic representations of operations by computing devices that may include conventional computer components, including a local processing unit, memory storage devices for the local processing unit, display devices, and input devices. Furthermore, such processes and operations may utilize conventional computer components in a heterogeneous distributed computing environment including, for example, remote file servers, computer servers, and memory storage devices. These distributed computing components may be accessible to the local processing unit by a communication network such as shown at 504. The processes and operations performed by such computing devices include the manipulation of data bits by a local processing unit and/or remote server and the maintenance of these bits within data structures resident in one or more of the local or remote memory storage devices. These data structures impose a physical organization upon the collection of data bits stored within a memory storage device and represent electromagnetic spectrum elements. A process or computer-implemented method may generally be defined as comprising a sequence of computer-executed steps leading to a desired result. These steps generally require physical manipulations of physical quantities. Usually, though not necessarily, these quantities may take the form of electrical, magnetic, or optical signals capable of being stored, transferred, combined, compared, or otherwise manipulated. It is conventional for those skilled in the art to refer to these signals as bits or bytes (when they have binary logic levels), pixel values, works, values, elements, symbols, characters, terms, numbers, points, records, objects, images, files, directories, subdirectories, or the like. It should be kept in mind, however, that these and similar terms should be associated with appropriate physical quantities for computer operations, and that these terms are merely conventional labels applied to physical quantities that exist within and during operation of the computer.
It should be understood that manipulations within the computer are often referred to in terms such as adding, comparing, moving, positioning, placing, illuminating, removing, altering, etc., which are often associated with manual operations performed by a human operator. The operations described herein are machine, programmatic operations performed in conjunction with various input provided by a human operator or user that interacts with the computer. The machines used for performing the operation of the present invention include local or remote general-purpose digital computers or other similar computing devices. In addition, it should also be understood that the programs, processes, methods, etc. described herein are not related or limited to any particular computer or apparatus nor are they related or limited to any particular communication network architecture. Rather, various types of general-purpose machines may be used with program modules constructed in accordance with the teachings described herein. Similarly, it may prove advantageous to construct a specialized apparatus to perform the method steps described herein by way of dedicated computer systems in a specific network architecture with hard-wired logic (such as FPGA 514) or programs stored in nonvolatile memory, such as read only memory.
