The subject application generally relates to data storage and retrieval, and, for example, to a key-value store that operates in conjunction with data storage streams and segments maintained in an append-only storage medium, and related embodiments.
Prior technology exists for storing a non-ordered associative array of pairs using append-only storage media, however, such prior technology does not support order between pairs, and so features, such as range queries and listing keys in sorted order, have remained similarly unsupported.
The technology described herein is illustrated by way of example and not limited in the accompanying figures in which like reference numerals indicate similar elements and in which:
Various aspects described herein are generally directed towards providing an ordered associative array of pairs (e.g., a key-value store) on top of append-only storage, which among other possible uses facilitates access to data storage streams. The technologies described herein are directed to storage of a non-ordered associative array of pairs. This disclosure provides an end-to-end implementation of how to store an ordered associative array of pairs using an append-only storage medium.
Reference throughout this specification to “one embodiment,” “an embodiment,” “one implementation,” “an implementation,” etc. means that a particular feature, structure, or characteristic described in connection with the embodiment/implementation is included in at least one embodiment/implementation. Thus, the appearances of such a phrase “in one embodiment,” “in an implementation,” etc. in various places throughout this specification are not necessarily all referring to the same embodiment/implementation. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments/implementations.
Aspects of the subject disclosure will now be described more fully hereinafter with reference to the accompanying drawings in which example components, graphs and operations are shown. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the various embodiments. However, the subject disclosure may be embodied in many different forms and should not be construed as limited to the examples set forth herein.
With regard to
In one aspect, the technology can efficiently organize data in a single, non-distributed database as an associated array of key-value pairs implemented on top of a storage medium that allows only data appends (data can only be added at the end, and once written, it may not be overwritten). In the implementation of
In one or more implementations, both the segment file, comprising a table segment that maintains the data, and an attribute index that contains information by which a table key maps to an offset location in the table segment where the value is stored can be implemented in an append only storage. For example, the attribute index can contain extended attributes maintained in an append-only, shadow segment file.
The implementation(s) described herein are non-limiting examples, and variations to the technology can be implemented. For instance, many of the examples herein are based on an attribute index that is accessed via a particular hash function and collision handling scheme, however other hash functions and/or collision handling schemes can be used. As such, any of the embodiments, aspects, concepts, structures, functionalities, implementations and/or examples described herein are non-limiting, and the technology may be used in various ways that provide benefits and advantages in data storage in general.
Turning to the drawings,
In
An attribute index 226 can map table keys to offset locations in a table segment, such as the table segment 220. Because table keys can have arbitrary lengths, the attribute index 226 can contain hash-mapped values based on a hash function 228. In one or more implementations, the attribute index 226 can comprise an associative array of 16-byte keys to 8-byte values, which can be stored in an append-only storage medium, e.g., an extended attribute shadow segment file. Access to the attribute index can be performed via an index reader 230 and an index writer 232 (e.g., an API).
For efficiency, a memory key index 234 can maintain a view of a subset of a table segment index, that is, data representing a number of {key, offset location} pairs, such as those most recently used. In general, the memory key index 234 can avoid needing to access the attribute index 226 on many read and update requests, as described herein. However, because data maintained in the memory key index 234 can be stored in volatile memory, such data is subject to loss once the process terminates. Recovery can be a resource-consuming operation and is thus not particularly desirable. Accordingly, asynchronous indexing 236 can be performed, which can operate in the background to update the attribute index 226 based on the data in the memory key index 234. Note that the index writer 232 and asynchronous indexing component 236 can be the same entity, although it may be beneficial to allow another entity to write to the attribute index 226, and thus these are shown in
Turning to updates to a table segment, for storage on non-ordered information, consider that a set of updates SU (each in the form {Key, Value, CompareVersion}) can be received for the table segment 220. This is represented in
Operation 304 evaluates whether the set of updates is conditional, meaning that at least one of the updates has its CompareVersion value set. If so, the update process branches to
Consider that in this example at least one of the updates has its CompareVersion value set, whereby the process branches to operation 402 which represents collecting those updates in the update set/batch that has a CompareVersion value set; this set may be the entire set S, or a proper subset CU of the set S.
At operation 404, if there are other pending conditional update(s) to any table key in the subset CU, then operation 404 can wait for the completion. Once none are pending, at operation 406, a first conditional update can be selected for update processing. Note that non-conditional updates, including for the same key, can execute in parallel, in which case each can update the value of the key (possibly for a very short amount of time), with one value prevailing as the final value. However, conditional updates can only overwrite the value of a key if the updater proves that the updater knows the previous value of the key (which is done by providing a version with each key update). Conditional updates cannot be executed in parallel, because the outcome of the previous update needs to be known before making another change. Because the version returned with each update cannot be guessed by the updater beforehand, conditional updates can be serialized. Any concurrent access results in at most one update executing successfully and the others failing.
Accordingly, conditional updates are validated. Operation 408 represents querying the memory key index 234 with the table key (it is feasible to use some hash function or the like to quickly locate the table segment offset location for the table entry, from which the {table key, value, version} is found. Note that it is feasible for the memory key index 234 to contain the {table key, value, version} data in addition to the segment table offset location, e.g., for efficiency, although for purposes of description herein, consider that the memory key index only contains {table key, offset} pairs (possibly arranged via some hash function on the table key). If not present as evaluated at operation 410, the value and version value can be looked up in the attribute index via the index reader 230 (
Once the version information is obtained, operation 414 can either allow (and perform) the update via operation 416 or disallow (operation 418) the update based on whether the obtained version value is what was expected (versus the compare version value). Operations 420 and 422 can repeat the process for other conditional updates.
Returning to
Turning to read requests, consider that a set of table keys SR can be received for the table segment 220 requesting a read operation. This is represented in
Operation 504 looks for the table key in the memory key index 234 (
Once the offset is obtained at operation 516, operation 518 can issue a segment read (block 238,
Turning to aspects of the attribute index and its structure, in one or more implementations, the attribute index 226 can be stored in an append-only attribute index file as an associative array of 16-byte keys to 8-byte values. In one or more implementations, the attribute index can comprise an append-only B+Tree implementation.
For the upper layers, the attribute index 226 can provide a mapping between a table key to an offset location in the corresponding table segment. A table key can be an arbitrary length byte sequence, while the offset can be a fixed-size, 8-byte number. As described herein, data corresponding to the table key and the offset can fit in the 16-byte to 8-byte attribute index file.
To this end, a hashing function 228 (e.g., SHA512, which produces a sixty-four byte hash value) can be used, which produces a fixed-size byte array/resulting hash value H. In one or more implementations, the resulting hash value H can be split into a series of hash parts HP[0 . . . n]; HP[0] has 16 bytes, while HP[1] HP[n] have 12 bytes. For a sixty-four byte hash, n can thus equal four; as will be understood, this provides five hash parts, which can map to nodes of the B+tree (as needed, to avoid collisions). However, in many instances there are no collision(s), whereby not all of the hash parts, and often only the primary hash part (HP[0]), can be used. Note that instead of using hash parts, it is feasible to use a series of different hash functions.
In general, when a table key is hashed, there is a possibility of a collision with another hashed table key value. As described herein, the primary hash part (HP[0]) can be checked for such a collision with another attribute index entry. If there is no collision, then the primary hash part, corresponding to a node in the B+tree, can be used as the key to the offset in the attribute index.
Otherwise, if there is a collision at the primary hash part, then the secondary hash part can be evaluated, (HP[1]). This corresponds to a branch under the node corresponding to the primary hash part. If there is no collision, then the primary and secondary hash parts can be used to differentiate between the two (hashed representations of) segment table keys. If a collision still exists, then the next secondary hash part, HP[2] can be used in the same manner, and so on, until there is no collision, or no hash parts remain. In the (unlikely) event that no hash parts remain, a linked list comprising backpointers can be used to differentiate the corresponding table key entries in the attribute index. While link lists are inefficient, with SHA512 and 5 hash parts, linked lists are rarely, if ever, going to be needed in practice.
As can be understood, in one or more implementations, there can be three types of entries in the index attribute structure 660 for offsets, child nodes and backpointers. Various information can be used to differentiate between these different types of hash key values in the index attribute structure 660. For example, in one implementation, keys starting with bit 1 can represent a primary hash to NodeId|Offset pointer, e.g., (1{127-bit primary hash)=NodeId|Offset.
Keys starting with bits 00 can represent back-pointers as described below, e.g., (00{62-bit ignored} {Offset1})=Offset2, where Offset2 can contain the previous entry in a linked list for the hash bucket that contains the entry at offset Offset1.
Keys starting with bits 01 represent secondary hashes to NodeId|Offset pointers, e.g., (01{NodeId} {SHIndex})=NodeValue. Note that if the NodeValue starts with bit 1, the remaining bits represent the Child NodeId for the secondary hash with Index for node NodeId. If the NodeValue starts with 0, the remaining bits represent the offset within the segment where the last entry for this hash bucket exists.
Consider that in this example, two table keys K1 and K2 can be different, but can have the same hashes/hash parts. As will be understood, backpointers can be used to keep track of both of their latest values. Further, another key K3 partially collides with K1 and K2; these keys share the same Primary Hash and the first Secondary Hash, but have a different second secondary hash. As such, these keys share Node1 and Node2, but K3 has a different offset due to its second Secondary Hash being different. Further, K4 is totally independent of K1, K2 and K3, and thus has its own, new entries. Note that the NodeId (e.g., Node1, Node 2 and so on) is a 30-bit number that is incremented whenever a new tree node is created underneath the first level (that is, every time the process uses a secondary hash to resolve primary hash collisions).
In
In order to differentiate between types of index entries, operation 706 can drop the first bit of the primary hash value HP[0], and replace the dropped bit with a 1; this is set to AK1 (for AKm, where m initially equals 1).
Operation 708 evaluates for a collision, that is, whether another table key K2 shares HP(0) with the current Key K. If there is no collision, the process can insert {AK1, OL) into the attribute index and stop (recall that OL is the offset location).
If there is a collision, operation 712 can increment the TableNodeId (TN) for this table segment (starting from an initial value of zero). The key, modified via the starting one bit into AK1, can be inserted along with the table node identifier into the attribute index at operation 714.
When there is a collision and child nodes are needed, operation 716 can concatenate (bitwise) TN and HP[1] to prefix them with bits {01}, and the result can be AK2. The process can continue to operation 802 of
At operation 802 of
If there is still a collision, operation 804 can evaluate whether hash parts remain that can differentiate between the colliding table keys. If hash parts remain, the process repeats from operation 712 of
If no hash parts remain, that is, HP[n] is reached and there are no more hash parts to use in an attempt to differentiate, operation 808 can insert {AK2, OL} into the attribute index. At this point there exists another table Key K2 that hares HP[0] . . . HP[n] with the table Key K. Because there are no more hash parts to differentiate between them, a linked list is used.
To this end, operation 810 can concatenate bits (00) with the offset location OL, with the result set to BackpointerSource. Operation 812 can find the offset corresponding to table key K2 that collides with Key K (they share all hash part components), with this K2 offset location set to BackpointerTarget. Operation 814 can insert {BackpointerSource, BackpointerTarget) into the attribute index.
Turning to index retrievals for storage of non-ordered information, as generally represented in the example operations of
With reference to
Operation 906 can drop the first bit of HP[0], and replace it with 1; the result can be set to AK1 (AKm, where for now m=1). Operation 908 can then perform a lookup in the attribute index for the value associated with AK1, which can be set to AV1 (AVm, where for now m=1). If AVm does not exist, there is no such entry and the process ends (e.g., with an error at operation 912 to notify the caller that no such entry was found).
If instead AVm does exist, and the value is an offset in the table segment, then operation 915 can return the offset value and stop. Otherwise the process can continue to operation 1002 of
Operation 1002 of
If AV2 does exist, and the value is an offset in the table segment as evaluated at operation 1010, operation 1012 can return the offset value OL and the process can end. Otherwise, if hash parts remain (HP[n] is not yet reached), operation 1014 can branch back to operation 1002. If no hash parts remain, the process can continue at operation 1102 of
If no hash parts remain, the linked list can be accessed to determine the offset location for the table key K. Operation 1102 can perform a table segment read at the last value retrieved (which is an offset) and can read the table key (RK) at that location. If RK matches K at operation 1104, then the offset result is found, and operation 1106 can return the offset value and the process can end.
If the keys do not match, operation 1108 can bitwise concatenate (00) and the last value retrieved (which was an offset) into backpointer source (BPS). Operation 1110 can perform a lookup in the attribute Index for the backpointer source (BPS), which can be set to backpointer target (BPT). Note that if BPT does not exist, operations 1112 and 1114 can end the process with a suitable error.
Otherwise, operation 1112 can return to operation 1102, passing the BPT to it as the last value retrieved. The linked list can be followed until a match with the table key is found (or no further backpointer exists).
As can be seen, index updates and retrievals to the attribute index can use a multi-part hashing scheme that maps to nodes in the attribute index, yet have a failsafe linked list mechanism in the unlikely event that two different table keys have the same hash value. Note that instead of the multi-part hashing scheme, the use of multiple different hash functions can similarly map to attribute index nodes.
Turning to asynchronous indexing, in regular updating, the operations of
During asynchronous indexing, the index writer 232 (which can incorporate or be coupled to the asynchronous indexing component 236) can maintain and retrieve the last offset that was indexed (and can durably persist this upon every change, so that this value cannot be lost). The index writer 232 can read the table entry updates between that last indexed offset and the end of the table segment, and can index these entries.
As described above, any time the attribute index is accessed, the table keys can be hashed using the hash function/hash part splitting operations described above. Thus, operation 1206 results in H and HP[0 . . . n] for each update key. Operation 1208 groups these hash values together in a single group.
Thereafter, the group can be subsequently broken down into smaller and smaller groups based on collisions, as can be determined by the values of HP[0 . . . n] for each key. For example, starting with HP[m] (where m=0 via operation 1210) and thereby considering only HP[0], any keys sharing HP[0] can be grouped together at operation 1212. Other keys can be sub-grouped into individual groups, as they have unique hashes. Then, following the logic of
Thus, after a first hash part is considered, operations 1216 and 1218, along with operations 1222 and 1224, can remove subgroups from further subgrouping when a group size reaches one. Otherwise subgrouping can continue via operation 1220.
Operation 1302 of
In the event that a subgroup with more than one key remains after all hash parts have been exhausted, the structure described with reference to
It should be noted that the index structure can be maintained when a key is updated and when a key is removed. This can involve removing entries from the attribute index.
After the changes are applied to the attribute index at operation 1310, the last indexed offset for the segment can be updated via operation 1312. The memory key index 234 can be notified of the change, so that any reads that blocked at step can be unblocked.
One or more aspects, such as those implemented in example operations of a method as in
The update request can comprise a compare version value, and aspects can comprise validating the compare version value with respect to an expected version value, and based on the validating, allowing the appending and the updating to proceed. Aspects can comprise waiting for another update request to complete. Aspects can comprise looking for the expected value in the memory key index, and in response to the expected value being determined not to be present in the memory key index, using the table key to access the attribute index to obtain an offset location in the table segment corresponding to the offset location from which the expected value is able to be obtained.
When the expected value is not found in the memory key index, aspects can comprise adding the expected value to the memory key index before the updating of the memory key index with the offset location of the table key. When the expected value is not found in the memory key index and the attribute index is configured as a tree structure, using the table key to access the attribute index can comprise hashing the table key into a hash value that corresponds to a node in the tree structure that comprises the offset location in the table segment from which the expected value is able to be obtained.
Aspects can comprise asynchronously indexing the memory key index to an attribute index stored in an extended append-only storage medium. Asynchronously indexing the memory key index to the attribute index can comprise hashing table keys in the memory key index into a series of hash parts corresponding to each table key, grouping data in the memory key index into subgroups based on the series of hash parts for each table key, and using the subgroups for indexing the memory key index to the attribute index.
Aspects can comprise receiving a read request comprising the table key, the read request requesting a requested value paired with the table key, and in response to the receiving the read request, looking for the requested value in the memory key index, and in response to the requested value being determined not to be present in the memory key index, using the table key to access the attribute index to obtain an offset location in the table segment corresponding to the offset location from which the requested value is able to be obtained. Using the table key to access the attribute index can comprise hashing the table key into a hash value that corresponds to a node in the tree structure that comprises the offset location in the table segment from which the requested value is able to be obtained.
One or more example aspects are represented in
Using the offset location data to return the requested value from the table segment in response to the read request can comprise issuing a segment read request in conjunction with the offset location data.
When the requested value is absent from the memory key index, and the attribute index is configured as a tree structure, using the table key to access the attribute index can comprise hashing the table key into a hash value that corresponds to a node in the tree structure that comprises the offset location data from which the requested value is able to be obtained. Hashing the table key into the hash value can comprise using a hash function to obtain the hash value, and dividing the hash value into a series of hash value parts, comprising a primary hash value part that corresponds to a node in the tree structure, and, in response to a collision with another primary hash value part of another hash value of another table key, a secondary hash value part that corresponds to a child node of the node in the tree structure.
Further example operations can comprise receiving an update request corresponding to the table segment, the update request comprising the table key and an updated value, appending the update request to the table segment by serializing the update request at a new offset location in the table segment, and updating the memory key index with the new offset location corresponding to the update request.
One or more aspects, such as can be implemented in a machine-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, are represented in
Using the primary hash function information obtained from the hashing the table key can comprise hashing the table key into a hash value, and dividing the hash value into a series of hash parts comprising the primary hash function information and one or more secondary hash parts corresponding to the secondary hash function information.
Using secondary hash function information obtained from hashing the table key to locate the child node can comprise determining that the child node corresponds to a linked list data structure.
Maintaining the attribute index can comprise hashing table keys in a memory key index into respective hash values, dividing the hash values into respective series of hash parts, grouping the table keys into subgroups based upon the respective hash parts, and updating the attribute index based on the subgroups.
Further operations can comprise using the offset location data in handling read operations and update operations.
As can be seen, the technology described herein provides a key-value store to a table segment in which append-only storage is used. An attribute index can be accessed via a hash function divided into parts (or series of hash functions), to provide a hash array mapped tree, comprising a search tree where each non-leaf node is a list of hashes that point to other nodes. At each level, a new hash is calculated based on the key; the leaf levels contain the data.
In general, techniques according to
Problems overcome by solutions according to this disclosure include, e.g., severe copy-on-write/write amplification which can occur when attempting to use a B+Tree on an append-only media. The solution presented herein does not suffer from copy-on-write/write amplification as it is more compact. Furthermore, performance reduction can result from indexing on the front end. To avoid this issue, solutions presented herein employ a cache for front-end validation and index asynchronously in the background. Solutions presented herein also generally separate, on one hand, an optimized solution for hash-based indexing and lookup from, on the other hand, a dedicated solution for storing order among keys. This separation provides for improved performance of the overall system, which, given the constraints of having to use append-only media in the presence of unreliable hardware prone to frequent crashes, can perform better than other implementations that have similar goals.
Prior to discussion of
System components can include the following: A “Segment Store” can receive and process Table Segment requests. A “Segment File” can provide an append-only storage medium. An “Attribute Index” can comprise an associative array of keys and values, e.g., 16 byte keys and 8 byte values, which can be stored in an append-only storage medium. Each Table Segment can have an Attribute Index. A “Table Segment Index (TSI)” can comprise multiple parts, including (1) a persisted part (TSI.P), and (2) a tail part (TSI.T). A “Sorted Key Index (SKI)” can also comprise multiple parts, including (1) a persisted part (SKI.P), and (2) a tail part (SKI.T).
In
In general, update operations for an update performed according to
If the update request 1701 is conditional, the update request 1701 can optionally be validated 1753 against existing data at the conditional update validator 1704. Next, the update request 1701 can be serialized 1754 to the segment file 1705, and tail portions 1707 and 1713 of the in-memory indices 1706 and 1712 can be updated 1755(1), 1755(2).
In another example embodiment which can be understood with reference to
If SU is conditional, i.e., if at least one update in the SU has “CompareVersion” set, then the following operations can be performed at conditional update validator 1704. First, a collection can be performed to collect every update U in SU that has CompareVersion set. The resulting collection can be referred to as CU. Next, if there are other pending conditional updates to any table key in CU, then this operation can wait for their completion. Finally, every update U in CU can be validated: first the TSI.T 1707 can be queried, and if not present, the value can be looked up in the TSI.P 1708 via an index reader, and it can be added to the TSI.T 1707. After that, we either allow or disallow the update to proceed based on the value that is found.
When the update proceeds, SU can be serialized 1754 and appended to the segment file 1705. The TSI.T 1707 can be updated with the key versions (offsets) of each key K in SU. The SKI.T 1708 can be updated with the keys in SU.
In an embodiment, asynchronous indexing can include reading 1761 change(s) from the segment file 1705 and indexing the changes by the index writer 1709. Persisted indices can be updated by updating the persisted portion 1708, via interaction 1762(1), and updating the persisted portion 1714, via interaction 1762(2). The persisted portion 1708 can store data in the attribute index 1711, as shown.
The sorted key index 1712 can retrieve and update its BTreeSet nodes and store them as table entries in the table segment. These table entries can be provided 1763(0) to the internal key translator 1715, and the internal key translator 1715 can label the table entries, e.g., as “Internal”. The BTreeSet Nodes can be retrieved via interaction 1763(1), and the BTreeSet Nodes can be updated via interaction 1763(2).
Asynchronous indexing can achieve the following goal: transfer TSI.T 1707 to TSI.P 1708 and transfer SKI.T 1713 to SKI.P 1714. Note that, as part of the update operations described with regard to
The index writer 1709 can be configured to record the last offset that it has indexed, e.g., by durably persisting the last offset upon every change, so that its value is not lost. The index writer 1709 can read all the table entry updates between the last indexed offset and the end of the table segment, and the index writer 1709 can index them in the following way. Updates can be read from the segment file 1705 in batches. Each batch can be parsed and updates can be deserialized in the form {Key, Value, Version}. Each key in the batch can be processed according to an index update procedure. When a batch is fully processed, the results can be durably persisted in the attribute index 1711. The index structure can be maintained both if a key is updated and when a key is removed. This may involve removing entries from the attribute index 1711. Finally, the last indexed offset can be updated to point to the end of the processed batch.
The TSI.T 1707 can be notified 1762(1) of changes, so that any previously blocked reads can be unblocked. The TSI.T 1707 can be free to release any memory associated with keys that have just been indexed, since they can now be retrieved from the TSI.P 1708. The SKI.T 1713 can also be notified 1762(2) of changes, so that it can also release any memory associated with keys that have just been indexed.
In another example embodiment which can be understood with reference to
In an aspect, the update in the group of updates can comprise a property to indicate whether the update is a conditional update. In response to the property indicating that the update in the group of updates is a conditional update, embodiments can collect all conditional updates in the group of updates, embodiments can wait for completion of another update if needed, and embodiments can validate the update and determine whether to allow the update based on a result of the validating.
Validating the update can comprise querying the tail portion 1707 of the table segment index 1706 to determine whether the tail portion 1707 of the table segment index 1706 comprises an expected value. Validating the update can further comprise, in response to the tail portion 1707 of the table segment index 1706 not comprising the expected value, querying a persisted portion 1708 of the table segment index 1706 to determine whether the persisted portion 1708 of the table segment index 1706 comprises the expected value.
In general, read operations according to
In another embodiment, read operations can include receiving 1851 a set of keys SU, included in read keys request 1801, wherein the set of keys SU is for a table segment S stored in a segment file 1705. Each key K in the set of keys SU can be looked up in TSI.T 1707. If a key K is present, its latest offset can be fetched from TSI.T 1707. Otherwise, the latest offset for the key K can be looked up in the TSI.P 1708, which is described further below in connection with index retrievals. If the attribute index 1711 is behind in terms of indexing (due to asynchronous indexing as discussed in connection with
In general, list key operations according to
At interaction 1954, the list processor 1903 can merge the results returned from tail portion 1713 and persisted portion 1714. Optionally, if a list entry was requested, the list processor 1903 can request 1955 values associated with the returned keys, using techniques described in connection with
In another embodiment, operations performed in connection with listing keys in order can include receiving 1951 the list keys/entries request 1901. The list keys/entries request 1901 can comprise, e.g., a request for listing all table keys between values K1 and K2, for a table segment S. K1 and K2 can comprise the lower and upper bounds, respectively. They may be inclusive or exclusive of those values, and they may also optionally indicate no low or upper bound. In response to receiving the list keys/entries request 1901, the tail portion 1713 can be queried 1953(1) for a sorted list of keys that satisfy the range K1 . . . K2. This query result can be named, e.g., TailResult. The persisted portion 1714 can also be queried 1953(2) for a sorted list of keys that satisfies the range K1 . . . K2. This query result can be named, e.g., PersistedResult.
Next, the query results (TailResult and PersistedResult) can be merged into a final result named, e.g., FinalResult. The merging operation can utilize, e.g., the following merge rules: First, if a key K exists in both PersistedResult and TailResult, and TailResult indicates the Key has been removed, do not include K in FinalResult. Second, otherwise, if key K exists in either PersistedResult or FinalResult, include K in FinalResult. Finally, it may be the case that FinalResult may end up being very large, so its size can be unfeasible to return it at once. As such, FinalResult can be capped to a maximum size.
Merged query results (FinalResult) can be returned to the caller. Both TailResult and PersistedResult are sorted, and FinalResult can be constructed from these as a sorted list as well. If FinalResult did not cover all the range requested by the caller, the caller can be given the option to send a subsequent request using the last key contained in FinalResult as K1 (i.e., List(Last_Key_of_FinalResult, K2)). By concatenating the two results, the caller can get the full list of keys, in order.
In a further embodiment which can be understood with reference to
A tail portion 1713 of a sorted key index 1712 can be queried for a first list of keys that satisfy the request 1901, wherein the tail portion 1713 of the sorted key index 1712 is stored in a first memory, such as in volatile memory. A persisted portion 1714 of the sorted key index 1712 can also be queried for a second list of keys that satisfy the request 1901, wherein the persisted portion 1714 of the sorted key index 1712 is stored in a second memory, such as on disk.
The first list of keys and the second list of keys can be merged into a merged list of keys, and the merged list of keys can be returned 1956(2) in response to the request 1901. Merging the first list of keys and the second list of keys into the merged list of keys can comprise including a key in the merged list of keys in response to the key being determined to exist in the first list of keys or in the second list of keys. A key can be omitted from the merged list of keys in response to the key being determined to be flagged as deleted from the first list of keys. Embodiments can furthermore prevent the merged list of keys from exceeding a size limit.
With regard to index structures for use with embodiments according to
In order to perform index updates, some embodiments can be understood by supposing existence of a table key K which exists at offset OF, and a goal to update the index to reflect that. In a first step, embodiments can generate the 16-byte hash H, mentioned above. If a first portion of the hash, e.g., a first 8 bytes, match a predefined, reserved constant (i.e., 0), then embodiments can select a new set of first 8 bytes (for example the constant value 1). This reserved value can be used in the next step.
In a next step, if there exists another key K2 which shares H with K, then we have a collision. Embodiments, can concatenate an 8-byte representation of 0 (the reserved value referenced in the first step) with OF and remember it as, e.g., BackpointerSource. Embodiments can find the offset of key K2 which collides with our Key K. Call this, e.g., BackpointerTarget. Embodiments can then insert {BackpointerSource, BackpointerTarget} (16 bytes, 8 bytes) into the attribute index 1711.
With regard to index retrieval operations for use with embodiments according to
In a second operation, embodiments can perform a lookup in the attribute index 1711 for the value associated with H. this value can be named, e.g., AV1. If AV1 does not exist, embodiments can stop here.
In a second operation, embodiments can perform a segment file 1705 read at offset AV1, to read the table key at that location (the table key at that location can be referred to as RK). If RK matches K, then the result is found, and the value associated with this key can be returned. Otherwise, embodiments can continue to the fourth operation.
In a fourth operation, embodiments can concatenate an 8-byte representation of 0 AV1 into a value named, e.g., BackpointerSource. Embodiments can perform a lookup in the attribute index 1711 for BackpointerSource (name it, e.g., BackpointerTarget). If BackpointerTarget does not exist, embodiments can stop here. Also, embodiments can go back to the third operation, ensuring BackpointerTarget is passed to the attribute index 1711 instead of AV1.
In an example implementation, the sorted key index (SKI) 1712 can comprise a tail portion 1713 and a persisted portion, as illustrated in
The persisted portion 1714 can comprise a disk-memory data structure that stores its elements in sorted order. Persisted portion 1714 can be durably stored in some medium so that it may be recovered in case of a failure. In an example implementation, a B+Tree Set (BTS) can be used for persisted portion 1714. A BTS is similar to a classic B+Tree, but it does not include values associated with keys. The persisted portion 1714 can be maintained using, e.g., only the segment file 1705 and the persisted portion 1808. In other words, the persisted portion 1714 can be maintained using other constructs described herein.
An example BTS is made of nodes. Leaf nodes contain keys in sorted order, while index nodes have pointers from keys to other nodes. BTS nodes can be stored alongside other table keys in the table segment. When the BTS needs the contents of a node, it can make a request to the table segment. When the BTS needs to update one or more nodes, it can make another request to the table segment. This poses the risk of key collision. The keys used by the BTS may have the same names as externally-provided keys, which is undesirable. To resolve this, the following safeguards can be implemented: First, BTS keys can be prefixed with a predefined constant (i.e., “Internal”). For example, an arbitrary BTS Node “1234” can be prefixed to have a key of “Internal1234”. Second External keys can be prefixed with a different, predefined constant (i.e., “External”). For example, key “MyKey” can be internally translated to “ExternalMyKey”. Third, when necessary, operations can be skipped for BTS nodes (internal). Fourth, all keys coming in, such as via update operations according to
Updates and queries to the BTS can be done according to the rules of any B+Tree, including node splits and mergers. Range requests (e.g., a request for all keys in range K1 . . . K2) can translate to direct BTS queries that lookup key K1 and then list all keys until K2. Since a BTS stores items in sorted order, range queries are straightforward to execute.
Operations that can be performed in connection with the sorted key index (SKI) 1712 can comprise, e.g., maintaining a tail portion 1713 of a sorted key index 1712 in a volatile memory, wherein the tail portion 1713 of the sorted key index 1712 comprises first keys organized in a sorted array. Inserting a key K in the tail portion 1713 of the sorted key index 1712 can comprise determining a sort location for the key K, and inserting the key K at the sort location. Determining the sort location for the key K can comprises determining a location at which any key K1<K is before K in the tail portion of the sorted key index and any key K2>K is after K in the tail portion of the sorted key index.
Example operations can further comprise updating the tail portion 1713 of the sorted key index 1712 to reflect a key update or a key deletion. The tail portion 1713 of the sorted key index 1712 can comprise flags associated with the first keys (the keys in the tail portion 1713), and the flags can indicate whether a key has been deleted.
Furthermore, operations that can be performed in connection with the sorted key index (SKI) 1712 can comprise, e.g., maintaining a persisted portion 1714 of the sorted key index 1712 in a non-volatile memory, wherein the persisted portion 1714 of the sorted key index 1712 comprises a tree data structure comprising leaf nodes, the leaf nodes comprising second keys organized in a sorted order. The tree data structure can comprises a B+Tree Set, as described herein.
In response to a range request for keys in a range [K1 . . . K2], operations can include generating a list comprising keys between K1 and K2 in one or more of the tail portion of the sorted key index or the persisted portion of the sorted key index, as described herein.
The system 2000 also comprises one or more local component(s) 2020. The local component(s) 2020 can be hardware and/or software (e.g., threads, processes, computing devices). In some embodiments, local component(s) 2020 can comprise an automatic scaling component and/or programs that communicate/use the remote resources 2010 and 2020, etc., connected to a remotely located distributed computing system via communication framework 2040.
One possible communication between a remote component(s) 2010 and a local component(s) 2020 can be in the form of a data packet adapted to be transmitted between two or more computer processes. Another possible communication between a remote component(s) 2010 and a local component(s) 2020 can be in the form of circuit-switched data adapted to be transmitted between two or more computer processes in radio time slots. The system 2000 comprises a communication framework 2040 that can be employed to facilitate communications between the remote component(s) 2010 and the local component(s) 2020, and can comprise an air interface, e.g., Uu interface of a UMTS network, via a long-term evolution (LTE) network, etc. Remote component(s) 2010 can be operably connected to one or more remote data store(s) 2050, such as a hard drive, solid state drive, SIM card, device memory, etc., that can be employed to store information on the remote component(s) 2010 side of communication framework 2040. Similarly, local component(s) 2020 can be operably connected to one or more local data store(s) 2030, that can be employed to store information on the local component(s) 2020 side of communication framework 2040.
In order to provide a context for the various aspects of the disclosed subject matter,
In the subject specification, terms such as “store,” “storage,” “data store,” data storage,” “database,” and substantially any other information storage component relevant to operation and functionality of a component, refer to “memory components,” or entities embodied in a “memory” or components comprising the memory. It is noted that the memory components described herein can be either volatile memory or nonvolatile memory, or can comprise both volatile and nonvolatile memory, by way of illustration, and not limitation, volatile memory 2120 (see below), non-volatile memory 2122 (see below), disk storage 2124 (see below), and memory storage 2146 (see below). Further, nonvolatile memory can be included in read only memory, programmable read only memory, electrically programmable read only memory, electrically erasable read only memory, or flash memory. Volatile memory can comprise random access memory, which acts as external cache memory. By way of illustration and not limitation, random access memory is available in many forms such as synchronous random access memory, dynamic random access memory, synchronous dynamic random access memory, double data rate synchronous dynamic random access memory, enhanced synchronous dynamic random access memory, SynchLink dynamic random access memory, and direct Rambus random access memory. Additionally, the disclosed memory components of systems or methods herein are intended to comprise, without being limited to comprising, these and any other suitable types of memory.
Moreover, it is noted that the disclosed subject matter can be practiced with other computer system configurations, comprising single-processor or multiprocessor computer systems, mini-computing devices, mainframe computers, as well as personal computers, hand-held computing devices (e.g., personal digital assistant, phone, watch, tablet computers, netbook computers, etc.), microprocessor-based or programmable consumer or industrial electronics, and the like. The illustrated aspects can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network; however, some if not all aspects of the subject disclosure can be practiced on stand-alone computers. In a distributed computing environment, program modules can be located in both local and remote memory storage devices.
System bus 2118 can be any of several types of bus structure(s) comprising a memory bus or a memory controller, a peripheral bus or an external bus, and/or a local bus using any variety of available bus architectures comprising, but not limited to, industrial standard architecture, micro-channel architecture, extended industrial standard architecture, intelligent drive electronics, video electronics standards association local bus, peripheral component interconnect, card bus, universal serial bus, advanced graphics port, personal computer memory card international association bus, Firewire (Institute of Electrical and Electronics Engineers 1594), and small computer systems interface.
System memory 2116 can comprise volatile memory 2120 and nonvolatile memory 2122. A basic input/output system, containing routines to transfer information between elements within computer 2112, such as during start-up, can be stored in nonvolatile memory 2122. By way of illustration, and not limitation, nonvolatile memory 2122 can comprise read only memory, programmable read only memory, electrically programmable read only memory, electrically erasable read only memory, or flash memory. Volatile memory 2120 comprises read only memory, which acts as external cache memory. By way of illustration and not limitation, read only memory is available in many forms such as synchronous random access memory, dynamic read only memory, synchronous dynamic read only memory, double data rate synchronous dynamic read only memory, enhanced synchronous dynamic read only memory, SynchLink dynamic read only memory, Rambus direct read only memory, direct Rambus dynamic read only memory, and Rambus dynamic read only memory.
Computer 2112 can also comprise removable/non-removable, volatile/non-volatile computer storage media.
Computing devices typically comprise a variety of media, which can comprise computer-readable storage media or communications media, which two terms are used herein differently from one another as follows.
Computer-readable storage media can be any available storage media that can be accessed by the computer and comprises both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer-readable storage media can be implemented in connection with any method or technology for storage of information such as computer-readable instructions, program modules, structured data, or unstructured data. Computer-readable storage media can comprise, but are not limited to, read only memory, programmable read only memory, electrically programmable read only memory, electrically erasable read only memory, flash memory or other memory technology, compact disk read only memory, digital versatile disk or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or other tangible media which can be used to store desired information. In this regard, the term “tangible” herein as may be applied to storage, memory or computer-readable media, is to be understood to exclude only propagating intangible signals per se as a modifier and does not relinquish coverage of all standard storage, memory or computer-readable media that are not only propagating intangible signals per se. In an aspect, tangible media can comprise non-transitory media wherein the term “non-transitory” herein as may be applied to storage, memory or computer-readable media, is to be understood to exclude only propagating transitory signals per se as a modifier and does not relinquish coverage of all standard storage, memory or computer-readable media that are not only propagating transitory signals per se. Computer-readable storage media can be accessed by one or more local or remote computing devices, e.g., via access requests, queries or other data retrieval protocols, for a variety of operations with respect to the information stored by the medium. As such, for example, a computer-readable medium can comprise executable instructions stored thereon that, in response to execution, can cause a system comprising a processor to perform operations, comprising determining a mapped cluster schema, altering the mapped cluster schema until a rule is satisfied, allocating storage space according to the mapped cluster schema, and enabling a data operation corresponding to the allocated storage space, as disclosed herein.
Communications media typically embody computer-readable instructions, data structures, program modules or other structured or unstructured data in a data signal such as a modulated data signal, e.g., a carrier wave or other transport mechanism, and comprises any information delivery or transport media. The term “modulated data signal” or signals refers to a signal that has one or more of its characteristics set or changed in such a manner as to encode information in one or more signals. By way of example, and not limitation, communication media comprise wired media, such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media.
It can be noted that
A user can enter commands or information into computer 2112 through input device(s) 2136. In some embodiments, a user interface can allow entry of user preference information, etc., and can be embodied in a touch sensitive display panel, a mouse/pointer input to a graphical user interface (GUI), a command line controlled interface, etc., allowing a user to interact with computer 2112. Input devices 2136 comprise, but are not limited to, a pointing device such as a mouse, trackball, stylus, touch pad, keyboard, microphone, joystick, game pad, satellite dish, scanner, TV tuner card, digital camera, digital video camera, web camera, cell phone, smartphone, tablet computer, etc. These and other input devices connect to processing unit 2114 through system bus 2118 by way of interface port(s) 2138. Interface port(s) 2138 comprise, for example, a serial port, a parallel port, a game port, a universal serial bus, an infrared port, a Bluetooth port, an IP port, or a logical port associated with a wireless service, etc. Output device(s) 2140 use some of the same type of ports as input device(s) 2136.
Thus, for example, a universal serial busport can be used to provide input to computer 2112 and to output information from computer 2112 to an output device 2140. Output adapter 2142 is provided to illustrate that there are some output devices 2140 like monitors, speakers, and printers, among other output devices 2140, which use special adapters. Output adapters 2142 comprise, by way of illustration and not limitation, video and sound cards that provide means of connection between output device 2140 and system bus 2118. It should be noted that other devices and/or systems of devices provide both input and output capabilities such as remote computer(s) 2144.
Computer 2112 can operate in a networked environment using logical connections to one or more remote computers, such as remote computer(s) 2144. Remote computer(s) 2144 can be a personal computer, a server, a router, a network PC, cloud storage, a cloud service, code executing in a cloud computing environment, a workstation, a microprocessor-based appliance, a peer device, or other common network node and the like, and typically comprises many or all of the elements described relative to computer 2112. A cloud computing environment, the cloud, or other similar terms can refer to computing that can share processing resources and data to one or more computer and/or other device(s) on an as needed basis to enable access to a shared pool of configurable computing resources that can be provisioned and released readily. Cloud computing and storage solutions can store and/or process data in third-party data centers which can leverage an economy of scale and can view accessing computing resources via a cloud service in a manner similar to a subscribing to an electric utility to access electrical energy, a telephone utility to access telephonic services, etc.
For purposes of brevity, only a memory storage device 2146 is illustrated with remote computer(s) 2144. Remote computer(s) 2144 is logically connected to computer 2112 through a network interface 2148 and then physically connected by way of communication connection 2150. Network interface 2148 encompasses wire and/or wireless communication networks such as local area networks and wide area networks. Local area network technologies comprise fiber distributed data interface, copper distributed data interface, Ethernet, Token Ring and the like. Wide area network technologies comprise, but are not limited to, point-to-point links, circuit-switching networks like integrated services digital networks and variations thereon, packet switching networks, and digital subscriber lines. As noted below, wireless technologies may be used in addition to or in place of the foregoing.
Communication connection(s) 2150 refer(s) to hardware/software employed to connect network interface 2148 to bus 2118. While communication connection 2150 is shown for illustrative clarity inside computer 2112, it can also be external to computer 2112. The hardware/software for connection to network interface 2148 can comprise, for example, internal and external technologies such as modems, comprising regular telephone grade modems, cable modems and digital subscriber line modems, integrated services digital network adapters, and Ethernet cards.
The above description of illustrated embodiments of the subject disclosure, comprising what is described in the Abstract, is not intended to be exhaustive or to limit the disclosed embodiments to the precise forms disclosed. While specific embodiments and examples are described herein for illustrative purposes, various modifications are possible that are considered within the scope of such embodiments and examples, as those skilled in the relevant art can recognize.
In this regard, while the disclosed subject matter has been described in connection with various embodiments and corresponding Figures, where applicable, it is to be understood that other similar embodiments can be used or modifications and additions can be made to the described embodiments for performing the same, similar, alternative, or substitute function of the disclosed subject matter without deviating therefrom. Therefore, the disclosed subject matter should not be limited to any single embodiment described herein, but rather should be construed in breadth and scope in accordance with the appended claims below.
As it employed in the subject specification, the term “processor” can refer to substantially any computing processing unit or device comprising, but not limited to comprising, single-core processors; single-processors with software multithread execution capability; multi-core processors; multi-core processors with software multithread execution capability; multi-core processors with hardware multithread technology; parallel platforms; and parallel platforms with distributed shared memory. Additionally, a processor can refer to an integrated circuit, an application specific integrated circuit, a digital signal processor, a field programmable gate array, a programmable logic controller, a complex programmable logic device, a discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. Processors can exploit nano-scale architectures such as, but not limited to, molecular and quantum-dot based transistors, switches and gates, in order to optimize space usage or enhance performance of user equipment. A processor may also be implemented as a combination of computing processing units.
As used in this application, the terms “component,” “system,” “platform,” “layer,” “selector,” “interface,” and the like are intended to refer to a computer-related entity or an entity related to an operational apparatus with one or more specific functionalities, wherein the entity can be either hardware, a combination of hardware and software, software, or software in execution. As an example, a component may be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of illustration and not limitation, both an application running on a server and the server can be a component. One or more components may reside within a process and/or thread of execution and a component may be localized on one computer and/or distributed between two or more computers. In addition, these components can execute from various computer readable media having various data structures stored thereon. The components may communicate via local and/or remote processes such as in accordance with a signal having one or more data packets (e.g., data from one component interacting with another component in a local system, distributed system, and/or across a network such as the Internet with other systems via the signal). As another example, a component can be an apparatus with specific functionality provided by mechanical parts operated by electric or electronic circuitry, which is operated by a software or a firmware application executed by a processor, wherein the processor can be internal or external to the apparatus and executes at least a part of the software or firmware application. As yet another example, a component can be an apparatus that provides specific functionality through electronic components without mechanical parts, the electronic components can comprise a processor therein to execute software or firmware that confers at least in part the functionality of the electronic components.
In addition, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless specified otherwise, or clear from context, “X employs A or B” is intended to mean any of the natural inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, then “X employs A or B” is satisfied under any of the foregoing instances.
While the invention is susceptible to various modifications and alternative constructions, certain illustrated implementations thereof are shown in the drawings and have been described above in detail. It should be understood, however, that there is no intention to limit the invention to the specific forms disclosed, but on the contrary, the intention is to cover all modifications, alternative constructions, and equivalents falling within the spirit and scope of the invention.
In addition to the various implementations described herein, it is to be understood that other similar implementations can be used or modifications and additions can be made to the described implementation(s) for performing the same or equivalent function of the corresponding implementation(s) without deviating therefrom. Still further, multiple processing chips or multiple devices can share the performance of one or more functions described herein, and similarly, storage can be effected across a plurality of devices. Accordingly, the invention is not to be limited to any single implementation, but rather is to be construed in breadth, spirit and scope in accordance with the appended claims.
Number | Name | Date | Kind |
---|---|---|---|
3617587 | Nayar et al. | Nov 1971 | A |
3825848 | Sternbeck et al. | Jul 1974 | A |
5826977 | Fowler et al. | Oct 1998 | A |
7340690 | Lau | Mar 2008 | B2 |
7430570 | Srinivasan et al. | Sep 2008 | B1 |
7610437 | Sinclair et al. | Oct 2009 | B2 |
7769717 | Federwisch et al. | Aug 2010 | B2 |
7984084 | Sinclair | Jul 2011 | B2 |
8285918 | Maheshwari | Oct 2012 | B2 |
8443263 | Selinger et al. | May 2013 | B2 |
8572091 | Sivasubramanian et al. | Oct 2013 | B1 |
8655825 | Roesch et al. | Feb 2014 | B2 |
8732403 | Nayak | May 2014 | B1 |
8873284 | Sinclair et al. | Oct 2014 | B2 |
8984248 | Morishita et al. | Mar 2015 | B2 |
9223693 | Sinclair et al. | Dec 2015 | B2 |
9336133 | Sinclair et al. | May 2016 | B2 |
9348746 | Sinclair et al. | May 2016 | B2 |
9407521 | Kulkarni | Aug 2016 | B1 |
9465731 | Sinclair et al. | Oct 2016 | B2 |
9514018 | Sikri | Dec 2016 | B2 |
9591316 | Bracha et al. | Mar 2017 | B2 |
9639589 | Theimer et al. | May 2017 | B1 |
9715434 | Xu et al. | Jul 2017 | B1 |
9734050 | Sinclair et al. | Aug 2017 | B2 |
9734911 | Sinclair et al. | Aug 2017 | B2 |
9753932 | Brow et al. | Sep 2017 | B1 |
9778855 | Sinclair | Oct 2017 | B2 |
9892803 | Reed | Feb 2018 | B2 |
9898482 | Bono | Feb 2018 | B1 |
9965215 | Vazhenin et al. | May 2018 | B1 |
10108543 | Duggal et al. | Oct 2018 | B1 |
10108544 | Duggal et al. | Oct 2018 | B1 |
10120613 | Sinclair et al. | Nov 2018 | B2 |
10133490 | Sinclair et al. | Nov 2018 | B2 |
10255179 | Ji et al. | Apr 2019 | B2 |
10338834 | Dighe et al. | Jul 2019 | B1 |
10430279 | Dittia et al. | Oct 2019 | B1 |
10565208 | Triou, Jr. et al. | Feb 2020 | B2 |
10628424 | Park et al. | Apr 2020 | B2 |
10705741 | Varadarajan et al. | Jul 2020 | B1 |
10795812 | Duggal et al. | Oct 2020 | B1 |
10860457 | Evenson et al. | Dec 2020 | B1 |
10867033 | Keren et al. | Dec 2020 | B2 |
10891228 | Burow | Jan 2021 | B2 |
10983715 | Sharoni et al. | Apr 2021 | B2 |
11016826 | Lehmann | May 2021 | B2 |
11086537 | Byun | Aug 2021 | B2 |
11194638 | Danilov et al. | Dec 2021 | B1 |
11314779 | Jain | Apr 2022 | B1 |
11354054 | Danilov et al. | Jun 2022 | B2 |
20040199524 | Rys et al. | Oct 2004 | A1 |
20050025152 | Georgiou et al. | Feb 2005 | A1 |
20050055519 | Stuart et al. | Mar 2005 | A1 |
20060036568 | Moore et al. | Feb 2006 | A1 |
20060058987 | Kumar et al. | Mar 2006 | A1 |
20070033325 | Sinclair et al. | Feb 2007 | A1 |
20070047635 | Stojanovic | Mar 2007 | A1 |
20070220518 | Verbowski et al. | Sep 2007 | A1 |
20080059724 | Stifter, Jr. | Mar 2008 | A1 |
20080082596 | Gorobets | Apr 2008 | A1 |
20080144079 | Pandey et al. | Jun 2008 | A1 |
20080184262 | Ginis et al. | Jul 2008 | A1 |
20080189477 | Asano et al. | Aug 2008 | A1 |
20080288713 | Lee et al. | Nov 2008 | A1 |
20080301135 | Alves et al. | Dec 2008 | A1 |
20090182784 | Rohit et al. | Jul 2009 | A1 |
20100083098 | Leme et al. | Apr 2010 | A1 |
20100088296 | Periyagaram et al. | Apr 2010 | A1 |
20100077013 | Clements et al. | May 2010 | A1 |
20100125553 | Huang et al. | May 2010 | A1 |
20100125794 | Hampton et al. | May 2010 | A1 |
20100174881 | Anglin et al. | Jul 2010 | A1 |
20100205163 | Eshghi et al. | Aug 2010 | A1 |
20100281081 | Stager et al. | Nov 2010 | A1 |
20100333116 | Prahlad et al. | Dec 2010 | A1 |
20110126099 | Anderson et al. | May 2011 | A1 |
20110131588 | Allam et al. | Jun 2011 | A1 |
20110145473 | Maheshwari | Jun 2011 | A1 |
20110161784 | Selinger et al. | Jun 2011 | A1 |
20110249147 | Ishii | Oct 2011 | A1 |
20120102503 | Meijer et al. | Apr 2012 | A1 |
20120109985 | Chandrasekaran | May 2012 | A1 |
20120151014 | Gokhale et al. | Jun 2012 | A1 |
20120198027 | Hetzler et al. | Aug 2012 | A1 |
20120259994 | Gillies et al. | Oct 2012 | A1 |
20130226931 | Hazel et al. | Aug 2013 | A1 |
20130275808 | McNeeney et al. | Oct 2013 | A1 |
20140006465 | Davis et al. | Jan 2014 | A1 |
20140089264 | Talagala et al. | Mar 2014 | A1 |
20140223115 | Dinkjian et al. | Aug 2014 | A1 |
20140325148 | Choi et al. | Oct 2014 | A1 |
20140365719 | Kuzmin et al. | Dec 2014 | A1 |
20150169449 | Barrell et al. | Jun 2015 | A1 |
20150172120 | Dwarampudi et al. | Jun 2015 | A1 |
20150205816 | Periyagaram et al. | Jul 2015 | A1 |
20150227602 | Ramu et al. | Aug 2015 | A1 |
20150261776 | Attarde et al. | Sep 2015 | A1 |
20150341212 | Hsiao et al. | Nov 2015 | A1 |
20150355862 | Hayes et al. | Dec 2015 | A1 |
20150363245 | Mutschler | Dec 2015 | A1 |
20160042008 | Tripathy et al. | Feb 2016 | A1 |
20160063080 | Nano et al. | Mar 2016 | A1 |
20160210061 | Soncodi et al. | Jul 2016 | A1 |
20160246713 | Choi et al. | Aug 2016 | A1 |
20160321287 | Luthra et al. | Nov 2016 | A1 |
20160337435 | Nigam et al. | Nov 2016 | A1 |
20160350324 | Wang et al. | Dec 2016 | A1 |
20160357476 | Chen et al. | Dec 2016 | A1 |
20170038978 | Li et al. | Feb 2017 | A1 |
20170075832 | Bhimani et al. | Mar 2017 | A1 |
20170075947 | Kurilov et al. | Mar 2017 | A1 |
20170123655 | Sinclair et al. | May 2017 | A1 |
20170177249 | Kurilov et al. | Jun 2017 | A1 |
20170177263 | Das et al. | Jun 2017 | A1 |
20170177546 | Heinz et al. | Jun 2017 | A1 |
20170212891 | Pundir et al. | Jul 2017 | A1 |
20170213127 | Duncan | Jul 2017 | A1 |
20170255392 | Nakashima | Sep 2017 | A1 |
20170289214 | Cho et al. | Oct 2017 | A1 |
20180101842 | Ventura et al. | Apr 2018 | A1 |
20180121307 | Braun et al. | May 2018 | A1 |
20180146018 | Chang et al. | May 2018 | A1 |
20180176244 | Gervais et al. | Jun 2018 | A1 |
20180184138 | Shaw et al. | Jun 2018 | A1 |
20180189175 | Ji et al. | Jun 2018 | A1 |
20180314727 | Epstein et al. | Nov 2018 | A1 |
20180329644 | Das et al. | Nov 2018 | A1 |
20180332325 | Kaitchuck | Nov 2018 | A1 |
20180332365 | Kaitchuck et al. | Nov 2018 | A1 |
20180332366 | Paduroiu | Nov 2018 | A1 |
20180332367 | Kaitchuck et al. | Nov 2018 | A1 |
20180336256 | Li et al. | Nov 2018 | A1 |
20180345140 | Posin | Dec 2018 | A1 |
20190004863 | Mainali et al. | Jan 2019 | A1 |
20190026301 | Wang et al. | Jan 2019 | A1 |
20190057138 | Knowles et al. | Feb 2019 | A1 |
20190129806 | Hsu et al. | May 2019 | A1 |
20190138494 | Inoue | May 2019 | A1 |
20190197173 | Tahara et al. | Jun 2019 | A1 |
20190278849 | Chandramouli | Sep 2019 | A1 |
20190327297 | Madani | Oct 2019 | A1 |
20190332318 | Gooding et al. | Oct 2019 | A1 |
20190340180 | Barsness et al. | Nov 2019 | A1 |
20190349422 | Dhruvakumar et al. | Nov 2019 | A1 |
20200034468 | Lei et al. | Jan 2020 | A1 |
20200089420 | Sharoni et al. | Mar 2020 | A1 |
20200174695 | Bazarsky et al. | Jun 2020 | A1 |
20200242103 | Paduroiu | Jul 2020 | A1 |
20200250172 | Busjaeger et al. | Aug 2020 | A1 |
20200310686 | Truong et al. | Oct 2020 | A1 |
20200320005 | Shulman et al. | Oct 2020 | A1 |
20200344299 | Sohail et al. | Oct 2020 | A1 |
20200394196 | Shivanna et al. | Dec 2020 | A1 |
20200404011 | Gervais et al. | Dec 2020 | A1 |
20210110328 | Hsiao et al. | Apr 2021 | A1 |
20210124746 | Klaedtke | Apr 2021 | A1 |
20210157520 | Bavishi et al. | May 2021 | A1 |
20210256029 | Danilov et al. | Aug 2021 | A1 |
20210342296 | Danilov et al. | Nov 2021 | A1 |
20210342354 | Danilov et al. | Nov 2021 | A1 |
20210365211 | Danilov et al. | Nov 2021 | A1 |
20210374021 | Santhakumar et al. | Dec 2021 | A1 |
20220035533 | Danilov et al. | Feb 2022 | A1 |
20220182724 | Gupta et al. | Jun 2022 | A1 |
Number | Date | Country |
---|---|---|
2 534 170 | Jul 2007 | CA |
2672879 | Apr 2008 | CA |
1708029 | Dec 2005 | CN |
104901958 | Sep 2015 | CN |
105591926 | May 2016 | CN |
2377038 | Dec 2002 | GB |
2002101585 | Dec 2002 | WO |
2004080067 | Sep 2004 | WO |
2009014993 | Jan 2009 | WO |
2015196281 | Dec 2015 | WO |
2015191120 | Dec 2015 | WO |
2018148149 | Aug 2018 | WO |
Entry |
---|
Office Action dated Apr. 20, 2022 for U.S. Appl. No. 16/944,094, 48 pages. |
M. Liu, M. Li, D. Golovnya, E. A. Rundensteiner and K. Claypool, “Sequence Pattern Query Processing over Out-of-Order Event Streams,” 2009 IEEE 25th International Conference on Data Engineering, 2009, pp. 784-795, doi 10.1109/ICDE.2009.95. (Year: 2009). |
Notice of Allowance dated Apr. 27, 2022 for U.S. Appl. No. 17/127,724, 19 pages. |
Office Action dated May 24, 2022 for U.S. Appl. No. 17/237,535, 62 pages. |
Aloysius K. Mok, Honguk Woo and Chan-Gun Lee, “Probabilistic Timing Join over Uncertain Event Streams,” 12th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA'06), 2006, pp. 17-26, doi: 10.1109/RTCSA.2006.52. (Year: 2006). |
T. Onishi, J. Michaelis and Y. Kanemasa, “Recovery-Conscious Adaptive Watermark Generation for Time-Order Event Stream Processing,” 2020 IEEE/ACM Fifth International Conference on Internet-of-Things Design and Implementation (IoTDI), 2020, pp. 66-78, doi: 10.1109/IoTDI49375.2020.00014. (Year: 2020). |
Office Action dated Mar. 21, 2022 for U.S. Appl. No. 16/864,905, 125 pgs. |
Edi Muskardin et al., “Implementation of Hashing Algorithms in Stream Mining”, International Conference on Smart Systems and Technologies (SST), Oct. 2018 , pp. 233-238. |
Notice of Allowance dated Apr. 11, 2022 for U.S. Appl. No. 16/944,089, 87 pages. |
J. C. Lee, J. Vance and R. Lysecky, “Hardware-Based Event Stream Ordering for System-Level Observation Framework,” in IEEE Embedded Systems Letters, vol. 6, No. 4, pp. 81-84, Dec. 2014, doi: 10.1109/LES.2014.2359154. (Year: 2014). |
Office Action dated Apr. 12, 2022 for U.S. Appl. No. 17/038,102, 48 pages. |
Notice of Allowance dated Jun. 28, 2021 for U.S. Appl. No. 16/884,647, 39 pages. |
Office Action dated Jun. 29, 2021 for U.S. Appl. No. 16/881,556, 24 pages. |
Notice of Allowance dated Aug. 4, 2021 for U.S. Appl. No. 17/200,652, 51 pages. |
Office Action dated Jul. 8, 2022 for U.S. Appl. No. 16/864,892, 33 pages. |
Office Action dated Jul. 15, 2022 for U.S. Appl. No. 16/864,905, 77 pages. |
Office Action dated Sep. 1, 2022 for U.S. Appl. No. 16/944,094, 25 pages. |
Office Action dated Jul. 21, 2022 for U.S. Appl. No. 17/070,029, 68 pages. |
Notice of Allowance dated Aug. 24, 2022 for U.S. Appl. No. 17/152,544, 55 pages. |
Office Action dated Sep. 15, 2022 for U.S. Appl. No. 17/383,425, 61 pages. |
Phaujdar, “Understanding Event Streams—A Comprehensive Guide 101_ Learn Hevo” [https://hevodata.com/learn/understanding-event-streams/] (Feb. 17, 2021) (Year: 2021). |
Dhanushka, “Understanding Kafka Topic Partitions” [https://medium.com/event-driven-utopia/understanding-kafka-topic-partitions-ae40f80552e8]—(Mar. 28, 2021) (Year: 2021). |
Splunk, “Comparing Pulsar and Kafka” [https://www.splunk.com/en_us/blog/it/comparing-pulsar-and-kafka-how-a-segment-based-architecture-delivers-better-performance-scalability-and-resilience.html]—(Dec. 5, 2017) (Year: 2017). |
Office Action dated Sep. 30, 2022 for U.S. Appl. No. 17/408,344, 66 pages. |
Office Action dated Oct. 1, 2021 for U.S. Appl. No. 17/127,724, 41 pages. |
Notice of Allowance dated Dec. 15, 2021 for U.S. Appl. No. 17/064,747, 54 pages. |
Office Action dated Nov. 10, 2021 for U.S. Appl. No. 16/944,089, 75 pages. |
Kleppmann, Making Sense of Stream Processing—The Philosophy Behind Apache Kafka and Scalable Stream Data Platforms; Mar. 4, 2016; retrieved from [https://hashingit.com/elements/research-resources/2016-03-04-making-sense-of-stream-processing.pdf] on Nov. 5, 2021, (Year: 2016), 183 pages. |
Notice of Allowance dated Feb. 4, 2022 for U.S. Appl. No. 16/915,762, 57 pages. |
Office Action dated Oct. 1, 2021 for U.S. Appl. No. 16/915,762, 57 pages. |
Office Action dated Jan. 21, 2022 for U.S. Appl. No. 16/864,892, 26 pages. |
Notice of Allowance dated Feb. 24, 2022 for U.S. Appl. No. 17/038,079, 55pgs. |
Azhar et al., “Efficient selection of access control systems through multi criteria analytical hierarchy process”, IEEE, doi: 10.1109/ICET.2012.6375419, 2012, pp. 1-8. (Year: 2012). |
Rox et al., “Construction and Deconstruction of Hierarchical Event Streams with Multiple Hierarchical Layers”, IEEE, doi: 10.1109/ECRTS.2008.13, 2008, pp. 201-210. (Year: 2008). |
Notice of Allowance dated Feb. 18, 2022 for U.S. Appl. No. 17/083,145, 70 pgs. |
Akidau et al., “MillWheel: Fault-Tolerant Stream Processing at Internet Scale” Proceedings of the VLDB Endowment, vol. 6, No. 11, 2013, 12 pages. |
Akidau et al., “The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing” Proceedings of the VLDB Endowment, vol. 8, No. 12,2015, 12 pages. |
“Execution Model” [https://beam.apache.org/documentation/runtime/model/]. The Apache Software Foundation. Retrieved Aug. 26, 2020, 5 pages. |
“Apache Beam Programming Guide” [https://beam.apache.org/documentation/programming-guide/]. The Apache Software Foundation. Retrieved Aug. 26, 2020, 69 pages. |
“What is Apache Flink?—Applications” [https://flink.apache.org/flink-applications.html#building-blocks-for-streaming-applications]. The Apache Software Foundation. Copyright 2014-2019, retrieved Aug. 26, 2020, 7 pages. |
“What is Apache Flink?—Architecture” [https://flink.apache.org/flink-architecture.html]. The Apache Software Foundation. Copyright 2014-2019, retrieved Aug. 26, 2020, 3 pages. |
“Stateful Function—Event-driven Application on Apache Flink” [https://flink.apache org/stateful-functions.html]. The Apache Software Foundation. Copyright 2014-2019, retrieved Aug. 26, 2020, 8 pages. |
“What is Apache Flink?—Operations” [https://flink.apache.org/flink-operations html]. The Apache Software Foundation. Copyright 2014-2019, retrieved Aug. 26, 2020, 3 pages. |
“Use Cases” [https://flink.apache.org/usecases.html]. The Apache Software Foundation. Copyright 2014-2019, retrieved Aug. 26, 2020, 5 pages. |
“Introduction” [http://kafka.apache.org/intro]. The Apache Software Foundation. Copyright 2017, retrieved Aug. 26, 2020, 6 pages. |
“Apache Kafka Quickstart” [http://kafka.apache.org/quickstart]. The Apache Software Foundation. Copyright 2017, retrieved Aug. 26, 2020, 6 pages. |
“Use Cases” [http://kafka.apache.org/uses]. The Apache Software Foundation. Retrieved Aug. 26, 2020, 3 pages. |
“Architecture Overview” [pulsar.apache.org/docs/en/concepts-architecture-overview/]. The Apache Software Foundation Copyright 2020, retrieved Aug. 26, 2020, 11 pages. |
“Messaging” [pulsar.apache.org/docs/en/concepts-messaging/]. The Apache Software Foundation. Copyright 2020, retrieved Aug. 26, 2020, 32 pages. |
“Pulsar Overview” [pulsar.apache.org/docs/en/concepts-overview/]. The Apache Software Foundation. Copyright 2020, retrieved Aug. 26, 2020, 2 pages. |
Office Action dated Jan. 29, 2021 for U.S. Appl. No. 16/256,083, 42 pages. |
Non-Final Office Action received for U.S. Appl. No. 17/152,558, dated Dec. 7, 2022, 59 pages. |
Notice of Allowance received for U.S. Appl. No. 16/864,905, dated Nov. 30, 2022, 347 pages. |
Tony Tung et al., “Topology Dictionary for 3D Video Understanding”, IEEE Transactions on Pattern Analysis and Machine Intelligence (vol. 34, Issue: 8, Aug. 2012), Dec. 2011, pp. 1645-1657. |
A. Bulut et al., “Distributed data streams indexing using content-based routing paradigm”, 19th IEEE International Parallel and Distributed Processing Symposium, Apr. 2005, pp. 1-10. |
Notice of Allowance received for U.S. Appl. No. 17/070,029, dated Dec. 2, 2022, 36 pages. |
Non-Final Office Action received for U.S. Appl. No. 17/223,263, dated Nov. 28, 2022, 64 pages. |
Non-Final Office Action received for U.S. Patent Application U.S. Appl. No. 17/337,940, dated Dec. 30, 2022, 68 pages. |
Notice of Allowance dated Nov. 15, 2022 for U.S. Appl. No. 16/864,892, 48 pages. |
Notice of Allowance dated Nov. 23, 2022 for U.S. Appl. No. 17/408,344, 35 pages. |
Notice of Allowance dated Nov. 17, 2022 for U.S. Appl. No. 16/944,094, 23 pages. |
Office Action dated Feb. 10, 2023 for U.S. Appl. No. 17/145,588, 78 pages. |
Notice of Allowance received for U.S. Appl. No. 17/383,425, dated Feb. 1, 2023, 37 pages. |
Non Final Office Action received for U.S. Appl. No. 17/976,574 dated Mar. 2, 2023, 55 pages. |
Notice of Allowance received for U.S. Appl. No. 17/337,940 dated Mar. 1, 2023, 70 pages. |
Notice of Allowance received for U.S. Appl. No. 17/223,263, dated Apr. 6, 2023, 32 pages. |
Number | Date | Country | |
---|---|---|---|
20220107931 A1 | Apr 2022 | US |