Patent Application
20200082012
The various aspects and embodiments described herein generally relate to a data analytics platform, and in particular, to a pipeline hinting system in which various modules in a data analytics pipeline can inform one another about module capabilities, condensing functions, anticipated data enrichments, temporal restrictions, and/or decision-making information to determine whether a given module should be passing constituent data or aggregate data.
In common parlance, the term “big data” is generally understood to refer to voluminous and complex data sets that conventional data processing systems cannot adequately handle. In practice, however, the term implicates challenges in many facets and dimensions, which include but are not limited to data capture, storage, analysis, query, search, visualization, and so on. Not only have data sets grown more and more voluminous, data sets are now generated with more diversity, speed, and noise than ever before. For example, data is generated whenever a person engages in online activity, carries a location-enabled smartphone while going about their day, or does anything else that involves a digital action or otherwise leaves a digital footprint. Moreover, with the increasing prevalence of technologies such as the Internet of Things, data is generated whenever “smart” machines communicate with one another, whether in the home, at the workplace, in industrial settings, or otherwise. Accordingly, because data sets are no longer limited to information stored in databases and other conventional data sources, modern data analytics should have the ability to handle unstructured data streams that can include images, videos, sound, binaries, text, and/or other data modalities, which may differ with respect to representation, distribution, scale, density, etc. (e.g., whereas text is usually represented as discrete sparse word count vectors, images may be represented according to pixel intensities, points of interest may be represented as spatial points associated with a given category, and so on). As such, data analytics tools should not treat disparate data sets equivalently, and normalization (e.g., converting all data to text data, which existing unstructured data platforms tend to require) can quickly become the dominating factor limiting a data ingest rate. There is therefore a need for mechanisms that have the ability to fuse and extract value from unstructured data.
Another important factor in data analytics systems is that the volume, variety, and velocity (as well as questionable veracity) of incoming data can quickly become overwhelming, which increases the difficulty to extract actionable intelligence or other value from the incoming data. What users need is a data analytics platform that can get up and running as fast as possible, does not require users to know what they do not know (or may want to know at some point in the future), and perform analytics on the data in a manner that enables users to rapidly explore data and build useful dashboards (or views) into the data in order to gain new insights, make predictions, make smarter decisions, and/or otherwise extract actionable intelligence from the data. Furthermore, although security is certainly one area in which data analytics has substantial value, the need for better data analytics extends into many other areas. As such, a data analytics platform should turn data into actual, usable, information and actionable intelligence and thereby enable organizations, individual users, and other entities to accomplish their mission, whatever that mission happens to be. Nonetheless, existing data analytics tools tend to fall short at least in areas that relate to speed, cost, flexibility, and complexity that increases the cognitive overhead on human operators. There is therefore a further need for a data analytics platform that enables high speed ingest and search, flexible data handling, and reduced cognitive overhead when users wish to engage the platform in order to extract meaning from large data sets.
The following presents a simplified summary relating to one or more aspects and/or embodiments disclosed herein. As such, the following summary should not be considered an extensive overview relating to all contemplated aspects and/or embodiments, nor should the following summary be regarded to identify key or critical elements relating to all contemplated aspects and/or embodiments or to delineate the scope associated with any particular aspect and/or embodiment. Accordingly, the following summary has the sole purpose to present certain concepts relating to one or more aspects and/or embodiments relating to the mechanisms disclosed herein in a simplified form to precede the detailed description presented below.
According to various aspects, a data analytics platform as described herein may enable a linear pipeline processing framework that substantially reduces the cognitive overhead on a human operator in crafting a data fusion pipeline compared to conventional architectures such as MapReduce. The linear pipeline processing framework allows an abstracted query language to define a data fusion pipeline assembly mechanism without explicit definition of data location, context, extraction, or normalization such that a human may think in terms of an assembly line, defining discrete operations without the need to perform a join on fully normalized data. The data analytics platform may remove the need for a user to be aware of data location or full normalization, while transparently leveraging concurrency to achieve high throughput.
According to various aspects, the data analytics platform described herein may comprise a single concurrent pipeline that contains four operator groups: storage, locality, selection, and renderer. The first operator group named “storage” is a storage subsystem that allows a data subset to be described via one or more identifiers that are expressed in a human-friendly (or human-readable) form. Each data entry in the data subset may include at least a byte array and a timestamp, wherein the timestamp may provide a temporal anchor to the data entry. In various embodiments, this temporal anchor may be derived from the data entry byte array and/or generated upon storage of the data entry. For example, the data entry may be given a timestamp at the time that the data entry is ingested and/or based on a time contained in the data entry itself, when available. The storage system may use the one or more identifiers to select relevant storage locations across N machines, each containing up to M storage arrays, and the storage system may extract data entries from the storage systems based on the temporal anchor such that the extracted data entries can be streams into a unified pipeline in a temporally ordered manner. The second operator group named “locality” may examine the totality of the abstracted query language and dynamically determine the optimal locality of execution for each component of the pipeline. As a result, the human need not think about where data is being processed as the locality operator group may distribute and execute the appropriate processing modules co-resident with the stored data and dynamically combine the data into a single processing stream. The third operator group named “selection” may enable the pipeline to establish linkages from data groupings to processing modules. In general, the pipeline may operate in a linear and concurrent manner, with each module operating only on the data groupings that the module is capable of operating on, passing on all other data down the pipeline. The final operator group named “renderer” may then receive the processed output and down select the data to only that which is needed in order to visualize or otherwise convey a fused and final output to the user. Accordingly, the unified, linear, and concurrent processing methodology set forth herein may allow for simplified abstraction of data processing while increasing the capability of a processing pipeline without dramatically increasing the complexity as presented to the user. The resulting methodology allows fusion of highly disparate data types such as text, sound, video, and others while allowing a human to think and operate in small discrete units.
According to various aspects, the data analytics platform described herein may further implement a pipeline hinting system in which various modules in a data analytics pipeline can inform one another about module capabilities, condensing functions, anticipated data enrichments (e.g., enumerated values that the modules expect to produce or consume), temporal restrictions, and/or decision-making information to determine whether a given module should be passing constituent data or aggregate data or to otherwise configure behavior associated with one or more modules in the data analytics pipeline. More particularly, the pipeline hinting system may include a first phase in which modules may inform other modules and/or be informed by other modules in the pipeline about the presence of any condensing modules, which may generally refer to the ability to aggregate data over a time range. As such, the condensing modules can change behavior associated with the pipeline in that any “upstream” modules preceding a condensing module may be configured to operate in a temporally discrete manner because the “downstream” condensing module(s) can aggregate data over a time range. However, when a given module detects that no downstream modules have the capability to condense temporally discrete data, the module can modify local behavior to output temporally concrete data and thereby become a condensing module. Furthermore, in various embodiments, the pipeline hinting system may include a second phase in which each module informs upstream modules (if any) about one or more enumerated data items that the respective module is operating on. As such, the upstream modules may determine whether downstream modules require that a data item be passed thereto in a fully processed and/or aggregated form. In response to a given module determining that a downstream module expects concrete output, the module may output a fully calculated result, whereas the module may not perform final calculations and instead pass discrete constituent data to downstream modules if the downstream module can handle discrete constituent data. In various embodiments, the pipeline hinting system may further include a third phase that involves determining whether a final renderer module has capabilities to perform second order temporal analytics. In the affirmative, upstream modules (e.g., search modules) may be hinted to use constituent data parts and to operate on a sliding temporal window when conducting data analytics, which may allow the final renderer module to re-frame discrete data at will given a new temporal window. On the other hand, if the final renderer module does not have capabilities to perform second order temporal analytics, the upstream modules may be configured to pass processed data to the final renderer module in a concrete aggregate form.
According to various aspects, a method for temporal data assembly may comprise receiving, at a data analytics platform operating on one or more machines, a plurality of data entries that each comprise at least a timestamp, a tag, and a byte array, assembling, by the data analytics platform, a data analytics pipeline configured to operate on a subset of the plurality of data entries associated with at least one tag, wherein the data analytics pipeline may include multiple modules arranged to operate on the byte array associated with each data entry in the subset of the plurality of data entries, and configuring, by the data analytics platform, the multiple modules to inform one another regarding capabilities associated therewith, wherein one or more of the multiple modules may be configured to determine whether to pass temporally discrete constituent data or temporally concrete aggregate data to a downstream module based at least in part on the capabilities associated with the downstream module.
According to various aspects, a data analytics system may comprise one or more storage devices configured to store a plurality of data entries that each comprise at least a timestamp, a tag, and a byte array and one or more processors coupled to the one or more storage devices, wherein the one or more processors may be configured to assemble a data analytics pipeline configured to operate on a subset of the plurality of data entries associated with at least one tag, the data analytics pipeline including multiple modules arranged to operate on the byte array associated with each data entry in the subset of the plurality of data entries, and to configure the multiple modules to inform one another regarding capabilities associated therewith, wherein one or more of the multiple modules may be configured to determine whether to pass temporally discrete constituent data or temporally concrete aggregate data to a downstream module based at least in part on the capabilities associated with the downstream module.
According to various aspects, a non-transitory computer-readable storage medium may have computer-executable instructions recorded thereon, wherein the computer-executable instructions may be configured to cause one or more processors to receive a plurality of data entries that each comprise at least a timestamp, a tag, and a byte array, assemble a data analytics pipeline configured to operate on a subset of the plurality of data entries associated with at least one tag, the data analytics pipeline including multiple modules arranged to operate on the byte array associated with each data entry in the subset of the plurality of data entries, and configure the multiple modules to inform one another regarding capabilities associated therewith, wherein one or more of the multiple modules are configured to determine whether to pass temporally discrete constituent data or temporally concrete aggregate data to a downstream module based at least in part on the capabilities associated with the downstream module.
Other objects and advantages associated with the aspects and embodiments disclosed herein will be apparent to those skilled in the art based on the accompanying drawings and detailed description.
A more complete appreciation of the various aspects and embodiments described herein and many attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings which are presented solely for illustration and not limitation, and in which:
Various aspects and embodiments are disclosed in the following description and related drawings to show specific examples relating to exemplary aspects and embodiments. Alternate aspects and embodiments will be apparent to those skilled in the pertinent art upon reading this disclosure, and may be constructed and practiced without departing from the scope or spirit of the disclosure. Additionally, well-known elements will not be described in detail or may be omitted so as to not obscure the relevant details of the aspects and embodiments disclosed herein.
The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments. Likewise, the term “embodiments” does not require that all embodiments include the discussed feature, advantage, or mode of operation.
The terminology used herein describes particular embodiments only and should not be construed to limit any embodiments disclosed herein. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. Those skilled in the art will further understand that the terms “comprises,” “comprising,” “includes,” and/or “including,” as used herein, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
Further, various aspects and/or embodiments may be described in terms of sequences of actions to be performed by, for example, elements of a computing device. Those skilled in the art will recognize that various actions described herein can be performed by specific circuits (e.g., an application specific integrated circuit (ASIC)), by program instructions being executed by one or more processors, or by a combination of both. Additionally, these sequences of actions described herein can be considered to be embodied entirely within any form of non-transitory computer-readable medium having stored thereon a corresponding set of computer instructions that upon execution would cause an associated processor to perform the functionality described herein. Thus, the various aspects described herein may be embodied in a number of different forms, all of which have been contemplated to be within the scope of the claimed subject matter. In addition, for each of the aspects described herein, the corresponding form of any such aspects may be described herein as, for example, “logic configured to” and/or other structural components configured to perform the described action.
According to various aspects, a data analytics platform as described herein may enable a linear pipeline processing framework that substantially reduces the cognitive overhead on a human operator in crafting a data fusion pipeline compared to conventional architectures such as MapReduce. The linear pipeline processing framework may define a data fusion pipeline assembly mechanism according to an abstracted query language without requiring data location, context, extraction, and/or normalization to be explicitly defined. As such, a human user operating or otherwise interacting with the data analytics platform may think in assembly line terms, defining discrete operations without the need to perform a join on fully normalized data. The data analytics platform may remove the need for the user to be aware of data location or full normalization, while transparently leveraging concurrency to achieve high throughput.
According to various aspects, the data analytics platform described herein may comprise a single concurrent pipeline that contains four operator groups: storage, locality, selection, and renderer. The first operator group named “storage” is a storage subsystem that allows a data subset to be described via one or more identifiers that are expressed in a human-friendly (or human-readable) form. Each data entry in the data subset may include at least a byte array and a timestamp, wherein the timestamp may provide a temporal anchor to the data entry. In various embodiments, this temporal anchor may be derived from the data entry byte array and/or generated upon storage of the data entry. For example, the data entry may be given a timestamp at the time that the data entry is ingested and/or based on a time contained in the data entry itself, when available. The storage system may use the one or more identifiers to select relevant storage locations across N machines, each containing up to M storage arrays, and the storage system may extract data entries from the storage systems based on the temporal anchor such that the extracted data entries can be streams into a unified pipeline in a temporally ordered manner. The second operator group named “locality” may examine the totality of the abstracted query language and dynamically determine the optimal locality of execution for each component of the pipeline. As a result, the human need not think about where data is being processed as the locality operator group may distribute and execute the appropriate processing modules co-resident with the stored data and dynamically combine the data into a single processing stream. The third operator group named “selection” may enable the pipeline to establish linkages from data groupings to processing modules. In general, the pipeline may operate in a linear and concurrent manner, with each module operating only on the data groupings that the module is capable of operating on, passing on all other data down the pipeline. The final operator group named “renderer” may then receive the processed output and down select the data to only that which is needed in order to visualize or otherwise convey a fused and final output to the user. Accordingly, the unified, linear, and concurrent processing methodology set forth herein may allow for simplified abstraction of data processing while increasing the capability of a processing pipeline without dramatically increasing the complexity as presented to the user.
According to various aspects, the data analytics platform described herein may further implement a pipeline hinting system in which various modules in a data analytics pipeline can inform one another about module capabilities, condensing functions, anticipated data enrichments (e.g., enumerated values that the modules expect to produce or consume), temporal restrictions, and/or decision-making information to determine whether a given module should be passing constituent data or aggregate data or to otherwise configure behavior associated with one or more modules in the data analytics pipeline. More particularly, the pipeline hinting system may include a first phase in which modules may inform other modules and/or be informed by other modules in the pipeline about the presence of any condensing modules, which may generally refer to the ability to aggregate data over a time range. As such, the condensing modules can change behavior associated with the pipeline in that any “upstream” modules preceding a condensing module may be configured to operate in a temporally discrete manner because the “downstream” condensing module(s) can aggregate data over a time range. However, when a given module detects that no downstream modules have the capability to condense temporally discrete data, the module can modify local behavior to output temporally concrete data and thereby become a condensing module. Furthermore, in various embodiments, the pipeline hinting system may include a second phase in which each module informs upstream modules (if any) about one or more enumerated data items that the respective module is operating on. As such, the upstream modules may determine whether downstream modules require that a data item be passed thereto in a fully processed and/or aggregated form. In response to a given module determining that a downstream module expects concrete output, the module may output a fully calculated result, whereas the module may not perform final calculations and instead pass discrete constituent data to downstream modules if the downstream module can handle discrete constituent data. In various embodiments, the pipeline hinting system may further include a third phase that involves determining whether a final renderer module has capabilities to perform second order temporal analytics. In the affirmative, upstream modules (e.g., search modules) may be hinted to use constituent data parts and to operate on a sliding temporal window when conducting data analytics, which may allow the final renderer module to re-frame discrete data at will given a new temporal window. On the other hand, if the final renderer module does not have capabilities to perform second order temporal analytics, the upstream modules may be configured to pass processed data to the final renderer module in a concrete aggregate form.
According to various aspects,
More particularly, referring to
According to various aspects, each data entry created at the ingester subsystem 120 based on the incoming raw data may comprise at least a byte array and a timestamp, wherein the timestamp may provide a temporal anchor used by the storage, locality, selection, and renderer operator groups mentioned above. Furthermore, in various embodiments, one or more data entries may optionally include one or more additional fields that can be used to provide more information about the data entry. For example, in various embodiments, the additional fields may include a tag field that associates a specific data entry with a specific tag in the indexer cluster 130, which may make the data entry easier to search at a later time. In another example, the additional fields may include a source address (e.g., a source IP address). In general, however, those skilled in the art will appreciate that any suitable fields may be defined to provide more detail about the data entry and thereby enable more robust data analytics. For example, in various embodiments, a given data entry may be structured as follows:
According to various aspects, referring still to
According to various aspects, as noted above, the indexer cluster 132 may generally comprise one or more indexers 132 that are configured to receive data entries from the ingesters 122 and to select appropriate locations to store the data entries within one or more storage systems 134. In particular, as will be described in further detail below in relation to at least
Accordingly, the indexer cluster 130 may generally be configured to act as a storage center in the data analytics platform 100, wherein the indexers 132 associated therewith may be responsible for storing, retrieving, and processing ingested data entries. Furthermore, in various embodiments, the indexers 132 may be configured to perform substantial initial processing when executing a query, first finding the appropriate data entries then pushing the data entries into a search pipeline in a temporally ordered manner. As noted above, the search pipeline (e.g., the locality operator group) may distribute as much of a query as possible to ensure that the indexers 132 can do as much initial processing in parallel as possible. As such, the storage system 134 may preferably include substantial high-speed low-latency storage resources and random access memory (RAM). For example, the indexers 132 may have the ability to utilize file system caches, meaning that the indexers 132 may have the ability to run multiple search queries over the same data from the file system caches without having to retrieve any data entries from disks associated with the storage systems 134, which may substantially improve search performance.
According to various aspects, as shown in
According to various aspects, as mentioned above, the webserver frontend 140 shown in
For example, in various embodiments, a particular search query may use the abstracted query language to specify a “upstream” data source (e.g., data entries associated with a particular tag), then specify one or more search modules 162 to analyze the data entries and filter out undesired data and/or extract interesting portions of the data, and finally specify a “downstream” renderer module 164 configured to down select the data output from the search modules 162 to only that which is needed in order to visualize or otherwise convey a fused and final output. For example, the following search query fetches data entries tagged “reddit” and then uses a json search module 162 to extract a field called “Body” before using an eval search module 162 to filter out any data entries with a Body field longer than twenty (20) characters, finally using a table renderer module 164 to display the contents of the Body fields in a table:
According to various aspects, a given search pipeline in the data analytics platform 100 may therefore include one or multiple search modules 162 that are arranged one after another in a substantially linear and concurrent manner, distributed and executed co-resident with the data on which the search module(s) 162 are configured to operate. Each search module 162 may operate on the results of the previous search module 162 in the search pipeline (e.g., the above example includes two search modules 162, json and eval, wherein the eval search module 162 operates on the results of the json search module 162 ). Furthermore, the renderer modules 164 take the results generated by the search modules 162 and produce a fused and final output suitable to convey to the user via the user interface 152 (e.g., in the above example, the table renderer module 164 is used to create a table suitable for displaying the contents of the enumerated Body fields). In general, a given search pipeline may include one renderer module 164, at the very end of the search pipeline. Accordingly, while the indexer cluster 130 in conjunction with the ingester subsystem 120 may generally implement the storage operator group mentioned above, the search pipeline may be dynamically assembled to at least partially implement the above-mentioned locality, selection, and renderer operator groups. For example, assuming that ingested data entries tagged “reddit” are stored in a particular data well, the json search module 162 configured to extract the Body field from such entries may be distributed and executed co-resident with such data entries by the locality operator group before the processed data entries are dynamically combined into a single processing stream for the eval search module 162. The selection operator group may then process the data in the pipeline in a manner that applies structures and adds intelligence to the unstructured data at runtime. For example, the selection operator group may be implemented via one or more tag descriptors that are handed to appropriate search modules 162, each of which may perform different actions on different data entries depending on the provided tag descriptor(s). Accordingly, the selection operator group may allow for feature extraction (e.g., extracting the Body element from raw JSON data), filtering (e.g., removing any data entries where the Body element does not contain specific data items), and/or alteration (e.g., processing the Body element to perform language translation). Furthermore, as noted above, the table renderer module 164 takes the results generated via the json and eval search modules 162 and down selects to contents of the Body fields in order to produce a fused and final output, thus implementing the renderer operator group mentioned above.
According to various aspects, the abstracted query language may include various enumerated values that are used by the search module(s) 162 and the renderer module(s) 164 forming the search pipeline. For example, in various embodiments, the enumerated values may include a source (SRC) enumerated value corresponding to the source of a given data entry, a tag (TAG) enumerated value representing the tag attached to the data entry at the ingester subsystem 120, and a timestamp (TIMESTAMP) enumerated value corresponding to the timestamp of the data entry. Furthermore, in various embodiments, the abstracted query language may allow user-defined enumerated values to be created and used in a search pipeline associated with a given search query. For example, in the search pipeline below, several enumerated values are created:
In the above example, the json search module 162 first parses JavaScript Object Notation (JSON) in raw data entries and pulls out a “Body” element, which is stored in a user-defined enumerated value named Body. The langfind search module 162 then accesses the Body enumerated values and attempts to analyze the language used, with the result put in a new enumerated value called lang. Next, the count search module 162 reads the lang enumerated values and counts how many times each value appears, storing the results in enumerated values named count. The rest of the search pipeline sorts the results in descending order based on the count enumerated values and creates a table from the lang and count enumerated values.
According to various aspects, the search modules 162 available in the data analytics platform 100 may each be configured to operate on data in a passthrough mode, meaning that the search modules 162 are configured to perform some action (e.g., filter, modify, sort, etc.) on the data entries and then pass the entries down the search pipeline. Furthermore, as noted above, each search module 162 may only operate on the data entries that the respective search module 162 is capable of or configured to operate on, passing on all other data down the search pipeline. In general, there can be many search modules 162 in a given search pipeline and each search module 162 may operate in a separate lightweight thread. As such, if there are N search modules 162 in a search pipeline, the search pipeline will spread out and use N threads. Furthermore, certain search modules 162 may cause distributed searches to collapse and/or sort. For example, a search module 162 that collapses may force the distributed search pipeline to collapse, meaning that the search module 162 as well as all downstream search modules 162 execute on the frontend 140. Accordingly, optimal performance may be achieved in use cases that put as many parallel search modules 162 as possible upstream of the first collapsing search module 162, as this approach may decrease pressure on the communication pipe and allow for greater parallelism in the data analytics platform 100.
According to various aspects, as mentioned above, the renderer modules 164 that are available in the data analytics platform 100 may be configured to receive data from the search module(s) 162 in the search pipeline and organize the received data in a manner suitable for display to the user via a user interface 152. In various embodiments, one or more of the renderer modules 164 may provide for a second order temporal index, which may allow a user to move around and zero in on time spans within the original search. Renderer modules 164 can optionally save search results in an appropriate storage location (not explicitly shown in
According to various aspects, as mentioned above, the search orchestrator 160 may assemble an unstructured data fusion pipeline (or “search pipeline”) when an appropriate search query is received or otherwise invoked at the webserver frontend 140, wherein the search query may be expressed according to an abstracted query language that does not require an explicit definition of data location, context, extraction, and/or normalization. Rather, the search orchestrator 160 may evaluate the totality of the abstracted query language and dynamically determine the optimal locality of execution for each component of the search pipeline, which may typically include one or more search modules 162 and a single renderer module 164. As a result, the human operator need not think about where data is being processed as the appropriate search module(s) 162 and renderer modules 164 may instead be distributed and executed co-resident with the stored data for dynamic combination into a single processing stream. For example, in various embodiments, the search orchestrator 160 may identify one or more search modules 162 specified in the search query and distribute at least a subset of the search modules 162 for execution on a machine in the indexer cluster 130 that is co-resident with the stored data. As such, the search module(s) 162 distributed for execution in the indexer cluster 130 may query one or more appropriate wells in the storage system(s) 134 based on the tag(s) further specified in the search query.
In this manner, the indexers 132 may perform the initial heavy lifting when executing the search query, first finding the appropriate data entries (e.g., data entries matching the tag(s) specified in the search query) and then pushing the data entries into the search pipeline. The search pipeline may distribute various search module(s) 162 to operate on the data entries retrieved from the storage system(s) 134 in parallel to the extent possible. Furthermore, the webserver frontend 140 may also participate in the search pipeline in that the webserver frontend 140 may perform some of the filtering, metadata extraction, and rendering of data. As such, when the search orchestrator 160 distributes the components of the search pipeline, at least the renderer module 164 at the end of the pipeline may be distributed for execution at the webserver frontend 140.
The search orchestrator 160 may further enable the search pipeline to establish linkages from data groupings to the search and renderer modules 162, 164, wherein the search pipeline may generally operate in a linear and concurrent manner, with each search module 162 only operating on the data groupings that the search module 162 is capable of operating on, passing on all other data down the search pipeline before the renderer module 164 receives the processed output and down selects the data to only that which is needed in order to visualize or otherwise convey a fused and final output to the user via the user interface(s) 152 at the client device(s) 150. Accordingly, the unified, linear, and concurrent processing methodology set forth herein may allow for simplified abstraction of data processing while increasing the capability of a processing pipeline without dramatically increasing the complexity as presented to the user. The resulting methodology allows fusion of highly disparate data types such as text, sound, video, and others while allowing a human to think and operate in small discrete units.
According to various aspects, as mentioned above, the data analytics platform 100 shown in
More particularly,
Referring to
According to various aspects, the flexible ingest framework described herein may allow for other topologies that may be more complex and/or secure than the topology 200 shown in
For example, in
Accordingly, the configuration shown in
According to various aspects, referring back to
According to various aspects, the storage orchestrator 260 may generally use the wells 270, 280, 290 to logically separate data entries of different types based on tags that are applied to the data entries at ingest points. For example, unique tags may be applied to syslog logs, Apache logs, network packets, video streams, audio streams, etc. to distinguish one data type from another. The wells 270, 280, 290 are the storage groupings that actually organize and store the ingested data entries. As such, each tag can be assigned to a certain one of the wells 270, 280, 290 so that data streams can be routed to faster or larger storage pools. For example, a raw pcap stream from a high bandwidth link may need to be assigned to a faster storage pool whereas log entries from syslog or a webserver may not require fast storage. A tag-to-well mapping is a one-to-one mapping, meaning that the same tag cannot be assigned to multiple wells. In various embodiments, a default well (e.g., well 270) may accept all tags, whereas any specifically configured wells (e.g., wells 280, 290) may accept data entries that are associated with certain tags. For example, the following snippet shows an indexer configuration with a well called “raw” that accepts data entries associated with pcap, video, testing, and apache tags and a default well that accepts all other data entries:
In various embodiments, tag names may be restricted to alphanumeric values, which may make the tags more human-friendly. For example, the tag names may not contain dashes, underscores, special characters, etc., which may promote a design whereby the tags are simple names that are easy for a human to type and reflect the type of data in use (e.g., “syslog” or “apache”). Furthermore, in various embodiments, tags are not assigned to the default well, which instead gets all data entries that are either untagged or associated with tags that have not been explicitly assigned to another well. For example, if a well named “syslog” has been assigned the tags “syslog” and “apache”, then all other tags will go to the default well. Although the ingester(s) 220 can still produce entries with tag names that are not explicitly defined, the data entries will be co-mingled with data entries associated with all other unassigned tags. Accordingly, the interaction between wells and tags is generally a management function that allows optimizing search, storage, and management of like data through describing a data subset via one or more human-friendly identifiers. According to various aspects, further detail relating to the storage topology associated with the indexer node 235 as shown in
According to various aspects, as mentioned above, the data analytics platform described herein may support arbitrarily deep ingester federation, meaning that a network classification may have multiple tiers or segments with ingesters appropriately deployed to relay data between networks or network segments that may have different levels of trust (e.g., from a public network to a private analysis network). For example,
According to various aspects,
As such, the federated ingest topologies 400, 500 described herein may be used to ingest data across geographically diverse regions when there may not be robust connectivity, provide an authentication barrier between network segments, control the tags that a data source group can provide, and/or reduce the number of connections to a given indexer (e.g., in
According to various aspects, referring now to
For example, referring first to
In general, the replication topologies may be logically separated into “Clients” and “Peers”, with each of the indexers 631-634 potentially acting as both a peer and a client. A client is generally responsible for reaching out to known replication peers and driving the replication transactions. When deploying a cluster in a replicating mode, the indexers 631-634 may be configured to initiate a TCP connection to any peers that may be acting as replication storage nodes, which refers to nodes that receive replicated data. Replication storage nodes may be allotted a specific amount of storage and are configured to not delete data until that storage has been exhausted. If a remote client node deletes a data shard as part of normal ageout, as described below, the data shard is marked as deleted and prioritized for deletion when the replication node hits the storage limit. The replication engine prioritizes deleted shards first, cold shards second, and oldest shards last. Furthermore, all replicated data may be compressed; if a cold storage location is provided the replication storage location may be configured to have at least the same storage capacity as the cold and hot storage combined.
According to various aspects,
According to various aspects,
According to various aspects,
According to various aspects,
According to various aspects, the replication engine used in the replication topologies described above may be configured as a best-effort asynchronous replication and restoration system designed to minimize impact on ingest and search. The replication engine may attempt a best-effort data distribution while focusing on timely assignment and distribution. This means that shards are assigned in a distributed first-come, first-serve order with some guidance based on previous distribution. The system does not attempt a perfectly uniform data distribution and replication peers with higher throughput (either bandwidth, storage, or CPU) may take on a greater replication load than peers with less. In a cluster topology intended to support data replication, the replication storage may be over-provisioned by a suitable amount (e.g., from 10-15%) to allow for unexpected bursts or data distribution that is not perfectly uniform.
In general, the replication engine may ensure that at least two core pieces of data are backed up, wherein the core pieces include tags and the actual entries to which the tags are assigned. Each indexer 631-634 independently maintains the mapping of tag names to storage wells, which are used for effective searching. Because the mappings of tag names to storage wells are relatively small, every indexer 631-634 may replicate the entire map to every other replication peer, although data may only be replicated once due to the larger storage requirements associated therewith. In various embodiments, the replication engine may be configured to coordinate with data ageout, migration, and well isolation policies, which are described in further detail below. For example, when a given one of the indexers 631-634 ages out data to a cold storage pool or deletes the data entirely, the data regions are marked as either cold or deleted on remote storage peers. The remote storage peers may use deletion, cold storage, and shard age when determining which data to keep and/or restore on a node failure. If a given one of the indexers 631-634 has marked data as deleted, the data will not be restored should the indexer 631-634 fail and recover via replication. Data that has previously been marked as cold will be put directly back into the cold storage pool during restoration. Post-recovery, the indexers 631-634 may be restored to the same state that the indexers 631-634 were in pre-failure when recovering using replication.
In various embodiments, as noted above, the data analytics platform described herein may implement one or more data ageout, migration, and/or well isolation policies either alone or in conjunction with the replication aspects described above. More particularly, the data analytics platform may support a well ageout system, whereby data management policies can be applied to individual wells 662. The ageout policies may allow for controlling data retention, storage utilization, and compression for each well 662, wherein each well 662 generally operates independently and asynchronously from all others. Each well 662 may further support a hot storage location and a cold storage location with a set of parameters used to determine how data is moved from one storage system to the other. In various embodiments, a storage architecture used in the data analytics platform may include several relatively small pools of high-speed storage that can tolerate random accesses as well as a high volume and low-cost storage pool to be used for longer term storage. Neither searching nor ingestion is impeded during ageout. However, if data is actively coming into a storage shard that is marked for ageout or is actively being queried, the ageout system may defer aging out the shard to a later time.
In various embodiments, the ageout policies used to control data retention, storage utilization, and compression for each well 662 may be defined via one or more parameters, which may include but are not limited to a time parameter, a total storage parameter, and a storage available parameter. For example, in various embodiments, the time parameter may be used to specify data retention policies to ensure compliance with or otherwise adhere to certain policies, contractual agreements, legal requirements, and/or other suitable requirements. The total storage parameter may be used to specify a storage bound for a well 622 and to instruct the data analytics platform to only ageout or discard data when the amount of stored data exceeds the specified storage bound. The storage availability parameter may be used to specify storage availability constraints based on the amount of storage left on a device, which may be useful to free storage on a device but discard data if the device ever drops below some availability threshold. In various embodiments, each constraint can be added to a single well 622, allowing for an intersection of rules. In general, the ageout policies may be configured to optimize data storage as data entries are transferred from hot pools to cold pools, localizing data entries that fall within the same or a similar time range and and/or data entries that have the same tag or tags that are otherwise related (e.g., different tags that are assigned to the same well 622), which may substantially reduce head movement on traditional spinning disks. Combined with compression, the optimization phase can substantially improve storage utilization and search performance on data entries stored in the cold pools. The ageout system can be further configured to delete old data, as appropriate.
According to various aspects, as noted above, the time-based ageout parameter allows for managing data based on time retention requirements (if any), wherein the time-based parameter may optionally be configured separately for hot and cold storage pools. For example, an organization may have requirements that all logs be kept for a certain duration (e.g., ninety days). As such, in one example, a first time-based ageout parameter may indicate that data entries are to be moved from a hot storage pool to a cold storage pool after a first time period (e.g., a number of days, weeks, months, etc.), and a second time-based ageout parameter may indicate that data entries are to be deleted from the cold storage pool after a second time period. In another example, a well 622 may be configured to only use a hot storage pool and a single time-based ageout parameter may be defined to indicate that data entries are to be deleted from the hot storage pool after a given time period.
According to various aspects, the total storage parameter may be used to allocate a specific amount of storage in a volume regardless of time spans. In this manner, the storage constraints may allow for configuring an indexer 631-634 that makes aggressive and full use of high speed storage pools that may have a limited size (e.g., Non-Volatile Memory Express (NVME) flash). The indexer 631-634 may keep data entries in the storage pool as long as the well 622 has not consumed more than the total storage parameter allows. The storage constraints may also prevent disruptions to data storage due to unexpected bursts of ingest. For example, if a given indexer (e.g., indexer 631) has one terabyte of high-speed flash storage that typically handles seven days of hot storage but an unexpected data event causes 600 GB of ingest in a single day, the indexer 631 can age out the older data to a cold pool without disrupting the ability of the hot pool to continue to take on new data. Data shards may be prioritized according to time, with oldest shards aged out first for both hot pools and cold pools. In various embodiments, the total storage constraint(s) may not be defined as instantaneous hard limit, meaning that data can be ingested and aged out substantially concurrently (e.g., the ingested data may be permitted to cause the total storage constraint to be temporarily exceeded while the age out is in progress).
According to various aspects, as noted above, storage constraints can also be applied to a given well 622 based on storage availability. For example, one or more wells 622 may be low priority and configured to only consume storage when available. Using the storage available parameter may therefore be used to specify a well 622 that is permitted to consume as much space as needed, provided that a defined minimum amount of available storage is maintained. For example, specifying a given value for hot storage availability for a given well 622 may instruct the well 622 to either migrate data from a hot storage pool to a cold storage pool (if available) or delete the oldest data shards in the hot storage pool in the event that available storage in the hot storage pool drops below the specified value. The storage available parameter may further apply to the underlying volume that may be hosting the associated storage location, meaning that if the volume is also hosting other wells 622 or other arbitrary file storage, the well 622 can pull back on storage usage to maintain at least the minimum available storage.
According to various aspects, referring now to
In various embodiments, the multiple webservers 740-1, 740-2 may have a substantially similar (if not identical) configuration, or at least connect to the same set of indexers 732-1 . . . 5 in a given indexer cluster 730. The client devices 750, 752, 754 may access an IP address, URL, or other suitable address associated with the load balancer 760, which may then direct the client devices 750, 752, 754 to one of the multiple webservers 740-1, 740-2 in a round-robin fashion. The load balancer 760 may be further configured to make sessions between the client devices 750, 752, 754 and the webservers 740-1, 740-2 sticky (e.g., by setting a cookie or token in the client device 750, 752, 754). As such, traffic between a given client device 750, 752, 754 may go to the same webserver 740 after the initial access request to ensure that the webserver 740 has the relevant state information associated with the requesting client device 750, 752, 754. In various embodiments, the load balancer 760 and the multiple webservers 740-1, 740-2 may run in substantially independent environments (e.g., on separate machines, as separate containers or virtual machine instances on a single machine or in an orchestrated cloud environment, etc.). As such, additional webservers 740 may be added to or removed from the distributed frontend 700 as needed.
According to various aspects, referring now to
In various embodiments, the storage operator group 810 may comprise a storage subsystem that allows for a data subset to be described via a series of human friendly identifiers or tags. For example, as described in further detail above, unstructured raw data 802 may be received at an ingester subsystem, which may be configured to bundle the unstructured raw data 802 into tagged data entries that each include at least a timestamp, a human-friendly identifier or tag, and a byte array. The storage operator group 810 may use the human-friendly identifiers or tags to select the appropriate storage locations for the tagged data entries across N machines, each of which may contain up to M storage arrays. For example, as described in further detail above, the storage locations may include a specific well configured to store data entries that have a given tag, wherein each well may be configured to store data entries associated with one or more specific tags. When needed, the storage operator group 810 may extract tagged data entries associated with a given search pipeline from the corresponding storage locations in a temporally ordered manner and stream the temporally ordered data entries into a unified pipeline, as depicted in
According to various aspects, the locality operator group 820 may examine the totality of the abstracted query language as used in a given search query and dynamically determine the optimal locality of execution for each pipeline component, which may generally include one or more search modules and one renderer module. As a result, the human user need not think about where data entries are being processed as the data analytics platform instead distributes the appropriate pipeline modules for execution at locations that are co-resident with the stored data entries, as depicted at 822. The locality operator group 820 may also dynamically combine the data entries into a single processing stream. For example, the locality operator group 820 may distribute one or more pipeline components to be executed on the same machine(s) storing the tagged data entries to be operated on in a given search pipeline (e.g., the machine(s) associated with the well(s) that are mapped to the tag(s) indicated in the search query). In another example, one or more pipeline components may be distributed and executed on a webserver frontend that acts as a focusing point for all searches and provides an interactive interface into the data analytics platform. For example, the webserver frontend may participate in the search pipeline and often perform at least the rendering function in addition some filtering, metadata extraction, and data condensing functions.
According to various aspects, the selection operator group 830 may be configured to establish linkages from data groupings to the pipeline components (e.g., search and renderer modules as distributed at 822). In particular, the selection operator group 830 may logically organize the pipeline components in a linear and concurrent fashion with each pipeline component operating only on the data groupings that it is capable of, passing on all other data down the pipeline. The final renderer operator group 840 receives processed data 832 from the selection operator group 830 and down selects the processed data to only that which is needed in order to visualize or otherwise convey a fused and final output 842 to the user.
Accordingly, the storage operator group 810, the locality operator group 820, the selection operator group 830, and the renderer operator group 840 may work in conjunction to implement a methodology in which highly disparate data types such as text, sound, video, and others can be fused into a single normalized stream while allowing a human to think and operate in small discrete units. For example, the various operator groups 810-840 may provide the ability to organize data entries having highly disparate data types that may otherwise be entirely incompatible (e.g., text logs, network data, voice, video, etc.) into a single normalized stream. As such, processing and feature extraction may be performed on the organized data entries and the output may be normalized into a single stream where the data entries having the different data types can eventually be rendered using a single renderer. For instance, one concrete example of the unstructured data fusion methodology may be where video from a secure door entry system is streamed along with data from proximity sensors and/or keycard access logs. The unstructured data fusion methodology may provide the ability to operate on all three data streams to provide a single view into building access, which may be used to detect tailgating or people without valid keycards loitering around a secure door, among other things. In another example, Bro text logs may contain one or more IP addresses and netflow binary items may contain one or more IP addresses, wherein the Bro text logs and the netflow binary items may have completely different data types. Nonetheless, the linear pipeline processing framework 800 shown in
According to various aspects,
In contrast, the unstructured data fusion pipeline 900 may stream the first temporally ordered data set 910 and the second temporally ordered data set 912 into a substantially linear pipeline in an assembly line fashion, where the assembly line may include one or more modules 920, 922, 924, 930, 940, 942 that each know the particular type of data to be handled thereby. As such, a human user may think about the search pipeline in a substantially linear manner rather than having to find the appropriate data entries, assemble the data entries into appropriate transportable packages, transporting the packages, and then unpacking the data entries to enable analysis thereon. For example, referring to
According to various aspects, with continued reference to
In various embodiments, the locality operator group 820 as shown in
In various embodiments, the various processing modules 920, 922, 924, 930, 940, 942 may implement the selection operator group 830 as shown in
Accordingly, the selection aspects described herein may cause each of the various processing modules 920, 922, 924, 930, 940, 942 to only operate on the data groupings that the processing modules 920, 922, 924, 930, 940, 942 have the capability to operate on, ignoring and passing all other data groupings down the pipeline 900. Furthermore, as noted above, the selection operator group 830 is used to organize the pipeline 900 into a linear and concurrent topology, wherein processing modules 920, 922, 924 operate in a linear manner and different instances of the processing modules 920, 922, 924 operate in a substantially concurrent manner. Furthermore, the output of the different instances of processing module 924 are linearly streamed to the condensing module 930, which operates in a linear and concurrent fashion with respect to the other processing modules 940, 942 down the pipeline 900. Finally, the renderer module 942 implements the renderer operator group 840 as shown in
According to various aspects, to further illustrate the fused and final output 950 that may be conveyed to the user, reference is made to
For example,
In another example,
In another example,
In still another example,
Those skilled in the art will appreciate that the examples shown in
According to various aspects, as mentioned above, the data analytics platform described herein may implement a pipeline hinting system in which various modules forming a data analytics pipeline can inform one another about module capabilities, condensing functions, anticipated data enrichments (e.g., enumerated values that the modules expect to produce or consume), temporal restrictions, and/or decision-making information to configure behavior associated with one or more modules in the data analytics pipeline (e.g., to pass constituent data or aggregate data to downstream modules depending on whether the downstream modules can handle temporally anchored data). For example, when creating a data analytics pipeline using an abstracted query language in the manner described in further detail above (e.g., where one or more search modules are specified to perform a given action on certain data sources and a single renderer module is specified to produce a fused and final output suitable for display to a user), adding modules to the data analytics pipeline may increase pipeline complexity in an exponential manner. As such, without a methodology to enable self-organization among the modules forming the data analytics pipeline, there is a limit to the analytics complexity that can be carried out.
Accordingly, the pipeline hinting system described in further detail herein may provide such a methodology to enable self-organization among the modules forming a data analytics pipeline and thereby remove the above-mentioned complexity barrier, which may advantageously allow more complex pipeline analytics. More particularly, as will be described in further detail below, the pipeline hinting system may be based on a multi-phase methodology in which various data analytics modules (e.g., search modules and renderer modules) can inform other modules in the pipeline and be informed by other modules in the pipeline as to module capabilities, condensing functions, anticipated data enrichments, temporal restrictions, and decision-making information such that each module may appropriately determine whether to pass or otherwise output constituent data or aggregate data.
More particularly, according to various aspects, the pipeline hinting system may generally be based around the idea that data entries flowing through a data analytics pipeline have a temporal anchor (e.g., a timestamp) that can potentially be preserved as the data entries flow through the data analytics pipeline before eventually arriving at a final renderer module, which may or may not have capabilities to perform second order temporal analytics. As such, in the following description, any references to a “temporal renderer” may refer to a renderer module that has capabilities to perform second order temporal analytics, whereas any references to a “non-temporal renderer” may refer to a renderer module that cannot perform second order temporal analytics. The pipeline hinting system may therefore be designed to keep the temporal anchor in place through to the final renderer module whenever possible and/or through as many modules as possible, which may generally depend on capabilities associated with each module forming the data analytics pipeline. For example, consider a data analytics pipeline in which a first module is configured to indicate an average number of data items that are flowing through at any given time and a second module is configured to identify a change in slope of a curve of the average (e.g., a variability or running an integral across the average). In context with this example, the pipeline hinting system may be configured such that the module running the average can look downstream and determine that the downstream module is operating on the same temporal window as the module running the average, so there is no need for the module running the average to collapse on anything. In other words, the upstream module will compute the average and ship the average downstream in a way that is structured on the sub-components of the averaging. As such, when the downstream module determines the change in the slope of the curve of the average, the sub-components (including the temporal anchor) are available, which may allow the downstream module to recombine over a different time window if needed. In contrast, if the upstream module is computing averages over a given time window and only passing the computed average downstream without also passing the sub-components from which the average was computed, the downstream module would be unable to recombine over a different time window because the sub-components are unavailable.
Accordingly, the pipeline hinting system described in further detail herein may provide a methodology in which the modules forming the pipeline may operate in a coordinated manner. For example, in various embodiments, one or more upstream modules may provide one or more downstream modules with any data sub-components used to perform a configured function while also informing the downstream modules as to how the configured function was performed. Consequently, the downstream modules may change behavior associated therewith if necessary (e.g., recombining the data sub-components over a different time window than the upstream modules). Similarly, in the other direction, the downstream modules may signal one or more requirements and/or capabilities to the upstream modules (e.g., requesting that the upstream modules operate over a given time window because the downstream modules need to operate over that time window, indicating an inability to change behavior to recombine over a time window that differs from the upstream modules, etc.). Accordingly, the upstream modules and the downstream modules may inform one another as to whether they are anchored in time and whether they can continue to be anchored in time. Furthermore, the upstream modules and the downstream modules may inform one another as to how data needs to be shipped or otherwise passed through the pipeline (e.g., in a concrete aggregate form that only includes a final calculated output, or in a temporally discrete constituent form in which any temporally anchored data sub-components are passed down the pipeline along with the final calculated output). For example, if the data analytics pipeline includes a “dumb” downstream renderer that cannot change its behavior (e.g., a non-temporal renderer that cannot perform second order analytics), the upstream modules may configure their behavior to ship finalized data in a concrete aggregate form without the data sub-components (e.g., only the average number of data items flowing through the data analytics pipeline at a given time).
According to various aspects, having given the above overview of the pipeline hinting system, the various phases of the pipeline hinting system will now be described in more detail. In the following description, there may be various references to data that is considered “concrete”, “temporally concrete”, “finalized”, “aggregated”, and/or any suitable combination thereof, which those skilled in the art will appreciate is intended to refer to data in which the totality of output is presented in a finalized form. Relatedly, those skilled in the art will appreciate that any references to data considered “discrete”, “temporally discrete”, “constituent”, and/or any suitable combination thereof is intended to refer to data processed within a given time window that also includes the temporally anchored data sub-components that were processed to produce the processed data.
According to various aspects, referring now to
According to various aspects, with specific reference to
According to various aspects, the first phase of the pipeline hinting system as illustrated in
According to various aspects, referring now to
According to various aspects, the second phase of the pipeline hinting system as illustrated in
According to various aspects, referring now to
According to various aspects, the third and final phase of the pipeline hinting system as illustrated in
Accordingly, because the non-temporal renderer 1650 does not have the ability to perform second order temporal analytics, processed data will be passed to the non-temporal renderer 1650 in a temporally concrete aggregate form, as depicted at 1640. However, the modules 1612, 1614, 1616, 1618 upstream of the condensing module 1610 may suitably pass temporally discrete data at 1630, 1632, 1634, 1636, as the condensing module 1610 has the ability to aggregate data over a time range before passing the temporally concrete data to the non-temporal renderer 1650. On the other hand, in the second data analytics pipeline 1600-2, the temporal renderer 1652 does have the ability to perform second order temporal analytics, whereby the upstream modules 1612, 1614, 1616, 1618, 1619 are all hinted to use constituent data parts and to operate on a sliding temporal window when conducting analytics. As such, the upstream modules 1612, 1614, 1616, 1618, 1619 may each ship temporally discrete data down the pipeline 1600-2, whereby the temporal renderer 1652 may receive the constituent data parts at 1642. In this manner, the temporal renderer 1652 may have the ability to re-frame the temporally discrete constituent data parts given a new temporal window that may differ from a temporal window used at the upstream modules 1612, 1614, 1616, 1618, 1619.
Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the aspects disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted to depart from the scope of the various aspects and embodiments described herein.
The various illustrative logical blocks, modules, and circuits described in connection with the aspects disclosed herein may be implemented or performed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices (e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, etc.).
The methods, sequences, and/or algorithms described in connection with the aspects disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of non-transitory computer-readable medium known in the art. An exemplary non-transitory computer-readable medium may be coupled to the processor such that the processor can read information from, and write information to, the non-transitory computer-readable medium. In the alternative, the non-transitory computer-readable medium may be integral to the processor. The processor and the non-transitory computer-readable medium may reside in an ASIC. The ASIC may reside in an IoT device. In the alternative, the processor and the non-transitory computer-readable medium may be discrete components in a user terminal.
In one or more exemplary aspects, the functions described herein may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a non-transitory computer-readable medium. Computer-readable media may include storage media and/or communication media including any non-transitory medium that may facilitate transferring a computer program from one place to another. A storage media may be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of a medium. The term disk and disc, which may be used interchangeably herein, includes CD, laser disc, optical disc, DVD, floppy disk, and Blu-ray discs, which usually reproduce data magnetically and/or optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.
While the foregoing disclosure shows illustrative aspects and embodiments, those skilled in the art will appreciate that various changes and modifications could be made herein without departing from the scope of the disclosure as defined by the appended claims. Furthermore, in accordance with the various illustrative aspects and embodiments described herein, those skilled in the art will appreciate that the functions, steps, and/or actions in any methods described above and/or recited in any method claims appended hereto need not be performed in any particular order. Further still, to the extent that any elements are described above or recited in the appended claims in a singular form, those skilled in the art will appreciate that singular form(s) contemplate the plural as well unless limitation to the singular form(s) is explicitly stated.
