This disclosure relates in general to the field of computer systems and, more particularly, to data analytics.
The Internet has enabled interconnection of different computer networks all over the world. While previously, Internet-connectivity was limited to conventional general purpose computing systems, ever increasing numbers and types of products are being redesigned to accommodate connectivity with other devices over computer networks, including the Internet. For example, smart phones, tablet computers, wearables, and other mobile computing devices have become very popular, even supplanting larger, more traditional general purpose computing devices, such as traditional desktop computers in recent years. Increasingly, tasks traditionally performed on a general purpose computers are performed using mobile computing devices with smaller form factors and more constrained features sets and operating systems. Further, traditional appliances and devices are becoming “smarter” as they are ubiquitous and equipped with functionality to connect to or consume content from the Internet. For instance, devices, such as televisions, gaming systems, household appliances, thermostats, automobiles, watches, have been outfitted with network adapters to allow the devices to connect with the Internet (or another device) either directly or through a connection with another computer connected to the network. Additionally, this increasing universe of interconnected devices has also facilitated an increase in computer-controlled sensors that are likewise interconnected and collecting new and large sets of data. The interconnection of an increasingly large number of devices, or “things,” is believed to foreshadow a new era of advanced automation and interconnectivity, referred to, sometimes, as the Internet of Things (IoT).
Like reference numbers and designations in the various drawings indicate like elements.
In some implementations, sensor devices 105a-d and their composite sensors (e.g., 110a-d) can be incorporated in and/or embody an Internet of Things (IoT) system. IoT systems can refer to new or improved ad-hoc systems and networks composed of multiple different devices interoperating and synergizing to deliver one or more results or deliverables. Such ad-hoc systems are emerging as more and more products and equipment evolve to become “smart” in that they are controlled or monitored by computing processors and provided with facilities to communicate, through computer-implemented mechanisms, with other computing devices (and products having network communication capabilities). For instance, IoT systems can include networks built from sensors and communication modules integrated in or attached to “things” such as equipment, toys, tools, vehicles, etc. and even living things (e.g., plants, animals, humans, etc.). In some instances, an IoT system can develop organically or unexpectedly, with a collection of sensors monitoring a variety of things and related environments and interconnecting with actuator resources to perform actions based on the sensors' measurements as well as with data analytics systems and/or systems controlling one or more other smart devices to enable various use cases and application, including previously unknown use cases. As such, IoT systems can often be composed of a complex and diverse collection of connected systems, such as sourced or controlled by a varied group of entities and employing varied hardware, operating systems, software applications, and technologies. Facilitating the successful interoperability of such diverse systems is, among other example considerations, an important issue when building or defining an IoT system.
As shown in the example of
Some sensor devices (e.g., 105a-d) in a collection of the sensor devices, may possess distinct instances of the same type of sensor (e.g., 110a-d). For instance, in the particular example illustrated in
Continuing with the example of
In some instances, prior to the sensor data being made available for consumption by one or more server systems (e.g., 120) or other devices, sensor data generated by a collection of sensor devices (e.g., 105a-d) can be aggregated and pre-processed by a data management system (e.g., 130). In some cases, a data management system 130 can be implemented separate from, and even independently of, server systems (e.g., 120) or other devices (e.g., 105a-d) that are to use the data sets constructed by the data management system 130. In such cases, data sets (generated from aggregate sensor data) can be delivered or otherwise made accessible to one or more server systems (e.g., 120) over one or more networks (e.g., 125). In other implementations, the functionality of data management system 130 can be integrated with functionality of server system 120, allowing a single system to prepare, analyze, and host services from a collection of sensor data sourced from a set of sensor devices, among other examples. In still other implementations, functionality of the data management system can be distributed among multiple systems, such as the server system, one or more IoT devices (e.g., 105a-d), among other examples.
An example data management system 130 can aggregate sensor data from the collection of sensor devices and perform maintenance tasks on the aggregate data to ready it for consumption by one or more services. For instance, a data management system 130 can process a data set to address the missing data issue introduced above. For example, a data management system 130 can include functionality for determining values for unobserved data points to fill-in holes within a data set developed from the aggregate sensor data. In some cases, missing data can compromise or undermine the utility of the entire data set and any services or applications consuming or otherwise dependent on the data set. In one example, data management system 130 can determine values for missing data based on tensor factorization. For example, in one implementation, data management system 130 can using a tensor factorization model based on spatial coherence, temporal coherence and multi-modal coherence, among other example techniques. Additionally, in instances where the data management system 130 is equipped to determine missing values in sensor data, the system can allow for sensors to deliberately under-sample or under-report data, relying on the data management system's ability to “fill-in” these deliberately created holes in the data. Such under-sampling can be used, for instance, to preserve and prolong battery life and the generally lifespan of the sensor devices, among other example advantages.
One or more networks (e.g., 125) can facilitate communication between sensor devices (e.g., 105a-d) and systems (e.g., 120, 130) that manage and consume data of the sensor devices, including local networks, public networks, wide area networks, broadband cellular networks, the Internet, and the like. Additionally, computing environment 100 can include one or more user devices (e.g., 135, 140, 145, 150) that can allow users to access and interact with one or more of the applications, data, and/or services hosted by one or more systems (e.g., 120, 130) over a network 125, or at least partially local to the user devices (e.g., 145, 150), among other examples.
In general, “servers,” “clients,” “computing devices,” “network elements,” “hosts,” “system-type system entities,” “user devices,” “sensor devices,” and “systems” (e.g., 105a-d, 120, 130, 135, 140, 145, 150, etc.) in example computing environment 100, can include electronic computing devices operable to receive, transmit, process, store, or manage data and information associated with the computing environment 100. As used in this document, the term “computer,” “processor,” “processor device,” or “processing device” is intended to encompass any suitable processing apparatus. For example, elements shown as single devices within the computing environment 100 may be implemented using a plurality of computing devices and processors, such as server pools including multiple server computers. Further, any, all, or some of the computing devices may be adapted to execute any operating system, including Linux, UNIX, Microsoft Windows, Apple OS, Apple iOS, Google Android, Windows Server, etc., as well as virtual machines adapted to virtualize execution of a particular operating system, including customized and proprietary operating systems.
While
The potential promise of IoT systems is based on the cooperation and interoperation of multiple different smart devices and sensors adding the ability to interconnect potentially limitless devices and computer-enhanced products and deliver heretofore unimagined innovations and solutions. IoT systems characteristically contain a significant number of diverse devices, many different architectures, diverse networks, and a variety of different use cases. Such diversity is the strength of IoT systems, but also presents challenges to the management and configuration of such systems.
In addition to having many different architectures, diverse networks, a variety of use cases, and a significant amount of devices with diverse characteristics, many IoT devices may additionally mandate low power constraints across a diverse set of IoT scenarios. Such IoT scenarios can include home automation system, smart city systems, smart farming applications, among other examples. For instance, with home automation an increasing number of IoT devices are being developed and entering the home. It can be impractical to have all of these varied devices connected to the central power source of the home (e.g., light sensors in the ceiling, smoke detectors in the ceiling, motion sensors around the doorway, etc.). Indeed, many IoT devices are being designed not to be reliant on a centralized AC power source, but rather batter power, to ensure the flexibility of their application. However, as a battery-powered device is reliant on the quality of its battery, such devices are prone to unpredictable power outages, as well as diminished performance as the battery capacity runs low, even when the battery is expected to power the device for months, years, etc. Maintaining power within an IoT system employing multiple battery-powered devices can thus place demands on the management of the system. Further, depending upon the number of battery-powered devices in the home (or other environment), an owner or manager of the property may be required to keep tabs on potentially dozens of the devices and bear the costs of repeatedly replacing such batteries. Further, part of the unpredictability of IoT device's power usage is the variability and adaptability of their activity. For instance, IoT devices in a smart city system may include sensor devices that sense such varied attributes as traffic, climate, weather, sunlight, humidity, temperature, stability of power supply and so on and so forth. Further, depending on the placement of each device, the variability of readings may differ dramatically, even between sensors of the same type. This implies that the uncertainty (or certainty) of each sensor reading may differ at different geolocations at different timestamps due to a family of factors. These scenarios all lead to a dilemma between system performance and power efficiency. Specifically, the more frequently readings are sampled at a device, the better accuracy that can be expected in terms of data analytics. However, as readings are sampled more frequently at the device, the higher the use of the device and its power source, thereby potentially diminishing the lifespan of the device and/or its power source.
In one implementation, given the changing variance of each sensor reading, a system can determine per-sensor and/or per data-instance variance to intelligently determine a corresponding sampling rate of a particular sensor during runtime, such that the number of samples is minimized (along with power consumption) while maintaining the integrity of the resulting sensor data set. For instance, the system may adopt a closed-loop client-server architecture for addressing the tradeoff between system performance and power efficiency using interactive sampling monitoring. Missing data within a set can be predictably (and reliably) and determined utilizing techniques such as interpolation, tensor factorization, as well as combinations of the two, such as described below. For instance, Discriminative Probabilistic Tensor Factorization (DPTF) can reliably estimate both missing data values and per instance variance for each sensor reading (either observed or predicted), rather than assuming a shared variance across all readings, as it traditionally done in data analysis systems.
Systems and tools described herein can address at least some of the example issues introduced above. For example, turning to
In one example, the system can include data management engine 130, a set of sensor devices (e.g., 105a-b), and server 120. The data set can be composed of sensor data (e.g., 235) generated by the collection of sensor devices (e.g., 105a-b). In one example, sensor devices 105a,b can include one or more processor devices 205, 210, one or more memory elements 215, 220, one or more sensors (e.g., 110a-b), and one or more additional components, implemented in hardware and/or software, such as a communications module 225, 230. The communications module 225, 230 can facilitate communication between the sensor device and one or more other devices. For instance, the communications modules 225, 230 can be used to interface with data management engine 130 or server 120 to make sensor data (e.g., 235) generated by the sensor device available to the interfacing system. In some cases, a sensor device (e.g., 105b) can generate sensor data and cause the data to be immediately communicated, or uploaded, to storage of another device or system (e.g., data management system (or “engine”) 130), allowing data storage capabilities of the sensor device to be simplified. In other instances, a sensor device (e.g., 105a) can cache or store the sensor data (e.g., 235) it generates in a data store (e.g., 240). The sensor data 235 in such instances can be made available to other systems (e.g., 120, 130) by allowing access to the contents of the data store 240, with chunks of sensor data being reported or uploaded to the consuming systems (e.g., 120, 130). A communications module 225, 230 can also be used to receive signals from other systems, such as suggested data sampling rates determined by the data management system 130, among other examples. Communications modules can also facilitate additional communications, such as communications with user devices used, for instance, to administer, maintenance, or otherwise provide visibility into the sensor device. In other implementations, sensor devices (e.g., 105a-b) can communicate and interoperate with other sensor devices, and communications module 225, 230 can include functionality to permit communication between sensor devices. Communications modules 225, 230 can facilitate communication using one or more communications technologies, including wired and wireless communications, such as communications over WiFi, Ethernet, near field communications (NFC), Bluetooth, cellular broadband, and other networks (e.g., 125).
In the particular example of
In one example, a data management engine 130 can include a missing data engine 260 embodied in software and/or hardware logic to determine values for missing data in the sensor data collected from each of and/or the collection of sensor devices 105a-b. For instance, in one implementation, missing data determination engine 260 can include tensor generation logic, tensor factorization logic, and/or interpolation logic, among other components implemented in hardware and/or software. In one example, the missing data determination engine 260 can process data sets or streams from each of the sensor instances (e.g., 110a,b) possessing missing data to determine one or more n-dimensional tensors 280 for the data. In some implementations, the data management engine 130 can utilize tensor factorization using corresponding tensors 280 to determine values for one or more missing data values in data received from the sensor devices. The missing data determination engine 260 can also utilize interpolation, in some instances, to assist in deriving missing data. For instance, interpolation can be used in combination with tensor factorization to derive missing data values in a data stream or set. In some cases missing data determination engine 260 can derive predicted values for all missing data in a particular data set or stream. In such instances, the data set can be “completed” and made available for further processing (e.g., in connection with services 290 provided by a server 120 or one or more other sensor devices). In other instances, tensor factorization can determine most but not all of the values for the missing data in a data set (e.g., from the corresponding tensor 280). In such instances, interpolation logic 275 can be used to determine further missing data values. Specifically, tensor factorization engine 270 can complete all missing values within the tensor representation. However, in some cases, values not comprehended within the tensor representation may be of interest (e.g., corresponding to geolocations without a particular deployed sensor type, instances of time without any observed sensor values, etc.). The interpolation logic 275 can operate on the partially completed data set 285 following tensor factorization learning. In other words, interpolation performed by interpolation engine 275 can be performed on the improved data set composed of both the originally-observed data values and the synthetically-generated missing data values (i.e., from tensor factorization). Interpolation can be used to address any missing data values remaining following tensor factorization to complete the data set 285 and make it ready for further processing.
A data management system 130 may additionally include a sampling rate management engine 260. The sampling rate management engine can be executable to determine the variance of data values generated by each of the sensors (e.g., using variance determination logic 265). Indeed, in some implementations, variance determination logic 265 can be configured to determine the variance on a per-sensor basis, as well as a per-instance (or per-data point) basis. Accordingly, sampling rate engine 260 can be used to determine the variability of the variance of each of the sensor instances (e.g., 110a,b) of the devices (e.g., 105a-c) to which it is communicatively coupled (e.g., over network 125). The variance measures determined by the variance determination logic 265 can be based on the accuracy, or degree of error, of the predicted missing data values derived by the missing data engine 260 for the same sensors. Indeed, tensor factorization can be utilized to derive the estimate variance measures for each of the data streams having missing data values. These variance measures can then be used (e.g., by sampling rate determination logic 270) to determine an optimized or minimized sampling rate, which could be communicated to and applied at each sensor (e.g., 110a,b) to allow the sensors to drop a portion of its data in an effort to preserve power and other resources of the sensor.
In one implementation, the system can be embodied as a non-interactive client-server system, in which a client (e.g., sensor device 105a,b) may randomly drop data points for power efficiency or other purposes while the server (e.g., the data management system) utilizing missing data determination logic to reliably reconstruct the full spectrum of the data. In an interactive client-server system (with bi-directional communication), the client (e.g., sensor device) is instructed explicitly by the server (e.g., data management system 130) with a specific determined probability, or rate (e.g., determined by sampling rate determination logic 270) at which the client can drop data while still allowing the missing data determination logic (e.g., 260) of the server (e.g., 130) to reliably reconstruct the full spectrum of the data (e.g., by reliably determining the values of the data dropped by the client).
To determine the rate at which a sensor device can drop data, the data management system 130 can determined, for each sensor (e.g., 110a,b), the variability of variance of data values generated by the particular sensor instance. In other words, the statistical variance (or uncertainty, confidence) can be determined at a per instance (i.e., per data point) level, such that at a certain location at a certain timestamp from a certain sensor type, variance determination logic (e.g., using discriminative probabilistic tensor factorization) can determine the variance of that corresponding data point (whether reported to or predicted by the missing data determination logic).
Accordingly, data management system 130 can interoperate with sensor devices (e.g., 105a,b) to provide an end-to-end architecture for interactive sampling monitoring, and in effect address low power constraints (among other issues), to allow sensors to randomly, opportunistically, or intelligently drop sensor data of any or all sensors, while the data management system 130 reconstructs the complete data from the intermittent (incomplete) data (e.g., to build data sets 285) and estimates the variance (error) for each data point (either observed or predicted). Further, the data management system 130 can periodically instruct (e.g., at a per data instant or longer frequency) one or more of the sensors (e.g., 110a,b) to dynamically adjust their respective sampling rate during runtime based on the corresponding changes in variance determined by the data management system 130.
A server system 120 can be provided to consume completed data sets 285 prepared by data management system 130. In one example, the server 120 can include one or more processor devices 292, one or more memory elements 295, and code to be executed to provide one or more software services or applications (collectively 290). The services 290 can perform data analytics on a data set 285 to generate one or more outcomes in connection with the service 290. In some cases, the service 290 can operate upon a data set 285 or a result derived by the data management system from the data set 285 to derive results reporting conditions or events based on information in the data set 285. In some examples, a service 290 can further use these results to trigger an alert or other event. For instance, the service 290 can send a signal to a computing device (such as another IoT device possessing an actuator) based on an outcome determined from the completed data set 285 to cause the computing device to perform an action relating to the event. Indeed, in some cases, other devices can host a service or an actuator that can consume data or data sets prepared by the data management system 130. In some cases, the service 290 can cause additional functionality provided on or in connection with a particular sensor device to perform a particular action in response to the event, among other examples.
While
Turning to the example of
As noted above, in some implementations, one or more sensor devices (e.g., 105a,b) in a system may include heterogeneous sensors (e.g., 110a, 110a′, 110b, 110b′). Upon data collection of sensor sj at each time step t, the sensor device dt uses a sampling probability pd
The data management system 130 may include computational logic to determine per instance variance estimation, for instance, using discriminative probabilistic tensor factorization (DPTF) (at 305) to predict variance (at 310) in a per instance (data point) manner (i.e., per device/per sensor/per time step instance). The per instance variance can then be used to generate a sampling probability (or rate) (at 315) for each sensor (e.g., 110a, 110a′, 110b, 110b′) on each device (e.g., 105a,b). The updated sampling probability (e.g., generated at 315) can then be sent back to the corresponding device (e.g., 105a,b). Upon successful receipt of the updated sampling probability, the device can determine whether to adopt the new sampling probability, and if adopted, can use the updated probability to determine, for the next or other subsequent data readings) whether or not to take or transmit the reading data back to the data management system.
In one example implementation, such as shown in the simplified block diagram 300 of
If the device (e.g., 105a) determines that a sampling probability applies to a given one of its sensors (e.g., 110a), before sending out (or in other implementations, even taking the reading), the device (e.g., 105a) can generate a random number (at 320) (e.g., with a value from 0-1) corresponding to the data instance and determine (at 325) whether the random number is greater or less than the identified sampling probability (e.g., sampling probability ps1 also with a value ranging from 0-1). In instances where the random number is greater than or equal to (or, alternatively, simply greater than) the sampling probability ps1, the device (e.g., 105a) can determine to send the corresponding data reading instance to the data management system 130. However, in instances where the device determines that the random number is less than (or, alternatively, less than or equal to) the sampling probability ps1, the device (e.g., 105a) can determine to drop the corresponding data reading instance, such that the data management system 130 never receives the reading and, instead, generates a replacement value for the dropped reading using missing data determination logic (e.g., utilizing discriminative probabilistic tensor factorization 305). In cases where the device (e.g., 105a) drops the data reading instance by cancelling the sending of the data, the device can store the dropped data reading in local memory (e.g., for later access in the event of an error at the data management system 130 or to perform quality control of missing data or variance estimate determined by the data management system 130, among other examples). In other instances, the device (e.g., 105a) can simply dispose of the dropped data.
Upon receiving an instance of reading data from a sensor (e.g., 110a), the data management system 130 can reconstruct missing data along with per instance variance, for instance, using discriminative probabilistic tensor factorization. In some cases, a tensor can be generated and user on a per-sensor device basis (e.g., with different tensors generated and used for each sensor), while in other instances, a single tensor can be developed for a collection of multiple sensors, among other implementations. The data management system 130 then uses the corresponding sensor's (e.g., 110a) per instance variance over time to determine the corresponding suggested sampling rate or sampling probability ps1 and thereby sampling rate (e.g., the probability multiplied by the sensor's native sampling frequency). For instance, a function can be determined utilizing machine learning techniques to determine the updated sampling rate corresponding to the latest per-instance variance determined for the sensor. Alternatively, control loop feedback (e.g., using a proportional-integral-derivative (PID) controller) can be utilized to iteratively derive and update the sampling rate from the history or per-instance variances determined for the sensor, among other examples. The newly determined sampling rate can then be returned, or fed back, to the corresponding device for application at the sensor within the closed loop of the architecture. Similar data sampling loops can be determined and applied for each of the sensors (e.g., (e.g., 110a, 110a′, 110b, 110b′)) coupled to the data management system by one or more networks. By determining the lowest sampling rate that can be applied at each device while preserving the data management system's ability to accurately reconstruct the deliberately dropped sensor data readings, the power and usage demands of the devices can be reduced, prolonging their lifespans.
Turning to
As noted above, discriminative probabilistic tensor factorization can be utilized both to reconstruct missing data values as well as derive per-instance variance for data generated by IoT sensors. In one example, to determine a tensor for a data stream or set, a 3-dimensional tensor can be defined by determining spatial coherence, temporal coherence, and multi-modal coherence of the data set. The tensor can represent the collaborative relationships between spatial coherence, temporal coherence, and multi-modal coherence. Coherence may or may not imply continuity. Data interpolation, on the other hand, can assume continuity while tensor factorization learns coherence, which may not be continuous in any sense. Spatial coherence can describes the correlation between data as measured at different points in physical space, either lateral or longitudinal. Temporal coherence can describe the correlation between data at various instances of time. Multi-modal coherence can describe the correlation between data collected from various heterogeneous sensors. The tensor can be generated from these coherences and can represent the broader data set, including unknown or missing values, with tensor factorization being used to predict the missing values.
Traditional techniques for determining missing data rely on data models based on one or more functions, f, each function being used to determine a respective value, y, from one or more respective variables, or features, x. In such models, the determination of the value y is dependent on x and the corresponding feature x must, therefore, be present for whichever data point (e.g., of y) we are to predict. In other words, features can be considered additional information that correlates with a particular set of data values. For example, in air quality inference, features may include population, temperature, weekday or weekend, humidity, climate, etc. upon which one or more other values are defined to depend. However, when a feature value is not available across space and time, values of other data dependent on the feature are not available. Consistent availability of features is not always comprehensive or available, resulting in errors when features are relied upon in interpolation of various data. Systems providing missing data tensor factorization based on spatio-temporal coherence with multi-modality can be performed without the use of features (although features can be used to supplement the power of the solution).
Coherence may not assume continuity in space and/or time, but instead learns collaboratively the coherence across space, time, and multimodal sensors automatically. Note that tensor representation does not assume continuity; namely, the results are the same even if, hyperplanes, e.g., planes in a 3D tensor, are shuffled beforehand.
While interpolation generally takes into account spatial continuity and temporal continuity, a data management engine may determine (or predict or infer) data values of multi-modality jointly and collaboratively using tensor factorization. As an example, in the case of a data set representing air quality samples, coarse dust particles (PM10) and fine particles (PM2.5) may or may not be correlated depending on spatial coherence, temporal coherence and other environmental factors. However, tensor factorization can learn their correlation, if any, without additional information or features (such as used by supervised learning techniques like support vector machines (SVMs) which mandate features), among other examples.
Turning to
As illustrated in
In one example, values of missing data (e.g., illustrated in
A multi-modal data set can be pre-processed through normalization to address variations in the value ranges of different types of data generated by the different sensors. In one example, normalization can be formulated according to:
Where μs denotes the mean and σs denotes the standard deviation of all observed values with a sensor type, or modality, s. In some cases, normalization can be optional.
Proceeding with the determination of missing data values in a data set, latent factors can be constructed and learned. Turning to
V
d,s,t
=D
d
·S
s
·T
t, where D ∈Rdk, S ∈Rsk,T ∈Rtk
Tensor factorization can address multi-modal missing data by generating highly accurate predictive values for at least a portion of the missing data. A tensor V with missing data can be decomposed into latent factors D, S, T.
In the absence of a feature for each data point (d, s, t), standard supervised machine learning techniques fail to learn a feature-to-value mapping. Tensor factorization, however, can be used to model data and infer its low rank hidden structure, or latent factors. Assuming there are latent factors for all device locations, sensor types and at all timestamps, the missing data can be modeled by learning latent factors from the (present) observed data. As a result, these latent factors can be utilized to make prediction and further optimizations. Given arbitrary latent factors of dimension k for each device location, sensor type and timestamp, predictions for a (missing) data point (d, s, t) can be determined according to the following formula:
d,s,t=ΞkDd,k*Ss,k*Tt,k (2)
Equations (1) and (2) can be used in combination to derive an objective function with latent factors. In some cases, using the mean-squared error between Equation (1) and (2) can be used to develop optimized training data, however, this approach can potentially over-fit the training data and yield suboptimal generalization results. Accordingly, in some implementations, a regularization term can be further applied to the objective function and applied to the latent factors, D, S, and T, to regularize the complexity of the model. For instance, an L2 regularization term, i.e. the Frobenius norm of latent factors, can be adopted to ensure differentiability through the objective function. As an example, regularization can be combined with normalization (e.g., Equation (1)) to yield:
Ξobserved(d,s,t)(
In Equation (3), λ is a value selected to represent a tradeoff between minimizing prediction error and complexity control.
To optimize Equation (3), stochastic gradient descent (SGD) can be used. For instance, an observed data point can be selected at random and can be optimized using the gradient of the objective function (3). For instance, an SGD training algorithm for latent factors can be embodied by as:
Resulting latent factors, D, S, T, can be regarded as a factorization of the original, observed dataset. For instance, as represented in
Through tensor factorization, missing data entries within the tensor can be recovered. However, in some cases, missing data values may lie outside the tensor in a multi-modal data set. For instance, if there are no values at all for a particular “plane” in the tensor, the corresponding latent factors do not exist (and effectively, neither does this plane within the tensor). In one example, planes of missing data in a tensor 605 can exist when there are no sensor readings at all devices at a particular time stamp. Additionally, planes of missing data in tensor 605 can result when there are no sensor readings at any time at a particular device location. Planes of missing data can be identified (before or after generation of the tensor 605) to trigger an interpolation step on the result of the tensor factorization. Bridging a spatial gap (e.g., a tensor plane) can be accomplished through interpolation to approximate the values for an unobserved device d′ as follows:
To bridge a gap in time, d′ can be generalized, for instance, by learning an objective function that minimizes the Euclidean distance between nearby time latent factors, among other example implementations.
In summary, a multi-modal data set composed of sensor data collected from a plurality of sensors on a plurality of sensor devices can be composed of observed data values as generated by the sensor devices. A subset of the data points in the original data set can be missing (e.g., due to sensor failure or malfunction, environmental anomalies, accidental or deliberate dropping of values, etc.). A tensor can be developed based on the original data set and serve as the basis of tensor factorization. From the tensor factorization, values for some or all of the originally missing data points can be determined, or predicted. In cases where the tensor factorization succeeds in determining values for each of the missing data points, the data set can be considered completed and made available for further processing and analysis. This may result when no empty “planes” are present in the tensor. When empty data point values remain following the tensor factorization an additional interpolation process can be performed in some instances on the updated data set (i.e., that includes the results of the tensor factorization but still some missing data values) to predict values for any remaining missing data points and produce the completed data set.
In some implementations, per instance variance estimation can be formulated in combination with a missing data reconstruction mechanism (e.g., described herein), as the variance calculation is intimately related to reconstruction error. In other words, the noisier a data point (or sensor) is, the less likely the missing data determination logic will be able to accurately reconstruct its values, resulting in a higher reconstruction error than other data points. Such as described herein, tensor factorization can be utilized to implement IoT multi-modal sensor missing data completion. Tensor factorization involves decomposition of a mode-n tensor (n-dimensional tensor) into n disjoint matrices, such as shown in
With Discriminative Probabilistic Tensor Factorization (DPTF), each data point instance can be modeled as an independent Gaussian distribution. To derive per instance variance; the unobserved per instance variance can be learned from a posterior distribution of data. For instance, tensor factorization for mean (i.e., missing data prediction) and variance can be performed simultaneously, with the output of each being used to formulate a posterior distribution for the data. The graphical model shown in
Equation 5: Estimation of mean (missing data prediction)
i
. . . i
=(V1)i
Equation 6: Estimation of variance
Equation 7: Prior distribution of mean latent factors
Equation 8: Prior distribution of variance latent variables
Equation 9: Likelihood distribution over latent variables.
In some implementations, tests can be conducted to verify, assess, or improve the function of the data management system. For instance, to verify the effectiveness of Discriminative Probabilistic Tensor Factorization (DPTF) logic of a data management system, a correlation can be calculated between predicted variance and the mean-squared-error of the missing data prediction. The mean-squared-error (MSE) can be first defined by the error of a missing data completion problem. That is, all observed data can be separated into disjoint training and testing data. The DPTF logic of the data management system can then be trained on the training data to capture instance wise distribution on the dataset. Thereafter, the expectation (mean) of instance-wise distribution can be used as its prediction, and MSE can be measured between the prediction and ground truth holdout data (e.g., the actual observed data as generated by the sensor and transmitted to the server). The interpretation of MSE can be regarded as the actual fitting level on the unobserved part of our model, while the variance can be regarded as the fitting level from the perspective of our model. Hence, the correlation between variance and MSE can be used to evaluate the feasibility of instance wise variance measurement. Baselines can be generated for use in the comparisons. Such baselines can include, for instance, random predictions, device information baselines (e.g., for a data point (device, sensor, timestamp), inverse of the number of records for the device in the training data can be used its prediction, based on the notion that more information available may imply more accurate prediction, sensor information baselines (e.g., similar to device information baselines, but defined as the inverse of the number of records for the sensor), and time information baselines (e.g., also to device information baselines, but defined as the inverse of the number of records for the timestamp), among other potential baselines.
While some of the systems and solution described and illustrated herein have been described as containing or being associated with a plurality of elements, not all elements explicitly illustrated or described may be utilized in each alternative implementation of the present disclosure. Additionally, one or more of the elements described herein may be located external to a system, while in other instances, certain elements may be included within or as a portion of one or more of the other described elements, as well as other elements not described in the illustrated implementation. Further, certain elements may be combined with other components, as well as used for alternative or additional purposes in addition to those purposes described herein.
Further, it should be appreciated that the examples presented above are non-limiting examples provided merely for purposes of illustrating certain principles and features and not necessarily limiting or constraining the potential embodiments of the concepts described herein. For instance, a variety of different embodiments can be realized utilizing various combinations of the features and components described herein, including combinations realized through the various implementations of components described herein. Other implementations, features, and details should be appreciated from the contents of this Specification.
Turning to
Processor 900 can execute any type of instructions associated with algorithms, processes, or operations detailed herein. Generally, processor 900 can transform an element or an article (e.g., data) from one state or thing to another state or thing.
Code 904, which may be one or more instructions to be executed by processor 900, may be stored in memory 902, or may be stored in software, hardware, firmware, or any suitable combination thereof, or in any other internal or external component, device, element, or object where appropriate and based on particular needs. In one example, processor 900 can follow a program sequence of instructions indicated by code 904. Each instruction enters a front-end logic 906 and is processed by one or more decoders 908. The decoder may generate, as its output, a micro operation such as a fixed width micro operation in a predefined format, or may generate other instructions, microinstructions, or control signals that reflect the original code instruction. Front-end logic 906 also includes register renaming logic 910 and scheduling logic 912, which generally allocate resources and queue the operation corresponding to the instruction for execution.
Processor 900 can also include execution logic 914 having a set of execution units 916a, 916b, 916n, etc. Some embodiments may include a number of execution units dedicated to specific functions or sets of functions. Other embodiments may include only one execution unit or one execution unit that can perform a particular function. Execution logic 914 performs the operations specified by code instructions.
After completion of execution of the operations specified by the code instructions, back-end logic 918 can retire the instructions of code 904. In one embodiment, processor 900 allows out of order execution but requires in order retirement of instructions. Retirement logic 920 may take a variety of known forms (e.g., re-order buffers or the like). In this manner, processor 900 is transformed during execution of code 904, at least in terms of the output generated by the decoder, hardware registers and tables utilized by register renaming logic 910, and any registers (not shown) modified by execution logic 914.
Although not shown in
Processors 1070 and 1080 may also each include integrated memory controller logic (MC) 1072 and 1082 to communicate with memory elements 1032 and 1034. In alternative embodiments, memory controller logic 1072 and 1082 may be discrete logic separate from processors 1070 and 1080. Memory elements 1032 and/or 1034 may store various data to be used by processors 1070 and 1080 in achieving operations and functionality outlined herein.
Processors 1070 and 1080 may be any type of processor, such as those discussed in connection with other figures. Processors 1070 and 1080 may exchange data via a point-to-point (PtP) interface 1050 using point-to-point interface circuits 1078 and 1088, respectively. Processors 1070 and 1080 may each exchange data with a chipset 1090 via individual point-to-point interfaces 1052 and 1054 using point-to-point interface circuits 1076, 1086, 1094, and 1098. Chipset 1090 may also exchange data with a high-performance graphics circuit 1038 via a high-performance graphics interface 1039, using an interface circuit 1092, which could be a PtP interface circuit. In alternative embodiments, any or all of the PtP links illustrated in
Chipset 1090 may be in communication with a bus 1020 via an interface circuit 1096. Bus 1020 may have one or more devices that communicate over it, such as a bus bridge 1018 and I/O devices 1016. Via a bus 1010, bus bridge 1018 may be in communication with other devices such as a user interface 1012 (such as a keyboard, mouse, touchscreen, or other input devices), communication devices 1026 (such as modems, network interface devices, or other types of communication devices that may communicate through a computer network 1060), audio I/O devices 1014, and/or a data storage device 1028. Data storage device 1028 may store code 1030, which may be executed by processors 1070 and/or 1080. In alternative embodiments, any portions of the bus architectures could be implemented with one or more PtP links.
The computer system depicted in
Although this disclosure has been described in terms of certain implementations and generally associated methods, alterations and permutations of these implementations and methods will be apparent to those skilled in the art. For example, the actions described herein can be performed in a different order than as described and still achieve the desirable results. As one example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In certain implementations, multitasking and parallel processing may be advantageous. Additionally, other user interface layouts and functionality can be supported. Other variations are within the scope of the following claims.
While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any inventions or of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.
Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.
The following examples pertain to embodiments in accordance with this Specification. One or more embodiments may provide a method, a system, a machine readable storage medium with executable code to identify a plurality of sensor data instances from a sensor device, determine at least one tensor for a data set based on the plurality of sensor data instances, determine a predicted value for each instance in the data set based on the tensor, determine a predicted variance for each instance in the data set based on tensor, and determine a sampling rate to be applied at the sensor device based on the predicted variances.
In one example, the sampling rate corresponds to a probability that sensor data dropped by the sensor device, and applying the sampling rate at the sensor device causes the sensor device to drop at least a portion of subsequent sensor data instances.
In one example, values of dropped sensor data instances are determined based on the tensor.
In one example, at least a portion of the values of dropped sensor data instances are determined through interpolation.
In one example, the plurality of sensor data instances correspond to instances in the data set and values of at least a portion of the instances of the data set are missing.
In one example, the sensor device is a particular one of a plurality of sensor devices and a respective tensor and a respective sampling rate are determined based on the corresponding tensor for each sensor of each of the plurality of sensor devices.
In one example, at least one of the plurality of sensor devices includes a plurality of sensors.
In one example, the tensor includes a 3-dimensional tensor with a spatial dimension, modality dimension, and temporal dimension.
In one example, the instructions, when executed, further cause the machine to determine, for each sensor data instance, a modality, a spatial location, and a timestamp of the sensor data instance.
In one example, tensor factorization is utilized to determine the predicted value and the predicted variance for each instance in the data set.
One or more embodiments may provide an apparatus including a sensor to detect attributes of an environment and generate sensor data instances describing the attributes, each sensor data instance corresponds to a reading of the sensor. The apparatus can include sampling logic to receive a signal over a network, where the signal indicates a sampling rate to be applied to the sensor, and apply the sampling rate to cause at least a portion of the sensor data instances to be dropped according to the sampling rate. The apparatus can include a transmitter to send undropped sensor data instances to a data management system.
In one example, the sampling logic is to receive a subsequent signal indicating an updated sampling rate to be applied to the sensor in response to a particular undropped sensor data instance sent to the data management system.
In one example, the sampling rate is based on a tensor corresponding to data generated by the sensor and each undropped sensor data instance cause the tensor and the sampling rate to be updated.
In one example, the apparatus includes a random number generator to generate, for each sensor data instance of the sensor, a random number, and applying the sampling rate includes determining a current value of the sampling rate, for each sensor data instance, comparing the sampling rate to the random number, and determining whether to drop the corresponding sensor data instance based on the comparing.
In one example, dropping a sensor data instance includes skipping the corresponding reading.
In one example, dropping a sensor data instance includes not sending the sensor data instance generated by the sensor.
In one example, the sensor includes a first sensor and the apparatus further includes at least a second additional sensor, and a respective sampling rate is received for each of the first and second sensors and updated based on respective sensor data instances generated by the corresponding sensor.
One or more embodiments may provide a method, a system, a machine readable storage medium with executable code to receive, over a network, a plurality of sensor data instances from a sensor device, determine a predicted value for each instance in the data set, determining a predicted variance for each instance in the data set, and determine a sampling rate to be applied at the sensor device based on the predicted variances.
In one example, at least one tensor for a data set can be determined based on the plurality of sensor data instances, and the predicted value and predicted variance for each instance in the data set are determined based on the at least one tensor.
In one example, a signal is sent to the sensor device indicating the determined sampling rate.
In one example, another data instance is received generated by the sensor device, the tensor is updated based on the other data instance, an updated sampling rate is determined based on the update to the tensor, and a signal is sent to the sensor device indicating the updated sampling rate.
One or more embodiments may provide a system including at least one processor, at least one memory element, and a data manager. The data manager can be executable by the at least one processor to receive, over a network, a plurality of sensor data instances from a sensor device, determine at least one tensor for a data set based on the plurality of sensor data instances, determine a predicted value for each instance in the data set based on the tensor, determine a predicted variance for each instance in the data set based on the tensor, and determine a sampling rate to be applied at the sensor device based on the predicted variances.
In one example, the system can include the sensor device, and the sensor device can apply the sampling rate to drop at least a portion of subsequent sensor data instances generated at the sensor device.
In one example, the data manager is further executable to predict values for the dropped portion of the subsequent data instances based on the tensor.
Thus, particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/US2015/000390 | 12/26/2015 | WO | 00 |