There are many applications where data is transmitted from a device to a server for further processing. For example, for Internet of Things (IoT), the number of devices that capture sensory information like images, audio, three-dimensional point-cloud, environmental data, etc. has increased dramatically over the past few years and it is predicted that the deployment of such devices will continue to increase. In many instances, devices, such as sensors that capture sensory data about their environment for IoT applications or other types of devices, periodically or continuously transmit data over a network to a remote computer for further processing.
Features of the present disclosure are illustrated by way of example and not limited in the following figure(s), in which like numerals indicate like elements, in which:
For simplicity and illustrative purposes, the principles of the present disclosure are described by referring mainly to examples thereof. In the following description, numerous specific details are set forth in order to provide an understanding of the examples. It will be apparent, however, to one of ordinary skill in the art, that the examples may be practiced without limitation to these specific details. In some instances, well known methods and/or structures have not been described in detail so as not to unnecessarily obscure the description of the examples. Furthermore, the examples may be used together in various combinations.
Throughout the present disclosure, the terms “a” and “an” are intended to denote one of a particular element or multiple ones of the particular element. As used herein, the term “includes” means includes but not limited to, the term “including” means including but not limited to. The term “based on” may mean based in part on.
When a large number of devices stream data to a remote computer over a network, such as in an IoT environment or other environments, it can lead to high bandwidth consumption and network congestion. Consequently, this can lead to a reduction in quality of service for all the devices connected to the network or for applications that rely on receiving the data in a timely manner. For example, in many instances, these devices stream data to a remote server for further processing. Applications may rely on the processed data for performing their functions. If there is a delay in streaming the data, which may be caused by network congestion, then the performance of the applications relying on the received data can degrade, especially for real-time applications.
Disclosed according to examples herein are apparatuses, systems, and methods for automatically controlling transmission rate from a device to a remote computer over a network based on a prediction of data being transmitted by the device. For example, a device may stream data to a remote computer. The data may be sensor data captured by a sensor of the device. A prediction is made as to a current value of the sensor data and then the prediction is compared to an actual current value measured by the sensor. If the predicted current value and the actual current value are similar, then the transmission rate may be reduced because it is assumed the transmitted data is not changing beyond what is expected. A machine learning predictor may be used to predict the sensor data. Also, a machine learning encoder may be used to encode data to a lower dimensional space prior to making predictions on the sensor data. Additionally, a comparison function that helps minimize false positives may be used to compare the predicted current sensor data with the actual current sensor data to determine whether the predicted and actual current sensor data are different for making transmission rate modification decisions. Through implementation of the apparatuses, systems, and methods disclosed herein, transmission may be automatically adjusted rate to alleviate potential network congestion, and thus improve quality of service for applications relying on the data transmitted from the device.
The machine-readable instructions 130 are described with respect to a flow diagram shown in
The processor 102 executes the instructions 131 to predict current data, shown as predicted current data 225 in
The predictor 232, for example, is a machine learning predictor. For example, a neural network may be trained to make the predictions for sensor data comprised of images or audio. Different machine learning functions may be used for making predictions for different types of data. In an example, the predictor 232 may be a deep learning model employing a deep neural network (DNN), such as convolutional neural network (CNN), long short-term memory (LSTM) in cascade with a fully-connected neural network (FCNN), or the like. The predictor 232 may be trained with a collection of annotated datasets that are used for supervised-learning tasks and based on the previous data 223. In an example, the predictor 232 is trained to identify people and/or objects in an image, and based on the previous data 223. The predictor 232 may predict that an identified object should be located in a particular location in a current image. If the object is determined to be at that location in the actual current image, then the predicted current image and the actual current image may be considered to be the same or similar.
Instructions 132 shown in
In an example, the comparison function 233 is a cosine similarity function. Cosine similarity is a metric used to determine how similar vectors are irrespective of their size. Mathematically, it measures the cosine of the angle between two vectors projected in a multi-dimensional space. In an example, pixels in the captured current image and the predicted current image are values in respective vectors, and the vectors are compared using cosine similarity. If the vectors are the same, the difference metric 226 (e.g., cosine similarity metric) is 1.0, and the difference metric 226 is 0 if they are orthogonal; otherwise the difference metric 226 is between 0 and 1 depending on how similar or different the vectors are. It will be apparent to one of ordinary skill in the art that other types of comparison functions may be used to compare the predicted current data 225 and current data 224.
Instructions 133 are executed by the processor 102 to determine, based on the difference metric 226, whether to modify a transmission rate at which the data, such as the sensor data 222, is transmitted to a remote computer. For example, the sensor 221 may stream the sensor data 222 to a remote computer for further processing. Also, multiple sensors may stream sensor data to the remote computer over the same network. If the predicted current data 225 and current data 224 are determined to be the same or similar, then the transmission rate for transmitting the sensor data 222 captured by the sensor 221 may be reduced to alleviate network congestion. Generally, if the predictor 232 is able to predict the current data 224, e.g., the predicted current data 225 and the current data 224 are the same or similar, then it is assumed that the current data 224 is what is expected and the transmission rate may be reduced. If the predicted current data 225 and the current data 224 are different, then the transmission rate may be increased. For example, if the sensor 221 is a security camera streaming images or video of an entrance to a home, the captured images may be the same or quite similar if no one is approaching the entrance to the home. Thus, there may not be a need to receive the images as frequently or as quickly. However, if the captured image includes a person approaching the entrance to the home, then the captured image including the person is different than the predicted image not including a person. Then, the transmission rate may be increased to receive images or video more frequently or quickly in order to identify the person approaching the entrance to the home as quickly as possible and provide proper notifications as quickly as possible.
Similar operations may be performed for other sensors transmitting data to the remote computer. In the example shown in
The machine-readable instructions 130 shown in
The encoder 231 encodes the sensor data 222 to a lower dimensional space. The encoder 231 reduces the number of dimensions by determining a subset of the dimensions less than the original that are principal dimensions. For example, for image sensor data, the encoder 231 determines a subset of the pixels in the image that are important, e.g., 1000 pixels, and uses that subset as the lower dimension vector to represent the image.
In an example, the encoder 231 is a machine learning encoder for performing dimensionality reduction. For example, the encoder is trained to learn the temporal dynamics of the sensor data 222, and to rely on features learned from the training to make more accurate assessments of the dimensions that are changing and may be of interest. For images or video or speech, the encoder 231 may be a neural network. For images or video, the encoder 231 may be trained with input images and an input parameter that specifies the number of dimensions to represent the image. For example, autoencoders are neural networks used for image or audio data that are trained to reconstruct their original inputs instead of classifying them. An autoencoder may structure a hidden layer of the neural network to have fewer neurons than the input/output layers. Thus, that hidden layer learns to produce a smaller representation of the original image, i.e., lower dimensional image. An autoencoder may be created through unsupervised learning because the input image is used as the target output.
The dimensionality reduction performed by the encoder 231 provides multiple advantages. It reduces the time and storage space required. Also, it can improve the performance of the machine learning model for the predictor 232 because the predictor 232 is focused on the dimensions that may be of interest.
The processor 102 executes instructions 302 shown in
The processor 102 executes instructions 303 to apply a comparison function to the predicted current data and encoded current data to determine a difference metric. For example, the predicted current data 225′ and the encoded current data 224′, which are in the lower dimensional space, are applied to comparison function 233 to determine the difference metric 226, similarly as is described with respect to
In another example, determination of whether to adjust the transmission rate is performed remotely at the remote computer 540. For example, sensor 521 captures time series sensor data, similar to the sensor data 222 discussed above, and transmits the sensor data to the remote computer 540 via the network 230. The remote computer 540 includes communication interface 544, processor 542, data store 543 and non-transitory computer readable medium 541. The remote computer 540 receives the sensor data via the network 230 and the communication interface 544, and stores the sensor data in the data store 543. The processor 542 executes machine-readable instructions, such as machine-readable instructions 130 or 330, stored in the computer readable medium 541. Also, the encoder 231, the predictor 232, the comparison function 233 and/or the mapping functions 234 shown in
In another example, the processing of sensor data to determine whether to adjust transmission rate to the remote computer 540 may be performed locally and remotely. For example, some sensors may be integrated or connected locally to a computer that can perform the processing to determine whether to adjust transmission rate, and some sensors may not be integrated or connected locally to such a computer, so the processing to determine whether to adjust transmission rate is performed at the remote computer 540 for those sensors.
At 601, data, such as time series sensor data 222 captured by sensor 221, is received. For example, the sensor data 222 may be stored locally, such as in buffer 230, and received by the processor 102 to determine whether to adjust the transmission rate, or the sensor data 222 may be received at the remote computer 540, and the remoter computer 540 determines whether to adjust the transmission rate.
At 602, previous sensor data is input to a machine learning predictor, such as predictor 232, to determine a prediction of the current sensor data. For example, as shown in
At 603, a difference metric is determined by applying a comparison function to the predicted current sensor data and received current sensor data. For example, as shown in
At 604, the difference metric is mapped to a transmission rate. For example, as shown in
Although described specifically throughout the entirety of the instant disclosure, representative examples of the present disclosure have utility over a wide range of applications, and the above discussion is not intended and should not be construed to be limiting but is offered as an illustrative discussion of aspects of the disclosure.
What has been described and illustrated herein is an example of the disclosure along with some of its variations. The terms, descriptions and figures used herein are set forth by way of illustration only and are not meant as limitations. Many variations are possible within the scope of the disclosure, which is intended to be defined by the following claims—and their equivalents—in which all terms are meant in their broadest reasonable sense unless otherwise indicated.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/US2019/057018 | 10/18/2019 | WO |