Patent Application
20220115148
The present disclosure relates to self-assessment of a machine learning model running on a device. The disclosure has particular, but not exclusive, relevance to self-assessment of a machine learning model running on an Internet of Things (IoT) device.
The Internet of things (IoT) describes a system of interconnected electronic devices, each of which has a unique identifier and a capability to transfer data over the Internet without requiring human intervention. Examples of IoT devices range from household appliances such as lighting fixtures, doorbells, audio speakers, televisions, washing machines and refrigerators, to energy or water meters, sensing devices and vehicles. Providing such devices with network connectivity allows for a wide range of functionalities to be implemented, for example performance monitoring, data gathering, real-time analytics and/or remote control of devices. In many cases, IoT devices make use of machine learning models to implement such functionalities.
In some cases, a single enterprise or owner may be responsible for a large number of IoT devices, for example hundreds, thousands, or tens of thousands of devices. Management of a large number of devices, including for example managing firmware versions running on the devices, managing device security, and training machine learning models running on the devices, requires significant resources and infrastructure. Cloud-based device management platforms, such as the Arm® Pelion® IoT platform, have been developed to reduce the burden of managing IoT devices, whilst providing the device owner/operator with a customizable level of control over the devices.
During the period in which an IoT device is deployed, properties of input data processed by the device may change. This may occur, for example, due to sensor degradation, human error during deployment or maintenance, or physical changes to the device and/or the environment in which device is deployed. If the device is arranged to process input data using a machine learning model, and the input data no longer sufficiently resembles the training data upon which the machine learning model was trained, the machine learning model may not be competent for use with the new input data. This may result in erroneous outputs from the machine learning model, which may in turn result in suboptimal performance or malfunctioning of the device. The suboptimal performance or malfunctioning may go undetected for a significant period of time, potentially having costly or dangerous consequences.
According to a first aspect, there is provided a system including a device management infrastructure arranged to process a set of training data using a data transformation model to generate first characteristic data indicative of values of one or more characteristics for the set of training data, and an electronic device communicatively coupled to the device management infrastructure. The device includes memory circuitry arranged to store a machine learning model trained using the set of training data, and a copy of the data transformation model. The device is arranged to process a set of input data using the copy of the data transformation model to generate second characteristic data indicative of values of said set of data characteristics for the set of input data. The device and/or the device management infrastructure is arranged to determine whether the first characteristic data and the second characteristic data satisfy one or more consistency criteria indicative of consistency between the set of training data and the set of input data.
According to second aspect, there is provided device management system. The device management system is arranged to process a set of training data for a machine learning model using a data transformation model to generate first characteristic data indicative of values of one or more characteristics for the set of training data, receive second characteristic data from an electronic device indicative of values of said one or more characteristics for a set of input data generated by the electronic device, and determine whether the first characteristic data and the second characteristic data satisfy one or more consistency criteria indicative of consistency between the set of training data and the set of input data.
According to a third aspect, there is provided a device including memory circuitry and one or more sensors. The memory circuitry is arranged to store a machine learning model; and a data transformation model. The device is arranged to receive first characteristic data from a device management system indicative of one or more characteristics for a set of training data used to train the machine learning model, generate a set of input data using the one or more sensors, process the generated set of input data using the data transformation model to determine second characteristic data indicative of values of a set of data characteristics for the set of input data, and determine whether the first characteristic data and the second characteristic data satisfy one or more consistency criteria indicative of consistency between the set of training data and the set of input data.
Further features and advantages of the invention will become apparent from the following description of preferred embodiments of the invention, given by way of example only, which is made with reference to the accompanying drawings.
Details of systems and methods according to examples will become apparent from the following description with reference to the figures. In this description, for the purposes of explanation, numerous specific details of certain examples are set forth. Reference in the specification to ‘an example’ or similar language means that a feature, structure, or characteristic described in connection with the example is included in at least that one example but not necessarily in other examples. It should be further noted that certain examples are described schematically with certain features omitted and/or necessarily simplified for the ease of explanation and understanding of the concepts underlying the examples.
An IoT device as described above typically include one or more firmware applications for implementing the functionality of the device. The firmware application is part of a firmware image written to a read-only memory (ROM) of a device, comprising low-level machine-readable instructions for implementing various functionalities of the device (for example, controlling hardware or performing real-time analytics). Firmware is typically installed at the time of manufacturing of a device, but may be updated during the life-cycle of the device, for example to add security patches or to improve or modify the functionality of the device. In cases where an IoT device implements a machine learning model, the machine learning model may be included as part of the firmware image, and updating the firmware image may include updating the machine learning model, for example after the machine learning model has undergone training.
As shown in
As shown in
In the example of
The device management system 104 obtains, at 402, a set of training data for training the machine learning model. The set of training data includes individual data points, each of which has one or more numerical components. Depending on the specific application, the set of training data may for example be collected from devices 106 which have already been deployed, retrieved from a database of historic data, collected automatically or manually from a laboratory or other test facility, or generated artificially from simulations. The training data may be labeled training data for use in training a supervised machine learning model or may be unlabeled training data for use in training an unsupervised machine learning model. The training data may alternatively be indicative of observed states of an environment, actions performed by a decision-making agent in said states of the environment, and rewards associated with the performance of those actions, for training the decision-making agent using reinforcement learning.
The device management system 104 trains, at 404, the machine learning model using the set of training data obtained at 402. Any suitable training method may be used, where the suitability of different methods will depend on the nature of the machine learning model. After the machine learning model has been trained (for example when the machine learning model satisfies predetermined performance criteria or convergence criteria, or when the entire set of training data has been used for training), the machine learning model is ready for deployment on the device 106. When deployed, the machine learning model will process input data to perform the task for which the machine learning model is trained. The machine learning model is expected to be competent to perform its intended function when processing input data that closely resembles the set of training data. Depending on the properties of the machine learning model, the machine learning model may also be able to generalize to input data which differs slightly from the training data. However, it is not expected that the machine learning model will perform adequately with input data having significantly different properties to those of the set of training data.
The device management system 104 processes, at 406, the set of training data using the data transformation model, to generate first characteristics data indicative of values of one or more characteristics for the set of training data. The characteristics may include, for example, mathematical moments such as mean, variance, skewness, kurtosis, and higher moments for data points in the set of training data, and/or other parameters of a distribution from which the data points are assumed to be sampled, estimated for example using the generalized method of moments (GMM). The characteristics may additionally, or alternatively, include maximum or minimum values for one or more components of the data points, and/or confidence intervals for one or more components of the data points. The characteristics are chosen to provide salient information about the underlying distribution from which it is assumed that the training data points are sampled.
The device management system 104 sends, at 408, the trained machine learning model to the device 106. Sending the trained machine learning model may involve sending an entire replacement machine learning model, or may instead involve sending updated parameter values for the machine learning model (for example, weights and biases where the model is based on a neural network architecture). The trained machine learning model may be transmitted to the device 106 as a differential update as described above, or may be transmitted as part of an entirely new firmware image. The device 106 receives the trained machine learning model at 410, and performs a firmware update such that the updated active firmware image on the device 106 incorporates the trained machine learning model. The device 106 reboots, and upon successful authentication of the firmware update by the bootloader on the device 106, begins to process input data generated by the sensors 304 using the trained machine learning model.
The device 106 processes, at 412, the input data generated by the sensors 304 using the copy of the data transformation model stored in the memory 308, to generate second characteristic data indicative of values of the one or more characteristics for the set of input data. The device 106 may apply the data transformation model in an iterative/streaming fashion, resulting in substantially continuous monitoring of the input data as the input data is generated, or the device 106 may apply the data transformation model in a batch fashion, for example by buffering input data generated by the sensors 304 in the memory 308, and processing the buffered input data intermittently using the data transformation model. The data transformation model may be applied periodically, for example every hour, every day, or at any other suitable frequency depending on the application. The data transformation model may alternatively be applied when a predetermined volume of input data has been generated, for example when a predetermined number of input data points have been generated, which may be appropriate when input data is not generated at a constant frequency. In another example, the device 106 may only be operational during certain times, for example only during daytime hours or only during the nighttime hours. In such cases, the device 106 may buffer input data during the time that the device 106 is operational, and then apply the data transformation model when the device 106 is not required to perform its usual functions. The microprocessor 306 of a given device 106 may be a relatively basic processor with limited processing resources, and accordingly may not be suitable for performing multiple tasks simultaneously. It therefore may be particularly advantageous to apply the data transformation model at a time when the microprocessor 306 is otherwise relatively inactive.
The device 106 sends, at 414, the second characteristic data to the device management system 104 and the device management system 104 receives the second characteristic data at 416. In cases where the data transformation model is applied in a batch fashion, the device 106 may send second characteristic data each time the data transformation model is applied. In cases where the data transformation model is applied in a streaming fashion, the device 106 may send second characteristic data periodically or when a certain amount of input data has been processed. By applying the data transformation model and/or sending the second characteristic data at a relatively low frequency compared with the frequency at which the input data is generated, the device 106 can save power and bandwidth use, both of which are important consideration for IoT devices. In any case, the data transformation model should be applied frequently enough to substantially mitigate costs or dangers associated with the device 106 operating with input data that is out of the range of competence of the on-board machine learning model.
The device management system 104 determines, at 418, whether the first characteristic data and the second characteristic data satisfy one or more consistency criteria. The consistency criteria are designed to measure whether the training data and the input data are sufficiently similar that the trained model is deemed competent for use with the input data. The consistency criteria may include, for example, a difference between a value of a characteristic for the training data and a value of the same characteristic for the input data being less than a specified threshold value. In this way, the consistency criteria can measure whether the set of input data sufficiently resembles the set of training data. The consistency criteria may include a predetermined distance between values of one or more characteristics for the two data sets being less than a specified value, or any other suitable metric for measuring a distance between distributions, such as a Kullback-Leibler divergence or other measure of divergence. Additionally, or alternatively, the consistency criteria may include values of one or more characteristics for the set of input data, for example a range or confidence interval, falling within limits depending on corresponding values for the set of training data. In this way, the consistency criteria can determine whether values for the set of input extend beyond a region for which the set of training data is deemed competent.
If the device management system 104 determines at 418 that the first characteristic data and the second characteristic data satisfy the one or more consistency criteria, the device 106 continues operating using the trained machine learning model, and routine returns to 412 to process the next set of input data at the next designated time.
If the device management system 104 determines at 418 that the first characteristic data and the second characteristic data do not satisfy the one or more consistency criteria, the device management system 104 determines, at 420, properties for a new set of training data for retraining the machine learning model running on the device 106. The device management system 104 determines the properties for the new set of training data in dependence on the second characteristic data and, optionally, the first characteristic data. In a first example, the device management system 104 may determine that the machine learning model should be retained, either from scratch or in a continued manner, using a new set of training data with properties consistent with those of the set of input data. This may be suitable if, for example, the device 106 is deployed in a new environment, and the properties of the input data generated by the sensors 304 in the new environment are not consistent with those of the training data (which may correspond to a different environment). The device management system 104 may, for example, send a request to the device 106 to send input data generated at the device 106 to the device management system 104, for use as new training data for the machine learning model. Alternatively, the device management system 104 may generate simulated training data with properties corresponding to those of the input data, or may output a request to a human user or automated system to collect new training data based on the determined properties.
In the example described above, the device management system 104 determines properties for the new set of training data such that the new set of training data resembles the input data generated by the sensors 304 at the device 106. However, in some cases at least a portion of the set of input data will resemble the original set of training data. In other words, the distribution of the set of input data may overlap or intersect with the distribution of the set of training data. In this case, it may not be efficient to use a new set of training data that resembles the entire set of input data, because the machine learning model is already competent for use with input data in the overlapping region of the distributions. In
The device management system 104 may determine properties for the new set of training data corresponding only to a portion of the input data, for example the non-overlapping region 506 shown in
The method described above with reference to
The device 106 determines, at 614, whether the first characteristic data and the second characteristic data satisfy one or more consistency criteria. Examples of consistency criteria are described above with reference to
If the device determines that the one or more consistency criteria are not satisfied, the device 106 sends, at 616, the second characteristic data to the device management system 104. The device management system 104 receives the second characteristic data at 618, and determines, at 620, properties for a new set of training data for retraining the machine learning model running on the device 106.
The method of
In the examples described above, the data processing system 104 determines properties for a new set of training data upon determining that the machine learning model is not competent for use with a set of input data, and furthermore may initiate retraining of the machine leaning model. In other examples, the data processing system 104 may perform other actions upon determining that the machine learning model is not competent for use with the input data. For example, the data management system 104 may send a signal to the device 106, causing the device 106 to shut down or otherwise alter its mode of operation. This may be valuable if, for example, use of the machine learning model with input data for which the machine learning model is not competent could have costly or dangerous consequences. Additionally, or alternatively, the device 106 may generate an alert for a human user, for example to be transmitted to the client system 102. In some examples, input data may be out of range due to human error during maintenance or deployment of the device 106. For example, if the sensors 304 of the device 106 include a camera, a human error could be leaving a lens cap on the camera when deploying the camera. In a further example, input data may become out of range due to sensor degradation, or obstruction of the sensors 304 for example due to dirt. If the device operator is alerted that there may be a problem with the device 106, the device operator can manually check the device 106 and perform any necessary maintenance of the device 106 if necessary. In order to provide assistance to the user, the generated alert may include information indicative of the nature of the discrepancy between the input data and the training data, and may even include suggestions as to the cause of the discrepancy.
The above embodiments are to be understood as illustrative examples of the invention. Further embodiments of the invention are envisaged. For example, although in the examples above, the device management system determines properties for the new set of training data, in other examples the device itself may determine properties for the new set of training data, and send a request to the device management system to retrain the model using training data having those properties. Furthermore, the methods described herein are not limited to IoT applications, and may be used in any case where a machine learning model running on a device is trained remotely. In a further example, a computer program product may be provided comprising machine-readable instructions which, when executed by processing circuitry of a system or device, cause the system or device to implement the methods performed by the device management system 104 or device 106 as described above.
It is to be understood that any feature described in relation to any one embodiment may be used alone, or in combination with other features described, and may also be used in combination with one or more features of any other of the embodiments, or any combination of any other of the embodiments. Furthermore, equivalents and modifications not described above may also be employed without departing from the scope of the invention, which is defined in the accompanying claims.
