Patent Application
20200211721
The present disclosure relates to identifying devices connected in a network, and more particularly, to methods for determining an identity of an unknown Internet-of-Things (IoT) device in a communication network.
Internet-of-Things (IoT) is a term used to describe various aspects related to the extension of the
Internet into the physical realm, by means of widespread deployment of spatially distributed devices with embedded identification, sensing, and/or actuation capabilities. IoT is enabled by the growth of the Internet and network-enabled objects. Until relatively recently, the Internet was primarily used to connect users to each other, and also to available information. With the growth of these network-enabled objects, the Internet is increasingly used to connect people to these objects and also to connect objects to each other. Some real-world examples of such objects are refrigerators, air-conditioners, audio systems, security cameras, and many other everyday devices embedded with electronics that enable these devices to be connected to a communication network.
IoT has been experiencing rapid growth in recent years and is expected to continue to proliferate, becoming an integral part of everyday communications. Among the challenges that IoT poses to organizations are security issues stemming from the proliferation of such devices and the ever increasing number of IoT-enabled organizational assets. In some cases, due to the diversity and the inherent mobility of a large portion of these IoT devices, organizations may find it difficult to maintain an accurate record of the IoT devices connected to their networks at a given time. It would therefore be useful for tracking IoT devices connected to a network if unknown IoT devices that are connected to the network can be accurately identified.
To determine the identity of an unknown IoT device connected to a network, one method proposed looking at Media Access Control (MAC) addresses of devices that are connected to the network. The MAC address is uniquely assigned to a device when it is manufactured. The prefixes of MAC addresses can be used to identify the manufacturer of a particular device. However, no standard exists to identify brands or types of devices. Although, it is possible that manufacturers have their own ad hoc strategy to identify models that are produced by them, this must be reversed engineered for each manufacturer. Furthermore, the strategies might not be generalized to other manufacturers or newer models.
Thus, it is desirable to provide a method of determining an identity of an unknown IoT device in a communication network which addresses the problems of existing prior art and/or to provide the public with a useful choice.
Various aspects of the present disclosure are described here. It is intended that a general overview of the present disclosure is provided and this, by no means, delineate the scope of the invention.
According to a first aspect, there is provided a method of determining an identity of an unknown Internet-of-Things (IoT) device in a communication network. The method includes receiving network traffic generated by the unknown IoT device, extracting device network behavior from the generated network traffic, and determining the identity of the unknown IoT device from a list of known IoT devices by applying a selected machine learning based classifier from a set of machine learning based classifiers to analyze the device network behavior. Each machine learning based classifier of the set is trained by a dataset including a plurality of features representing network behavior of a respective known IoT device from the list and the known IoT device's identity. The plurality of features is associated with the corresponding device network behavior of the generated network traffic.
The network traffic may include a number of communication sessions having respective unlabeled feature vectors representing the device network behavior of the unknown IoT device. Each machine learning based classifier of the set may include a single session classifier associated with a respective known IoT device in the list. The single session classifier outputs a probability. Each machine learning based classifier of the set may include a classification threshold for comparing with the probability to determine if the session being analyzed is generated by a particular device in the known IoT device list. Each machine learning based classifier of the set may include a session sequence size which defines the number of communication sessions to analyze.
Analyzing the device network behaviour may include (i) analyzing the unlabeled feature vector of one of the communication sessions using the single session classifier of the selected machine learning based classifier to output the probability, (ii) comparing the probability with the classification threshold, and (iii) if the probability is higher than the classification threshold, (iv) classifying the communication session as being generated by a particular IoT device from the known IoT device list associated with the single session classifier, and (v) determining the identity of the unknown IoT device from the classification.
The method may further include selecting a next machine learning based classifier in the set if the probability is not higher than the classification threshold, using the single session classifier of the next selected machine learning based classifier to analyze the unlabeled feature vector and repeating steps (ii) to (v).
Alternatively, analyzing the device network behaviour may include (i) analyzing unlabeled feature vectors of consecutive communication sessions using the single session classifier of the selected machine learning based classifier to output corresponding probabilities, (ii) comparing each of the probabilities with the respective classification thresholds, (iii) if any of the probabilities are higher than the respective classification thresholds, (iv) classifying those communication sessions as being generated by a particular device from the known IoT device list associated with the single session classifier, and (v) determining the identity of the unknown IoT device based on the classification.
The method may further include selecting a next machine learning based classifier in the set if a majority of the probabilities is not higher than the respective classification thresholds, selecting a next machine learning based classifier in the set and using the single session classifier of the next selected machine learning based classifier to analyze the unlabeled feature vectors and repeating steps (ii) to (v).
The method may further include selecting the machine learning based classifier from the set in sequence starting from the machine learning based classifier having the lowest session sequence size to the highest session sequence size for analyzing the unlabeled feature vectors of the consecutive communication sessions.
The identity of each of the known IoT devices may include the device's make and model.
According to a second aspect, there is provided a method of creating a training dataset for a machine learning based classifier to be used for determining an identity of an unknown device in a communication network. The method includes generating network traffic from a plurality of IoT devices with known identities, extracting a plurality of features from the network traffic which are relevant to represent network behaviour of each one of the plurality of IoT devices, associating the extracted plurality of features with the corresponding identity of each one of the plurality of IoT devices, and creating the training dataset based on the association.
The method may further include converting the network traffic into communication sessions and extracting the plurality of features from each communication session.
The plurality of features may be extracted from network, transport and application layers of the network.
According to a third aspect, there is provided an apparatus for determining an identity of an unknown Internet-of-Things (IoT) device in a communication network. The apparatus is arranged to receive network traffic generated by the unknown IoT device. The apparatus includes a network feature extractor arranged to extract device network behaviour from the generated network traffic. The apparatus also includes a processor arranged to determine the identity of the unknown IoT device from a list of known IoT devices by applying a selected machine learning based classifier from a set of machine learning based classifiers to analyze the device network behaviour. Each machine learning based classifier of the set is trained by a dataset including a plurality of features representing network behaviour of a respective known IoT device from the list and the known IoT device's identity. The plurality of features is associated with the corresponding device network behaviour of the generated network traffic.
The apparatus may form part of a communication network which also includes a plurality of IoT devices which forms a fourth aspect.
An exemplary embodiment will now be described with reference to the accompanying drawings in which:
One or more embodiments of the present disclosure will now be described with reference to the figures. The use of the term “an embodiment” in various parts of the specification does not necessarily refer to the same embodiment. Features described in one embodiment may not be present in other embodiments, nor should they be understood as being precluded from other embodiments merely from the absence of the features from those embodiments. Various features described may be present in some embodiments and not in others.
Additionally, figures are there to aid in the description of the particular embodiments. The following description contains specific examples for illustration. The person skilled in the art would appreciate that variations and alterations to the specific examples are possible and within the scope of the present disclosure. The figures and the following description should not take away from the generality of the preceding summary.
In the present embodiment, machine learning techniques are applied to network traffic data obtained from a list of known IoT devices in order to train a set of classifiers to accurately determine, from the list of known IoT devices, the identity of unknown IoT devices that are connected to a network by analyzing the network behaviour of the unknown IoT devices.
Additionally, since non-IoT devices are often also connected to the network, the present disclosure also distinguishes non-IoT devices from IoT devices by determining the identity of the non-IoT devices connected to the network. Therefore, in a broader aspect, the described embodiment is able to determine the identity of network-enabled devices connected to the network.
Network-enabled devices may include IoT and non-IoT devices. As opposed to non-IoT devices such as PCs, laptops, tablets and smartphones, IoT devices are typically resource-constrained task-oriented previously-unconnected appliances, fortified with various sensors and actuators. These IoT devices are designed to facilitate the automation and efficiency of numerous daily processes in virtually every aspect of modern life, such as home automation, manufacturing, healthcare, transit, and so forth. For instance, smart sockets are an example of IoT devices, as they have very limited computing power (in terms of CPU, memory, etc.), they support a specific predefined task (i.e., enable remote connection/disconnection of power, monitor power consumption) and they facilitate the automation of power saving.
In a preferred embodiment, there is provided a method of determining the identity of an unknown network-enabled device from a list of known network-enabled devices by applying a selected machine learning based classifier from a set of machine learning based classifiers to analyze the device network behaviour. Each machine learning based classifier of the set is trained by a dataset which includes a plurality of features representing network behaviour of a respective known network-enabled device from the list and the known device's identity. The plurality of features is associated with the corresponding device network behaviour of the generated network traffic.
To elaborate further, the description of the preferred embodiment is divided into two parts—the first part discusses how a set of classifiers can be trained using machine learning techniques to determine the identity of network-enabled devices from a list of known network-enabled devices, and the second part discusses how the trained machine learning based classifier determines the identity of unknown network-enabled devices communicating in a network.
To train the set of classifiers, a training data set is first created from network traffic data of known network-enabled devices. The network traffic data is collected as such.
As mentioned, the network-enabled devices 102 may be IoT devices 103 or non-IoT devices 104. Table 1 provides an exemplary list of network-enabled devices 102 including their “make and model” and the number of TCP sessions collected for each device. The devices are indicative of devices that are commonly connected to a system's wireless network.
At step 210, the network feature extractor tool 123 reconstructs *.pcap files containing TCP packets 201 to TCP sessions 211. Each TCP packet 201 is converted to a TCP session 211. Each TCP session 211 comprises unique 4-tuples consisting of source and destination IP addresses and port numbers, from the point of requesting a connection (SYN flag) to the end of the requested connection (FIN flag).
At step 220, features 221 are extracted from each TCP session 211. Features 221 represent unique properties of the TCP session 211 which defines the behaviour of the TCP session 211 in the network traffic. In the present embodiment, the data is extracted from the network, transport, and application layers of each TCP session 211.
In some embodiments, the features 221 extracted from the TCP may include destination port, packet sizes, number of packets with PUSH bit set, and average duration of a handshake.
The method 200 also uses third party information gathered from publicly available external databases. In the present embodiment, third party information from Alexa Rank and Geo IP are used. At step 230, behavioral features 231 from across different protocols and network layers of the third party information are added to respective features 221 extracted from each TCP session 211. Each TCP session 211 is characterized by a feature vector 232 comprising of features from both the TCP session 211 and corresponding third party information gathered from Alexa Rank and GeoIP.
It has been found that some features are regarded to be more valuable for modeling of the device behaviour. The following table illustrates the top 40 features which are regarded as being more valuable.
At step 240 of
Each device 102 is therefore represented by a set of labeled feature vectors 232 in the training dataset 241. The number of labeled feature vectors 232 representing each device 102 depends on the number of TCP sessions 211 recorded for the device 102.
The device identification model is a set of machine learning based classifiers. The proposed method of
The following notations are used in the embodiments of the present disclosure.
In some embodiments, the test set DStest may be omitted and a labeled/training dataset 241 may be divided chronologically into two mutually exclusive sets consisting of a single-session training set DSs and a multi-session training set DSm. In other words, there will not be a final stage for evaluating the performance of the multi-session classifier.
At step 410, a single-session classifier Ci is induced for each device di in the set of known devices D. D represents the set of known devices to be identified based on their network traffic. A set of single-session classifier C is obtained using the single-session training set DSs. To train Ci for device di, DSs is transformed into a binary dataset such that all labeled feature vectors of sessions that belong to di are labeled as di, and labeled feature vectors of sessions that do not belong to di is labeled as “other”. Thus, given a feature vector (hereinafter referred to as unlabeled feature vector) extracted from a session that emanated from an unknown device, each single session classifier Ci is applied to the unlabeled feature vector to obtain a vector of posterior probabilities (p1s, . . . , pns).
At step 420, the optimal classification threshold (cut-off value) tri* for labeling a given session s with probability pis as di or “other” is determined. The multi-session dataset DSm is used to evaluate the performance of the set of single session classifiers C, and for setting the optimal threshold values tri*. Each optimal threshold tri* was selected such that the accuracy of each single-session classifier Ci is optimized for identifying device di.
At step 430, the optimal session sequence size si* for each single-session classifier Ci is determined. The optimal session sequence size si* is obtained as such. First, for each device di represented in the multi-session training set DSm, the set of single-session classifiers C is applied to all labeled feature vectors to obtain the classification results. Then, the classification results of each optimized classifier is analyzed using the optimal classification threshold tri* and multi-session dataset DSm. The optimal session sequence size si* is then the minimal number of consecutive session classifications whereby a majority vote will provide zero false positives and zero false negatives on the entire DSm.
Table 2 is an exemplary performance (i.e. False Negative Rate and False Positive Rate) of the single-session classifiers in determining identity of IoT devices after being optimized with tri* and their optimal si*.
From Table 2, it is shown that some devices (e.g. security camera, socket, refrigerator) require lower optimal session sequence size si* for an accurate identification. From a macro point of view, the network behaviour of different network-enabled devices 102 varies according to the device. Some devices (e.g. security cameras) generate network traffic that is more ‘recognizable’ than the network traffic generated by other devices (e.g. thermostat). Since the network traffic is captured in the feature vectors of each device as described in
Algorithm 1 illustrates how the program calculates si* for each device di.
The multi-session classifier therefore comprises single-session classifiers Ci, and the corresponding optimal threshold values tri* and optimal session sequence size si*. For every device di there is a classifier Ci with an optimal classification threshold tri, and if a majority voting on its si* consecutive classifications is performed, the result of the majority voting determines whether sessions that emanated from a given IP were generated by di with 100% accuracy.
Device Identification Using the Trained Classifier
Given a stream of unlabeled feature vectors that emanated from an IP and generated by an unknown network-enabled device 150 in the communication network 100 of
At step 510, the set of single-session classifiers Ci is sorted according to ascending si* values.
At step 520, the stream of unlabeled feature vectors is applied to a single-session classifier Ci corresponding to device di with the lowest si* value. The single-session classifier Ci classifies si* consecutive sessions of the unlabeled feature vectors to be originating from device di or not.
At step 530, determine whether a majority of the si* sessions were classified as device di. If the answer is yes, then at step 540, establish the identity of the unknown device 150 that originated the stream of sessions to be device di. If the answer is no, then steps 520 and 530 are repeated for the next single-session classifier with the next lowest si* value.
The device inspection order is organized by ascending si* values so that the algorithm starts to inspect devices with the lowest si* value first and follows through with increasing i* values. The search for the identity of the unknown network-enabled device 150 can be optimized in this manner.
Another way to optimize the search algorithm is to take into account the prior probability of a device being observed. In practice, this means sorting the set of classifiers by descending order of prior probabilities. For example, if a smartwatch is more probable to connect to the network than a smart refrigerator, then the classifier that determines whether the stream originated from a smartwatch would be applied before the smart refrigerators classifier.
Algorithm 2 illustrates the program for device classification.
At step 610, the TCP packets 651 originating from the unknown network-enabled device 150 are first converted to corresponding TCP sessions 652. This is achieved in the same manner as how the TCP packets 201 of the known network-enabled devices 102 are converted into TCP sessions 211 in step 210.
At step 620, classification of smartphones is performed on a TCP session by analyzing the “user agent” property string that is found in HTTP packets. The analysis has a 100% accuracy for identifying smartphones. If the unknown network-enabled device 150 is identified as a smartphone, the process 600 is completed. If the unknown network-enabled device 150 is not identified as a smartphone, then the process 600 continues to step 630.
At step 630, the TCP sessions 652 are then converted to corresponding unlabeled feature vectors 653 in the same way that the features 221 are extracted from TCP sessions 211 and formed into feature vectors 232 in step 220 and 230. However, in process 600, no third party information is added to the TCP sessions 652.
At step 640, a single session (or corresponding unlabeled feature vector) is classified using a single-session classifier. The accuracy for determining that a session originated from a PC based on a single classification of the session is found to be good. If the unknown network-enabled device 150 is identified as a PC, then the process 600 is completed. If the unknown network-enabled device 150 is not identified as a PC, then the process 600 continues to step 650.
At step 650, the device identification process 500 illustrated in
The exemplary process 600 therefore determines the identity of non-IoT devices 104 (i.e. smartphones and PCs) first before using the device identification process 500 to determine the identity of the IoT devices 103. By sieving out non-IoT devices 104 such as smartphones and PCs first, the exemplary process 600 reduces the number of unknown network-enabled devices' identity to be determined. In a communication network, where the majority of network traffic may be generated by non-IoT devices 104 such as smartphones and PCs, the difference can be significant. The exemplary process 600 is therefore more efficient in determining the identity of IoT devices 103 in such a network.
At step 710, the computer system 120 receives network traffic 711, in the form of TCP packets, generated by the unknown IoT device 150a.
At step 720, the device network behaviour 721 of the unknown IoT device 150a is extracted from the network traffic 711. The extraction is performed in the same manner as the extraction of features 221 from known devices 102 described in step 210 of method 200. Therefore, TCP packets originating from the network traffic 711 of the unknown IoT device 150a is first converted to corresponding TCP sessions. Features from each TCP session are extracted using the network feature extractor tool 123 of the computer system 120 and arranged in corresponding unlabeled feature vectors. Each TCP session is therefore characterized by an unlabeled feature vector comprising features extracted from the network traffic of the unknown IoT device 150a. The end product of step 720 is a set of unlabeled feature vectors representing the device network behaviour 721 of the unknown IoT device 150a.
At step 730, a selected machine learning based classifier 731a from a set of machine learning based classifiers 731 is applied to the set of unlabeled feature vectors to analyze the device network behaviour 721. The analysis is performed utilizing the device identification process described in
At step 740, the identity of the unknown IoT device is determined from the list of known IoT devices 103 based on results of the analysis in step 730.
Evaluation
The device identification process 600 is evaluated for its performance characteristics using the test set DStest that was partitioned out in
The performance of the device identification process 600 for classifying whether a device is IoT or non-IoT (i.e., smartphone or PC) is presented in Table 3. Using the device identification process 600, classification accuracy for smartphones is 100% while the classification of PCs is almost perfect. Therefore, the identity of unknown non-IoT devices can be determined quickly and with near perfect accuracy.
Having accurately classified the non-IoT devices (i.e., smartphones and PCs), Algorithm 2 is applied on DStest set for evaluating the performance for IoT device classification. Since Algorithm 2 is optimized to derive the type of an IoT device by analyzing a minimal number of consecutive sessions, in a worst case scenario it needs to analyze maximum (si*) consecutive sessions. In order to properly evaluate the performance of process 600, Algorithm 2 is rerun multiple times with each time omitting the first session of the sequence from the previous run. This is performed to compensate for a possible bias that may occur when the sequence begins with different sessions. Given the test set DStest in chronological order, used for evaluating the process 600, let DSitest be a subset of sessions in DStest originated from di, and let DSitest[a] be the ath session originated from di in DSitest. For each device di ϵ D (i.e. the set of known network-enabled devices 102), the evaluation is repeated by applying Algorithm 2 (i.e. the device identification process of
It is determined from Table 4 that the accuracy of Algorithm 2 in determining the identity of devices on DStest is high.
Algorithm 1 is then executed once again, this time on DStest. The si* value previously obtained from DSm is compared to the si* value obtained from DStest after executing Algorithm 1. Classification accuracy measures on DStest and the recalculated si* value is shown in Table 5.
In conclusion, to obtain better results for all devices in DStest, an si* which is 4.333 times higher than the ones that are computed by Algorithm 1 on DSm is preferable.
Although the present disclosure has been described with reference to specific exemplary embodiments, various modifications may be made to the embodiments without departing from the scope of the invention as laid out in the claims. For example, various methods and processes described may be operated on any computer systems with the proper software tools to execute the instructions. Features may be extracted from the TCP sessions using any feature extraction tool that is readily available. Furthermore, network traffic need not be TCP packets only. Other protocols from a different layer of the network traffic may be utilized as long as it embodies network behaviour of a device. For example, HTTP, DNS and SSL protocols on the transaction level can be recorded. Consequently, features from different protocols and levels of the network traffic may be extracted for use to represent device network behaviour.
Algorithms 1 and 2 are provided for illustrating exemplary methods and steps. The exemplary methods and processes may be executed using other computing languages that are known to the skilled person and can be readily achieved by the skilled person.
Furthermore, exemplary process 700 may be expanded to include identifying other non-IoT devices such as laptops, and tablets.
Various embodiments as discussed above may be practiced with steps in a different order as disclosed in the description and illustrated in the Figures. Modifications and alternative constructions apparent to the skilled person are understood to be within the scope of the disclosure.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10201701692Y | Mar 2017 | SG | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/SG2018/050089 | 2/27/2018 | WO | 00 |
