Systems and methods for predicting the likelihood of cyber-threats leveraging intelligence associated with hacker communities

Abstract

Various embodiments of a system and methods for reasoning about enterprise-related external cyber threats using a rule-leaning approach are disclosed.

Description

FIELD

The present disclosure generally relates to cyber threat prediction; and more particularly involves a rule-based approach for generating actionable warnings of cyber threats that support timely situational awareness operations, efficient resource allocations, and proactive countermeasure prioritizations.

BACKGROUND

With the widespread use of technology, cyber-security has become an important issue of concern for both commercial organizations and governments. With the recent incidents of data breaches at Equifax, Verizon, Gmail and others, organizations are looking at methods to proactively identify if they will be the target of future attacks. A 2017 Verizon investigation report stated that 75% of breaches were perpetrated by outsiders exploiting known vulnerabilities. Monitoring the activity of threat actors is a key aspect of predicting cyber-attacks.

Cyber threats also present major issues for cryptocurrency. Cryptocurrencies are digital currencies that primarily use the blockchain concept to record transactions. Perhaps the most well-known one is Bitcoin. In recent years, an increasing adoption of cryptocurrencies has been observed in a wide range of businesses as well as in the malicious actor community in the (D2web) forums. According to recent reports, the market capitalization of cryptocurrencies is estimated to exceed 400 billion dollars, after peaking at over 700 billion dollars. With the high reliance on technology, increasing adoption from businesses and traders, and due to the inherent anonymity associated with transactions and wallet owners, malicious threat actors (including hackers and scammers) aiming for financial gain have been highly motivated to hack and scam to gain control over cryptocurrency wallets and perform transactions.

The fast-evolving nature of cyberattacks, as well as the high direct and indirect cost of remediation, calls for organizations to seek proactive defense measures. Hackers may communicate prior to attacks using D2web hacking websites and it is generally desirable to formulate cyber threat predictions based on hacker communications. However, generating transparent and explainable predictions that allow human experts to understand the reasoning leading to such predictions is yet challenging.

Further, although cybersecurity research has demonstrated that many of the recent cyberattacks targeting real-world organizations could have been avoided, proactively identifying and systematically understanding when and why those events are likely to occur is challenging. Information associated with the D2web is constantly changing, and conventional methods of predicting cyber threats are too reliant upon specific data points associated with hacker communications.

It is with these observations in mind, among others, that various aspects of the present disclosure were conceived and developed.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a simplified block diagram showing a computer-implemented system for rule-based cyber threat prediction;

FIG. 1B is a simplified illustration of exemplary modules or components of a framework for implementing the rule-based cyber threat prediction;

FIG. 2 is a simplified illustration showing a timeline of observations and actions;

FIG. 3 is an illustration showing a process flow for APT logic rule-based cyber threat prediction;

FIG. 4 is an illustration depicting a scenario for past and future cyber incidents;

FIG. 5 is an image that shows a thread that considers the scenario shown in FIG. 4, including past and future incidents;

FIG. 6 is a graphical representation of the time taken to generate rules when varying Δt;

FIG. 7 is a set of graphs illustrating the precision-recall curve discussed herein;

FIG. 8 is an image of an exemplary word cloud; and

FIG. 9 is an example schematic diagram of a computing device that may implement various methodologies of the proposed framework.

Corresponding reference characters indicate corresponding elements among the view of the drawings. The headings used in the figures do not limit the scope of the claims.

DETAILED DESCRIPTION

Aspects of the present disclosure relate to a computer-implemented system and associated methods for predicting cyber threats using rule-based approaches. More specifically, at least a learning module and a prediction module may be executed by a computing device. The learning module may be configured, via machine learning or otherwise, to learn rules correlating indicators of cyber threats with real-world attack events, and the computing device may execute the prediction module to apply the rules from the learning module to real-world/additional data to generate warnings regarding possible impending cyber threats. In some embodiments, the learning module may include a logical programing framework defined as annotated probabilistic temporal logic (APT logic) that correlates hacker activity to real-world cyber events previously observed. The learning module may further incorporate other learning approaches, such as application of knowledge representation and reasoning (KRR).

Introduction:

To conduct a cyber-attack, malicious actors need to identify vulnerabilities, get resources to exploit those vulnerabilities, identify vulnerable targets and then successfully deploy their attacks. Darkweb (D2web) forums and marketplaces can provide an environment for analysis of anonymous discussions related to software vulnerabilities and purchase/sale of exploits available for the same. A cyber threat intelligence firm that collects and maintains threat intelligence recently reported intelligence on cyber-attacks before such attacks were actually carried out by malicious actors. Two such instances are shown in Table 1 below. For example, the Microsoft office vulnerability (CVE-2017-11882) was disclosed by the National Institute of Standards and Technology (NIST) on July 31st with no public knowledge of an available exploit. The interest in the vulnerability and exploit peaked in multiple D2web forums between 20-23rd of November as identified by the firm, three days before an actual attack was observed in the wild on November 26^th.

TABLE 1
Attacks predicted
Attack                     Timeline (2017)
WannaCry Ransomware        Discussions on D2web: April 14. Attack
                           observed: May 12.
Cobalt targeting Microsoft Disclosure by NIST: July 31. Hacker
Office vulnerability CVE-  discussion on D2web: Nov 20-23.
2017-11882                 Attack Observed: Nov 26.

Cyber Threat Related Terms:

A number of terms shall now be described which may be used to define cyber threat parameters and forms of cyber threat information. For example, the term CVE may be used to abbreviate the phrase, Common Vulnerability Exposure representing a potential vulnerability that a hacker or other bad actor may desire to exploit. A CVE may define a unique identifier assigned to a software vulnerability report from NIST which maintains a database of all the vulnerabilities publicly available in a National Vulnerability Database (NVD). The term CPE may be used to abbreviate the phrase, Common Platform Enumeration, and may relate to a list of specific software and hardware products that are vulnerable for a given CVE.

The “darkweb” or “deep web” (collectively “D2web”) refers to the portion of the Internet not indexed by search engines and hence generally cannot be accessed by standard web browsers. Specialized browsers, such as Tor, are required to access these websites. Widely used for underground communication, “The Onion Router” (Tor) is free software dedicated to protect the privacy of its users by obscuring traffic analysis. The network traffic in Tor is guided through a number of volunteer-operated servers (also called “nodes”). Each node of the network encrypts the information it blindly passes on neither registering where the traffic came from nor where it is headed, thereby preventing any tracking of the traffic.

Hackers may utilize the D2web to form communities for identifying technology exploits and sharing information related to cyber threats. Hacker community discussions, general communications, or other such information may include Information retrieved from both marketplaces—where users advertise to sell information regarding vulnerabilities or exploits targeting the vulnerabilities and forums—where hackers or other bad actors engage in discussions on discovered vulnerabilities, among others.

EXEMPLARY EMBODIMENTS

Referring to FIG. 1A, a computer-implemented system 100 is shown which may be utilized for implementing various embodiments of a rule-based cyber threat prediction framework (“framework”) 101 described herein. Embodiments of the framework 101 are referred to herein as Embodiment A, B, C and D. Each of the embodiments may generally be implemented using the components of system 100 described herein, yet each of the embodiments may include slightly different approaches for rule-learning and generating cyber threat warnings. For example, Embodiment A applies annotated probabilistic temporal logic to learn rules for predicting cyber threats, whereas Embodiment D utilizes knowledge representation and reasoning (KRR); i.e., KRR logic may be applied as described herein. It is contemplated that each features of each embodiment may be utilized by other embodiments, such that e.g., the features of Embodiment A may be utilized by other embodiments, and Embodiment A may incorporate features of other embodiments, etc.

As indicated, the framework 100 may include a plurality of components or modules executed or otherwise implemented using at least one computing device 102 (equipped with at least one or more of the features of the computing device 200 of FIG. 9). For example, the computing device 102 may be configured to execute or otherwise perform operations associated with a data extraction and pre-processing module 104, a learning module 106, a prediction module 108, and a consolidation module 110 defined by the framework 101. In some embodiments, one or more of the modules 104, 106, 108, and 110, may be implemented as code and/or machine-executable instructions executable by the computing device 102 that may represent one or more of a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements, and the like. In other words, embodiments of the framework 101 described herein may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware or microcode, the program code or code segments to perform the necessary tasks (e.g., a computer-program product) may be stored in a computer-readable or machine-readable medium, and a processor(s) associated with the computing device 102 may perform the tasks defined by the code.

As further shown, the system 100 may further include at least one device 112 in operable communication with the computing device 102. In some embodiments, the computing device 102 may access or be in operable communication with the device 112 to obtain cyber threat data 114 from the dark web or deep web 116 for rule-based learning as described herein. The device 210 may include any electronic device capable of accessing/tracking the cyber threat data 114 from the dark web or deep web 116. In addition, the system 100 may include a client application 120 which may be configured to provide aspects of the framework 101 to any number of client devices 122 via a network 124, such as the Internet, a local area network, a wide area network, a cloud environment, and the like.

Referring to FIG. 1B, additional detail regarding the flow of data and implementation of the framework 101 is illustrated. As shown, in some embodiments, an application programming interface (API) referred to herein as a D2Web API 130, may be executed by any computing device (e.g., computing device 102 as part of the data extraction and pre-processing module 104) to obtain a first dataset from hacker community discussions 132. Hacker community discussions 132 or hacker communications may be obtained from any number of forums, marketplaces or other platforms of the deep/dark web.

As further, shown, information from the hacker community discussions 132 may generally be applied to the learning module 106 along with ground truth data 134 to derive a plurality of rules 136, using one or more rule-based approaches (such as APT logic), by correlating indicators of cyber threats with real-world attack events. The indicators of threats may be annotated from a collection of communications of the hacker community discussions 132 while the real-world attack events may be annotated from a collection of cyberattack attempts observed by a Data Provider or other data source providing the ground truth data 134. In some embodiments, the framework 101 may use a single indicator-extracting approach. That is, indicators may include mapping mentions of software vulnerabilities (CVEs) to affected software vendors and product names (CPEs). These indicators may be annotated with dates as to when the corresponding vulnerabilities are mentioned, then used as preconditions in the rule-learning approach. It is contemplated that any number of threat-intelligence platforms may be leveraged for generating the rules 136, and that extracting indicators may involve capturing aggregate discussion trends. In another embodiment, the output of the learning module 106 may be an APT-logic program i.e., a set of APT rules.

Other embodiments of indicators are contemplated to accommodate the changing volume of cyber threat intelligence data. For example, many applications of event forecasting, the volume of signals from the monitored sensors is assumed to remain the same across the learning and the forecasting phases. However, this assurance does not exist with cyber threat intelligence data. A key driver to this is the ephemeral nature of many D2web sites. This ephemeral nature is due to a variety of reasons: law enforcement actions, malicious hackers going “dark”, operational security measures employed by cyber criminals, and differences induced by the addition of newer data sources. Changes to the volume of incoming cyber threat intelligence data may directly impact the number of warnings, thus impacting the system's performance. Accordingly, the indicators that are evaluated may be based on volume of discussion trends exceeding a threshold computed from a sliding time window.

Once the rules 136 are generated, the framework 101 may then utilize the rules 136 to form real-world predictions regarding possible cyber threats/attacks based on additional or new communications extracted from the hacker community discussions 132. In other words, the prediction module 108 may use the output of the learning module 106, e.g., the APT-logic program, and the indicators annotated from the communications accessed from the hacker community discussions 132. The prediction module 108 may trigger one or more of the rules 136 if some indicators are observed matching preconditions of the rules 136 in the APT-logic program or otherwise. If a match is identified between an indicator (present within new communications extracted from the hacker community discussions 132) and a rule, the prediction module 108 may generate a warning with metadata including the corresponding indicators and hacking discussions. In other words, a second dataset may be accessed from the hacker community discussions 132 using the D2Web API 130 or otherwise, and the prediction module 108 may apply information from the second dataset to the rules 136 to generate warnings 140 with any number of predictions about the possible cyber threats/attacks. These warnings 140 may be transmitted to or otherwise made available to other devices/organizations, such as a security operations (OPS) center 142.

Further detail regarding the framework 101 and embodiments thereof shall now be described.

Functionality of Data Extraction and Pro-Processing Module (104) for Embodiments A and B

In some embodiments, the data extraction and pre-processing module 104 may configure the computing device 102 to crawl the dark/deep web to collect the hacker community discussions 132. Data from the hacker community discussions 132 may be accessed using the D2Web API 130, or otherwise accessed and a database (not shown) may be formed comprising forum discussions and items offered for sale in marketplaces in the D2web. Light weight crawlers and parsers may be leveraged t constructed to collect hacker community discussions 132 data from forums and marketplaces. To accommodate collection of cyber-security relevant data, machine learning models may be employed to filter data of the hacker community discussions 132 related to drugs, weapons, and other discussions not relating to cyber-security.

The D2web API 103 may be configured to supply tags with each post using a tagging algorithm. Each tag may belong to one of three categories: financial, software, or general topic tags. The tagging algorithm may be used to post content. It may leverage known document similarity techniques to derive a vector representation of each post, compute its similarity with vector representations of a set of tags, and identify the tags that are most similar to that post based on the similarity score. The data extraction and pre-processing module 104 may use the count of tag mentions per day, i.e., when it exceeds certain threshold, it is viewed as a spiking activity. This threshold may be tag-dependent; meaning each tag has a threshold, determined based on the average and the standard deviation of its mentions per day in the historical data.

Ground Truth for Embodiments A and B (or Other Embodiments)

To construct rules and evaluate the performance of the learned model, real-world attack data may be provided by the IARPA CAUSE program. This real-world attack data, or historical known attack data, may function as the ground truth (GT) data 134 for the learning module 106 of Embodiments A and B or other embodiments. The GT data 134 may include time series data points of real-world attacks gathered from two participating hypothetical organizations. One of the two organizations may include a defense industrial base (referred to as Armstrong) while the other may define a financial services organization (referred to as Dexter). Each data point is a record of a detected deliberate malicious attempt to gain unauthorized access, alter or destroy data, or interrupt services or resources in the environment of the participating organization. Those malicious attempts may relate to real-world events detected in the wild, in uncontrolled environment, and by different security products such as anti-virus, intrusion detection systems, and hardware controls. Each GT data 134 record includes: ID, Format Version, Reported Time, Occurrence Time, Event Type, and Target Industry. The attack types that are included in the GT data 134 include:

• • Malicious Email. A malicious attempt is identified as a Malicious Email event if an email is received by the organization, and it either contains a malicious email attachment, or a link (embedded URL or IP address) to a known malicious destination.
  • Malicious Destination. A malicious attempt is identified as a visit to a Malicious Destination if the visited URL or IP address hosts malicious content.
  • Endpoint Malware. A Malware on Endpoint event is identified if malware is discovered on an endpoint device. This includes, but is not limited to, ransomware, spyware, and adware.

Table 2 summarizes the time periods and the number of records for each attack type for the participating organizations.

	TABLE 2
	Sensor Dates
GT Provider	Attack Type	Start	End	Gaps	Total No. of Records
Dexter	Malicious Email	Apr. 4, 2016	Jul. 28, 2016	N/A	114
	Endpoint Malware	Apr. 4, 2016	Jul. 28, 2016	N/A	35
	Malicious Destination	Mar. 15, 2016	Apr. 4, 2016	N/A	88
Armstrong	Malicious Email	Oct. 3, 2016	Sep. 29, 2017	N/A	147
	Endpoint Malware	Apr. 4, 2016	Sep. 29, 2017	Sep. 27, 2016	141
				to
				Jun. 4, 2017
	Malicious Destination	Apr. 4, 2016	Sep. 29, 2017	Aug. 22, 2016	32
				to
				Jun. 4, 2017

Functionality of Data Extraction and Pre-Processing Module (104) for Embodiment C

In this embodiment a wider variety of cyber threat intelligence sources may be leveraged or accessed using the same D2Web API 130 from sources spanning hacker communities around the globe, including environments such as Chan sites, social media, paste sites, grey-hat communities, Tor (dark-web), surface web, and even highly access-restricted sites (deepweb). This includes over 400 platforms and over 18 languages. Non-English postings may be translated to English using various language translation services. In addition, crawling uses customized lightweight crawlers and parsers for each site to collect and extract data. To ensure collection of cybersecurity-relevant data, machine learning models may be used to retain discussions related to cybersecurity and filter out or remove irrelevant data.

Ground Truth: In this embodiment, the ground truth includes a collection of historical records of malicious emails originated from sources that are outside of a Data Providers network. An email is considered malicious if it either has a piece of malware in its attachments, or a link (URL or IP address) to a destination serving malicious content, e.g., malware or phishing. FIG. 2 shows a month-wise distribution of malicious emails observed by Data Provider from January-October, 2019, the last data update from Data Provider. The Data Provider records include events generated by detectors of other attack types such as malicious-destination and endpoint-malware. However, the system has primarily been tested on malicious-email events as the other event types are observed with significantly lower frequency, approaching 0 in some months.

Extracting Indicators of Cyber threat: In this embodiment, two approaches may be used to extract indicators of threats: (1) annotating software vendor and product names corresponding with the software vulnerabilities mentioned in hacker discussions, and (2) annotating spikes in the volume of entity tags identified from the context of those discussions.

Functionality of Data Extraction and Pre-Processing Module (104) for Embodiment D

In this embodiment any commercially-available API may be used to supply multi-sourced threat intelligence from sources spanning hacker communities around the globe, including environments such as Chan sites, social media, paste sites, grey-hat communities, Tor, surface web, and even highly access-restricted sites. This includes over 400 platforms and over 18 languages. Non-English postings may be translated to English using the Google translation API. For testing purposes the data may be focused from Jan. 1-Oct. 31, 2018.

Extracting Hacker Activity: The threat intelligence sources used may supply a vast amount of textual content over time. A commercial natural language processing API such as TextRazor, which leverages a wide-range of machine learning techniques (including Recurrent Neural Networks) to recognize entities from the context of postings, may be utilized. Each extracted entity may be associated with a confidence score quantifying the amount of confidence in the annotation. A lower bound may be set on confidence score to only retain entities that are relevant. This approach may use the extracted entities (single entities and itemsets) to learn temporal rules correlating hacker activity with attacks targeting the Data Provider.

Ground Truth: In this embodiment, the ground truth may be a collection of historical records of malicious emails received by the Data Provider from outside sources. An email may be considered malicious if it either has a piece of malware in its attachments, or if it has a link (URL or IP address) to a destination serving malicious content, e.g., malware or phishing.

Possible Implementations of the Learning Module (106)

In some embodiments, the learning module 106 may involve CVE-CPE mapping. In this embodiment, any database storing or having access to the hacker community discussions 132 may be queried using API calls associated with the D2Web API 130 to access forum discussions or marketplace items with software vulnerability mentions (in terms of a CVE number). To identify a CVE number, regular expressions may be used to match the CVE number format. After identifying any such vulnerabilities using the CVE number, corresponding targeted platform products (CPEs) may be identified, including software and/or hardware. In this manner, each of the identified CVEs may be mapped to its respective CPEs. CVEs may also be mapped to nation-state threat actors who are known to frequently exploit a given vulnerability. For instance, a well-known threat actor like the North Korean group Hidden Cobra is known for affecting multiple vulnerabilities in various applications. A list of such threat actors may be created and define what vulnerabilities the threat actors exploited in order to map any identified CVE to these actors. These CPE and nation-state actor mappings may be used as pre-conditions during the rule-learning phase of the learning module 106 discussed herein.

In some embodiments, the learning module 106 may be configured to generate the rules 136 in the form of APT rules using annotated probabilistic temporal (APT) logic known. The rule learning may take software vulnerability discussions gathered from the D2web (both from forums and marketplaces) and then may map them to a CPE and known nation-state actors. These mappings may then be used to construct rules by using actual attack data (GT 134) made available through the IARPA CAUSE program or otherwise. APT logic is described in further detail below. An example of one embodiment of a framework 150 similar to the framework 101 is shown in FIG. 3. In this embodiment, an APT learning module 152 may be employed to learn APT rules which may be leveraged by an APT prediction module 154 to formulate predictions or warnings based on the APT rules.

In logic, the syntax of the representation language specifies all the sentences that are well formed in the knowledge base, while the semantics define the truth of each sentence with respect to each possible world. The syntax and semantics of APT-logic programs applied to the domain shall now be described.

Annotated Probabilistic Temporal Logic (APT-LOGIC—Syntax)

The existence of a first order logical language , with a finite set _cons of constant symbols (which stand for objects), a finite set _pred of predicate symbols (which stand for relations), and an infinite set _var of variables may be assumed. The allowable sentences of may be defined as set forth below.

Conventions. Constant and predicate symbols may begin with lowercase letters. For example, the constant symbols set_forums_1, debian, malicious-email and the predicate symbols mention_On, attack and multiple_attacks might be used. Differently, variables may be uppercase letters that could be used as arguments of predicates, such as mention_On(X,Y), attack(X) and multiple_attacks(X,Y).

Terms and Atoms. A term is any member of _cons∪_var, while ground terms may be present only in _cons. Atomic sentences (atoms) are formed from a predicate symbol followed by a parenthesized list of terms. Each predicate symbol pϵ_pred has an arity that fixes the number of arguments. If tr₁, tr_n are (ground) terms, and p∈_pred, then p(tr₁, . . . , tr_n) is also a (ground) atom.

Herbrand base (Conditions and Actions). B may be used to denote the Herbrand base of , or its finite set of all ground atoms. Then, B may be divided into two disjoint sets: B_{conditions} and B_{actions}, so that B≡B_{conditions}∪B_{actions}. B_{conditions} comprehends the atoms allowed only in the premise of the APT rules, representing conditions or all users' actions performed and collected from dark-web forums, for instance: mention_on(set_forum_1,debian). On the other hand, B_{conditions} comprehends the atoms allowed only in the conclusion of the rules, representing actions or all malicious activities registered and reported by Armstrong company in its own facilities, for instance: attack(malicious-email).

Regular Formulas. Complex sentences (formulas) are constructed recursively from simpler ones, using parentheses and three logical connectives: (¬, negation, ∨ disjunction, ∧ conjunction). A (ground) atom is a (ground) formula, and if F and G are (ground) formulas, then F∨G, F∧G and ¬F are also (ground) formulas. As specified for atoms, formulas representing conditions may be located only in the premise of APT rules, while formulas representing actions may be located only in the conclusion of those rules.

Time Formulas. If F is a (ground) formula, t is a time point, then F_t is a (ground) time formula stating that F is true at time t. If ϕ, ρ are (ground) time formulas, then ¬ϕ, ¬ρ, ϕ∨ρ and ϕ∧ρ are also (ground) time formulas. Throughout, Greek letters ϕ, ρ p will be used for time formulas and capital letters F, G for regular formulas.

Probabilistic Time Formulas. If ϕ is a (ground) time formula and [l, u] is a probability interval ⊆[0, 1], then ϕ:[l, u] is a (ground) probabilistic time formula (ptf). Intuitively, ϕ:[l, u] says ϕ is true with a probability in [l, u], or using the complete notation, F_t:[l, u] says F is true at time t with a probability in [l, u]. To illustrate the ptf's generation process, consider the image presented in FIG. 2. The image is showing a timeline divided in past incidents (for tϵ) and future incidents (for tϵ).

Past conditions and actions may be annotated with [1,1], since they refer to facts that have been already observed in the past. A primary goal is to learn the relationship between those past incidents to predict highly likely actions in the future. This goal is accomplished by narrowing the probability boundaries of future actions derived from APT rules that have higher probabilities when compared to their priors. The matrix below illustrates how ptf's corresponding to facts are derived in the knowledge base using the timeline of FIG. 2. Other ptf's that express uncertainty will be added later.

$\hspace{1em}\left[\begin{array}{ccc}{F}_1\text{:}\left[1,1\right]& G_2\text{:}\left[1,1\right]& F_3^{\prime }\text{:}\left[1,1\right]\\ G_4^{\prime }\text{:}\left[1,1\right]& G_5\text{:}\left[1,1\right]& F_6\text{:}\left[1,1\right]\end{array}\right]$

The knowledge base of ptf's illustrated in the matrix above could also be specified as a conjunction of time formulas, as shown below:F₁∧G₂∧F′₃∧G′₄∧G₅∧F₆

APT Rules and Programs. Suppose condition F and action G are (ground) formulas, Δt is a time interval, [l, u] is a probability interval and ƒ r∈F is a frequency function symbol (these symbols will be defined together with formal APT semantics).

Then

$\begin{array}{cc}FG\text{:}\left[\Delta t,l,u\right]& \end{array}$ is an (ground) APT (Annotated Probabilistic Temporal) rule. The rule checks the probability that G is true within Δt units after F becomes true. Consider for instance, the following APT rule in a cyber domain:

$\mathrm{mention\_on}\left(\mathrm{set\_forum}\_1,\mathrm{debian}\right)\mathrm{attack}\left(\mathrm{malicious}-\mathrm{email}\right)\text{:}\left[3,0.4,0.5\right]$

This rule is informing the probability of Armstrong company being attacked by a malicious-email, within three time units after users mention “debian” on a set of forums (forums_1), is between 40% and 50%. APT Rules with tight boundaries like this one, where the lower bound value considerably exceeds the prior probability of having the corresponding cyberattack (action), produce precise information that can be leveraged by companies to allocate their limited resources and patch vulnerabilities. Naturally, an APT logic program is a finite set of APT rules and ptf's, that unless specified otherwise, may be used as ground in this context.

Annotated Probabilistic Temporal Logic (APT-LOGIC—Semantics)

A formal declarative semantics for APT-logic programs will now be described.

World: A world is any set of ground atoms that belong to B. The power set of B (denoted 2^B) is the set of all possible worlds (Herbrand interpretations) that describe possible states of the domain being modeled by an APT-logic program. A few possible worlds in this domain are listed below:

• • {mention_on(set_forums_1,debian),mention_on(set_forum_1, (microsoft-office)}
  • {mention_on(set_forums_1,debian)}
  • {attack(malicious-email)}
  • {multiple_attacks(malicious-email,2)}
  • { }

A world w satisfies a ground formula F (denoted wF), if the following four conditions hold:

• • If F=a for some ground atom a, then a∈w;
  • If F=¬G for some ground formula G, then w does not satisfy G;
  • If F=F₁∧F₂ for some ground formulas F₁ and F₂, then w satisfies F₁ and w satisfies F₂;
  • If F=F1∨F₂ for some ground formulas F₁ and F₂, then w satisfies F₁ or w satisfies F₂.

Thread: It may be assumed that all applications are interested in reasoning about a large but fixed size interval of time, and that τ={1, . . . , t_max} denotes the set of time points the present system is interested in. A thread is a mapping Th(1, . . . , t_max)→2^B that models the domain using worlds that evolve over time, or in this specific case, over τ. Th(i) specifies that according to the thread Th, the world at time i will be Th(i). Given a thread Th and a ground time formula ϕ, then Th satisfies ϕ (denotes Thϕ) if:

• • If ϕ≡F_t for some ground time formula F_t, then Th(t) satisfies F;
  • If ϕ≡¬ρ for some ground time formula ρ, then Th does not satisfies ρ;
  • If ϕ≡ρ₁∧ρ₂ for some ground time formulas ρ₁∧ρ₂, then Th satisfies ρ₁ and Th satisfies ρ₂;
  • If ϕ≡ρ₁∨ρ₂ for some ground time formulas ρ₁ and ρ₂, then Th satisfies ρ₁ or Th satisfies ρ₂;

For additional explanation, reference is made to FIGS. 4-5, where FIG. 5 presents a thread that considers the scenario illustrated in FIG. 4 including past and future incidents.

This thread is showing how users are posting on darkweb forums and how cyber-attacks are conducted over time until time point 8. As observed, there is a malicious-email attack suffered by Armstrong company at t=7. This is the type of action the present disclosure wants to predict using APT logic. For the sake of simplicity, it will be assumed in the present disclosure that the existence of a single thread Th corresponds to the historical corpus of data. The thread is used to consider the case when a ground time formula ϕ entails another ground time formula. Thus, ϕiff Thϕ then Thρ.

Frequency Functions: One of the ways APT-logic separates itself from past work is the introduction of the frequency functions. The basic intuition behind a frequency function is to represent temporal relationships within a thread, checking how often a world satisfying formula F is followed by a world satisfying formula G. Formally, a frequency function f r maps quadruples of the form (Th, F, G, Δt) to [0,1] of real numbers.

Presently, there are two different ways to define a frequency function from the historical data. The first one is the Point Frequency Function (pfr), which specifies how frequently the action G follows the condition F in exactly Δt time points, expressing what is desired where there is a precise temporal relationship between events. The second one is the Existential Frequency Function (pfr), which specifies how frequently the action G follows the condition F within Δt time points, allowing the action to fall within some specified period of time rather than after exactly Δt units of time.

Although both frequency functions are capable to quantify the temporal relationship between conditions and actions within this thread, it was realized that they strongly rely on the value of t_max. This fact may produce some problems for the present model, since different values for t_max in a retraining process can deeply interfere in the accuracy of predictions. Thus, the present disclosure discloses alternative definitions for both frequency functions that could avoid this weakness. In order to accomplish that, the present system first may specify how a ptf can be satisfied in the model. If the present systems considers the ptf F_t: [l, u], and some A′ϵA, where A is the set of all ptf's satisfied by the thread Th, then ThF_t[l, u] iff:

• • If F=a for some for some ground a, then ∃a_t:[l′, u′]ϵA s.t. [l′, u′][l, u];
  • If F_t[l, u]=¬F′_t[l, u] for some ground formula F′, then ThF′_t:[1-u, 1-l];
  • If F_t[l, u]=F′_t[l, u]∧F″_t[l, u] for some ground formulas F′ and F″, then ThF′_t[l, u] and ThF″_t[l, u];
  • If F_t[l, u]=F′_t[l, u]∨F″_t[l, u] for some ground formulas F′ and F″, then ThF′_t[l, u] or ThF″_t[l, u];

The new formulations of both frequency functions, starting with pfr in Equation 1 are disclosed below:

$\begin{array}{cc}\mathrm{pfr}\left(\mathrm{Th},F,G,\Delta t\right)=\left[\frac{t\left|\mathrm{Th}\right|=F_t\text{:}\left[l,u\right]\bigwedge \sum _{\mathrm{Th}|=G_{t+\Delta t}\text{:}\left[l^{\prime },u^{\prime }\right]}{l}^{\prime }}{\sum _{t\left|\mathrm{Th}\right|=F_t\text{:}\left[l,u\right]}u},\frac{t\left|\mathrm{Th}\right|=F_t\text{:}\left[l,u\right]\bigwedge \sum _{\mathrm{Th}|=G_{t+\Delta t}\text{:}\left[l^{\prime },u^{\prime }\right]}{u}^{\prime }}{\sum _{t\text{:}\mathrm{Th}|=F_t\text{:}\left[l,u\right]}l}\right]& \left(1\right)\end{array}$

In this context, the ptf's F_t:[l, u] and G_t+Δt:[l′, u′] will represent facts already observed in the past and can be annotated with [1,1]. This formalization is general enough to capture situations when the ptf's are representing uncertainty in the interval [0,1]. In addition, the values of both frequency functions may be calculated using probability intervals instead of point probabilities. Finally, the new formal definition of ef r in Equation 2 is set forth below:

$\begin{array}{cc}\mathrm{efr}\left(\mathrm{Th},F,G,\Delta t\right)=\left[\frac{t\left|\mathrm{Th}\right|=F_t\text{:}\left[l,u\right]\bigwedge \sum _{\mathrm{Th}|={\bigcup }_{i=1}^{\Delta t}{G}_{t+i}\text{:}\left[l^{\prime },u^{\prime }\right]}{l}^{\prime }}{\sum _{t\left|\mathrm{Th}\right|=F_t\text{:}\left[l,u\right]}u},\frac{t\left|\mathrm{Th}\right|=F_t\text{:}\left[l,u\right]\bigwedge \sum _{\mathrm{Th}|={\bigcup }_{i=1}^{\Delta t}{G}_{t+i}\text{:}\left[l^{\prime },u^{\prime }\right]}{u}^{\prime }}{\sum _{t\text{:}\mathrm{Th}|=F_t\text{:}\left[l,u\right]}l}\right]& \left(2\right)\end{array}$

Satisfaction of APT Rules and Programs. In the qualitative case of a formal logic, a Herbrand interpretation is defined as a mapping B→false, true. In the quantitative case of APT logic, the mapping range is the interval [0,1] of real numbers. If T is considered as the set of time points that comprise Th, the mapping B×T→[0,1] can be thought of as the membership function characterizing a “fuzzy” subset of B over Th.

As done before for the frequency functions, the membership function is measured using probability intervals. Then, the fuzzy subset of B over Th will be the mapping B×T→[0,1]×[0, 1], which assigns probability intervals to the worlds over the time points of the single thread Th. Then Th satisfies an APT Rule:

$\begin{array}{cc}FG\text{:}\left[\Delta t,l,u\right]\left(\mathrm{denotes}\mathrm{Th}|=FG\text{:}\left[\Delta t,l,u\right]\right)\mathrm{iff}\text{:}\mathrm{fr}\left(\mathrm{Th},F,G,\Delta t\right)\subseteq \left[l,u\right]& \left(3\right)\end{array}$

Equation 3 checks if the probability interval calculated by the frequent function f r(Th, F, G, Δt) is within the range [l, u] of the APT rule to be satisfied. The present system specifies in Equation 4 the entailment relationship between both frequency functions:

$\begin{array}{cc}\mathrm{If}\mathrm{Th}|=\stackrel{\Delta t}{\bigcup _{i=1}}\left\{FG\text{:}\left[i,l_i,u_i\right]\right\}\mathrm{then}\mathrm{Th}|=FG\text{:}\left[\Delta t,\max \left(l_i\right),\min \left(u_i\right)\right]& \left(4\right)\end{array}$

Annotated Probabilistic Temporal Logic (Embodiment D)

In one embodiment of the system, a subset of the previously introduced APT-logic syntax and semantics may be used. The existence of a set of atoms A may be assumed and partitioned into two disjoint sets: condition atoms, denoted a_condition, and action atoms, denoted a_action. a_condition describes hacker activities and may have a single predicate spike(ƒ), i.e., ƒ∈A_var, and A_var is a finite set of variables a_action. describes external targeted attacks and may have a single ground atom a_attack. Condition atoms a_condition may be connected using conjunction (i.e., ∧) to form condition formulas. A world is any set of ground atoms. A thread Th is a mapping of worlds to discrete time points—and time granularity may be fixed to days. Time points are represented by natural numbers, ranging from 1, . . . , t_max. A thread Th may satisfy a formula F at a time point t (denoted Th(t)F) iff: ∀_a∈F(Th(t)a). The existence of a single thread Th may be assumed from which the APT-logic program may be computed.

To compute the probability of Th(t+Δt)a_attack conditioned on Th(t)F, the point frequency function (pfr) concept may be used, which is a mapping of quadruples of the form (Th, F, a_attack, Δt) to a probability p. This mapping can be modeled in APT-logic programming defining a set of pfr rules of the form:

$\begin{array}{cc}F{a}_{\mathrm{attack}}\text{:}\left[\Delta t,p\right]& \left(1\right)\end{array}$

wherein F and a_attack may be assigned as the pre- and post-conditions of the rule, respectively. Additionally, the probability of a rule may be determined based on the fraction of times its post-condition is satisfied at t+Δt after the times its pre-condition is satisfied at t. |⋅| may be used to denote set cardinality:

$\begin{array}{cc}\mathrm{pfr}\left(\mathrm{Th},F,a_{\mathrm{attack}},\Delta t\right)=\frac{\left\{t\left|\mathrm{Th}\left(t\right)\right|=F\bigwedge \mathrm{Th}\left(t+\Delta t\right)|=a_{\mathrm{attack}}\right\}}{\left\{t\left|\mathrm{Th}\left(t\right)\right|=F\right\}}& \left(2\right)\end{array}$

The goal of the system at this stage may be to learn pfr rules whose probability is higher than the prior probability of, a_attack (i.e., pfr (pfr Th, F, a_attack, Δt)>pfr(Th, ϕ, Th a_attack, Δt)). If this condition is not met, the system may not consider the temporal correlation between F and a_attack to be significant, hence excluding it from the APT-logic program.

Extracting Entity Tags (Embodiment C)

The threat intelligence sources that may be used in this embodiment may supply a vast amount of textual content over time. A commercial natural language processing API may be utilized that leverages a wide-range of machine learning techniques (including Recurrent Neural Networks) to recognize entities from the context of postings. Each extracted entity is associated with a confidence score quantifying the amount of confidence in the annotation. A lower bound on confidence score may be set to only retain entities that are relevant. Two steps may be taken to extract the final indicators: (1) annotating spikes in the volume of individually extracted tags, and (2) for those tags, identifying sets that frequently spike together.

Annotating spiking tags: To gain an understanding of abnormal hacker activities that could possibly correlate with the attack events, abnormal activities may be defined, and those definitions used as preconditions of APT-logic rules. These definitions may or may not correlate with actual attack events, but the APT-logic program may only contain the rules whose precondition is found to correlate with the attack events. To identify such abnormalities, common entity tags may be considered that appear in most of the days, i.e., appear in 90 days or more as training periods are always 180 days. An entity may be regarded as abnormal if it is observed on a given day with a spiking volume—spikes may be determined when the count of times an entity is observed exceeds a moving median added to a multiplier of a moving standard deviation.

For instance, let F be an itemset i.e.:F={spike(ƒ₁), . . . ,spike(ƒ_n)|∀i∈{1, . . . ,n}:ƒ_i∈a_var}

The existence of three utility functions may be assumed:

• • 1. count(ƒ, t), which returns the number of time an entity ƒ is extracted on day t,
  • 2. median(ƒ, t, window), which returns the median of set S:S={count(ƒ,i)|i∈{t−window, . . . ,t}}
  • 3. stDiv(ƒ, t, window), which returns the standard deviation of S.

The thread Th satisfies a predicate spike(f) at some time point t, denoted Th(t) spike(f) iff:count(ƒ,t)>(median(ƒ,t,window)÷(multiplier×stDiv(ƒ,t,window)))

Extracting Entity Tags (Embodiment D)

Identifying Threat Indicators (Annotating abnormalities): In order to gain an understanding of abnormal hacker activities that possibly correlate with the attack events, abnormal activities may be defined and used as preconditions in this approach. They may or may not correlate with actual attack events, but the APT-logic program may contain the rules whose precondition is found to correlate with the attack events. To identify such abnormalities, common entities may be considered.

Common entitles: Common entities may be defined as entities that appear in most of the days. An entity may be regarded as abnormal if it is observed on a given day with a spiking volume—spikes are determined when the count of times an entity is observed exceeds a moving median added to a multiplier of a moving standard deviation.

For instance, let F be an itemset i.e., {spike(ƒ₁), . . . , spike(ƒ_n)|∀iϵ{1, . . . , n}: ƒiϵA_var}. The existence of three utility functions may be assumed: (1) count(ƒ, t), which returns the number of time an entity ƒ is extracted on day t, (2) median(ƒ, t, window), which returns the median of set S={count(ƒ, i)|iϵ{t−window, . . . , t}}, and (3) stDiv(ƒ, t, window), which returns the standard deviation of S.

The thread Th satisfies a predicate spike(ƒ) at some time point t, denoted Th(t) spike(ƒ) iff:count(ƒ,t)>(median(ƒ,t,window)+(multiplier H stDiv(ƒ,t,window)))

Frequent Itemset Mining: As explained, preconditions could be atoms or a formula (i.e., an itemset). A primary consideration is precondition formulas that are frequently satisfied in the historical data. An Apriori algorithm may be used in this consideration. The Apriori algorithm may take as input a database of transactions—the annotated hacker abnormal activities may be grouped by days, each day corresponding to a transaction. The algorithm may then produce all itemsets of hacker activities that are frequently observed together. The identified itemsets may be considered as preconditions for the APT-logic program.

Rule Learning—Rule-Learning Algorithms

Computing Probability Intervals. The probability intervals related to all pairs [l, u] specified in the present disclosure are derived using the standard deviation of the corresponding point probability in a binomial distribution, considering the formula in Equation 5.

$\begin{array}{cc}\sigma =\frac{\sqrt{\mathrm{support\_F}\star p\star \left(1-p\right)}}{\mathrm{support\_F}}& \left(5\right)\end{array}$

where support_F is the number of times the precondition or F is observed.

Rule Learning—APT Programs

The algorithms disclosed herein only add to the logic programs the rules with lower bounds exceeding the prior probability of the rule heads happening in any random time period of the same length as the rule's Δt days in the case of efr, and 1 day in the case of pfr.

If a rule is triggered due to a vulnerability mention, it may or may not result in generating warnings. The warning generation and fusion is less straightforward. In the case of efr, if more than one rule is triggered on the same day, the rule that predicts the attacks with the highest point probability will result in a warning. Additionally, if the validity time period of a new warning falls within a validity time period of an existing warning, the new warning may be cancelled, and the probability of the existing warning may be updated if the new warning predicts the attack with a higher probability.

In the case of pfr, the problem is to identify whether a triggered rule should generate warnings, and the number of warnings to generate. When there are no rules predicting attacks on a given day, no warnings are generated. As mentioned before, the present system may not assume the Markov property (i.e., the preconditions of different rules are independent if they happen on different days). Therefore, when two rules are triggered on different days, and they predict the same attack type that will occur on the same day, they both result in warnings; if both are qualified to generate warnings. It was also assumed that rule heads are only dependent on rule bodies. Therefore, if two rules are triggered on the same day, both predict the same attack type on the same day, and one predicts x number of attacks while the other predicts y number of attacks, then they will generate x+y warnings; if both are qualified to generate warnings. A pfr r∈R is qualified to generate x number of warnings (i.e., the head of r is attackNumber(attType, x), if (1) it is triggered, and (2) there is no other rule r′∈R is triggered on the same day, with the same rule head and Δt as r's, and r′'s point probability is greater than r's.

Sliding Window for Existential Frequency Rules—(Embodiment B)

The current semantic structure of APT-logic may not capture the concept of efr whose precondition atoms occur in any order within a sequence of Δt time-points. However, the efts sought to be obtained use such semantic. To do so, the APT-logic was used, but a new thread Th′ was made by assigning the atoms that Th satisfies at each time-point t to Th′(t), and for each of the next Δx time-points, according the logic below:∀_{a∈A,t∈[l, . . . ,tmax]}(Th(t)a→(∀_{x∈[t, . . . ,min(t+Δx,tmax)]}(Assign(Th(x),a)))

Where the function Assign(Th′(x), a) assigns an atom a to the thread Th′ at time-point x (i.e., Th′(x)a).

Item Sets for Existential Frequency Rules (Embodiment B)

The preconditions of the efrs are frequent itemsets obtained from running the Apriori algorithm. The input to the Apriori algorithm is a thread-like dataset, i.e., a sequence of events in discrete time-points. The output is all combinations of events that are satisfied by the thread and occur with a frequency exceeding a minimum support. A thread Th′ was generated, and run the Apriori algorithm on that thread, which returns a set of frequent itemsets (denoted freqItemsets). Then a thread Th_itemsets was made containing only atoms (frequent itemsets) that Th′ satisfies at each time-point as follows:∀_{I∈freqItemsets,t∈[l, . . . ,tmax]}(I⊆Th′(t)→(Assign(Th_itemsets(t),I)))Computing Device

Referring to FIG. 9, a computing device 200 may be used to implement various aspects of the framework 101 described herein. More particularly, in some embodiments, aspects of the framework 101 may be translated to software or machine-level code, which may be installed to and/or executed by the computing device 200 such that the computing device 200 is configured to learn rules and formulate predictions associated with cyber threats as described herein. It is contemplated that the computing device 200 may include any number of devices, such as personal computers, server computers, hand-held or laptop devices, tablet devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, digital signal processors, state machines, logic circuitries, distributed computing environments, and the like.

The computing device 200 may include various hardware components, such as a processor 202, a main memory 204 (e.g., a system memory), and a system bus 201 that couples various components of the computing device 200 to the processor 202. The system bus 201 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. For example, such architectures may include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus also known as Mezzanine bus.

The computing device 200 may further include a variety of memory devices and computer-readable media 207 that includes removable/non-removable media and volatile/nonvolatile media and/or tangible media, but excludes transitory propagated signals. Computer-readable media 207 may also include computer storage media and communication media. Computer storage media includes removable/non-removable media and volatile/nonvolatile media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules or other data, such as RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that may be used to store the desired information/data and which may be accessed by the general purpose computing device. Communication media includes computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. For example, communication media may include wired media such as a wired network or direct-wired connection and wireless media such as acoustic, RF, infrared, and/or other wireless media, or some combination thereof. Computer-readable media may be embodied as a computer program product, such as software stored on computer storage media.

The main memory 204 includes computer storage media in the form of volatile/nonvolatile memory such as read only memory (ROM) and random access memory (RAM). A basic input/output system (BIOS), containing the basic routines that help to transfer information between elements within the general purpose computing device (e.g., during start-up) is typically stored in ROM. RAM typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processor 202. Further, a data storage 206 stores an operating system, application programs, and other program modules and program data.

The data storage 206 may also include other removable/non-removable, volatile/nonvolatile computer storage media. For example, data storage 206 may be: a hard disk drive that reads from or writes to non-removable, nonvolatile magnetic media; a magnetic disk drive that reads from or writes to a removable, nonvolatile magnetic disk; and/or an optical disk drive that reads from or writes to a removable, nonvolatile optical disk such as a CD-ROM or other optical media. Other removable/non-removable, volatile/nonvolatile computer storage media may include magnetic tape cassettes, flash memory cards, digital versatile disks, digital video tape, solid state RAM, solid state ROM, and the like. The drives and their associated computer storage media provide storage of computer-readable instructions, data structures, program modules and other data for the general purpose computing device 200.

A user may enter commands and information through a user interface 240 (displayed via a monitor 260) by engaging input devices 245 such as a tablet, electronic digitizer, a microphone, keyboard, and/or pointing device, commonly referred to as mouse, trackball or touch pad. Other input devices 245 may include a joystick, game pad, satellite dish, scanner, or the like. Additionally, voice inputs, gesture inputs (e.g., via hands or fingers), or other natural user input methods may also be used with the appropriate input devices, such as a microphone, camera, tablet, touch pad, glove, or other sensor. These and other input devices 245 are in operative connection to the processor 202 and may be coupled to the system bus 201, but may be connected by other interface and bus structures, such as a parallel port, game port or a universal serial bus (USB). A monitor 260 or other type of display device may also be connected to the system bus 201. The monitor 206 may also be integrated with a touch-screen panel or the like.

The computing device 200 may be implemented in a networked or cloud-computing environment using logical connections of a network interface 203 to one or more remote devices, such as a remote computer. The remote computer may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the general purpose computing device. The logical connection may include one or more local area networks (LAN) and one or more wide area networks (WAN), but may also include other networks. Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets and the Internet.

When used in a networked or cloud-computing environment, the computing device 200 may be connected to a public and/or private network through the network interface 203. In such embodiments, a modem or other means for establishing communications over the network is connected to the system bus 201 via the network interface 203 or other appropriate mechanism. A wireless networking component including an interface and antenna may be coupled through a suitable device such as an access point or peer computer to a network. In a networked environment, program modules depicted relative to the general purpose computing device, or portions thereof, may be stored in the remote memory storage device.

Certain embodiments are described herein as including one or more modules 212. Such modules 212 are hardware-implemented, and thus include at least one tangible unit capable of performing certain operations and may be configured or arranged in a certain manner. For example, a hardware-implemented module 212 may comprise dedicated circuitry that is permanently configured (e.g., as a special-purpose processor, such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC)) to perform certain operations. A hardware-implemented module 212 may also comprise programmable circuitry (e.g., as encompassed within a general-purpose processor or other programmable processor) that is temporarily configured by software or firmware to perform certain operations. In some example embodiments, one or more computer systems (e.g., a standalone system, a client and/or server computer system, or a peer-to-peer computer system) or one or more processors may be configured by software (e.g., an application or application portion) as a hardware-implemented module 212 that operates to perform certain operations as described herein.

Accordingly, the term “hardware-implemented module” encompasses a tangible entity, be that an entity that is physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a certain manner and/or to perform certain operations described herein. Considering embodiments in which hardware-implemented modules 212 are temporarily configured (e.g., programmed), each of the hardware-implemented modules 212 need not be configured or instantiated at any one instance in time. For example, where the hardware-implemented modules 212 comprise a general-purpose processor configured using software, the general-purpose processor may be configured as respective different hardware-implemented modules 212 at different times. Software may accordingly configure a processor 202, for example, to constitute a particular hardware-implemented module at one instance of time and to constitute a different hardware-implemented module 212 at a different instance of time.

Hardware-implemented modules 212 may provide information to, and/or receive information from, other hardware-implemented modules 212. Accordingly, the described hardware-implemented modules 212 may be regarded as being communicatively coupled. Where multiple of such hardware-implemented modules 212 exist contemporaneously, communications may be achieved through signal transmission (e.g., over appropriate circuits and buses) that connect the hardware-implemented modules. In embodiments in which multiple hardware-implemented modules 212 are configured or instantiated at different times, communications between such hardware-implemented modules may be achieved, for example, through the storage and retrieval of information in memory structures to which the multiple hardware-implemented modules 212 have access. For example, one hardware-implemented module 212 may perform an operation, and may store the output of that operation in a memory device to which it is communicatively coupled. A further hardware-implemented module 212 may then, at a later time, access the memory device to retrieve and process the stored output. Hardware-implemented modules 212 may also initiate communications with input or output devices.

Consolidation: One other possible component of the framework 101 may involve functionality to fuse warnings from various heterogeneous models (including DARKMENTION), populate any missing warning fields according to the program requirement, and generate the final version of each warning. This completed warning may then be submitted to the Security Operations Center. Each warning submitted may be available to view and drill down into using a Web UI and Audit Trail analysis capability. This audit trail may go from the submitted warning all the way through model fusion, the individual models, and each individual model's raw data used to generate a warning. In the case of DARKMENTION, this would include the D2web postings/items with the CVE's mentioned highlighted.

Experimental Results—(Embodiment A)

In this section, evidence is provided regarding the viability of the present system through a series of experiments. The warnings that are generated by this model, along with warnings from other models, are being evaluated by the Security Operations Centers (SOCs) on a monthly basis. However, since the external evaluations are aggregated for all models, and DARKMENTION was not operationally deployed until after the time periods those reports cover, DARKMENTION was internally evaluated. Now, the experimental settings that were followed will be explained and the evaluation metrics used to evaluate the warnings generated from the present model.

Experimental Settings

Evaluations were performed on the warnings targeting Armstrong that were submitted during July, August, and September of 2017. The results are aggregated per months for the experiments on Armstrong data while aggregated on periods of 7-days for Dexter. The latter started from July 1 to Jul. 28, 2016. These time windows differ because the Armstrong dataset covers longer period of time as compared to the period covered by Dexter, and there is no more GT data about Dexter that is going to be provided or evaluated by the program. The reported records of Malicious Destination for Dexter only cover a time period that ends before the testing time period starts, hence they are not evaluated.

Evaluation Metrics

To evaluate the accuracy of the present system, three metrics were used: recall, which corresponds to the fraction of GT events that have matching warnings from the total number of GT events; precision, which is the fraction of warnings that have matching GT events from the total number of the generated warnings; and F1, which is the harmonic mean of recall and precision. Table 3 summarizes the evaluation metrics.

TABLE 3
Evaluation metrics
Metric    Formula
Recall    No                        .                                                                                                  ⁢                        of                                            ⁢                                                                                          ⁢                      matched                      ⁢                                                                                          ⁢                      GT                      ⁢                                                                                          ⁢                      events                                                                                      No                        .                                                                                                  ⁢                        of                                            ⁢                                                                                          ⁢                      GT                      ⁢                                                                                          ⁢                      events
Precision No                        .                                                                                                  ⁢                        of                                            ⁢                                                                                          ⁢                      matched                      ⁢                                                                                          ⁢                      warnings                                                                                      No                        .                                                                                                  ⁢                        of                                            ⁢                                                                                          ⁢                      warnings
F1        2                    *                                                                  precision                        *                        recall                                                                    precision                        +                        recall

Matching Warnings and GT Events. The matching problem is to find whether a warning w earns credit for predicting a GT attack event g. If w predicts an attack with different type than g's, or w predicts an attack on a different day than the occurrence day of g, then they do not match. Otherwise, they may or may not match based on whether or not w or g have already been paired up with another GT event or a warning, respectively.

To join together warnings with GT events in parings while ensuring that resultant pairings are mutually exclusive, the Hungarian assignment algorithm was used. Intuitively, the algorithm takes as input an n*n matrix representation of (−1*lead-time). Lead-time is the time between warnings and GT events that are qualified to match. Then, the algorithm returns a solution S that maximizes the total lead-time. Here, S is a set of pairs, each maps a warnings to a GT event such that the pairs are guaranteed to be mutually exclusive. The database stored the pairs that were returned by the algorithm.

Results

It was found that the present system outperforms a baseline system that randomly generates x number of warnings on each day such that each value of x has a chance proportional to its frequency of occurrence in the historical data. The baseline was repeated for 100 runs and took the average of each metric. In the real-time deployment of DARKMENTION, human experts can evaluate the warnings by leveraging the other capabilities of the system through a Web UI dashboard. However, in those experiments any triggered rule may be counted, which may not necessarily be important given other details. Nevertheless, the present system scored significantly higher than the baseline system as shown in Table 4.

TABLE 4
The evaluation results. A simple baseline model that generates x number
of warnings on each day based on the prior probability of each possible
value of x that was seen in the training data.
	Testing	#GT-	DARKMENTION	Baseline* (average of 100 runs)	% increase
Dataset	type	starts	events	#warnings	Precision	Recall	F1	#warnings	Precision	Recall	F1	in F1
Armstrong	M-E	Jul-17	24	32	0.313	0.417	0.357	11.759	0.417	0.205	0.271	32%
		Aug-17	11	3	1.000	0.273	0.429	11.966	0.289	0.315	0.299	43%
		Sep-17	13	18	0.167	0.231	0.194	12.793	0.249	0.249	0.247	−21%
	M-D	Jul-17	4	12	0.167	0.500	0.250	3.534	0.099	0.091	0.090	178%
		Aug-17	9	23	0.174	0.444	0.250	3.121	0.232	0.086	0.120	108%
		Sep-17	3	10	0.100	0.333	0.154	2.948	0.071	0.075	0.068	126%
	E-M	Jul-17	14	10	0.300	0.214	0.250	8.552	0.326	0.200	0.242	3%
		Aug-17	18	45	0.200	0.500	0.286	9.155	0.324	0.168	0.217	32%
		Sep-17	17	21	0.286	0.353	0.316	8.879	0.247	0.127	0.164	93%
Dexter	M-E	1-Jul-16	2	13	0.150	1.000	0.267	2.720	0.157	0.205	0.169	58%
		8-Jul-16	7	10	0.300	0.714	0.588	2.610	0.633	0.253	0.348	69%
		15-Jul-16	9	5	0.333	0.222	0.267	2.770	0.619	0.188	0.276	−3%
		22-Jul-16	4	2	0.500	0.250	0.333	3.050	0.469	0.355	0.385	−14%
	E-M	1-Jul-16	1	2	0.500	1.000	0.667	1.700	0.189	0.330	0.226	195%
		8-Jul-16	3	4	0.250	0.333	0.286	1.750	0.243	0.167	0.186	54%
		15-Jul-16	3	1	1.000	0.333	0.500	1.740	0.281	0.190	0.217	130%
		22-Jul-16	4	2	0.300	0.250	0.333	1.780	0.383	0.208	0.257	30%

Runtime Analysis—(Embodiment B)

FIG. 6 shows that when the length Δt increases, the time taken for algorithm to generate the rules increases linearly. This is because that the number of time-points for which algorithm may use to check the satisfaction of the consequence increases. This shows that the approach can scale well with datasets having a large number of time-points and tags. However, the Apriori algorithm runs in exponential time, and for all analysis in this study, the number of items was restricted to 2 in each itemset.

Experimental Results (Embodiment C)

Experimental Setup

Training/testing splits: To produce the APT-logic program, an APT-EXTRACT algorithm may be used on the ground truth data and on the spiking tags observed in the 6-month-period preceding the testing month. Then, for each day in the testing month, the system may generate warnings by matching the spiking tags observed on that day with preconditions of rules in the APT-logic program. If a match exists, a warning may be generated for the day corresponding with the value of Δt of the triggered rule.

Time-series forecasting baseline: The IARPA may be used as a baseline model that reads a training data of the Data Providers ground truth events and may model weekly/daily time seasonality using a simple, constant base-rate model that calculates the average frequency of events from the training data. Using this approach, the model is fitted to ground truth data from all the months prior to the testing month and the model may be used to generate warnings for the testing month.

Evaluation (Pairing ground truth events with warnings): To receive a score, each warning may be paired up with a single ground truth event occurring within the same day, or one day after the attack prediction date, i.e., 1-to-1 relationships. To do so, a Hungarian assignment algorithm may be used to solve the warning-to-ground truth assignment problem, with the objective to maximize warning-to-attack lead time. The results of the Hungarian algorithm (i.e., warning-to-ground truth assignments) may be used to evaluate the performance of the system.

TABLE 5:
Evaluation metrics. TP—true positives,
FP—false positives, FN—false negatives,
TN—true negative.
Metric    Formula
Precision TP                                          TP                      +                      FP
Recall    TP                                          TP                      +                      FN
F1        2                    ·                                                                  precision                        ·                        recall                                                                    precision                        +                        recall

Evaluation metrics: Standard evaluation metrics, namely precision, recall, and F1, may be used. Precision is the fraction of warnings that match ground truth events, recall is the fraction of ground truth events that are matched, and F1 is the harmonic mean of precision and recall. Table 5 above summarizes the used metrics. Using these metrics, a performance comparison is presented between the system and the baseline model. Additionally, a fused model is shown that can benefit from the temporal correlations and statistical characteristics captured by the system and the baseline model, respectively.

Experimental Results

Fusion: A simple combining strategy may be used to test the performance of a fused model. The warnings from the two models, i.e., the system and the baseline, may first be combined. The warnings may be grouped by their generation date and prediction data. Then, half of the warnings may be removed from each group in order to leverage the power of the individual approaches while limiting their intersection, i.e., removing half of the duplicate warnings.

Parameter tuning: The condition on what rules to be considered in the APT-logic program, i.e., rules whose probability is higher than the prior probability of the postcondition, may not guarantee the highest performance. Therefore, the classical Grid search method may be used to find optimal minimum thresholds on rule probability and support (i.e., the numerator of Equation 2). The parameter values that maximize F1 inform the decision on what set of rules are most useful for real-world production system.

Performance comparison: FIG. 7 shows the precision-recall curve for each of the testing months. By itself, the present approach performs comparable to the baseline in terms of F1—specifically providing higher precision in the case of lower recall. It should be noted that when the approach is combined with the baseline, the results improve further. The combined approach can significantly outperform the baseline in terms of both precision and recall, yielding a recall increase of at least 14% while maintaining precision. Furthermore, the baseline does not allow for a tradeoff between precision and recall while the present approach produces warnings with probability values, enabling not only better tradeoff between performance metrics, but also a metric approximating the importance of each warning, and allowing the human analyst to prioritize investigation.

Transparent predictions: This approach may support transparent predictions i.e., the user knows why certain warnings is generated. The user may trace back to the rule corresponding to a warning, and view its precondition. Table 6 shows a few examples of preconditions of rules that generated warnings preceding attack incidents. The user can further pinpoint the collection of hacker discussions that are responsible for the warning. For example, FIG. 8 shows an exemplary word cloud generated from a collection of posts resulting in a warning submitted on August 23th. The warning predicts an event on August 25th, i.e., Δt of 2. An event of malicious email is then observed by the Data Provider on August 26th.

TABLE 6
Examples of preconditions of rules that would have
generated warnings preceding attack incidents
			Warning	Lead time
Precondition	Probability	σ	date	(days)
spike(Credit card) Λ	0.88	0.07	Aug. 26	1
spike(Gmail)
spike(Email) Λ	0.86	0.08	Aug. 16	1
spike(Security hacker)
spike(Google Play)	0.92	0.04	Aug. 13	2

Setup

Training/testing splits. The performance of the approach taken by this embodiment has been tested on each month starting from July-October, 2018. To produce the APT-logic program, an APT-EXTRACT algorithm may be used on the ground truth data and on the abnormal hacker activities observed in the 6-month-period preceding the testing month. Then, for each day in the testing month, this approach may generate warnings by matching the abnormal activities observed on that day with preconditions of rules in the APT-logic program. If a match exists, a warning may be generated for the day corresponding with the Δt of the triggered rule.

Time-series forecasting baseline: The IARPA used may perform a baseline model that reads a training data of the Data Providers ground truth events and may model weekly time seasonality using a simple, constant base-rate model that calculates the average frequency of events from the training data. Using this approach, the model may be fitted to ground truth data from all the months prior to the testing month and the model may be used to generate warnings for the testing month.

Pairing ground truth events with warnings: To receive a score, each warning may be paired up with a single ground truth event occurring within the same day, or one day after the attack prediction date, i.e., a 1-to-1 relationships. To do so, the Hungarian assignment algorithm may be used to solve the warning-to-ground truth assignment problem, with the objective of maximizing warning-to-attack lead time. The results of the Hungarian algorithm may then be used to evaluate the performance of the proposed approach as well as the baseline model.

Evaluation

Evaluation metrics: Standard evaluation metrics such as precision, recall, and F1 may be used. Precision is the fraction of warnings that match ground truth events, recall is the fraction of ground truth events that are matched, and F1 is the harmonic mean of precision and recall. Using these metrics, a performance comparison may be presented between the proposed approach and the baseline model. Additionally, it may be shown that a fused model can benefit from the temporal correlations and statistical characteristics captured by the proposed approach and the baseline model, respectively.

Fusion: A simple combining strategy may be used to test the performance of a fused model. The warnings from the two approaches, i.e., the approach of this embodiment and the baseline, may be combined. The warnings may be grouped by their generation date and prediction data. Then, half of the warnings may be removed from each group. The goal is to leverage the power of the individual approaches while limiting their intersection, i.e., removing half of the duplicate warnings.

Grid search: The condition on what rules are to be considered in the APT-logic program, i.e., rules whose probability is higher than the prior probability of the postcondition, may not guarantee the highest performance. Therefore, the classical Grid search method may be used to find optimal minimum thresholds on rule probability and support. The parameter values that maximize F1 may inform the decision on what set of rules are most useful for real-world production system.

TABLE 7
Examples of preconditions of rules that would have
generated warnings preceding attack incidents.
		Warning	Lead time
Precondition	Probability	date	(days)
spike(Credit card) Λ spike(Gmail)	0.88	Aug. 26	1
spike(Email) Λ spike(Security	0.86	Aug. 16	1
hacker)
spike(Google Play)	0.92	Aug. 13	2

Result

Performance comparison. FIG. 7 shows the precision-recall curve for each of the testing months. Overall, the warning generation approach of this embodiment performs comparably to the baseline approach. It should be noted that the baseline may not allow for a tradeoff between precision and recall while this approach may produce warnings with probability values, enabling not only better tradeoff between performance metrics, but also a metric approximating the importance of each warning. That should allow the human analyst to prioritize investigation. In addition, the figure demonstrates high potential for the fused approach as it clearly outperforms the individual predictors.

Transparent predictions: The approach of this embodiment supports transparent predictions i.e., the user knows why certain warnings are generated. The user can trace back to the rule corresponding to a warning, and view its precondition. Table 1 shows a few examples of preconditions of rules that generated warnings preceding attack incidents. The user can further pinpoint the collection of hacker discussions that are responsible for the warning. For example, FIG. 8 shows the word cloud generated from the collection of posts resulting in warnings submitted on August 23th. The warning predicts an event on August 25th, i.e., Δt of 2. An event of malicious email is then observed by the Data Provider on August 26th.

In some embodiments, rule-based learning, i.e., implementation of the Learning Module 106 may involve Knowledge Representation and Reasoning (KRR). KRR supports formally explainable reasoning, which is a desirable feature for many applications, including predicting cybersecurity incidents. Yet much of KRR is too rigid for real-world applications. On the other hand, it is hard to incorporate knowledge that is not statistically present in the training data (e.g., expert knowledge) to Machine Learning (ML) models, and for some models (e.g., neural networks and SVMs), it is hard to explain the output of ML.

In some embodiments, concept drift may be considered. Hacking tactics advance rapidly corresponding to advances in cybersecurity, i.e., new vulnerabilities are discovered, new exploits are integrated with malware kits, attack signatures are identified, etc. Likewise, the attacks observed in the wild and the activities of hackers in the hacker community websites such as social media are always evolving. This change in the underlying data distribution for both the hacker discussions and the predicted events is known as “concept drift”. To account for potential impact of concept drift, in each month, in some embodiments a learner is run on data from the previous 6 months, and the resulting rules may be used to predict events in the examined month.

In some embodiments, warning generation by the prediction module 108 may be executed daily by first acquiring all CVEs mentioned in the last 24 hours within the D2web streaming data. The CPE groups/nation-state actors for these mentioned CVEs may then be obtained. Next, based on the APT-rules—the prediction module 108 may try to match the CPE/nation-state actor mappings to a particular rule. If a match exists, the prediction module 108 may predict if and when an attack exploiting the vulnerabilities will occur by generating a warning. The warning fields may be populated using the information contained in the rule, such as the probability, event type, and target organization.

In some embodiments, some of the non-functional requirements related to the generated warnings (i.e., timely, actionable, accurate, and transparent), may be maintained by the system 100 over time. Further, due to various factors relating to both intelligence data (i.e. the ephemeral nature of D2web sites) and enterprise data (i.e. data from a Security Information Event Manager or SIEM, which can be subject to schema differences due to policy changes over time), further requirements are examined for this approach.

It should be understood from the foregoing that, while particular embodiments have been illustrated and described, various modifications can be made thereto without departing from the spirit and scope of the invention as will be apparent to those skilled in the art. Such changes and modifications are within the scope and teachings of this invention as defined in the claims appended hereto.

Claims

1. A method of predicting likelihood of exploitation for cyber threats before they occur, the method, comprising: providing a processor in communication with a tangible storage medium;storing instructions that are executed by the processor to perform operations comprising:accessing a first dataset defining communications from forums and marketplaces associated with a hacker community;deriving a plurality of temporal rules by correlating a plurality of indicators generated from the first dataset and ground truth information associated with known cyberattacks by: filtering out data that is not related to cybersecurity;retaining data that is related to cybersecurity;recognizing entities from the context of postings and assigning a confidence score;using the confidence score and potential impact of concept drift to filter for relevant entities; andusing machine learning to derive the temporal rules;the plurality of indicators including mappings between a vulnerability and a platform known to be susceptible to the vulnerability; andpredicting a cyber threat, including:identifying an indicator of the plurality of indicators from a second dataset, the second dataset defining additional communications from the hacker community and the indicator being a precondition to a corresponding rule of the plurality of temporal rules, and applying information associated with the indicator to the corresponding rule of the plurality of temporal rules to output at least one prediction of a future attack associated with the cyber threat.

2. The method of claim 1, further comprising generating the plurality of rules by deriving a set of probability boundaries of future actions using an annotated probabilistic temporal logic rules framework and narrowing the set of probability boundaries.

3. The method of claim 2, wherein one of the plurality of rules defines a probability value for the attack associated with the cyber threat occurring within a predetermined time interval of a condition being true.

4. The method of claim 3, wherein a point frequency function of the annotated probabilistic temporal logic rules framework is applied to output a frequency value for the attack following identification of the indicator from the second dataset in an exact time interval and defines a predetermined precise temporal relationship between the attack and the indicator.

5. The method of claim 4, wherein the frequency value for the attack following the indicator in an exact time interval is calculated using a probability interval.

6. The method of claim 2, wherein an existential frequency function of the annotated probabilistic temporal logic rules framework is applied to output a frequency value for the attack following identification of the indicator within a predetermined number of time points and defines a specified temporal relationship between the attack and the indicator.

7. The method of claim 6, wherein the frequency value for the attack following the indicator within a predetermined number of time points is calculated using a probability interval.

8. The method of claim 1, wherein a plurality of rule-learning approaches are applied to learn a set of temporal correlations between the first dataset and the known cyberattacks.