This invention related generally to systems and methods for detecting and preventing internal and external threats to technology infrastructure, information assets and intellectual property of enterprises and other organizations, and more particularly to assessing threats based on a mix of behavioral and direct indicators.
The rapid detection of security threats is critical for organizations to prevent compromise of their computer systems, data, networks and applications. Organizations, whether commercial, educational or governmental, and other enterprises store and transfer the majority of their data in digital form in computer systems and databases. Much of this data is valuable confidential commercial information such as business plans or trade secret information, or private information about individual employees or members that is not intended for public view, and any exposure or manipulation of this information could cause the organization or employees great financial or reputational damage. Organizations are frequently challenged by the attacks that involve fraud, data destruction or theft, intellectual property theft, or national security implications. Some attackers may be backed by nation states or groups with political agendas and apply more sinister attacks intended to gain control of or to damage critical infrastructures.
Organizations typically employ a multi-layered network topology that separates various components of their IT infrastructure from the Internet or other external networks. Internal network workstations and servers are generally protected from direct attack from external sources by network proxy servers; external network traffic is typically terminated by such servers at “demilitarized network zones” (DMZ); and the incoming traffic is filtered through a firewall. External attackers normally attempt to penetrate an organization's defenses that are set up at the organization's network perimeter, and many security approaches attempt to prevent network access by such external attacks. Once external attackers breach the network perimeter and get onto the internal network, they become much more difficult to detect and defend against. They may unleash malware or attempt to access internal data, and typically operate under the guise of an internal user by either hijacking an existing user's account or by creating a new user account. Inside attackers are more insidious and pose threats that are more difficult to detect and defend against because the inside attackers are perceived to be rightful users of the organization's computer network systems. They may have legitimate IT accounts, and their unauthorized and illicit activities may generally fall within authorized areas of responsibility for insiders, but otherwise exceed what is normal behavior. For instance, illicit behavior by an employee customer service representative such as granting a customer an inappropriately large refund, or by an insider accessing and manipulating customer information or other sensitive data may be difficult to detect as a threat.
Many approaches to external threat detection utilize signatures of known attacks to identify and create models for providing alerts upon detecting activities having similar signatures. In order to define signatures for any new threats, the underlying components of the associated threat vectors must be studied in detail and signatures of these threat vectors must be made available to a threat detection system. There are several major shortcomings of these signature-based threat detection approaches. The development of signatures for new threats requires an in-depth analysis on an infected system, which is time consuming and resource intensive, and may be too slow to address quickly evolving threats. Signatures also do not adapt well to changes in threat vectors. Moreover, signature-based defenses cannot protect against zero-day attacks that exploit previously unknown vulnerabilities, and are ineffective for detecting insider threats originating from within an organization.
Identifying insider attacks typically involves constructing various profiles for the normal behaviors of insiders, detecting anomalous deviations from these profiles, and estimating the probabilities of threat risks of these anomalies. However, constructing profiles that accurately characterize normal insider behavior is difficult and an inexact art. Moreover, organizations in different industries may have different profile models for behavior considered normal. For example, the health care industry has models for normal activities that are different from those for the financial and retail industries due to inherent differences between the industries. Applying the same profile models to different industries can lead to false results. Moreover, many profiles are constructed using statistical approaches for observables that are assumed often incorrectly to be normally distributed when they are not. Using such profiles for detecting anomalies that represent possible threats can produce erroneous results and lead to many false positive alerts that can overwhelm security analysts. Balancing between the risk of missing an actual threat by using high confidence levels for detection to minimize false positives and using an overly permissive approach that floods security analysts with alerts is a difficult trade-off.
There is a need for systems and methods that address these and other known problems in reliably detecting, evaluating and assessing threat risks to protect organizations from data breaches, attacks and other injuries. In particular, there is a need for more accurate threat modeling and risk evaluation approaches that reliably identify threats and evaluate threat risks within an organization's IT infrastructure while minimizing false positive alerts. It is to these ends that this invention is directed.
The invention addresses the foregoing and other problems of detecting and assessing threat risks to organizations through a multi-tiered hierarchical process that aggregates threats in order of increasing complexity into composite threat risks to complex use cases across multiple domains, and amplifies risks to enhance risk detection along kill chains that afford early detection of an attack before serious damage can be done. Threat indicators may be quantified using a risk scoring process that assigns a threat risk score to each indicator to enable the different indicators to be aggregated into composite risks and assessed.
In one aspect of the invention, a system and a method are provided in which direct and behavioral threat indicators are combined to indicate various different singular threats, and the various singular threats are combined to provide composite threat indications for complex use cases for detecting composite threats. A composite risk score is determined for a composite threat, and used to detect a possible attack by comparing the composite risk score to a predetermined threshold. Composite and singular threats are aggregated to create complex use cases across multiple domains to amplify risks along a kill chain to increase the likelihood of early detection of an attack.
In other aspects, the invention classifies and normalizes entity risk scores for different entities and at different levels of the enterprise to enable comparison of the risks associated with different entities, and aggregates the entity risks to determine organizational risk scores for different departments, divisions and groups of the enterprise.
A process in accordance with the invention may be embodied in executable instructions in computer readable physical media that control the operations of one or more computers in the computer network infrastructure of an enterprise.
Applicant's prior U.S. application Ser. No. 14/811,732, filed Jul. 28, 2015, the disclosure of which is incorporated by reference herein, is directed to identifying threats to an enterprise by detecting anomalous behaviors of individuals operating within the IT infrastructure (computer network) of the enterprise by comparing their behaviors to behavioral profiles to detect anomalies that indicate a possible threat. This invention extends the invention of said prior application by identifying, evaluating and assessing events and behaviors assembling both behavioral indicators and direct indicators to form composite threat risks, and by quantifying (scoring) the composite threat risks in ways that enhance the detection of actual threats and that enable risks for different types of threats and for different entities and organizations to be assessed and compared.
At the next level of the hierarchy the singular threats 112 are combined to form composite threats 130 and complex use cases across multiple domains, as will be described in more detail below. Composite threats comprise more complex combinations of singular actions and events that are more strongly indicative of likely threats. They may comprise, for instance, data exfiltration (unauthorized data removal or theft) 132, account hijacking 134, lateral movement of data 136, and malware infection 138. Possible combinations of singular threats that may form composite threats are indicated in the figure by the arrows. For example, data consumption 114 and data egress 116 may combine to create the threat of data exfiltration 132, whereas data egress 116, anomalous access 120 and proxy blocks 122 may combine to create the threat of malware infection 138.
At the next level, entity risks 140 may be determined, as will be described in more detail below, by aggregating, as shown, dynamic threat risks from singular threats 112 and composite threats 130 attributed to a specific entity, such as a user 142, an application 144, or a system 146. Entity risks may further be aggregated to determine the organizational risk 150 for departments or groups such as sales 152 and engineering 154 within the organization, as well as for the enterprise 156 itself.
In accordance with the invention, threat indicators can be quantified to create risk scores for each specific threat such that they may be combined in meaningful ways to create topical and cumulative threat scores. Each threat indicator may be assigned a risk factor, fI, for a specific threat based, for instance, upon metadata for the type of threat to which the risk factor is assigned. For example, a failed login happens to everyone occasionally, and a single occurrence of a failed login is not necessarily indicative of a threat. Therefore, a failed login event may be assigned a risk factor such as 0.3 meaning that based upon empirical metadata for a single failed login there is a 30% probability that it represents an actual threat. However, if instead ten failed login events are observed, while this is also possibly normal, it is less likely that this number of failed logins is due to chance. In this instance, it may be appropriate that a risk score of 0.8 be assigned to the threat, representing an 80% chance that these events are due to malicious activity. Thus, the risk score that is determined for this particular threat indicator is based not only on its risk factor, but also is a function of the number of occurrences of events represented by the threat indicator. This is illustrated in
In
P
T=1−(1−fDPD)πI(1−fIPI)
Each threat may have a different aging factor that may be selected empirically based upon the type of threat, previous observations and other information such as experience with the particular type of threat. Effectively, the aging factor decreases the weight accorded to historical determinations of threat score to avoid overly biasing a current risk score determination with the scores for previous threats which may have been based upon different conditions and different threat indicators. For some threats, the aging factor, fD, may be zero, in which case the threat score becomes a topical (current) threat score, {acute over (P)}T:
{acute over (P)}
T=1−πI(1−FIPI).
As noted earlier, the threat score depends upon the number of events observed. Accordingly, a large number of low-risk anomalies may result in a high risk score.
Combining singular threats to create composite threats for complex use cases across multiple domains creates a kill chain that facilitates the detection of threats. By creating a kill chain comprising a sequence of events that represent composite threats in the order in which they would occur in an attack, risks along the kill chain are amplified which affords earlier prediction and detection of threats. By complex use cases across multiple domains is meant being able to correlate multiple different threats that may not be directly related but combine to indicate a possible threat that otherwise may go undetected. For instance, in an email channel traditional security tools that look for phishing emails may not alone generate a sufficiently high probability to warrant an alarm. However, when combined with indicators from other domains such as the findings from anti-virus software or other malware scanners, the ability to detect the threat can be enhanced. The ability afforded by the invention of creating different risk models by combining different types of risks to create composite risks scores for more complex use cases and for more complicated attacks is something that previously was unavailable and is particularly advantageous. As will be described in more detail below, this also affords particularized threat detection models tailored for different industries and different market segments that permit more accurate identification and characterization of threat risks, which substantially reduce false positives.
A composite risk score, RC, may be computed from the cumulative threat scores, PT, determined as indicated above to exponentially increase with the number of threats observed along the kill chain as:
R
C=πT(1+PT)−1
The composite risk score combines risk scores of all threats in the composite threat in a manner that exponentially amplifies resulting score if there is more than one non-zero risk score.
In
For example, for a data exfiltration use case, e.g., threat 448 of
Behavioral indicators are independent measurements of an observable for a specific time series. For example, for customer service transactions, hourly and day of the week counts of transactions, as well as the amount of each transaction may be chosen. For monitoring database activity, the number of concurrent users per hour and daily volume of data consumed by each user may be employed for specific use cases. The proper time series may be selected based upon the volume of observations and the expected volatility, with preference given to time periods that reflect patterns of life, such as working hours or workdays. In combining behavioral indicators for a threat model, it is desirable for accurate risk scoring that the threat indicators selected be independent or at least not strongly correlated. Otherwise, since indicators combine to produce a composite risk score, dependent indicators can bias the resulting score and produce inaccurate results. Accordingly, prior to inclusion in a model, potential candidate indicators should be tested pairwise for dependency by determining their correlation using, for example, Pearson's correlation function.
Referring to
As indicated in
R
E
=R
T·[(1+RS)·(1+RI)−1]
In the account hijacking example above, if User 3 has a static risk RS=0.9, which is a high risk, and an inherent risk, RI=0.3, which is a medium risk, the overall risk of account hijacking for this user would be 6.37·[(1+0.9)·(1+0.3)−1]=9.36. Both static and inherent risk scores have to be normalized before they can be used in the above relationship to calculate entity risk, RE, and to compare the risk levels of different entities.
In accordance with a preferred embodiment of the invention, normalization is preferably accomplished as a two-step process—classification and then normalization. First, raw risk scores may be classified by stack ranking the raw scores for each entity class (user, system and application). Next risk thresholds may be established to enable classifying the raw scores into discrete ranks or classification levels based upon percentages, where the discrete ranks reflect the enterprise's judgment as to the seriousness of the risks. Classification of risk scores by levels is useful for reporting and remediation within an entity class. Classification, however, may not enable risk scores of different entities to be compared or used directly to indicate an overall organizational risk. Accordingly, following classification raw scores are normalized by converting them to a normalized risk score.
Referring to
In a preferred embodiment, the invention normalizes risk factors by using a non-linear function that minimizes the impact of low risk entities, gradually increases in a medium risk range, and emphasizes the contribution to overall risk of high and critical risk entities to facilitate rapid detection of more serious attacks. Preferably, the invention uses a normal cumulative distribution function for normalization. This distribution, which is illustrated in
where μ and σ are approximated using robust estimates from the top 50% of the distribution: μ=mediani(xi), σ=mediani(|xi−medianj(xj)|). This normalization formula is fairly stable in regards to a change in underlying raw risk scores, and parameters can be recalculated when the new risk vector is added or risk landscape has changed significantly.
Once entity risks are known, appropriate steps may be taken to mitigate threats, such as educating high risk users on safe browsing to protect against malware or restricting their access to critical assets. Additionally, risk indications may be further aggregated to determine the risks to a department, such as sales or engineering, or to a division or other group, as well as to the enterprise as a whole.
Organizational risk 150 (
While the foregoing has been with respect to particular embodiments of the invention, it will be appreciated that changes to these embodiments may be made without departing from the principles of the invention, the scope of which is defined by the appended claims.
This application claims the benefit of U.S. application Ser. No. 62/110,031, filed Jan. 30, 2015, the disclosure of which is incorporated by reference herein.
| Number | Date | Country | |
|---|---|---|---|
| 62110031 | Jan 2015 | US |