Patent Application
20070234424
The invention pertains to identification of Internet worm propagation.
Malicious computer worms (or Internet worms) are a danger to any computer that is accessible via a computer network, such as the Internet. A computer worm is a self-replicating program, similar to a computer virus. However, unlike a virus, which attaches itself to and infects an executable file on a computer, a worm is self-contained and does not need to be part of another program to propagate itself.
Worms are often designed to exploit the file transmission capabilities of many computers. A worm uses a network to send copies of itself to other systems and it does so without any necessary human intervention, such as forwarding by email, which is a common method of spreading a virus. Scan-based worms use a form of scanning (transmission of packets) from an infected host to a potential new host as a propagation technique. Based on the potential host's response to this scan (i.e., does the potential host respond positively, or does the response indicate that the potential host will not accept additional packets from the infected host), the infected host determines whether to spread the worm to the potential host. It is also possible that a worm can be carried in a single packet. In this situation, the infected host transmits the packet to another address without the need for a response from the potential new host.
Typical approaches to preventing a worm outbreak involve worm detection, dissection and signature development. Signature development occurs once the worm has been identified, and a common pattern is found which can be used to identify the worm. This signature must then be propagated throughout the network, either to a firewall running security software or to each individually connected computer running a certain security program. Once the security program receives the signature, the database of signatures the security program recognizes as malicious is updated, and the computer running the security program is protected against the identified worm. But this approach does not address the case of previously unidentified worms for which no signature has been identified.
Previously unidentified, fast spreading worms are a reality, as amply demonstrated by worms such as the Slammer worm. The release and propagation of the Slammer worm in 2003 was a revolutionary event in the study of computer worm propagation. It not only demonstrated in an unprecedented way the scale and disruption that is possible in the real world with a relatively compact worm, it also showed the ineffectiveness of current techniques in detecting and countering these new fast spreading worms. More specifically, in the early phase of Slammer propagation, it doubled in size every 8.5 seconds. It reached a maximum scan rate of 55 million addresses per second and was able to infect more than 90 percent of vulnerable hosts within 10 minutes. In the end, even though Slammer carried no malicious payload and its main damage was in network resource (bandwidth and CPU) consumption, it served as a wake-up call to network administrators and the computer security industry.
With these kinds of fast spreading worms, the traditional approach of signature-based detection is no longer sufficient. Worms can infect all vulnerable hosts well before a signature can be identified. Several approaches have been proposed utilizing non-signature based detection means. One such approach detects a worm by monitoring the correlation between the incoming and outgoing packets at a network connection. More specifically, this approach studies the correlation of the payloads and packet headers of the incoming and outgoing packets. However, this correlation is not always reliable. Specifically, the technique was most effective against earlier worms that used a fixed destination port, or a portion of the network address specifying the port where the packet is received on the network connection, which made correlation studies easier as a single destination port could be monitored across the network. However, recent worm attacks randomize the destination port on the network connection. This renders monitoring of destination port incoming and outgoing packets and studying the correlation between the two packet types less reliable for worm detection.
Another non-signature based approach involves detecting a worm by identifying the exponential growth trend of scanning rates on a particular network connection. However, this process requires studying the growth trend over a given interval of time. Different worms have different propagation times. For example, a worm may inhabit a host computer for an hour before propagating to a new host. If the wrong interval of time is chosen to study the growth trend, then relevant information relating to the growth trend is missed and a worm cannot be effectively detected.
What is needed is a fast method to detect worms lacking known signatures. This method should be accurate and robust (i.e., it must quickly and accurately identify different propagation characteristics of different worms), and work quickly enough so that a worm can be detected at the inception of the worm spread, before its propagation hits its exponential growth rate.
In accordance with the principles of the present invention, a new worm detection technique is presented that utilizes a process to detect the outbreak of a new worm without knowing the signature of the worm. Changes in the traffic pattern of unsolicited packets are detected, and any changes in traffic patterns are analyzed to determine if they are consistent with changes in traffic associated with worm propagation. More specifically, traffic arrival patterns are monitored, primarily for unsolicited traffic, i.e., traffic coming into a computer network connection that was not first requested. Next, changes in the traffic patterns are analyzed. During this analysis, certain patterns of growth rates relating to the unsolicited traffic that are indicative of the presence of worm propagation are searched for, such as an exponential growth rate of unsolicited traffic from numerous senders. When such a pattern is detected, it is assumed that a worm is present so that measures can be implemented to halt its progress.
In accordance with the present invention, a non-signature based method for detecting Internet worms is presented. By monitoring and analyzing traffic patterns at a network connection, a worm can be detected.
Conventional security software identifies malicious packets based on the signature of the packet, or a unique identifier for each packet. However, new worms are being designed which can infect millions of hosts well before a signature can be found. For example, client 130 is infected with a previously unidentified worm such that client 130 becomes a scanner, meaning the worm sends out a scan, which is a series of packets intended to poll a potential host computer. Client 130 scans clients 102, 104 and 106. Since the worm has no known signature, neither the firewall nor the desktop security program would recognize that the packets are malicious. If clients 102, 104 and 106 respond positively to the scan, client 130 passes the worm to clients 102, 104 and 106, and they become new hosts.
The system now proceeds to step 205 where any changes in traffic arrival patterns are determined. Though all changes in traffic arrival patterns may not be due to worm propagation, worm propagation usually results in traffic arrival pattern changes with certain similar characteristics. As described in further detail below, the system uses cumulative summing, or CUSUM, a common statistical analysis tool used to detect changes in data sets, to study the arrival rates to determine any changes. CUSUM will detect a trend of increasing unsolicited packet arrival rate.
The process continues to decision step 207, in which, if CUSUM has detected a change in the arrival rates, the process continues to step 210. If CUSUM has failed to detect a change, flow returns to step 200.
If a change is detected in Step 205, the system proceeds to Step 210 where the changes are analyzed to determine if the changes are related to worm propagation. Specifically, the changes are analyzed to determine whether the changes have some exponential growth patterns in arrival rates. A Maximum Likelihood Estimation (MLE) is used to produce a non-stationary Poisson process and estimate its rate. Poisson processes are commonly used in statistical analysis to examine the number of times an event happens during a given time interval, where the probability for the event occurring is constant with respect to time. An alarm will trigger when the MLE yields a significant increase in propagation rate with a high level of confidence.
Steps 205 and 210 are further explained herein below. In step 205, first the inter-arrival times of the unsolicited packets are determined. Tn denotes the arrival time of the n-th unsolicited packet in a t-sample (a sample taken at most once every t seconds), and Xn=Tn−Tn-1, is the inter-arrival time where T0=0. It is assumed that the inter-arrival times {Xn: 1≦n≦nw} before the worm starts are independently and identically distributed with mean μ, where Tnw represents the time of the first worm scan. After a worm arrives, the inter-arrival times {Xn: nw≦n≦∞} should have a decreasing mean that is less than μ. This shift in the distribution of inter-arrival times may be considered a change point in statistical terms and CUSUM is designed for detecting changes from one distribution to another such as this change in inter-arrival times.
The CUSUM scheme can be applied as follows. Set S0=0 and define
Sn=max(0,Sn-1+μ−Xn−pμ), n=1, 2, . . .
where p is dependant on the expected drop in mean inter-arrival times due to a worm. Typically, pμ is set to about half the size of the drop in mean inter-arrival time that is crucial to detect a change in arrival rates quickly. A change of inter-arrival time is signaled whenever Sn exceeds a certain threshold h. The theory behind CUSUM is that, if the mean of Xn shifts from μ to something smaller than μ−pμ at sample nw then Sn will tend to accumulate positive increments after nw and thus eventually cross the threshold h and signal a change. In practice, μ is not known, as arrival times can vary due to network conditions; but an estimate, such as an Exponentially Weighted Moving Average (EWMA) can be used in its place. The EWMA is based on the median of an initial sample of inter-arrival times.
Choosing the threshold parameter, h, requires trading off between detection delay (i.e., sensitivity) and the false detection rate. Small values of h provide quick detection when changes are present but also give more false alarms. The threshold h can be calculated from the expected time between false alarms, known as the Average Run Length (ARL) in quality control.
As seen in the flowchart, the CUSUM process used in step 205 is not used to directly trigger a worm alarm, but only as a first stage toward worm detection. As previously noted, if the CUSUM value Sn exceeds threshold h, the process proceeds to step 210 in which the detected changes are analyzed and a worm propagation model is estimated. However, if a new worm outbreak is in progress, it is probable that some time has elapsed between the outbreak and the CUSUM signal. When step 205 detects an unusual increase in unsolicited network traffic, there are three relevant cases that this increase might indicate. Let Tn0 denote the most recent time (prior to the current signal) when the CUSUM transitioned from a value of 0 to a positive value. If a worm exists, its arrival is most likely earlier than Tn0 (hereinafter Case 1). However, it is possible for a worm to arrive between Tn0 and the CUSUM signaling time (hereinafter Case 2). This happens rarely and the lag between worm infection and the CUSUM signal transitioning from 0 to a positive value will most likely be small, e.g., on the order of second. Of course, it is also possible that no worm exists (hereinafter Case 3), which statistically is the most likely case. Let us first focus on the statistical estimation of the worm propagation model based on Case 1. It will be shown below that this also includes Case 3 and also serves as a good approximation for Case 2.
Scanner arrivals in a t-sample before a worm outbreak are well-modeled as a Poisson process with rate b(t) that changes slowly with time. Scanners that arise from a fresh worm outbreak can be modeled as a non-stationary Poisson process with rate:
λ(t)=aer(t−tw)I(t≧tw)
where tw is the time when the first worm scan arrives; a is the expected number of worm scanner arrivals in the first second; r is the exponential propagation rate; and I(x) is an indicator function having value 1 when x is true and 0 otherwise. It is assumed that any background scanners (non-malicious scanners) and the ones caused by a new worm are independent. The superposition of background and worm scanners is thus modeled as a non-stationary Poisson process with rate:
λ(t)=b(t)+aer(t−tw)I(t≧tw).
Because the background traffic is approximately stationary, its rate b(t) can be estimated easily using local averaging. Propagation characteristics are described by the parameters a and r that depend on the efficiency of the worm and the size of the network being monitored. Although a is not identifiable (i.e., cannot be estimated statistically) when tw is unknown, the exponential rate r is identifiable. A worm alarm is triggered when the data indicates with high confidence that r is significantly higher than a small tolerable rate r0.
For simplicity, assume that the worm starts at 0 (i.e., tw=0), unsolicited scanners arrive at times T1, T2, . . . according to a Poisson process with rate λ(t)=b+aert, t≧0, and the corresponding CUSUM sequence S1, S2, . . . remains below the threshold h until some arrival Tn0 (n0≧1) when the CUSUM exceeds h and therefore causes flow to proceed to step 210 in which the change is to be further analyzed.
With respect to step 210, let us define
Theorem 1. Let T1, T2, . . . denote consecutive arrival times from a Poisson process with positive rate λ(t)=b+aert beginning at t=0. Define
Corollary 1. Under the conditions of Theorem 1 and assuming that a>0, the parameters
The exception a=0 corresponds to no worm and in this case the propagation rate r has no meaning. Fortunately, for the purpose of worm detection, r is the most interesting parameter and it can be estimated by maximum likelihood inference as discussed next.
Let
the maximum likelihood estimates (MLE) are defined as
({circumflex over (r)},
Let θ=(r,
Theorem 2. Under the conditions of Theorem 1, if θ is bounded, then as n→∞,
{circumflex over (θ)}→θ,
in probability and
√{square root over (n)}({circumflex over (θ)}−θ)→N(0,I(θ)−1),
in distribution where I(θ) is the information matrix,
and can be estimated consistently by
The MLE {circumflex over (r)} and its estimated asymptotic variance are used repeatedly in the second stage to test whether r is significantly positive. In particular, r>r0 is tested against r≦r0, where r0 (say 0.0001) is the maximal rate that can be ignored. Let se({circumflex over (r)}) be the asymptotic standard error of {circumflex over (r)}, that is,
se({circumflex over (r)})=√{square root over ([Î−1]11/n)}.
Since Zn≡({circumflex over (r)}−r0)/se({circumflex over (r)}) is asymptotically normally distributed with mean 0 and variance 1 under the null hypothesis r=r0, the second stage declares a worm outbreak when Zn>qc, where qc is a threshold such as the 99.99 percentile of the standard Normal distribution. For example qc=3.8 is the 99.99% quantile of the Normal distribution.
In most CUSUM monitoring applications, the CUSUM statistic is reset to zero after a signal is triggered. In the present algorithm, however, a large CUSUM is required for the step 210 of
Although scanner arrivals, for the most part, resemble a locally stationary Poisson process, outliers do occasionally occur in arrival traces. These are cases in which the inter-arrival time between scanners is abnormally large for one reason or another. These outliers never trigger a false alarm because the MLE does not yield a large r in step 210. However, the outliers can easily lead to a CUSUM signal and thus needlessly trigger the MLE computations.
To reduce the impact of outliers in creating such false alarms, the algorithm may implement the following random tail-draw technique. Let μn-1 be the most recent exponentially weighted moving average (EWMA) estimate of E(Xn). If Xn lies outside of the 0.01% and 99.99% percentiles of the exponential(μn-1) distribution, then it is replaced with a random draw {tilde over (X)}n from the corresponding distribution for the purpose of calculating Sn.
A trace of the Slammer Worm outbreak was used to test the algorithm.
It should be clear to persons familiar with the related arts that the process, procedures and/or steps of the invention described herein can be performed by a programmed computing device running software designed to cause the computing device to perform the processes, procedures and/or steps described herein. These processes, procedures and/or steps also could be performed by other forms of circuitry including, but not limited to, application-specific integrated circuits, logic circuits, and state machines.
Having thus described a particular embodiment of the invention, various alterations, modifications, and improvements will readily occur to those skilled in the art. Such alterations, modifications, and improvements as are made obvious by this disclosure are intended to be part of this description though not expressly stated herein, and are intended to be within the spirit and scope of the invention. Accordingly, the foregoing description is by way of example only, and not limiting. The invention is limited only as defined in the following claims and equivalents thereto.
