Multistage system and method for analyzing obfuscated content for malware

Abstract

A malware detection system configured to detect suspiciousness in obfuscated content. A multi-stage static detection logic is utilized to detect obfuscation, make the obfuscated content accessible, identify suspiciousness in the accessible content and filter non-suspicious non-obfuscated content from further analysis. The system is configured to identify obfuscated content, de-obfuscate obfuscated content, identify suspicious characteristics in the de-obfuscated content, execute a virtual machine to process the suspicious network content and detect malicious network content while removing from further analysis non-suspicious network content.

Description

FIELD OF INVENTION

The present invention relates generally to computing systems, and more particularly to systems and methods of detecting and classifying malicious content.

GENERAL BACKGROUND

Over the last decade, malicious software (malware) has become a pervasive problem for Internet users. In some situations, malware is an exploit, in the form of a program or other object, which is embedded within downloadable content and designed to adversely influence or attack normal operations of a computer. Examples of different types of exploits may include bots, computer viruses, worms, Trojan horses, spyware, adware, or any other programming that operates within an electronic device (e.g. computer, tablet, smartphone, server, router, wearable technology, or other types of electronics with data processing and network capability) without permission by the user or an administrator. In some situations, malware is obfuscated to hide the purpose of the malware, while in other situations, non-malicious software is obfuscated. For instance, software may be obfuscated for non-malicious purposes including reducing the size through compression or protecting against unauthorized access through encryption.

Obfuscation of malicious objects may increase false negatives when a malware detection system analyzes the objects. For instance, exploits that are located in an obfuscated object may be hidden from detection by the malware detection system. As an illustrative example, an exploit may be placed within an executable object whose contents have been obfuscated, and carried into an electronic device undetected by conventional malware detection software, such as antivirus programs (e.g. an exploit inserted into an object's JavaScript and the contents obfuscated to hide malicious code patterns).

Known malware detection systems employing anti-virus scanning approaches are adequate for detecting malware in un-obfuscated content but often fail to detect malware in obfuscated content. This results in a high level of false negatives. Other known malware detection systems detect malicious objects in either obfuscated or not obfuscated content by executing the content in runtime environments often established by virtualization technology and isolated from production environments for security purposes, an approach which often avoids high levels of false negatives. Unfortunately, such systems typically require substantial computing resources to establish run-time (virtual) environments required to safely process the content for an adequate period of time to observe malicious behavior. Hence, it would be desirable to provide a malware analysis scheme capable of efficiently detecting malware in obfuscated content, while avoiding high levels of false positives and high computing resource demands.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the disclosure are illustrated by way of example and not by way of limitation to the figures of the accompanying drawings. Like references in the figures indicate similar elements:

FIG. 1 is an exemplary block diagram illustrating a communication system implementing a plurality of malware detection (MCD) systems each having a multi-stage static analysis system in accordance with one embodiment.

FIG. 2 is an exemplary block diagram of logic associated with one of the multi-stage static analysis systems of FIG. 1 in accordance with one embodiment.

FIG. 3A is an exemplary block diagram of the preprocessor of FIG. 1 in accordance with one embodiment.

FIG. 3B is an exemplary block diagram of the post-processor of FIG. 1 in accordance with one embodiment.

FIG. 4 is an exemplary block diagram of a flowchart illustrating a method of operation of a multi-stage static analysis system in accordance with one embodiment.

DETAILED EMBODIMENT

Various embodiments of the disclosure provide a multistage malware analysis scheme that takes into account whether content is obfuscated in automatically analyzing that content for malware. Embodiments of the analysis scheme attempt to de-obfuscate such content using an emulator, and then statically analyze the de-obfuscated content for suspiciousness prior to processing only that content found to be suspicious in a dynamic analyzer. The dynamic analyzer processes the suspicious content in a virtual runtime environment (e.g., using one or more virtual machines configured with a software profile suitable for the type of content). In this way, embodiments of the invention provide efficient detection of malware embedded in obfuscated content, while avoiding high levels of false positives and high computing resource demands (e.g. CPU processing time, temporary memory space, network bandwidth, etc.).

In one embodiment, the multistage malware analysis scheme initially determines whether content is either suspicious or obfuscated using a preprocessor that statically analyzes the content. If determined to be suspicious and not obfuscated, the content is next analyzed using a dynamic analyzer. If determined to be obfuscated by the preprocessor, the content is de-obfuscated by an emulator into a form that may be statically analyzed using a post-processor. If the post-processor determines that the de-obfuscated content from the emulator is suspicious, the corresponding original obfuscated content whose de-obfuscated form was found to be suspicious is next analyzed using a dynamic analyzer. If the de-obfuscated form from the emulator is determined to be not suspicious by the post-processor, the content is filtered from further analysis and not analyzed by the dynamic analyzer.

In embodiments of the disclosure, a multistage malware detection system may include a preprocessor, an emulator, and one or more post-processors. The preprocessor receives content for static analysis, for example, from a network interface, and may include an obfuscation detector and suspiciousness determining logic. The obfuscation detector determines whether any objects in the content are obfuscated. The suspiciousness determining logic determines whether any objects in the content are suspicious in that they have one or more characteristics (e.g., attributes) associated with malware. In the emulator, the content provided by the obfuscation detector is emulated to produce a de-obfuscated form. The de-obfuscated content is further processed by one or more post-processors which determine the suspiciousness of the de-obfuscated content. Each post-processor includes suspiciousness determining logic. If suspiciousness as determined by the preprocessor or post-processor is below a threshold of suspiciousness, the content may be, in one embodiment, deemed benign and filtered from further analysis and not provided to the dynamic analyzer. If the suspiciousness from the post-processor is above the threshold, the content is scheduled for dynamic analysis.

In various embodiments, the suspiciousness determining logic of the preprocessor and post-processor may include one or more of (depending on the embodiment) an indicator match engine, a protocol anomaly checker, or heuristics engine (e.g. a rules checker, etc.). The preprocessor and post-processor may have the same component checkers and engines or different components. More specifically, the indicator match engine of the preprocessor determines suspiciousness based on matches of the content with a first set of indicators of suspiciousness stored in memory. The post-processor may have a second set of indicators of suspiciousness that is distinct from the first set malicious indicators used by the preprocessor or may utilize identical indicators. The protocol anomaly checker checks the content against applicable protocols (e.g., communication protocols) or industry standards (e.g. industry protocols) to determination suspiciousness based on deviations from the protocol(s). The heuristics engine performs a statistical analysis on the content to assess whether the content has (or characteristics of objects therein have) a statistical correlation with known malware. The rules checker determines whether detected characteristics of the content violate one or more rules such as, for example, access control rules or rules with respect to blacklisted or whitelisted addresses. The rules in some embodiments may be defined by a network or security administrator and entered via a suitable user interface. The rules are developed from past experience with detected malware and malware families.

In some embodiments, the preprocessor and post-processor can be combined into a single component which receives content from both the network interface and the emulator, and statically analyzes the content. In other embodiments, these components may be implemented as separate components, and even as separate electronic devices each including processing units which, for example, may be located within a single data center or may be located geographically distant from one another and connected by a communication link or a network (cloud).

In embodiments of the invention, processing and analysis of the suspicious content in the dynamic analyzer can be prioritized based on level of suspiciousness. The level of suspiciousness, depending on the embodiment, can be expressed as a score or weighting associated with a confidence level (e.g., probability) that the content is malicious. Accordingly, content queued for dynamic analysis and associated with a higher weight or score may be processed by the dynamic analyzer ahead of queued content with a lower weight or score. In some embodiments, both the preprocessor and post-processor include score generation logic for associating a score with each object constituting the original, un-obfuscated content analyzed by the preprocessor and with each object in the de-obfuscated form of the originally obfuscated content analyzed by the post-processor, respectively. In other embodiments, score generation logic can be interposed between the post-processor and the dynamic analyzer or within a dynamic analysis scheduler so as to generate a single weighting or score for all output from the preprocessor and post-processor.

More specifically, an embodiment of a preprocessor determines suspiciousness of content based on indicators of suspiciousness. The preprocessor conducts static analysis operations to identify characteristics of the content that may indicate malware by matching or finding a high correlation (above a threshold) with patterns of indicators associated with malware or families of malware. In some embodiments, the indicators of malware may be abstracted patterns of known malicious or benign content identifiers correlated to maliciousness (e.g. whitelist or blacklist of patterns). Herein, the static analysis operations are generally referred to as “malware checks”. Content may be classified as “suspicious” by the preprocessor when one or more characteristics associated with malware are identified during static analysis operations conducted on the content indicate a probability that the content includes malware above a select threshold. This determination may be stored as a first suspiciousness score. Obfuscation (e.g. compression, encryption, etc.) of the content may itself be an indicator of suspiciousness. If the suspiciousness assigned to the content after preprocessing is below the threshold, the content will be filtered from further analysis and not presented for dynamic analysis. If the suspiciousness content after preprocessing is above the threshold, the content will be further processed. Obfuscated content will be provided by the preprocessor to the emulator, and un-obfuscated content will be provided by the preprocessor to the dynamic analyzer.

An embodiment of an emulator will process (e.g., simulate execution of) the obfuscated content provided by the preprocessor to produce a de-obfuscated representation of the obfuscated content. The emulator is configured to emulate the operations associated with the processing of the content in the context of an emulated computer application. As an optional feature, emulation logic may process a pre-determined list of functions of an application, which, in some embodiments, can be selectively modified by an IT or security administrator. In particular, the emulator intercepts (or “hooks”) function calls, events and messages (e.g., API calls via specific application specific interfaces) and, in response sends values appropriate to the content and context of the emulation as inputs to the emulator (e.g. timing events, keyboard events, etc.). The de-obfuscated representation of the network content produced by the emulation logic will be processed by the post-processor. The information exchanged during the “hooking” operations as well as output data from the content under analysis are provided to the post-processor where they are statically analyzed for malware characteristics or indicators.

In an embodiment of the post-processor, the de-obfuscated representation produced by the emulator is processed and suspiciousness is determined, as noted above, based on a second set of indicators of suspiciousness. The second set of indicators of suspiciousness may be identical to the first set of indicators of suspiciousness or may have indicators peculiar to the de-obfuscated representation of the content. The second suspiciousness determination may lead to and may be stored as a second suspiciousness score. In an embodiment of the disclosure, if the suspiciousness determined for the de-obfuscated representation of the content by the post-processor is below a threshold of suspiciousness, the content may be filtered from further analysis. In an embodiment of the disclosure, the second threshold for suspiciousness may be identical to the first threshold for suspiciousness. In an alternative embodiment of the disclosure the second threshold for suspiciousness may be different from the first threshold for suspiciousness. If the suspiciousness determined for the de-obfuscated content is above the threshold of suspiciousness, the content is scheduled, in a scheduler, to be analyzed by a dynamic analyzer.

In an embodiment of the scheduler, the suspiciousness score(s) generated in the preprocessor and post-processor is used to determine a level of priority associated with corresponding content for use in scheduling the content for analysis by the dynamic analyzer. The scheduler may use for these purposes, depending on the embodiment, the first suspiciousness score assigned to the obfuscated content, the second suspiciousness score assigned to the de-obfuscated content, both suspiciousness scores, the individual indicators of suspiciousness used by the preprocessor and post-processor to determine the priority, or a combination of one or more of the foregoing. The level of priority is determined in the scheduler by assigning a higher priority to content that is more suspicious.

According to yet another embodiment of the disclosure, a forensic malware detection method or system may receive content for analysis that has been identified in advance as having a high level of probability of containing malware, and may implement a multistage malware detection scheme in accordance with any of the above embodiments to confirm or verify the existence of malware in the content under analysis.

I. General Architecture

Referring to FIG. 1, an exemplary block diagram of a communication system 100 deploying a plurality of malware content detection (MCD) systems 110₁-110_N (N>1, e.g. N=3) communicatively coupled to a management system 120 via a network 125 is shown. In general, management system 120 is adapted to manage MCD systems 110₁-110_N. For instance, management system 120 may be adapted to cause one or more malware identifiers, each of which being information representative of prior detected malware, to be shared among some or all of the MCD systems 110₁-110_N for use in malware checks. Such sharing may be conducted automatically or manually uploaded by an administrator. Also, such sharing may be conducted freely among the MCD systems 110₁-110_N or subject to a subscription basis.

Herein, according to the embodiment illustrated in FIG. 1, a first MCD system 110₁ is an electronic device that is adapted to analyze information associated with network traffic routed over a communication network 130 between at least one server device 140 and at least one client device 150. More specifically, the first MCD system 110₁ is configured to conduct static analysis of information (e.g., an object that is part of message(s) transmitted via the network traffic) received via communication network 130 and, where applicable, classify the object with different “malicious” scores. An object may be classified with a first level (e.g. “suspicious”—assigned a score less than or equal to a first threshold) when at least one characteristic (or pattern of characteristics) identified during static analysis operations conducted on the object indicates a certain level of probability that the object includes malware. Similarly, the object may be classified with a second level (e.g. “malicious”—assigned a score greater than or equal to a second threshold greater than the first threshold) when at least one characteristic observed during static analysis operations conducted on the object indicates a certain greater level of probability that the object includes malware.

In an embodiment, static analysis is conducted by static analysis engine 170 within the first MCD system 110₁. The static analysis engine 170 is configured to conduct a first static analysis operation on an object extracted from the network traffic to determine whether the object is suspicious in that it has a certain probability of being malicious. As part of that analysis the static analysis engine 170 determines whether the object contains obfuscated content. The static analysis engine 170 may determine that the object is obfuscated based on detecting, in the content, indicia signaling that the content has been compressed, encrypted or otherwise encoded. A determination that the content has been obfuscated can be used to increase the probability that it is suspicious and support a classification of maliciousness. If the object is found to not be suspicious (e.g. benign), no further analysis need be performed on the object. On the other hand if the object is found to be suspicious the static analysis engine 170 will provide the object to an emulator 260 to render the obfuscated code to de-obfuscated format. The emulator processes the obfuscated content into de-obfuscated form by “hooking” to the Application Programming Interfaces (APIs) calls from the content processed in an emulated application and substituting the appropriate values when functions are called during the processing of the content. The de-obfuscated content obtained after emulation is analyzed to determine if the content correlates with one or more malware characteristics. The level of correlation (e.g., exact pattern match, or for example, matching a majority of the pattern) may be “factory set” or may be adjustably set (e.g. by an administrator). In some embodiments an object of network content may have one or more embedded objects (e.g. sub-objects). The malware checks are conducted by the static analysis engine 170 on the one or more embedded objects within the object in order to potentially classify the overall object as “malicious,” depending on whether any embedded object within the object is determined to have at least one characteristic associated with malware or whether the suspiciousness of the embedded objects in the aggregate is sufficient to determine whether the overall object requires further analysis.

The communication network 130 may include a public network such as the Internet, in which case an optional firewall 155 (represented by dashed lines) may be interposed between communication network 130 and client device 150. Alternatively, the communication network 130 may be a private computer network such as a wireless telecommunication network, wide area network, or local area network, or a combination of networks.

The first MCD system 110₁ is shown as being coupled with the communication network 130 (behind the firewall 155) via a network interface 160. The network interface 160 operates as a data capturing device (referred to as a “tap” or “network tap”) that is configured to receive data traffic propagating to/from the client device 150 and provide content from the data traffic to the first MCD system 110₁.

In an embodiment, the network interface 160 receives and duplicates the content that is received from and provided to client device 150 normally without an appreciable decline in performance in the server device 140, the client device 150, or the communication network 130. The network interface 160 may duplicate any portion of the content, for example, one or more objects that are part of a data flow or part of the payload contained within certain data packets, or the like.

In some embodiments, the network interface 160 may capture metadata from network traffic intended for client device 150. This metadata may be used, at least in part, to de-obfuscate a corresponding file. For instance, the metadata may include one or more keys that can be used to de-obfuscate the file. The metadata may be advantageously used by the scheduler 180 to retrieve and configure a virtual machine (VM) to correspond to the pertinent features of the client device 150.

It is contemplated that, for any embodiments where the first MCD system 110₁ is implemented as a dedicated appliance or a dedicated computer system, the network interface 160 may include an assembly integrated into the appliance or computer system that includes network ports, network interface card and related logic (not shown) for connecting to the communication network 130 to non-disruptively “tap” data traffic propagating through firewall 155 and provide a copy of the data traffic to the static analysis engine 170. In other embodiments, the network interface 160 can be integrated into an intermediary device in the communication path (e.g., firewall 155, router, switch or other network device) or can be a standalone component, such as an appropriate commercially available network tap. In virtual environments, a virtual tap (vTAP) can be used to duplicate objects from virtual networks.

Referring still to FIG. 1, first MCD system 110₁ may include a static analysis engine 170, a database 175, a scheduler 180, a storage device 185, and a dynamic analyzer 190. In some embodiments, the network interface 160 may be contained within the first MCD system 110₁. Also, static analysis engine 170, scheduler 180 and/or dynamic analyzer 190 may be software modules executed by one or more processors 200 that receive one or more objects within the content and perform a multi-stage static analysis on the objects, which may involve accessing one or more non-transitory storage mediums operating as database 175, storage device 185 and/or reporting module 195. In some embodiments, the static analysis engine 170 may be one or more software modules forming a static framework 172, where such software modules are executed by a processor 200. The scheduler 180 and the dynamic analyzer 190 may be one or more software modules executed by the same or a different processor, where these different processors are possibly located at geographically remote locations, located within the same processor package (e.g. different processor cores) and/or communicatively coupled for example via a network.

In general, the static analysis engine 170 performs static analysis on an object to determine whether it may include exploits, vulnerable functions, and/or malicious patterns. More specifically, as illustrated in FIG. 1, the static analysis engine 170 receives a duplicate copy of one or more objects associated with network traffic received from the network interface 160 and statically scans the content of the object(s) for malware identifiers. In other words, the static analysis engine 170 performs a first static analysis by comparing the contents of each object with known characteristics of malware, which may include finding correlations with characteristics associated with malware, to determine the existence of malware. Characteristics associated with malware may include exploit patterns, names of vulnerable functions, heap spray patterns, etc. Alternatively, in in-line deployments, the network interface 160 may provide the original (rather than a duplicate) of the portion of content to the static analysis engine 170 for malware analysis. While the balance of this specification may refer to the duplicate, this alternative embodiment may serve to implement the concepts later described herein.

If the comparison reveals a correlation between the contents of the object and one or more malware identifiers (hereinafter referred to as a “match”), which denotes a malicious event, the static analysis engine 170 determines whether the corresponding object is “suspicious” and assigns a score to the object. The malware checks applied by the static analysis engine 170 may be based on data and/or rules stored in the heuristics database 175.

Upon detecting a match, the score generation logic 174 may assign a score (e.g., an indicator of the likelihood of the analyzed object including malware) to the object analyzed with the static analysis engine 170. The score generation logic 174 will determine a score for the analyzed content using the results of the preprocessor 250 or post-processor 270. This may be performed in some embodiments by assigning a score to individual objects found in the content, and then mathematically combining the object-level scores to obtain an overall score for the content. Thereafter, the content and score are routed from the scanning engine to the dynamic analyzer 190 for use in further analysis to confirm the presence of malware within the content. The content may be scheduled for analysis in the dynamic analyzer 190 by a scheduler 180.

The static analysis engine 170 conducts emulation operations in an emulator 260, where the processing of an object is emulated and an output associated with such processing may be statically scanned to determine if portions of the output match any of the pre-stored malware identifiers. The emulator may be, for example, a “light weight” run-time environment. The run-time environment is described as “light weight” because it does not actually run applications that normally are used in processing objects. The run-time environment may be provided by an emulated application. For example, a JavaScript™ object may expect a browser application, and therefore, for emulation, the MCD system 110₁ relies on an emulator to present certain simulated or virtualized features and system responses associated with that application as expected by the object being analyzed. The MCD system 110₁ then monitors characteristics of the object within the emulator, for example, by intercepting or “hooking” function calls and comparing the hooked calls with those expected of similar objects when processed. Any unexpected function calls are regarded as anomalies that may indicate the object is malicious. The object is then assigned a score based on the presence or absence of such anomalies, and, in some embodiments, the type of the call or other characteristics of the observed anomalies.

After static analysis, the content may be presented to the dynamic analyzer 190 for more in-depth dynamic analysis using virtual machine technology for processing of the object in a more complete run-time environment in which one or more applications are executed (not just emulated). The ensuing processing may be conducted for a period of time that is either fixed or variable, and may be set manually by an administrator or automatically by the MCD system 110₁, for example, based on the type of object, queue length of objects to be analyzed or other analysis parameters. For this purpose, the static analysis engine 170 communicates with the scheduler 180.

The scheduler 180 may retrieve and configure a virtual machine (VM) with appropriate computer software (e.g. operating system and applications) and features to correspond to the pertinent characteristics of a client device. In one example, the scheduler 180 may be adapted to configure the characteristics of the VM to correspond only to those features of the client device 150 that are affected by the data traffic copied by the network interface 160. The scheduler 180 may determine the features of the client device 150 that are affected by the content by receiving and analyzing the network traffic from the network interface 160. Such features of the client device 150 may include ports that are to receive the content, certain device drivers that are to respond to the content, and other devices coupled to or contained within the client device 150 that can respond to the content. In another embodiment of the disclosure, where the network content is processed forensically, the scheduler 180 will configure the VM to correspond to the features indicated by analyzing content received by the network interface 160. Content received by the network interface 160 may include metadata which may be analyzed to determine protocols, application types and other information that may be used to retrieve and configure the VM by the scheduler 180.

In another embodiment of the disclosure, the static analysis engine 170 may determine the features of the client device 150 that are affected by the network traffic by receiving and analyzing the content from the network interface 160. The static analysis engine 170 may then transmit the features of the client device to the scheduler 180 or dynamic analyzer 190.

The dynamic analyzer 190 is adapted to execute one or more VMs to simulate the receipt and/or execution of different “malicious” objects within content under analysis (analyzed content) within a run-time environment as expected by the type of object, as noted above. The run-time environment may be one selected to correspond to one that is appropriate to the content, or that is prevalently provided by client devices, or, in alternative embodiments, one that can be provided by the client device 150 in particular. Furthermore, the dynamic analyzer 190 analyzes the effects of such content upon the run-time environment, such as the client device 150. Such effects may include unusual network transmissions, unusual changes in performance, and the like. This detection process is referred to as dynamic malicious content detection.

The dynamic analyzer 190 may flag the malicious content as malware according to the observed behavior of the VM. The reporting module 195 may issue alerts indicating the presence of malware, and using pointers and other reference information to identify which content may contain malware. Additionally, the server device 140 may be added to a list of malicious network content providers, and future network transmissions originating from the server device 140 may be blocked from reaching their intended destinations, e.g., by firewall 155.

In an alternative embodiment of the disclosure the static framework 172, includes a controller, a deconstruction engine, and a post-processor. The controller receives the content to be analyzed and detects the type of the content. The controller determines, based on whether the controller can access the native code of the content, an analysis technique to be used by the deconstruction engine. The deconstruction engine may perform a variety of techniques to analyze the content received from the controller. If the controller determines that the content cannot access the native code of the content, the deconstruction engine will select an analysis technique that implements a de-compiler to access the native code of the content. The deconstruction process returns the native code (e.g. plain-text, etc.) as a deconstructed representation of the content. The deconstructed representation is processed in the post-processor 270 which receives the de-constructed content from the de-constructor. The post-processor determines if the de-constructed content matches any known malware characteristics. The malware characteristics may include malicious obfuscation such as, for example, functional obfuscation, using the frequency, n-gram, length of strings or numbers and size of decompiled files. Matches to the malware characteristics are indicators of suspiciousness and may amount to maliciousness. If these values exceed a certain threshold, the file is classified as malicious. Each component of the static framework 172 may be stored in persistent storage 230 and executed by the processors 200.

Of course, in lieu of or in addition to static analysis operations being conducted by MCD systems 110₁-110_N, it is contemplated that cloud computing services 135 may be implemented to perform such static analysis: de-obfuscate content for static analysis, perform emulation of obfuscated content to render de-obfuscated content, and conduct static scans on one or more objects within the de-obfuscated content as described herein. In accordance with this embodiment, MCD system 110₁ may be adapted to establish secured communications with cloud computing services 135 for exchanging information. The dynamic analysis may also be conducted in the cloud.

Referring now to FIG. 2, a block flow diagram of logic associated with MCD system 110₁ is shown. MCD system 110₁ includes one or more processors 200 that are coupled to communication interface logic 210 via a first transmission medium 215. Communication interface logic 110 enables communications with other MCD systems 110₂-110_N and management system 120 of FIG. 1. According to one embodiment of the disclosure, communication interface logic 210 may be implemented as a physical interface including one or more ports for wired connectors. Additionally, or in the alternative, communication interface logic 210 may be implemented with one or more radio units for supporting wireless communications with other electronic devices.

Processor 200 is further coupled to persistent storage 230 via transmission medium 225. According to one embodiment of the disclosure, persistent storage 230 may include static analysis engine 170, which includes a preprocessing malware check logic 250, an emulation logic 260, and a post-processing malware check logic 270. Of course, when implemented as hardware, logic 250, 260 and/or 270 would be implemented separately from persistent storage 230.

Preprocessing malware check logic 250 includes one or more software modules to conduct static analysis of content within an object to determine whether such content includes characteristics of malware, such as information associated with known malware exploits. The preprocessing malware check logic 250 may conduct, in some embodiments, a malware signature matching operation. The term “signature” designates an indicator of a set of characteristics and/or behaviors exhibited by one or more exploits that may not be unique to those exploit(s). Thus, a match of the signature may indicate to some level of probability, often well less than 100%, that an object constitutes an exploit. In some contexts, those of skill in the art have used the term “signature” as a unique identifier or “fingerprint,” for example, of a specific virus or virus family (or other exploit), which is generated for instance as a hash of its machine code, and that is a special sub-case for purposes of this disclosure. As an example, malware check logic 250 may be initially invoked to conduct static analysis of content in the object, which includes conducting pattern comparison operations (e.g. bitwise comparison, byte-wise comparison, etc.) between accessible object content and malware identifiers associated with known exploit patterns (e.g. heap spray, shell code, etc.) and vulnerable functions obtained from data store 290. Accessible object content is content that is not obfuscated; obfuscation is detected by the obfuscation detection logic 320. The data store 290 may store patterns correlated with characteristics and patterns of characteristics of known malicious content. The patterns stored in data store 290 may be generated in a signature generator 295. The patterns stored in data store 290 may be modified by an administrator remotely or locally. The correlation engine 310 may access the data store 290 for malware identifiers associated with known exploit patterns. When the preprocessing malware check logic 250 fails to have code-level access for a targeted object as a result of obfuscation within the object, it is processed by the emulation logic 260.

Emulation logic 260 (emulator) includes one or more software modules that are invoked when the preprocessing malware check logic 250 (preprocessor) fails to have code-level access for a targeted object within the analyzed object. Emulation logic 260 is configured to emulate operations of the object and, perhaps in combination with preprocessing malware check logic 250, monitor for anomalous characteristics. The monitoring may be accomplished by “hooking” certain functions associated with that object (e.g., application programing interface (API) function calls, etc.) as described above, and controlling what data is specifically returned in response to corresponding function calls (e.g., substituting function input values). After receipt of the returned data, operations by the object are monitored by the static analysis engine. For instance, the output from the object may be monitored and analyzed by the post-processing malware check logic 270 (post-processor) to determine if a portion of the output matches any malware characteristics. In some embodiments, the emulation logic 260 or the post-processing malware check logic 270 may access data store 290 for purposes of retrieving, from an API call repository, expected API calls typical of the type of content being processed, and may store “hooked” API calls therein for later reference.

Subsequently, after failing to detect any suspicious characteristics and emulating the object as described herein, the post-processing malware check logic 270 may conduct a secondary static analysis in which content within objects associated with the de-obfuscated object (e.g., code associated with one or more embedded objects within the object that is accessible after emulating the object) is compared to the malware identifiers.

Referring to FIG. 3A, the preprocessing malware detection logic 250 (preprocessor) may include a suspiciousness determining logic 350. The suspiciousness determining logic 350 may further include a heuristics engine 351, a rules checker 352, and a protocol anomaly detector 353. The heuristics engine 351 identifies patterns correlating with malware in the content being analyzed. The correlations of the heuristics engine 351 are the result of statistical analyses and experiential knowledge pertaining to previously detected malware and malware families. The statistical analyses for the heuristics engine 351 may be conducted in the same preprocessor 250 or post-processor 270. The rules checker 352 would identify suspiciousness by associating compliance of the content analyzed with access control rules (e.g. blacklists, whitelists, etc.), that are defined by an administrator locally or defined by a remote malware-related service provider and downloaded for use. The protocol anomaly detector 353 associates suspiciousness with non-compliance with applicable protocols or applicable standards. Analyzed content's deviation from the corresponding protocol may be a characteristic associated with maliciousness. The obfuscation detection logic 320 determines if content is obfuscated through analysis of the content. Analysis in the obfuscation detection logic 320 may include extracting features at various representation levels (i.e. source code, lexical token, etc.).

Referring to FIG. 3B, the post-processing malware detection logic 270 (post-processor) includes a suspiciousness determining logic 370. The post-processing suspiciousness determining logic 370 components may be identical (as described above) preprocessing suspiciousness determining logic 350.

Referring to FIG. 4, an exemplary diagram of a flowchart illustrating a multi-stage static analysis of content (e.g., analyzed object) is shown. Herein, a first static analysis is conducted to determine if objects within analyzed content has a correlation with (generally referred to as “matches”) known malware exploit patterns, vulnerable function calls, or malicious patterns (block 400). If a suspicious identifier is matched, it is contemplated that a score that identifies the likelihood of the presence of malware within the analyzed object may be produced and in some embodiments, provided for use in subsequent VM-based analysis of the object (blocks 480 and 490).

In the event that no suspicious characteristic is detected in the first stage, an analysis will be conducted to determine if the content is obfuscated (block 420). If the content analyzed is determined not to be obfuscated (block 430) the content is filtered from further analysis. In the event that the content is determined to be obfuscated (block 430), a second stage static analysis (block 445) is conducted after emulated processing of the object within the context of simulated operations associated with a targeted application (block 440). For example, the emulation simulates operations that may occur if a browser application were actually, or virtually, being executed. One reason for such emulation is to detect malware exploits that are executed only upon detecting a certain targeted application. If an identifier associated with malware is detected, a score that identifies the likelihood of the presence of malware within the analyzed object may be produced and provided for subsequent VM-based analysis of the content (blocks 450, 480 and 490) and in some embodiments, in classifying the content following such VM based analysis.

If the suspicious characteristic is detected and the object is not obfuscated, a determination is made as to whether further analysis is to be conducted within the VM-based dynamic analyzer (block 490). If no further static analysis is to be conducted and a determination is made to analyze the object in the VM-based analyzer, the analyzed object (or at least a portion of the analyzed object) is provided to dynamic analyzer 190 of FIG. 1 for VM-based analysis.

In the event the static analysis does not find a suspicious characteristic and the object is not obfuscated, the analysis will end. In an alternative embodiment the endpoint 495 is a virtual endpoint simulating a destination during a forensic analysis. A forensic analysis is one that analyzes content that is suspected to have caused malicious behavior on a device.

In general, one embodiment is directed to a multi-stage static analysis that filters non-suspicious objects from further analysis and emulates processing of the analyzed object with results of the same being provided to the dynamic analyzer 190 for subsequent VM-based analysis. This multi-stage static analysis is configured to improve accuracy and efficiency in malware analysis. As alternative embodiments, it is contemplated that any or all of the above-described static analyses may be conducted concurrently.

In some embodiments, based on a combination of the respective scores resulting from the static scans exceeding a threshold, the analyzed object may be classified as malicious, non-malicious, or, possibly, in need of further analysis, e.g., in VM-based analysis before classification may be made. It can be understood that, even if classified as a result of the static analysis, the object may be subjected to VM-based analysis to further identify characteristics and behaviors for use in later deterministic checks, for identifier comparison, for blocking, for remediation or for confirmation of static analysis results.

In the foregoing description, the invention is described with reference to specific exemplary embodiments thereof. It will, however, be evident that various modifications and changes may be made without departing from the broader scope of the invention as set forth in the claims.

Claims

1. A method for detecting malicious content conducted by a preprocessor and a post-processor being software executed by a hardware processor, the method comprising: receiving an object including content over a network;determining, in the preprocessor, whether the content of the object is suspicious and whether the content is obfuscated by at least determining whether there is a high correlation between a portion of the content of the object and a first set of indicators of suspiciousness;responsive to determining that the content is suspicious and is not obfuscated, transmitting the suspicious content to a dynamic analyzer, the dynamic analyzer including one or more virtual machines that processes at least the content and monitors for anomalous behaviors occurring during the processing of the content;responsive to determining that the content of the object is obfuscated, transmitting the obfuscated content to an emulator that is different than the dynamic analyzer, the emulator being configured to produce a de-obfuscated representation of the content that is different from the non-obfuscated content and send the de-obfuscated representation of the content to the post-processor,determining, in the post-processor, whether the de-obfuscated representation of the content from the emulator is suspicious by at least determining whether there is a high correlation between a portion of the de-obfuscated representation of the content and a second set of indicators of suspiciousness that differs from the first set of indicators of suspiciousness, andproviding the de-obfuscated representation of the content for further analysis by the dynamic analyzer; andremoving non-suspicious content from further analysis.

2. The method of claim 1, wherein the content of the object is determined to be suspicious based on a correlation of characteristics of the content with a first set of indicators of suspiciousness as detected by the preprocessor and a correlation of characteristics of the content with a second set of indicators of suspiciousness as detected by the post-processor.

3. The method of claim 1, further comprising sending content removed from further analysis to a destination device.

4. The method of claim 1, wherein the determining whether the content of the object is obfuscated occurs when the preprocessor fails to have code-level access for at least a portion of the content.

5. The method of claim 1, further comprising transmitting the suspicious content to the dynamic analyzer in a remote network to determine if the suspicious content is malicious.

6. The method of claim 1 wherein the suspicious content is prioritized for analysis in the dynamic analyzer.

7. The method of claim 2, wherein the first set of indicators of suspiciousness are modifiable.

8. The method of claim 2, wherein the content of the object is determined to be malicious when the correlation of characteristics of the content with the first set of indicators of suspiciousness exceeds a threshold of maliciousness.

9. The method of claim 2, wherein the first set of indicators of suspiciousness in the preprocessor are identical to the second set of indicators of suspiciousness in the post-processor.

10. The method of claim 2, further comprising combining result of the correlation of characteristics of the content with the first set of indicators of suspiciousness from the preprocessor and the correlation of characteristics of the content with the second set of indicators of suspiciousness from the post-processor to determine if the content of the object is malicious.

11. The method of claim 9, wherein the combination of result of the correlation of characteristics of the content with the first set of indicators of suspiciousness and the correlation of characteristics of the content with the second set of indicators of suspiciousness are weighted based on the set of indicators of suspiciousness the correlation is based.

12. The method of claim 9, wherein the determining whether the de-obfuscated content is suspicious includes intercepting one or more function calls associated with the de-obfuscated representation of the content and controlling a return of data in response to the one or more function calls.

13. A system to detect when an object including content is malicious, comprising: a preprocessor that is configured, upon execution by a hardware processor, to determine (i) whether content of an object under analysis is suspicious and not obfuscated by at least determining whether there is a high correlation between a portion of the content of the object and a first set of indicators of suspiciousness, (ii) whether the content is obfuscated, or (iii) whether the content is to be removed from further analysis when non-suspicious and not obfuscated, the preprocessor to provide the suspicious content to a dynamic analyzer for determining whether the suspicious content is malicious when the content is suspicious and not obfuscated;an emulator communicatively coupled to the preprocessor, the emulator is configured, upon execution by the hardware processor, to receive the content, in response to the preprocessor determining that the content is obfuscated, and process the content to produce a de-obfuscated representation of the content that is different than the content that is not obfuscated; anda post-processor that is configured, upon execution by a hardware processor, to a receive the de-obfuscated representation of the content from the emulator, (ii) determine if the de-obfuscated representation of the content is suspicious by at least determining whether there is a high correlation between a portion of the de-obfuscated representation of the content and a second set of indicators of suspiciousness that differs from the first set of indicators of suspiciousness, and (iii) provide the de-obfuscated representation of the content to the dynamic analyzer if the de-obfuscated representation of the content is suspicious or remove the de-obfuscated representation of the content from further analysis if the de-obfuscated representation of the content is non-suspicious.

14. The system of claim 13, wherein the preprocessor is configured to determine the suspiciousness of the content by establishing a correlation with a first set of indicators of suspiciousness.

15. The system of claim 13, wherein the post-processor is configured to determine the suspiciousness of the de-obfuscated representation of the content by establishing a correlation with a second set of indicators of suspiciousness.

16. The system of claim 13 further comprising a scheduler that is configured to receive the suspicious content from the preprocessor or the suspicious, de-obfuscated representation of the content from the post-processor to schedule for analysis in the dynamic analyzer that determines if the suspicious content is malicious.

17. The system of claim 13 further comprising a reporting module configured to combine the correlations, with indicators of suspiciousness from the preprocessor and post-processor and indicators of maliciousness from the dynamic analyzer, to indicate maliciousness.

18. The system of claim 15, wherein the second set of indicators of suspiciousness used by the post-processor is identical to the first set of indicators of suspiciousness used by the preprocessor.

19. The system of claim 16, wherein a scheduler is configured to prioritize content for analysis in the dynamic analyzer based on the correlation of the content with maliciousness.

20. The system of claim 16, wherein the dynamic analyzer is located in a remote network to determine if the content is malicious.

21. The system of claim 17, wherein the preprocessor determines whether the content of the object is obfuscated when the preprocessor fails to have code-level access for at least a portion of the specimen of content.