Patent Application
20190207961
Embodiments of the present invention detect malware using a transparent proxy.
Spyware is a type of malware that may be installed on computers and collects bits of information at a time about users without their knowledge. The presence of spyware is typically hidden from the user and may be difficult to detect. Typically, spyware is secretly installed on the user's personal computer. While the term spyware suggests software that secretly monitors the user's computing machine, the functions of spyware extend well beyond simple monitoring. Spyware programs may collect various types of personal information, such as Internet surfing habits and sites that have been visited, but may also interfere with user control of the computer in other ways, such as installing additional software and redirecting Web browser activity.
In response to the emergence of spyware, a small industry has sprung up dealing in anti-spyware software. Running anti-spyware software has become a widely recognized element of computer security practices for computers, especially those running Microsoft Windows. A spyware program is rarely alone on a computer. An affected machine usually has multiple infections. Users frequently notice unwanted behavior and degradation of system performance. A spyware infestation may create significant unwanted CPU activity, disk usage and network traffic. Stability issues, such as applications freezing, failure to boot and system-wide crashes, may also be common. Spyware, which interferes with networking software commonly causes difficulty connecting to the Internet.
Many programmers and some commercial firms have released products dedicated to remove or block spyware. Anti-spyware software programs may be used solely for detection and removal of spyware software that has already been installed onto the computer. With this spyware protection software, the user may schedule weekly, daily or monthly scans of the computer to detect and remove any spyware software that has been installed on the computer. This type of anti-spyware software scans the contents of the windows registry, operating system files and installed programs on the computer and may provide a list of any threats found, allowing the user to choose what should be deleted and what should be kept.
Such programs may inspect the contents of the Windows registry, the operating system files, and installed programs, and remove files and entries which match a list of known spyware components and signatures. Real-time protection from spyware works similarly to real-time anti-virus protection: the software scans disk files at download time and blocks the activity of components known to represent spyware. In some cases, diction programs may also intercept attempts to install start-up items or to modify browser settings. Because many spyware and adware are installed as a result of browser exploits or user error
Like most anti-virus software, many anti-spyware/adware tools require a frequently-updated database of threats and signatures. As new spyware programs are released, anti-spyware developers discover and evaluate them, making “signatures” or “definitions” which allow the software to detect and remove the spyware. As a result, anti-spyware software is of limited usefulness without a regular source of updates. Some vendors provide a subscription-based update service, while others provide updates free. Updates may be installed automatically on a schedule or before doing a scan, or may be done manually.
Current solutions to spyware are notoriously weak. The model that has been adopted is the on-host anti-virus model. Anti-spyware solutions attempt to determine the presence of spyware by searching for known signatures of the spyware executable by searching the disk. This approach suffers from multiple problems. First, like anti-virus engines, spyware signatures need to be comprehensive and kept up to date. While the updating mechanisms generally work fine, even the best spyware solutions have not been able to be comprehensive, in other words they often times miss many forms and instance of spyware. The second problem is that many forms of spyware do not run as an isolated executable. They are often packaged with some other executable, such as a free music player or content viewer. Spyware tool vendors usually do not include these other commercial entities on the list of spyware. In other instances, spyware runs as browser helper objects—an extension to the browser itself. Third, on-host solutions may be easily compromised by the very content they are trying to detect. The convergence of root-kits with spyware is making on-host solutions unreliable for reporting on what is on the computing machines. The root-kits may effectively hide portions of the file system and running process from any user-level process that is inquiring. As a result, the antispyware and other on-host defenses may not be trusted to fulfill their function reliably.
Current approaches to identifying software include tools such as fpdns, Nessus, and Nmap. However, generation of signatures using these tools is manual and arduous as the signatures are extremely fragile. As a result, automatic fingerprinting algorithms are developed leveraging other work such as those used to fingerprint computer worms. On-the-fly fingerprinting is used to positively identify the source of all network flows, and therefore detect known and unknown forms of malicious software. The passive network tap approach is a clientless/agentless approach that is far more amenable to commercialization than on-host approaches that require massive deployment roll-outs, which many IT departments may be reluctant to purchase. Thus, techniques that simply plug into a network router or gateway and are able to classify network traffic to detect both known and unknown malicious software have a clear advantage over on-host approaches.
Finally, in the commercial sector, network extrusion detection products are emerging as a sub-category in network defense. Existing tools provide facilities for packet capture, such as using Cisco standard PCAP libraries for examining network flows as well as TCP Dump-like utilities for examining packets. However, these tools suffer from an overwhelming amount of data that an analyst must review in order to determine whether an infection has occurred in the network, and as such are not very useful for detecting malicious software threats. Rather, they are used more often for forensic analysis after an infection has been detected by a host-based detection agent.
Therefore, there is a need to develop a method and system for implementing malware detection and reporting. In particular, there is a need for the capability to capture malware without on-host installation of anti-spyware software on every computing machine. Frameworks based on examining out-bound network connections for malware detection are more efficient and easier to manage and are considered in the embodiments of this invention.
Passive detection analysis techniques analyzing outbound traffic fall short in detecting all known and unknown malware on an enterprise computer machines. To show this a test was carried out on an enterprise with HTTP and Voice over IP (VoIP) protocol including fully centralized, peer-to-peer, and hybrid implementations of the SIP protocol. The results aimed at identifying malware generated by Google feed by analyzing the normal, legitimate behavior of known applications. In one example tested scenario, sixty four percent of the port numbers of the detected malware belong to HTTP and HTTPS. Twenty-eight percent belong to Internet Relay Chat (IRC), and eight percent belong to other applications.
Passive detection may identify a good fraction of the malware that is collected in an enterprise network, but may not identify all of them. In this test, the passive analysis techniques were not able to detect more than sixty percent of malware. On the other hand, the active content challenge techniques as presented in the embodiments of this invention have hundred percent detection rate with the tested malware corpus.
The range of the ports that the malware attempts to exploit has shifted towards HTTP and HTTPS traffic from pure IRC traffic that was prevalent couple of years ago. Extrusion detection approaches today largely depend on malware to use anomalous ports for command and control or to exfiltrate data. However, current malware is shifting its command and control and data link to the http/https protocol where firewalls and current extrusion detection approaches may be ineffective.
Furthermore, passive analysis, although still effective, may become obsolete in the future. Using purely passive techniques, only below forty percent of malware may be detected, while using the proposed embodiments of the present invention with active content challenges, hundred percent of the malware may be detected.
Indeed, purely passive network analysis is theoretically and practically unable to recognize malware that “mimics” good/known web browser behavior using the original software's libraries because malware s often uses communication libraries of the vetted application to masquerade its communication patterns and bypass the passive analysis. This limitation of the passive techniques may be generalized for known applications and protocols. This limits the effectiveness of purely passive detection techniques for a next generation malware.
To address the limits of the passive techniques and to ferret out malware that mimics the traffic behavior of legitimate applications, embodiments of the present invention utilize active detection mechanism(s). The active detection mechanism(s) may also be called Active Content Challenges and may be implemented using a transparent proxy.
The transparent network proxy (130 and 202) may intercept outbound requests and issue Active Content Challenges to the requesting application. The principle is similar to Turing puzzles and CAPTCHAs, however, rather than trying to distinguish a human from software, the objective is to distinguish legitimate software from malware. Thus, unlike existing mechanism that demand end-users to be involved in the identification process by solving a puzzle, the approach in this embodiment requires no user involvement or application modification.
Using passive analysis, all out-bound network traffic may be analyzed. Prior art has focused on detecting in-bound malicious software or traffic. Approaches have included using known signatures, generating signatures from repetitive features that would be present in worm-based attacks to anomalous flow detection. Because the approach used in embodiments of the present invention characterize network traffic, traffic searching may not need to be performed for finding malicious traffic. Rather, network traffic may be classified by its originating program based on a fingerprint developed using network-level analysis including header information such as timing and source/destination addresses and ports. Moreover, the HTTP headers' ordering and special HTTP headers may be harnessed to identify known browser applications and decide which bucket that program falls in, including: traffic from an already encountered program, known netted malware and unknown traffic.
The monitor module 303 may examine traffic 309 including first data 310 originating from at least one application (201, 400, and 600) running on a computing machine (100 and/or 200) towards a remote server (141, 142, . . . 149, 204, 402 and/or 602). The application (201, 400, and 600) may expect a server response (408, 606 and 607) from the remote server (141, 142, . . . 149, 204, 402 and/or 602). The application (201, 400, and/or 600) may be one of the following: a Web browser, a VOIP application, a peer-to-peer application, a database client, a database server or the like. The monitor module 303 may output the received data traffic on a data interface 310 to the protocol determination module 304.
The protocol determination module 304 may be configured to identify the protocol type 307 used for the first data 310 generated from one of the applications (201, 400 and/or 600). The protocol determination module 304 may use protocol fingerprinting to identify the protocol type 307. The protocol type 307 may be one of the following: a HTTP, a VOIP protocol, a Session Description Protocol, a Session Initiation Protocol, a Real Time Transport Control Protocol or the like.
The challenge generation module 305 may: produce a challenge 311 for the application (201, 400 and/or 600) based upon the protocol type 307; send the challenge 311 to the application; and maintain a state related to the data 310 and the challenge 311. The challenge generation module 305 may be further based upon the state of the application (201, 400 and/or 600) when the state of the application (201, 400 and/or 600) is transmitting the first data 310. The challenge 311 may be non-disruptive to the application (201, 400 and/or 600) and may be transparent to users. As an example, the application (201, 400 and/or 600) may be a Web browser 801 and the challenge 805 may be an encoded redirect request that is triggered after a page is processed by the targeted Web browser 801 engine.
The response determination module 306 may make a determination if an automatic non-interactive application response 308 is received in response to the challenge 311 from application (201, 400 and/or 600). The application response 308 may not involve the end application user.
The data control module 302 may allow the first data received on interface 300 to continue to the remote server (141, 142, . . . 149, 204, 402, and/or 602) through interface 301 when the determination is valid. The first data control module 302 may block the first data to continue to the remote server (141, 142, . . . 149, 204, 402 and/or 602) when the determination is invalid.
The data control module 302 may receive an allowance 312 to allow the server response 606 from the remote server (141, 142, . . . 149, 204, 402 and/or 602) received on interface 301 to be transmitted to the application 600 through interface 300 when the determination is valid. The data control module 302 may block the server response 607 from the remote server (141, 142, . . . 149, 204, 402 and/or 602) to be received by the application 600 when the determination is invalid.
The transparent proxy 313 may also include a malware classification module 350 to determine malicious software 355 running on a computing machine (100, 109, 110, 119 and 200) and a reporting module 340 to report 345 malicious software running on the computing machine (100, 109, 110, 119 and 200). The malware software 355 is determined by analyzing the first data 301, the application type (201, 400, and 600), and the automatic non-interactive application response (405 and 605).
By making use of the application's rendering engine, the transparent proxy may effectively distinguish between malicious and benign network traffic. The active content challenge 404 may not be visible to the users at all since its content may only be related to the application. If the application successfully replies to the active content challenge by sending message 405, the transparent proxy may determine that the data was generated from a valid application. Malware requests may be revealed if the malware software is not able to generate a valid response to the active content challenge. If a valid response is not received within a predetermine time period, the transparent proxy 401 may detect the malware and may block the data message 407 and report that malware is detected on the computing machine running the application as shown in scenario 2 in
The end-user may not need to be involved in the process. A seamless user experience may be offered with virtually no delays in most cases. Malware may be revealed if it is unable to generate a valid response to the active content challenge but rather attempts to re-connect to the same or another alternate server unsuccessfully. Therefore, in general, the core of this approach frustrates the communication of the malware by injecting traffic that malware is incapable of parsing and generating a valid response contrary to the legitimate application.
By making use of the application's rendering engine, the transparent proxy may effectively distinguish between malicious and benign network traffic. The active content challenge 604 may not be visible to the users at all since its content is only related to the application. If the application successfully replies to the active content challenge by sending message 605, the transparent proxy may determine that the data was generated from a valid application. Malware requests may be revealed because it may not be able to generate a valid response to the active content challenge. If a valid response is not received within a predetermine time period, the transparent proxy 601 may detect the malware and may block the server response 607 and report that a malware is detected on the computing machine running the application as shown in scenario 2 in
The end-user may not need to be involved in the process. A seamless user experience may be offered with virtually no delays in most cases. Malware may be revealed if it is unable to generate a valid response to the active content challenge but rather attempts to re-connect to the same or another alternate server unsuccessfully. In general, the core of this approach frustrates the communication of the malware by injecting traffic that malware may be incapable of parsing and generating a valid response contrary to the legitimate application.
Malicious software running in an enterprise may exfiltrate data or command and control communication channels of an application. The transparent proxy 313 may classify and identify the malicious software by examining program network traffic. Current approaches search for malicious software by identifying signatures of known attacks, but the approach presented in embodiments of the present invention validate all out-going network sessions based on their application network behavior. This approach is two pronged: (1) the network characteristics of applications may be passively examined using a set of transparent proxies located on the network edges that use protocol fingerprinting; and (2) in addition to pure passive monitoring, the transparent proxy 313 may create active content challenges to software applications to differentiate legitimate from sophisticated malware. This approach may be non-disruptive to applications and transparent to application users. The process attempts to reveal the sophisticated malware by both classifying applications that are known to be good from applications that are either unknown or known to be malicious.
The network-based proxy 313 may automatically examine all network traffic 309 emanating from the enterprise through Internet egress points. The transparent proxy may also be located on a workstation or laptop and simply examine all outgoing traffic from the workstation or laptop. The transparent proxy successfully differentiates different browsers (Internet Explorer, Firefox, Opera) from malicious software by passively analyzing their outbound traffic.
Sophisticated malware that imports browser components, such as Internet Explorer, may be detected using Active Content Challenges. The transparent proxy may also be capable of classifying and identifying malicious software that may affect VoIP protocols, including Session Initiation Protocol (SIP), Session Description Protocol (SDP), Real-time Transport Control Protocol (RTCP), Real-time Transport Protocol (RTP), or the like. The transparent proxy in the embodiments of the present invention could differentiate different VoIP implementations including Asterix, sipX, PBX, and Skype.
The modules implemented in the transparent proxy may be a hardware based network apparatus that sits in parallel with the Internet gateway router to identify all traffic emanating from the enterprise network 120 and identify all outgoing sessions as authorized, malicious, or unknown, and then take actions accordingly. For example, the transparent proxy may route malicious or unknown traffic to a separate honey pot server, or simply block the connection, or report the malware software in the network. The process may provide a solution to identifying any and all malware in an enterprise that attempt to make outside connections via the Internet.
By categorically identifying network traffic emanating to the Internet as known good, malicious or unknown, the transparent proxy may identify computing machines within an enterprise that are compromised and may be under the command and control of an outside entity or malicious software. Some differences over the on-host-based solutions include:
1) Embodiments may have an agentless/clientless passive network solution that obviates the need for an enterprise-wide roll-out, which may be very expensive. Also, with an agentless solution, there may be no need to worry about stepping on or being stepped on by other client-side software such as anti-virus software. There may be no need to worry about affecting the reliability of users' computing machines.
2) There may not be a need for update signatures on all the computing machines in an enterprise. Some embodiments may not require signatures, though signatures may enhance the identification of known malware. Any signature updates occur may only need to be applied to one machine—the transparent proxy. In embodiments of the present invention, signatures may be automatically created through an automated fingerprinting approach for network traffic.
3) Fingerprinting all network traffic emanating from the enterprise network may identify all known good, known bad and unknown traffic. In this approach, signatures may no longer be necessary in finding malware, and merely provide additional information about the found netted malware.
4) Since the transparent proxy may not need to be on each computing machine, it may not be susceptible to being subverted or manipulated by rootkits or other on-host process-hiding evasion techniques. Rather, the transparent proxy may detect the netted malware when it attempts to communicate to the Internet preventing it from exfiltrating data, propagate, receive new targets or even update itself to the new version. All of the aforementioned activities may be detected and potentially further analyzed by moving the malware to execute on a controlled environment.
As an example,
In addition to passive analysis for HTTP, the active content challenge process may be performed on the existing Voice Over IP clients and servers and their legitimate protocol behavior and characteristics. To that end, a passive analysis may be performed on the network behavior of the VOIP clients identified by their order of headers and other communication characteristics.
Furthermore, variations in the way that clients implement the “FROM” and “TO” fields are identified to characterize some of the clients.
As shown here, active content challenge algorithms may be developed for various different protocols such as HTTP, HTTPS, RTP, VoIP protocols and the like. The transparent proxy framework may be updated on a continuing basis for adding new outbound network protocols. Passive fingerprinting techniques may be sufficient to differentiate multiple VoIP protocols, as well as standard malware that may attempt to impersonate VoIP. However, sophisticated malware may re-engineer, or co-opt an off-the-shelf VoIP product to exfiltrate data. To handle this case, active content challenges may be implemented that actively query VoIP requests with puzzles that may only be solved by non-coopted legitimate software.
The passive and active challenge approaches may be integrated on the same transparent proxy for both HTTP and VOIP protocols on a single platform. A framework may be created for specifying protocols that the transparent proxy would intercept and issue active challenges as required. The list of protocols analyzed by the transparent proxy may include HTTP, VOIP, and any other networking protocol as required.
The transparent network proxy may be developed on a hardware module. For example the transparent proxy may be implemented as a network appliance in a rack-mounted form factor. The transparent network proxy may be considered as a hardware refactored for use in a variety of different networked environments including Military, Government agency, commercial, and academic environments.
Active Content Challenges may be applied to applications using various protocols including: HTTP and Voice Over IP (VOIP) protocols. VOIP protocols may include the following four sub-protocols:
1. Session Initiation Protocol (SIP), used to initiate and setup the call.
2. Real-time Transport Protocol (RTP), the Internet-standard protocol for the transport of real-time data.
3. Session Description Protocol (SDP), which describes many elements of the RTP session.
4. Real-time Transport Control Protocol (RTCP) is employed to carry the control information: sender and receiver session identifiers.
Active content challenges may be designed for all of the above sub-protocols because a VOIP session may be established in three different, but equally functional, logical network topologies: fully centralized, peer-to-peer, and hybrid. For each scenario and depending on the client used by the sender and the receiver, a different active content challenge may be designed. An example set of supported clients may have the most popular desktop SIP clients and soft-phones including X-ten Lite, SJPhone, ekiga, linphone and may be targeted for the following protocols: SIP, SIP/SDP, RTCP, RTP.
The transparent proxy framework 313 may include both passive and active malware detection algorithms. It may be able to quickly sift through traffic attempting to detect if the flow is generated from a malicious client using a combination of passive and active techniques. Each of these techniques comes with a cost in terms of processing, memory, and resources, which, for certain operating conditions (e.g. large network lines) may quickly become prohibiting. There may be a need for evaluating what is the optimal strategy to detect malware for a given flow depending on the resources available and to achieve that, there may be a need to integrate passive network analysis with Active Content Challenge algorithms.
The transparent proxy detection engine may include a framework that allows additional protocol specifications. This component may be important to enable:
a) Including new protocols in the list of supported protocols
b) Incorporating additional behavioral rules for passive and active content filtering
c) Formalizing and optimizing characteristics of inspected protocols and defining them in a concise and clean fashion.
The framework may be extensible by enabling the inclusion of novel protocols and behavioral signatures to be potentially generated by third parties. The same framework for developing HTTP, HTTPS, and VOIP protocols may be extended to accommodate other protocols. The transparent network proxy may be a standalone hardware box that is stable in operational environments, enabling a robust system capable of running in diverse network environments. It may include a protocol specification framework to extend the detection approach to other network protocols to the transparent proxies.
Refactoring the system may start with a review of the system design to identify and eliminate any redundancies, to consider alternative architectures or methods, re-coding functionality as necessary, then stubbing and unit testing, followed by system testing the code for robustness. In addition, the transparent proxy processes may be optimized for the state-of-the-art hardware including network cards, memory, and processor. The transparent proxy may be a stand-alone box in a rack-mounted network server form factor that is able to meet the operational requirements of enterprise-sized networks.
The transparent proxy may include multiple modules. Alternative communication methods may be considered between the modules such as using shared memory for data transfer or using the method of setting up server pipes. Architecturally, appropriate abstractions may also be considered to ensure similar functionality may be re-used as well-tested robust code. A detailed unit testing may be performed for each module by stubbing out other modules and subjecting the unit under test to the full range of possible input a module may experience, as well as performing security tests to ensure operation is robust against attack.
It should be noted that references to “an” embodiment in this disclosure are not necessarily to the same embodiment, and they mean at least one. In this specification, “a” and “an” and similar phrases are to be interpreted as “at least one” and “one or more.”
Many of the elements described in the disclosed embodiments may be implemented as modules. A module is defined here as an isolatable element that performs a defined function and has a defined interface to other elements. The modules described in this disclosure may be implemented in hardware, hardware in combination with software, firmware, wetware (i.e hardware with a biological element) or a combination thereof, all of which are behaviorally equivalent. For example, modules may be implemented as a software routine written in a computer language (such as C, C++, Fortran, Java, Basic, Matlab or the like) or a modeling/simulation program such as Simulink, Stateflow, GNU Octave, or LabVIEW MathScript. Additionally, it may be possible to implement modules using physical hardware that incorporates discrete or programmable analog, digital and/or quantum hardware. Examples of programmable hardware include: computers, microcontrollers, microprocessors, application-specific integrated circuits (ASICs); field programmable gate arrays (FPGAs); and complex programmable logic devices (CPLDs). Computers, microcontrollers and microprocessors are programmed using languages such as assembly, C, C++ or the like. FPGAs, ASICs and CPLDs are often programmed using hardware description languages (HDL) such as VHSIC hardware description language (VHDL) or Verilog that configure connections between internal hardware modules with lesser functionality on a programmable device. Finally, it needs to be emphasized that the above mentioned technologies are often used in combination to achieve the result of a functional module.
The disclosure of this patent document incorporates material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, for the limited purposes required by law, but otherwise reserves all copyright rights whatsoever.
While various embodiments have been described above, it should be understood that they have been presented by way of example, and not limitation. It will be apparent to persons skilled in the relevant art(s) that various changes in form and detail can be made therein without departing from the spirit and scope. In fact, after reading the above description, it will be apparent to one skilled in the relevant art(s) how to implement alternative embodiments. Thus, the present embodiments should not be limited by any of the above described example embodiments. In particular, it should be noted that, for example purposes, the above explanation has focused on the example(s) of a transparent proxy located on a network gateway or firewall. However, one skilled in the art will recognize that embodiments of the invention could be located on any network computer workstation or laptop and monitor outgoing traffic. In another embodiment, the transparent proxy may be implemented in a distributed framework including multiple hardware components, or be installed on an existing network gateway and firewalls.
In addition, it should be understood that any figures which highlight the functionality and advantages, are presented for example purposes only. The disclosed architecture is sufficiently flexible and configurable, such that it may be utilized in ways other than that shown. For example, the steps listed in any flowchart may be re-ordered or only optionally used in some embodiments.
Further, the purpose of the Abstract of the Disclosure is to enable the U.S. Patent and Trademark Office and the public generally, and especially the scientists, engineers and practitioners in the art who are not familiar with patent or legal terms or phraseology, to determine quickly from a cursory inspection the nature and essence of the technical disclosure of the application. The Abstract of the Disclosure is not intended to be limiting as to the scope in any way.
Finally, it is the applicant's intent that only claims that include the express language “means for” or “step for” be interpreted under 35 U.S.C. 112, paragraph 6. Claims that do not expressly include the phrase “means for” or “step for” are not to be interpreted under 35 U.S.C. 112, paragraph 6.
This application claims the benefit of U.S. Provisional Application No. 61/168,053, filed Apr. 9, 2009, entitled “Malware Detector,” which is hereby incorporated by reference in its entirety.
This invention was made with government support under Contract No. W911NF-08-C-0139 awarded by US Army AMC. The government has certain rights in the invention.
| Number | Date | Country | |
|---|---|---|---|
| 61168053 | Apr 2009 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 15359026 | Nov 2016 | US |
| Child | 16295498 | US | |
| Parent | 14482793 | Sep 2014 | US |
| Child | 15359026 | US | |
| Parent | 12757675 | Apr 2010 | US |
| Child | 14482793 | US |
