Embodiments of the disclosure relate to the field of malware detection. More specifically, one embodiment of the disclosure relates to a hypervisor-based, malware detection architecture.
In general, virtualization is a technique for hosting different guest operating systems concurrently on the same computing platform. With the emergence of hardware support for full virtualization in an increased number of hardware processor architectures, new software virtualization architectures have emerged. One such virtualization technique involves adding a software abstraction layer, sometimes referred to as a virtualization layer, between the physical hardware and a virtual machine (referred to as “VM”).
A VM is a software abstraction that operates like a physical (real) computing device having a particular operating system. A VM typically features pass-through physical and/or emulated virtual system hardware, and guest system software. The virtual system hardware is implemented as software components in the host (e.g., virtual central processing unit “vCPU” or virtual disk) that are configured to operate in a similar manner as corresponding physical components (e.g., physical CPU or hard disk). The guest system software, when executed, controls operations inside the VM, such as the execution and allocation of virtual resources, so that the VM operates in a manner consistent to operations of the physical computing device. As a result, the software virtualization architecture allows for a computing device, which may be running one type of “host” operating system (OS), to support a VM that operates like another computing device that is running another OS type.
Over the last few years, while efforts have been made to improve functionality of VMs, the overall logical architecture of the virtualization layer has experienced little change. The virtualization layer includes a hypervisor, the most privileged component of the virtualization software stack, which runs on top of the hardware resources. The virtualization layer functions similar to an OS kernel—abstracting the underlying hardware resources and isolating software components running on the hypervisor.
While able to access low-level context data (e.g., register values, etc.) from the guest operating system (guest OS) residing in the VM, the virtualization layer is unable to discern higher level context concerning the guest processes, such as the particular type and/or version of the application associated with the active guest process running in the VM. Stated differently, from the context data, the virtualization layer is unable to discern whether the active process pertains to a particular type/version of web browser application (e.g., FireFox® browser, version 24.7) or a particular type/version of a Portable Document Format (PDF) reader (e.g., Adobe® Reader, version 10) for example. Additionally, although the virtualization layer has access to stored data within the entire virtual memory, without additional metadata, it is unable to discern whether that stored data is associated with a process stack or a critical OS data structure.
As a result, the virtualization layer relies on additional data delivery schemes to obtain metadata associated with guest processes running in the VM. One data delivery scheme instruments an event monitoring process as part of or operating in conjunction with the guest OS. This software component, sometimes referred to as a “guest agent,” is configured to provide the virtualization layer with metadata that may assist in the handling of exploit detection. For instance, depending on the OS-type, a particular guest agent may be instrumented into or operating in conjunction with the guest OS and, in response to at least one selected event, provides metadata to the virtualization layer.
Despite having an immense value in exploit detection, the guest agent remains generally unsecure as there are no mechanisms within the software virtualization architecture to protect the integrity, confidentiality and availability of the guest agent. When operating inside the guest OS, the guest agent is at the same privilege level as potential malware being processed within the VM. Hence, the guest agent is highly susceptible to a malicious attack. The loss of proper guest agent functionality may result in a loss of semantic information for the virtualization layer, which may degrade its exploit detection and guest process protection abilities.
Embodiments of the disclosure are illustrated by way of example and not by way of limitation in the figures of the accompanying drawings, in which like references indicate similar elements and in which:
Various embodiments of the disclosure are directed to added functionality of the virtualization layer to protect one or more processes running in a virtual machine from being compromised through a malicious attack. One of the processes (sometimes referred to as a “guest agent”) is an instance of a software component that is instrumented as part of or operating in conjunction with either a guest operating system (OS) kernel or a guest OS software application. Alternatively, the guest agent may be implemented as a separate software component. Any of these implementations reside in a guest environment within the software virtualization architecture of a particular computing device.
The virtualization layer and the guest agent process are configured to monitor, perhaps on a continuous basis, for the presence of malicious activity inside the virtual machine. The presence of malicious activity may be detected from events that occur inside the virtual machine during its execution. According to one embodiment of the disclosure, the events may be traces of malicious activity that are detected during execution of the virtual machine, where the events may be caused by current processing of an object received over a network or uploaded from a stored medium. According to another embodiment of the disclosure, the events may be caused by a guest OS and one or more guest applications simulating processing of the object within an intended computing device.
For VM-based operations, the virtualization layer monitors events occurring during execution of the virtual machine and the guest agent process contributes certain metadata associated with these events. Provided to the virtualization layer in real-time or subsequently after detection of the events, the metadata, namely information that augments an understanding of particular events that suggest the presence of malware (e.g., origin of data being processed that is rendering the particular events, relationship of such data with other data, etc.), assists in the detection and classification of any uncovered anomalous behaviors associated with the events. It is contemplated that the virtualization layer may be configured to gather the metadata in lieu of the guest agent process.
Herein, the virtualization layer is a logical representation of at least a portion of a host environment. The host environment features a light-weight hypervisor (sometimes referred herein as a “micro-hypervisor”) operating at a high privilege level (e.g., host mode, ring “0”). In general, the micro-hypervisor operates similar to a host kernel. The host environment further features a plurality of software components, generally operating as user-level virtual machine monitors (VMMs), which provide host functionality and operate at a lower privilege level (e.g. host mode, ring “3”) than the micro-hypervisor.
According to one embodiment of the disclosure, the virtualization layer provides enhanced security of a software component (e.g., a guest agent) operating within a virtual machine, by protecting the integrity, confidentiality and availability of that software component. As an illustrative example, the integrity of the guest agent process is protected by limiting, at certain times, access to memory pages pertaining to code and/or data structures for the guest agent process. The confidentiality of the guest agent process is also protected by controlling access to some of these memory pages, notably those pages containing sensitive or confidential data (e.g., personal user data, financial data, etc.). Lastly, the availability of the guest agent process may be protected by guaranteeing some amount of execution time within the virtual machine.
Herein, according to one embodiment of the disclosure, the virtualization layer is configured with a first security mechanism to protect the integrity and confidentiality of a guest agent process by altering permissions set forth in the nested page tables, which may be directed to memory pages containing code (and/or data structures) pertaining to the guest agent process and/or metadata captured by that guest agent process. As a result, the page permissions associated with the second memory address translation (GPA-to-HPA) may be altered to be more stringent than the page permissions associated with the first memory address translation (GLA-to-GPA).
As an illustrative example, responsive to detecting that a guest agent process is “inactive” (e.g., not currently executing), permission adjustment logic within the virtualization layer may be configured to tighten page permissions associated with the second memory address translation. This temporarily precludes a certain type or types of accesses to memory pages that may contain code (and/or data structures) pertaining to the guest agent and/or metadata captured by that guest agent. As described below, the permission adjustment logic may be implemented within the guest monitor component, which is operating in concert with the micro-hypervisor.
According to one embodiment of the disclosure, the guest agent may be detected as being “inactive” upon (i) observing a guest process switch (e.g., change in the first data store, such as the processor control register CR3) and (ii) determining that the value loaded into the first data store does not correspond to the address space of the guest agent process. Likewise, the guest agent process may be detected as being “active” upon (i) observing a guest process switch and (ii) determining that the value loaded into the first data store corresponds to the address space of the guest agent process. As a result, in response to a guest process switch, when the guest monitor component of the virtualization layer observes the guest agent process is “inactive”, the memory pages associated with the guest agent process may be read-only or may be hidden by removing read/write/execute permissions for these memory pages in the nested page tables (EPTs). The removal of the read/write/execute permissions for the above-identified memory pages effectively renders those memory pages completely inaccessible inside the VM. Similarly, when the guest monitor component of the virtualization layer observes the guest agent process is active, the page permissions are returned to less stringent permissions as may be found in the first memory address translation (e.g., read/write/execute or any combination thereof). Additionally, by tightening page permissions for an input/output memory management unit (IOMMU), we can also protect the agent's memory pages from DMA accesses.
It is contemplated that the virtualization layer observes state changes as described above. These state changes may constitute observed CR3 changes, but also may constitute privilege level (ring) changes in the guest OS and modify the EPT permissions in response to those privilege level changes. For instance, the virtualization layer protects the integrity, but also protects the confidentiality of memory pages of the agent by relaxing the EPT permissions when the process is executing in user mode (guest ring 3) and tightening the EPT permissions when the process is executing in kernel mode (guest ring 0). For this embodiment, the virtualization layer causes VM exits on all ring transitions between ring 0 and ring 3 (i.e. switches between user and kernel mode or back).
Using this scheme, the agent could store a crypto key in its memory that not even the OS kernel could read. In particular, each time the OS kernel is active (in ring 0), the memory page would be read-protected. Also, each time the agent is executing (in ring 3), the memory page would be readable again.
More specifically, access to memory may be conducted in accordance with the use of the MMU or the IOMMU. A processor may be configured to conduct a memory page access based on a GLA-GPA-HPA translation process in the MMU. So by restricting the permissions in the nested page tables (EPT), the virtualization layer can restrict processor accesses to particular memory pages when certain guest processes are active/inactive. Additionally, device accesses (via a direct memory access) do not go through the MMU. Rather, such accesses are conducted through the IOMMU, which is positioned in the PCI root complex between the devices and the memory controller. The IOMMU uses page tables similar to the EPT, which can translate device-physical addresses to host-physical addresses and also control access permissions. DMA transactions rely on “read” and “write” operations, as there is no “execute” operations. Using the IOMMU, the virtualization layer can protect memory pages against DMA reads or writes. This could also be changed during a guest process switch, but typically guest agent pages may be set to be inaccessible to DMA for the lifetime of the agent process.
Indicating access privileges to memory pages associated with the guest agent, the page permissions include three access privileges: read permission “r” in which the contents of memory may be retrieved and made available in a perceivable format to the user; a write permission “w” in which the contents of memory may be retrieved and subsequently altered by the user; and an execute permission “x” in which the contents of memory may be retrieved and executed. Herein, when the guest agent process is inactive, for enhanced security, the write “w” permission may be removed from nested page table entries that are associated with memory pages containing code (and/or data structures) pertaining to the guest agent and/or metadata captured by that guest agent. Additionally, the execute “x” permission may be removed as well. At that time, the memory pages containing code (and/or data structures) pertaining to the guest agent and/or metadata captured by that guest agent include only read permission “r”. It is contemplated that, when inactive, the memory pages associated with the guest agent process may be hidden (invisible inside the VM) by removal of all page permissions to these memory pages.
As an illustrative example, if the first memory address translation (GLA-to-GPA) is set as “rwx” (read/write/execute) and the second memory address translation (GPA-to-HPA) is altered and configured as “rx” (read/execute), then the effective end-to-end permissions will be “rx”. For this example, the write “w” permission is removed by the permission adjustment logic in the virtualization layer (e.g., in guest monitor component) as paging control logic in the hardware processor conducts a logical AND of the permissions for both memory address translations. As a result, if the memory pages used for the GLA-to-GPA address translation include permissions “rwx” and the memory pages used for the GPA-to-HPA address translation include permissions “rx”, the resultant permission for the guest agent will be “rx”.
It is contemplated that page table manipulation is a privileged operation that requires ring-0 (kernel) privileges. The configuration of the guest page tables (GVA-to-GPA) may be conducted inside the guest kernel (Windows® kernel running in guest ring-0). The configuration of the nested page tables (GPA-to-HPA) may be performed by paging control logic inside the host kernel (hypervisor running in host ring-0), but may be handled by permission adjustment logic within the guest monitor component which has access to the paging control mechanism within the micro-hypervisor via an Application Programming Interface (API).
Additionally, operating as a user-level process in the guest OS, the guest agent process can be disabled like any other process. Given the complexity of internal data structures with the guest OS and the lack of disclosure of such internal data structures to third parties, the virtualization layer is not adapted to interfere with the scheduling or process management of the guest OS without risking a complete system breakdown. Accordingly, in accordance with a second security scheme, the kernel services operating in the guest OS are configured with an interrupt service routine (ISR) that guarantees execution time (availability) for the guest agent process inside the guest OS without modifying OS-internal data structures, such as process control blocks, scheduler run queue, or the like.
More specifically, an interrupt service routine (ISR) component may be deployed as a part of kernel services of the guest OS kernel. Additionally, all data structures related to interrupt delivery are protected, including an Interrupt Descriptor Table (IDT), a Global Descriptor Table “GDT”, a Task State Segment (TSS), and/or the code/data/stack pages of the ISR. Such protection may be accomplished by controlling page permissions associated with ISR's memory pages in the nested page tables in accordance with the first security mechanism as described above.
In order to allocate guaranteed execution time to the guest agent process, interrupt injection logic is deployed within the guest monitor component and is configured to inject a virtual interrupt into the guest OS kernel, such as at a predetermined rate for example. The virtual interrupt will cause execution to vector to the protected ISR within the guest OS kernel at the next suitable point (e.g., the next instruction boundary executed inside the guest OS), provided the guest OS has interrupts enabled. The protected ISR can then perform selected operations, depending on whether the guest agent process is still running or has been disabled. These selected operations may include, but are not limited or restricted to (i) checking for the integrity of certain critical OS or guest agent data structures, (ii) verifying if the guest agent is still running, and/or (iii) restarting the guest agent process if necessary. Depending on the frequency of these virtual interrupts, the virtualization layer can control how often the guest agent process is provided control inside the guest environment. Unfortunately, the entire guest agent process cannot run as an ISR, because ISRs have to be short-running code paths. Fortunately, an attacker cannot disable the ISR because it would render the guest OS unstable.
In the following description, certain terminology is used to describe features of the invention. For example, in certain situations, the terms “component” and “logic” are representative of hardware, firmware or software that is configured to perform one or more functions. As hardware, a component (or logic) may include circuitry having data processing or storage functionality. Examples of such circuitry may include, but are not limited or restricted to a hardware processor (e.g., microprocessor with one or more processor cores, a digital signal processor, a programmable gate array, a microcontroller, an application specific integrated circuit “ASIC”, etc.), a semiconductor memory, or combinatorial elements.
A component (or logic) may be software in the form of one or more software modules, such as executable code in the form of an executable application, an API, a subroutine, a function, a procedure, an applet, a servlet, a routine, source code, object code, a shared library/dynamic load library, or one or more instructions. These software modules may be stored in any type of a suitable non-transitory storage medium, or transitory storage medium (e.g., electrical, optical, acoustical or other form of propagated signals such as carrier waves, infrared signals, or digital signals). Examples of non-transitory storage medium may include, but are not limited or restricted to a programmable circuit; semiconductor memory; non-persistent storage such as volatile memory (e.g., any type of random access memory “RAM”); or persistent storage such as non-volatile memory (e.g., read-only memory “ROM”, power-backed RAM, flash memory, phase-change memory, etc.), a solid-state drive, hard disk drive, an optical disc drive, or a portable memory device. As firmware, the executable code may be stored in persistent storage.
The term “object” generally refers to a collection of data, whether in transit (e.g., over a network) or at rest (e.g., stored), often having a logical structure or organization that enables it to be classified for purposes of analysis for malware. During analysis, for example, the object may exhibit certain expected characteristics (e.g., expected internal content such as bit patterns, data structures, etc.) and, during processing, a set of expected behaviors. The object may also exhibit unexpected characteristics and a set of unexpected behaviors that may offer evidence of the presence of malware and potentially allow the object to be classified as part of a malicious attack.
Examples of objects may include one or more flows or a self-contained element within a flow itself. A “flow” generally refers to related packets that are received, transmitted, or exchanged within a communication session. For convenience, a packet is broadly referred to as a series of bits or bytes having a prescribed format, which may, according to one embodiment, include packets, frames, or cells. Further, an “object” may also refer to individual or a number of packets carrying related payloads, e.g., a single webpage received over a network. Moreover, an object may be a file retrieved from a storage location over an interconnect.
As a self-contained element, the object may be an executable (e.g., an application, program, segment of code, etc.) or a non-executable. Examples of non-executables may include a document (e.g., a Portable Document Format “PDF” document, Microsoft® Office® document, Microsoft® Excel® spreadsheet, etc.), an electronic mail (email), downloaded web page, or the like.
The term “computing device” should be construed as electronics with the data processing capability and/or a capability of connecting to any type of network, such as a public network (e.g., Internet), a private network (e.g., a wireless data telecommunication network, a local area network “LAN”, etc.), or a combination of networks. Examples of a computing device may include, but are not limited or restricted to, the following: an endpoint device (e.g., a laptop, a smartphone, a tablet, a desktop computer, a netbook, a medical device, or any general-purpose or special-purpose, user-controlled electronic device configured to support virtualization); a server; a mainframe; a router; or a security appliance that includes any system or subsystem configured to perform functions associated with malware detection and may be communicatively coupled to a network to intercept data routed to or from an endpoint device.
The term “malware” may be broadly construed as information, in the form of software, data, or one or more commands, that are intended to cause an undesired behavior upon execution, where the behavior is deemed to be “undesired” based on customer-specific rules, manufacturer-based rules, and any other type of rules formulated by public opinion or a particular governmental or commercial entity. This undesired behavior may include a communication-based anomaly or an execution-based anomaly that would (1) alter the functionality of an electronic device executing an application software in a malicious manner; (2) alter the functionality of an electronic device executing that application software without any malicious intent; and/or (3) provide an unwanted functionality which is generally acceptable in other context.
The term “interconnect” may be construed as a physical or logical communication path between two or more computing platforms. For instance, the communication path may include wired and/or wireless transmission mediums. Examples of wired and/or wireless transmission mediums may include electrical wiring, optical fiber, cable, bus trace, a radio unit that supports radio frequency (RF) signaling, or any other wired/wireless signal transfer mechanism.
The term “computerized” generally represents that any corresponding operations are conducted by hardware in combination with software and/or firmware. Also, the term “agent” should be interpreted as a software component that instantiates a process running in a virtual machine. The agent may be instrumented into part of an operating system (e.g., guest OS) or part of an application (e.g., guest software application). The agent is configured to provide metadata to a portion of the virtualization layer, namely software that virtualizes certain functionality supported by the computing device.
Lastly, the terms “or” and “and/or” as used herein are to be interpreted as inclusive or meaning any one or any combination. Therefore, “A, B or C” or “A, B and/or C” mean “any of the following: A; B; C; A and B; A and C; B and C; A, B and C.” An exception to this definition will occur only when a combination of elements, functions, steps or acts are in some way inherently mutually exclusive.
Referring to
The computing devices 140i (i=1, 2, 3) illustratively communicate by exchanging packets or messages (i.e., network traffic) according to a predefined set of protocols, such as the Transmission Control Protocol/Internet Protocol (TCP/IP). However, it should be noted that other protocols, such as the HyperText Transfer Protocol Secure (HTTPS) for example, may be advantageously used with the inventive aspects described herein. In the case of private network 120, the intermediary computing device 1401 may include a firewall or other computing device configured to limit or block certain network traffic in an attempt to protect the endpoint devices 1403 from unauthorized users.
As illustrated in
The virtualization layer 185 features a micro-hypervisor (not shown) with access to the physical hardware and one or more host applications running in the user space (not shown), which operate in concert to provide additional security to one or more software components (hereinafter “software component(s)”) operating within the virtual machine(s) 170. The software component(s) may include guest agent 175 or software components within the guest operating system (OS) kernel such as an interrupt service routine (not shown). This additional security may be achieved by protecting the integrity, confidential and availability of the software component(s).
Referring now to
The hardware processor 210 is a multipurpose, programmable device that accepts digital data as input, processes the input data according to instructions stored in its memory, and provides results as output. One example of the hardware processor 210 may include an Intel® x86 central processing unit (CPU) with an instruction set architecture. Alternatively, the hardware processor 210 may include another type of CPU, a digital signal processor (DSP), an application specific integrated circuit (ASIC), or the like.
According to one implementation, the hardware processor 210 may include one or more control registers 212 (e.g., CR3 register). The number of control registers 212 may depend on the number of processor cores within the hardware processor 210. For instance, each core may have its own CR3 register.
Herein, according to one embodiment of the disclosure, a first control register (CR3) is configured to store a guest page table pointer, which identifies a memory address location for a guest page table hierarchy for the currently active guest process, namely the guest page tables associated with a currently running guest process that is under control of the guest OS (e.g., Windows®-based process). Additionally, a second control register is configured to store a nested page table pointer (EPTP), which identifies a memory address location for a nested page table hierarchy for the currently active virtual machine, where the nested page table may apply page permission restrictions on certain memory pages in the nested page table, namely the memory pages corresponding to the guest agent process or any other protected process that are not active.
Herein, one or more of the control registers 212 may be context-switched between the host mode and the guest mode. Hence, when the hardware processor 210 is executing in guest mode, the CR3 register is loaded with the guest page table pointer as described above. However, in response to a VM Exit and the hardware processor 210 is executing in host mode, the CR3 register 212 points to the host page tables of the currently running hyper-process. The EPTP is not used in host mode unless multiple VMs are layered (e.g., a hypervisor is running inside the VM).
The nested page tables (EPT) control the GPA-HPA translation, and as such, configure the physical memory layout of the VM. They have no direct connection with guest processes. According to this embodiment of the disclosure, the EPT may be configured responsive to a guest process switch. As a result, different guest processes can see memory differently.
Reconfiguring the EPT responsive to an observed guest process switch can be performed in accordance with a number of approaches. One approach involves changing the translations and/or permissions in the currently active EPT. Another approach involves activating a different EPT (e.g., by loading a new nested page table pointer (EPTP) into the EPTP register), which may be preferable given that the switching frequency between “active” guest processes. The same switching operations are applicable to the DMA page tables in the IOMMU.
The network device(s) 240 may include various input/output (I/O) or peripheral devices, such as a storage device for example. One type of storage device may include a solid state drive (SSD) embodied as a flash storage device or other non-volatile, solid-state electronic device (e.g., drives based on storage class memory components). Another type of storage device may include a hard disk drive (HDD). Each network device 240 may include one or more network ports containing the mechanical, electrical and/or signaling circuitry needed to connect the endpoint device 1403 to the network 120 to thereby facilitate communications over the system network 110. To that end, the network interface(s) 230 may be configured to transmit and/or receive messages using a variety of communication protocols including, inter alia, TCP/IP and HTTPS.
The memory 220 may include a plurality of locations that are addressable by the hardware processor 210 and the network interface(s) 230 for storing software (including software applications) and data structures associated with such software. The hardware processor 210 is adapted to manipulate the stored data structures as well as execute the stored software, which includes an operating system (OS) 300 that includes a (guest) OS kernel 301 and one or more guest OS applications 302; user mode processes 320; a micro-hypervisor 360; and/or hyper-processes 370.
Herein, the hyper-processes 370 are instances of software program code (e.g., user-space applications operating as user-level VMMs) that are isolated from each other and run in separate (host) address spaces. In communication with the micro-hypervisor 360, the hyper-processes 370 are responsible for controlling operability of the endpoint device 1403, including policy and resource allocation decisions, maintaining logs of monitored events for subsequent analysis, managing virtual machine (VM) execution, and managing malware detection and classification.
The micro-hypervisor 360 is disposed or layered beneath the guest OS kernel 301 of the endpoint device 1403. The micro-hypervisor 360 is the only component that runs in the most privileged processor mode (host mode, ring-0). As part of a trusted computing base of most components in the computing platform, the micro-hypervisor 360 is configured as a light-weight hypervisor (e.g., less than 10 K lines of code), thereby avoiding inclusion of complex, potentially exploitable virtualization code in an operating system (e.g., x86 virtualization code).
The micro-hypervisor 360 generally operates as the host kernel that is devoid of policy enforcement; rather, the micro-hypervisor 360 provides a plurality of mechanisms that may be used by the hyper-processes 370 for controlling operability of the virtualization architecture. These mechanisms may be configured to control communications between separate protection domains (e.g., between two different hyper-processes 370), coordinate thread processing within the hyper-processes 370 and virtual CPU (vCPU) processing within the VM 170, delegate and/or revoke hardware resources, and control interrupt delivery and DMA, as described below.
The guest OS kernel 301, portions of which are resident in memory 220 and executed by the hardware processor 210, functionally organizes the endpoint device 1403 by, inter alia, invoking operations in support of guest applications executing on the endpoint device 1403. The guest OS kernel 301 may include, but is not limited or restricted to the following: (1) a version of the Windows® series of operating systems; (2) a version of the MAC OS® and IOS® series of operating systems; (3) a version of the Linux™ operating system; or (4) a version of the Android™ operating system, among others. Suitable application programs 320 may include Adobe Reader® and/or Microsoft Word®.
The guest user mode processes 320 constitute instances of the guest applications running their separate address space. Events (monitored behaviors) of an object that is processed by one of the user mode processes are monitored by a guest agent process, which provides metadata to at least one of the hyper-processes 370 and the micro-hypervisor 360 for use in malware detection.
Referring now to
A. Guest Environment
As shown, the guest environment 160 comprises at least one virtual machine 170, which analyzes an object 335 for the presence of malware or continuously analyses execution inside the virtual machine 170 for traces of malicious activity such as execution of user pages from kernel code, disabling certain protection features such as paging, overwriting buffers beyond their bounds, or the like. As shown, the virtual machine 170 features a guest OS kernel 301 that is running in the most privileged level (host mode, ring-0 305) along with one or more processes which are instances of software applications 320 (hereinafter “guest application process(es)”) that are running in a lesser privileged level (host mode, ring-3 325). The guest application process(es) 320 may be based on the same software application, different versions of the same software application, or even different software applications, provided the guest software applications 320 may be controlled by the same guest OS kernel 301 (e.g., Windows® kernel).
It is contemplated that malware detection on the endpoint device 1403 may be conducted as a background process by one or more processes embodied as software components running with the virtual machine 170. These processes include a static analysis process 330, a heuristics process 332 and a dynamic analysis process 334, which collectively operate to detect suspicious and/or malicious behaviors by the object 335 during execution within the virtual machine 170. Notably, the endpoint device 1403 may feature data processing being implemented as its primary processing (e.g., in the foreground having majority use of endpoint resources) while malware detection may be implemented as background processing (i.e., minor use of endpoint resources).
In the alternative, however, the malware detection components could be implemented as part of or operate in conjunction with the same guest process, different modules in the guest OS kernel (or all in the same module), or different hyper-processes in the virtualization layer 185.
As used herein, the object 335 may include, for example, a web page, email, email attachment, file or universal resource locator. Static analysis may conduct a brief examination of characteristics (internal content) of the object 335 to determine whether it is suspicious, while dynamic analysis may analyze behaviors associated with events that occur during virtual execution of the object 335, especially a detected extended page table (EPT) violation where the object 335 is performing a memory access that the page permissions of the nested page tables prohibit (e.g., write data to a memory page that is “write protected”—namely the page without write “w” permission; execution from a page that is marked as non-executable in the nested page tables). These events are further made available to the threat protection component, as described below.
According to one embodiment of the disclosure, when applicable, the static analysis process 330 and the heuristics process 332 may conduct a first examination of the object 335 to determine whether it is suspicious and/or malicious. The static analysis process 330 and the heuristics process 332 may employ statistical analysis techniques, including the use of vulnerability/exploit signatures and heuristics, to perform non-behavioral analysis in order to detect anomalous characteristics (i.e., suspiciousness and/or malware) without execution (i.e., monitoring run-time behavior) of the object 335. For example, the static analysis process 330 may employ signatures (referred to as vulnerability or exploit “indicators”) to match content (e.g., bit patterns) of the object 335 with patterns of the indicators in order to gather information that may be indicative of suspiciousness and/or malware. The heuristics module 332 may apply rules and/or policies to detect anomalous characteristics of the object 335 in order to identify whether the object 335 is suspect and deserving of further analysis or whether it is non-suspect (i.e., benign) and not in need of further analysis. These statistical analysis techniques may produce static analysis results (e.g., identification of communication protocol anomalies and/or suspect source addresses of known malicious servers) that may be provided to reporting module 336. The reporting module 336 generates a report (result data in a particular format) for transmission to a remotely located computing device such as MDS 1402 or another type of computing device.
More specifically, the static analysis process 330 may be configured to compare a bit pattern of the object 335 content with a “blacklist” of suspicious exploit indicator patterns. For example, a simple indicator check (e.g., hash) against the hashes of the blacklist (i.e., exploit indicators of objects deemed suspicious) may reveal a match, where a score may be subsequently generated (based on the content) by the threat protection component 376 to identify that the object may include malware. In addition to or in the alternative of a blacklist of suspicious objects, bit patterns of the object 335 may be compared with a “whitelist” of permitted bit patterns.
The dynamic analysis process 334 may conduct an analysis of the object 335 during processing (or analysis of events monitored during execution of the virtual machine 170), where the guest agent process 175 monitors the run-time behaviors of the object 335 and capture any resulting events that occur during run time. The events are provided to the host environment 180 via any communication channel, such as events stored within a ring buffer 340 of the guest agent 175 for example, for subsequent routing to and analysis by the threat protection component, as described below. In an embodiment, the dynamic analysis process 334 normally operates at least partially contemporaneously not generally wait for results from the static analysis process 330 and/or the heuristics process 332. During processing of the object 335, certain events may trigger page table violations that result in a VM exit to the host environment 180 for further analysis by the threat protection component 376.
1. Guest OS
In general, the guest OS 300 manages operability of the virtual machine 170, where some of these operations involve network connectivity, memory translation and interrupt service delivery and handling of these incoming service requests. More specifically, the guest OS kernel 301 of the guest OS 300 may receive an input/output (I/O) request from the object 335 being processed by one or more guest software applications 320, and in some cases, translates the I/O request into instructions. These instructions may be used, at least in part, by virtual system hardware (e.g., vCPU) to drive one or more network devices, such as a network interface card (NIC) for example, for establishing communications with other network devices. Upon establishing connectivity with the private network 120 and/or the public network 110 of
Another operation supported by the guest OS 300, such as the guest OS kernel 301 for example, involves the management of guest page tables, which are used as part of the two-step address translation process, to translate a guest-linear address (GLA) to a guest-physical address (GPA). The GLA along with the value of a processor control register (e.g., CR3 register) is used to produce GPA, which operates as an index for the recovery of the host physical address (e.g., the actual address for the data in physical memory).
Lastly, kernel services 310 within the guest OS kernel 301 is configured with an Interrupt Service Routine (ISR) 315 that supports one or more different types of interrupts, including network-based interrupts, graphics-based interrupts and kernel services interrupts. Since the guest agent process 175 may be turned off or halted through malicious attack prompted during processing of the object 335 within the VM 170, the kernel services interrupts are invoked by the guest monitor component 374, as described below, to ensure processing of the guest agent process 175 within the VM 170.
Issued by the guest monitor component 374, the kernel services interrupt represents a virtual interrupt that causes kernel services 310 to conduct a plurality of checks. One of these checks is directed to an analysis of the operating state of the guest agent process 175 (i.e., halted, disabled, in operation, etc.). Another check involves an evaluation of data structures associated with the guest agent process 175 or other software components within the VM 170 to determine whether such data structures have been tampered. Another check involves an evaluation of critical guest OS data structures, such as a system call table (not shown) to determine if entry points for any of the system calls have been maliciously changed.
2. Guest Agent
According to one embodiment of the disclosure, the guest agent 175 is a software component configured to provide the virtualization layer 185 with metadata that may assist in the handling of exploit detection. Instrumented into a guest software application 320, guest OS kernel 301 or operating as a separate module, the guest agent 175 is configured to provide metadata to the virtualization layer 185 in response to at least one selected event.
Herein, the guest agent 175 comprises one or more ring buffers 340 (e.g., queue, FIFO, shared memory, buffer and/or registers), which records certain events that may be considered of interest for malware detection. Examples of these events may include information associated with a newly created process (e.g., process identifier, time of creation, originating source for creation of the new process, etc.), information about the type and location of certain data structures, information associated with an access to certain restricted port or memory address, or the like. The recovery of the information associated with the stored events may occur through a “pull” or “push” recovery scheme, where the guest agent 175 may be configured to download the metadata periodically or aperiodically (e.g., when the ring buffer 340 exceeds a certain storage level or in response to a request). The request may originate from the threat protection component 376 and is generated by the guest monitor component 374.
B. Host Environment
As further shown in
Running on top of the micro-hypervisor 360 in ring-3 352, a plurality of processes being instances of host applications (referred to as “hyper-processes” 370) communicate with the micro-hypervisor 360. Some of these hyper-processes 370 include master controller component 372, guest monitor component 374 and threat protection component 376. Each of these hyper-processes 372, 374 and 376 represents a separate software component with different functionality and is running in a separate address space. As these hyper-processes 370 are isolated from each other (i.e. not in the same binary), inter-process communications between the hyper-processes 370 are handled by the micro-hypervisor 360, but regulated through policy protection by the master controller component 372.
1. Micro-Hypervisor
The micro-hypervisor 360 may be configured as a light-weight hypervisor (e.g., less than 10 K lines of code) that operates as a host OS kernel. The micro-hypervisor 360 features logic (mechanisms) for controlling operability of the computing device, such as endpoint device 1403 as shown. The mechanisms include inter-process communication (IPC) logic 362, resource allocation logic 364, scheduling logic 366 and interrupt delivery control logic 368, where all of these mechanisms are based, at least in part, on a plurality of kernel features—protection domains, execution context, scheduling context, portals, and semaphores (hereinafter collectively as “kernel features 369”) as partially described in a co-pending U.S. Patent Application entitled “Microvisor-Based Malware Detection Endpoint Architecture” (U.S. patent application Ser. No. 14/929,821), the entire contents of which are incorporated herein by reference.
More specifically, a first kernel feature is referred to as “protection domains,” which correspond to containers where certain resources for the hyper-processes 370 can be assigned, such as various data structures (e.g., execution contexts, scheduling contexts, etc.). Given that each hyper-process 370 corresponds to a different protection domain, a first hyper-process (e.g., master controller component 372) is spatially isolated from a second (different) hyper-process (e.g., guest monitor component 374). Furthermore, the first hyper-process is spatially isolated (with the address space) from the virtual machine 170 as well.
A second kernel feature is referred to as an “execution context,” which features thread level activities within one of the hyper-processes (e.g., master controller component 372). These activities may include, inter alia, (i) contents of hardware registers, (ii) pointers/values on a stack, (iii) a program counter, and/or (iv) allocation of memory via, e.g., memory pages. The execution context is thus a static view of the state of a thread of execution.
Accordingly, the thread executes within a protection domain associated with that hyper-process of which the thread is a part. For the thread to execute on a hardware processor 210, its execution context may be tightly linked to a scheduling context (third kernel feature), which may be configured to provide information for scheduling the execution context for execution on the hardware processor 210. Illustratively, the scheduling context may include a priority and a quantum time for execution of its linked execution context on the hardware processor 210.
Hence, besides the spatial isolation provided by protection domains, the micro-hypervisor 360 enforces temporal separation through the scheduling context, which is used for scheduling the processing of the execution context as described above. Such scheduling by the micro-hypervisor 300 may involve defining which hardware processor may process the execution context (in a multi-processor environment), what priority is assigned the execution priority, and the duration of such execution.
Communications between protection domains are governed by portals, which represent a fourth kernel feature that is relied upon for generation of the IPC logic 362. Each portal represents a dedicated entry point into a corresponding protection domain. As a result, if one protection domain creates the portal, another protection domain may be configured to call the portal and establish a cross-domain communication channel.
Lastly, of the kernel features, semaphores facilitate synchronization between execution context on the same or on different hardware processors. The micro-hypervisor 360 uses the semaphores to signal the occurrence of hardware interrupts to the user applications.
The micro-hypervisor 360 utilizes one or more of these kernel features to formulate mechanisms for controlling operability of the endpoint device 200. One of these mechanisms is the IPC logic 362, which supports communications between separate protection domains (e.g., between two different hyper-processes 370). Thus, under the control of the IPC logic 362, in order for a first software component to communicate with another software component, the first software component needs to route a message to the micro-hypervisor 360. In response, the micro-hypervisor 360 switches from a first protection domain (e.g., first hyper-process 372) to a second protection domain (e.g., second hyper-process 374) and copies the message from an address space associated with the first hyper-process 372 to a different address space associated with the second hyper-process 374. The same mechanism can also be used for communicating between two execution controls in the same protection domain, in which case no address space switch occurs.
Another mechanism provided by the micro-hypervisor 360 is resource allocation logic 364. The resource allocation logic 364 enables a first software component to share one or more memory pages with a second software component under the control of the micro-hypervisor 360. Being aware of the location of one or more memory pages, the micro-hypervisor 360 provides the protection domain associated with the second software component access to the memory location(s) associated with the one or more memory pages.
Also, the micro-hypervisor 360 contains scheduling logic 366 that, when invoked, selects the highest-priority scheduling context and dispatches the execution context associated with the scheduling context. As a result, the scheduling logic 366 ensures that, at some point in time, all of the software components can run on the hardware processor 210 as defined by the scheduling context. Also, the scheduling logic 366 re-enforces that no component can monopolize the hardware processor 210 longer than defined by the scheduling context.
Lastly, the micro-hypervisor 360 contains an interrupt delivery control logic 368 that, when driven by the micro-hypervisor 360, any interrupts that occur are also delivered to the micro-hypervisor 360.
2. Master Controller
Referring still to
Additionally, the master controller component 372 is responsible for the allocation of resources. Initially, the master controller component 372 receives access to most of the physical resources, except for access to security critical resources that should be driven by high privileged (host mode, ring-0) components, not user space (host mode, ring-3) software components such as hyper-processes 370. For instance, while precluded for access to the memory management unit (MMU) or the interrupt controller, the master controller component 372 may be configured to control selecting which software components are responsible for drive certain network devices.
The master controller component 372 is platform agnostic. Thus, the master controller component 372 may be configured to enumerate what hardware is available to a particular process (or software component) and to configure the state of the hardware (e.g., activate, place into sleep state, etc.).
By separating the master controller component 372 from the micro-hypervisor 360, a number of benefits are achieved. One inherent benefit is increased security. When the functionality is placed into a single binary (executable), which is running in host mode, any vulnerability may place the entire computing device at risk. In contrast, each of the software components within the host mode is running in its own separate address space.
3. Guest Monitor
Referring still to
In response an extended page table (EPT) violation, which causes a VM exit for the guest OS 300 to the virtualization layer 185, the guest monitor component 374 identifies that an unpermitted operation was attempted on a memory page associated with the nested page table. The presence of the trap may prompt the guest monitor component 374 to obtain and forward metadata associated with the EPT violation (as monitored by the guest agent 175) to the threat protection component 376. Based on the metadata, the threat protection component 376 determines if the event was malicious or not.
If the event was benign, although the page is access protected, the guest monitor component 374 may be responsible for emulating the attempted access. For instance, for an EPT violation triggered for a write-protection violation that is determined to be benign, the guest monitor component 374 would need to simulate the write access and its side effects.
As an illustrative example, it is noted that there are certain events that cause a VM exit (a transition of execution from the guest mode to the host mode). The guest monitor component 374 can configure, on an event basis, which events should trigger a transition from the guest mode to the host mode. One event may involve the execution of a privileged processor instruction by a vCPU within the virtual machine 170. In response to execution by the vCPU of a privileged instruction, the micro-hypervisor 360 gains control of the platform and generates a message to the guest monitor component 374, which is responsible for handling the event.
The guest monitor component 374 also manages permissions of the nested page tables under control of the virtualization layer. More specifically, the micro-hypervisor 360 includes a mechanism (i.e. paging control logic 365) to populate the nested page tables or ensures that no hyper-process can delegate resources that it does not have access to. In particular, no hyper-process is able to grant access to micro-hypervisor memory regions. The guest monitor component 374 features permission adjustment logic 375 that alters the page permissions. One technique in altering the page permissions may involve selecting a particular nested page table among multiple nested page tables, which provides the same memory address translation but is set with page permissions for the targeted memory pages that differ from page permissions for other nested page tables. Some of the functionality of the permission adjustment logic 375 may be based, at least in part, on functionality within paging control logic 365 that is accessible via an API (not shown).
The guest monitor component 374 also includes interrupt injection logic 377, which is responsible for handling the injection of virtual interrupts to the ISR 315 within the kernel services 310. The virtual interrupts are intended for the ISR agent 315 to assume control over certain operations of the virtual machine 170.
4. Threat Protection Component
As described above and shown in
After analysis, the detected events are correlated and classified as benign (i.e., determination of the analyzed object 335 being malicious is less than a first level of probability); suspicious (i.e., determination of the analyzed object 335 being malicious is between the first level and a second level of probability); or malicious (i.e., determination of the analyzed object 335 being malicious is greater than the second level of probability). The correlation and classification operations may be accomplished by a behavioral analysis logic 380 and a classifier 385. The behavioral analysis logic 380 and classifier 385 may cooperate to analyze and classify certain observed behaviors of the object (based on events) as indicative of malware. In particular, the observed run-time behaviors by the guest agent 175 are provided to the behavioral analysis logic 380 as dynamic analysis results. These events may include metadata and other information associated with an EPT violation that causes a VM exit to the virtualization layer that is delivered as an event to the guest monitor component 374. As a result, the guest monitor component 374 receives metadata associated with the events from the guest agent 175 and routes the same to the threat protection component 376.
At this time, the static analysis results and dynamic analysis results may be stored in memory 220, along with any additional metadata from the guest agent 175. These results may be provided via coordinated IPC-based communication to the behavioral analysis logic 380, which may provide correlation information to the classifier 385. Additionally or in the alternative, the results and/or events may be provided or reported via a network device initiated by the guest OS kernel to the MDS 1402 for correlation. The behavioral analysis logic 380 may be embodied as a rules-based correlation engine illustratively executing as an isolated process (software component) that communicates with the guest environment 160 via the guest monitor component 374.
In an embodiment, the behavioral analysis logic 380 may be configured to operate on correlation rules that define, among other things, patterns (e.g., sequences) of known malicious events (if-then statements with respect to, e.g., attempts by a process to change memory in a certain way that is known to be malicious) and/or non-malicious events. The events may collectively correlate to malicious behavior. The rules of the behavioral analysis logic 380 may then be correlated against those dynamic analysis results, as well as static analysis results, to generate correlation information pertaining to, e.g., a level of risk or a numerical score used to arrive at a decision of (deduce) maliciousness.
The classifier 385 may be configured to use the correlation information provided by behavioral analysis logic 380 to render a decision as to whether the object 335 is malicious. Illustratively, the classifier 385 may be configured to classify the correlation information, including monitored behaviors (expected and unexpected/anomalous) and access violations, of the object 335 relative to those of known malware and benign content.
Periodically or a periodically, rules may be pushed from the MDS 1402 to the endpoint 1403 to update the behavioral analysis logic 380, wherein the rules may be embodied as different (updated) behaviors to monitor. For example, the correlation rules pushed to the behavioral analysis logic 380 may include, for example, rules that specify a level of probability of maliciousness, whether a running process or application program has spawned processes; requests to use certain network ports that are not ordinarily used by the application program; or attempts to access data in memory locations not allocated to the guest application running the object. Alternatively, the correlation rules may be pulled based on a request from an endpoint device 1403 to determine whether new rules are available, and in response, the new rules are downloaded.
Illustratively, the behavioral analysis logic 380 and classifier 385 may be implemented as separate modules although, in the alternative, the behavioral analysis logic 380 and classifier 385 may be implemented as a single module disposed over (i.e., running on top of) the micro-hypervisor 360. The behavioral analysis logic 380 may be configured to correlate observed behaviors (e.g., results of static and dynamic analysis) with known malware and/or benign objects (embodied as defined rules) and generate an output (e.g., a level of risk or a numerical score associated with an object) that is provided to and used by the classifier 385 to render a decision of malware based on the risk level or score exceeding a probability threshold. The reporting module 336, which executes as a user mode process in the guest OS kernel 301, is configured to generate an alert for transmission external to the endpoint device 1402 (e.g., to one or more other endpoint devices, a management appliance, or MDS 1402) in accordance with “post-solution” activity.
According to one embodiment of the disclosure, the virtualization layer 185 provides enhanced security of a software component operating within a virtual machine by protecting the integrity, confidentiality and availability of that software component. For instance, the protected software component (e.g., guest agent process 175) may be a separate instance (as shown in
More specifically, as shown in
As shown in
The second memory address translation (GPA-to-HPA) 420 is conducted through use of nested page tables 445, sometimes referred to as extended page tables (EPTs), under the control of the virtualization layer 185 (e.g., the micro-hypervisor 360 and guest monitor component 374 of
In general, the integrity and confidentiality of the guest agent process 175 is protected via the first security mechanism that is configured to limit (or restrict) access, at certain times, to certain memory pages 450 that may contain code (and/or data structures) pertaining to the software component and/or metadata captured by that software component. These certain times are determined by monitoring for a change in the CR3 register 440 to/from a value corresponding to the protected software component.
According to one embodiment of the disclosure, access is mitigated to the guest agent process 175 by tightening page permissions for the memory pages 450, which are part of the nested page tables 445 associated with the GPA-to-HPA address translation 420, when the protected software component is inactive (e.g., non-execution state).
When in operation, in order to provide enhanced security to the computing device and protect the receipt of metadata into the virtualization layer, the first security mechanism is configured to (1) identify when a guest process change has occurred and (2) responsive to an identified guest process change, determine whether a different nested page table hierarchy from nested page tables 445 is needed for that active process. As described below in detail, the guest process change may be observed by detecting a guest process switch (e.g., a change in a specific data store such as processor control register “CR3” 440) or guest address space switch. In response to a guest process change, a change of the current nested page table hierarchy may be determined. The different nested page table hierarchies may correspond to different protection schemes, which are tailored for the particular active process. These protection schemes may remove some or all of the page permissions associated with the memory pages for one or more processes that are currently inactive.
For instance, no change may be necessary where the guest process switch identifies a change from a normal (non-protected) guest process to another normal guest process. However, a change may be necessary where the guest process switch identifies a change from a normal guest process to a protected guest process, between different protected guest processes, or from a protected guest process to a normal guest process.
In response to a memory access to virtual memory, a two-step memory address translation occurs within the virtualization architecture. As described below, a first memory address translation includes the GLA-to-GPA translation 400. Herein, the first memory address translation (GLA-to-GPA) 400 is conducted based on operations by the guest OS kernel 301 and a memory management unit “MMU” (not shown) of the physical hardware, which utilize a first data store (e.g., the CR3 register 440) in accordance with x86 processor architectures) to identify a memory address location for one or more guest page tables 430 associated with this particular memory access. The guest page tables 430 are data structures whose access is under control of the guest OS (e.g., Windows® OS, Linux™ OS, etc.) and accessible by the MMU that conducts the memory address translation.
More specifically, according to one embodiment of the disclosure, the guest OS kernel 301 is configured to generate and manage a plurality of guest page tables 430, namely a first guest page table hierarchy (e.g., at least a first guest page table 435) that is associated with a first process and a second guest page table hierarchy (e.g., at least a second guest page table) that is associated with another (second) process. According to one embodiment, a single guest page table or a plurality of guest page tables may be referred to as a “guest page table hierarchy”. The first data store 440 (e.g., CR3 register, specific memory address, etc.) is loaded with a guest page table pointer, namely a base (root) address pointer to a memory address location for a particular guest page table hierarchy associated with the active process that is accessing virtual memory.
Responsive to a memory address (GLA-to-GPA) translation conducted for a first (active) process, the MMU relies on the guest page table pointer stored in the first data store by the guest OS kernel 301 to locate the data structures for the first guest page table hierarchy. Hence, the MMU relies on the first guest page table hierarchy to translate a memory access by the first process from a guest-linear address (GLA) to the guest physical address (GPA). As the guest OS switches from the first process (now inactive) to a second process (now active), the guest OS kernel 301 alters the contents of the first data store 440 by storing a guest page table pointer that points to a second guest page table hierarchy corresponding to the second process.
Stated differently, according to one embodiment of the disclosure, the guest OS kernel 301 is configured to (i) modify the content within a guest page table hierarchy (one or more guest page tables) associated with the GLA-to-GPA translation 400 for any process and/or modify the access privileges for certain memory pages that are part of the guest page table hierarchy. The guest OS kernel 301 is also configured to change the guest page table pointer within the first data store in order to switch, when necessary, from one guest page table hierarchy associated with one process to another guest page table hierarchy associated with another process that is now active.
Additionally, a second memory address translation includes the GPA-to-HPA translation 420. The second memory address translation (GPA-to-HPA) is conducted through use of nested page tables 445 under the control of the virtualization layer 185 (e.g., the micro-hypervisor and guest monitor component described above). Sometimes referred to as an extended page table (EPT), a particular nested page table (e.g., nested page table 450) is referenced (pointed to) by an address pointer referred to as an “EPT pointer” (EPTP). The EPTP is stored in a second data store 455 (e.g., another processor control register other than CR3) that is accessible to one or more software components within the virtualization layer 185.
More specifically, according to one embodiment of the disclosure, the virtualization layer 185 is configured to (i) modify the content of one or more nested page tables 445 associated with the GPA-to-HPA translation 420 and/or the access privileges for certain memory pages that are part of these nested page tables 445, and (ii) change the EPTP in the second data store 455 to switch, when necessary and for the same virtual machine, from one nested page table to another nested page table. According to one embodiment, each nested page table may represent different protection levels (e.g., different page permissions). The EPTP corresponds to a base (root) address pointer for one or more nested page tables (i.e., nested page table hierarchy) for the currently active process.
Responsive to an observed guest process switch and based, at least in part, on which guest process is currently active, the virtualization layer 185 may be configured to select which of the nested page tables 445 are used for GPA-to-HPA translation 420. By selecting among a plurality of nested page tables based on the switched, active guest process, the access privileges (e.g., page permissions) for certain memory pages associated with a protected process (e.g., guest agent process) may be restricted when the active process is not that protected process. Similarly, the access privileges for certain memory pages may be set to unrestricted (normal) access privileges when the active process is that protected process (or one of the group of protected processes).
As an illustrative example, a first nested page table corresponds to the guest agent process being active and a second nested page table that corresponds to other processes. Hence, for the first nested page table, the page permissions for memory pages associated with the guest agent process are available and have not been overridden. For the second nested page table, however, the page permissions for memory pages associated with the guest agent process are protected by partially or completely removing certain page permissions (e.g., write permission removed, execute permission removed, both write and execute permissions removed, or all page permissions removed).
As another illustrative example, multiple processes may be protected, where some of these processes may have different protection schemes. For instance, one nested page table hierarchy corresponds to a first protected process (e.g. a guest agent process), while a second nested page table hierarchy corresponds to a second protected process (e.g., anti-virus process) and a third nested page table hierarchy corresponds to a third protected process (e.g., intrusion protection system). A fourth nested page table hierarchy corresponds to other processes. Hence, whenever a particular process is active in response to a guest process switch (and review of the content of the first data store such as the CR3 register), the corresponding nested page table hierarchy may be configured to remove (override) certain page permissions associated with the memory pages for any processes that are not currently active.
The nested page tables (EPT) cannot just override (relax/tighten) page permissions for a particular guest-physical address (GPA). Because EPT translates GPA-to-HPA, it can also be used to translate to a different HPA, depending on which process is active. As an illustrative example, an unprotected process is active. Then, the respective nested page tables may translate a particular GPA (G) to the corresponding HPA (H). However, if a protected process is active, the respective (different) nested page tables may translate the same GPA (G) to a different HPA (H′). Where “H” and “H′” are different pages in memory (e.g. RAM). So, depending on the active guest process, the virtualization layer can activate a different nested page table (EPT) to (1) set different (from normal) permissions and/or (2) set different (from normal) translations.
Referring to
Thereafter, in response to a VM Exit, the virtualization layer determines whether the VM Exit is due to a change in the guest page table (GPT) pointer (block 510). If not, the virtualization layer conducts VM Exit handling unrelated to nested page table selection and, upon completion, the virtualization layer returns control to the VM (blocks 520 and 560). However, if the virtualization layer determines that the VM Exit is due to a change in the guest page table (GPT) pointer, the virtualization layer determines whether the content of the GPT pointer indicates that the newly active guest process is a protected process, such as the guest agent process for example (block 530).
Where the newly active guest process is a protected process, the virtualization layer ensures that a nested page table hierarchy (e.g., one or more nested page tables) associated with the protected process is used by the MMU to complete the GPA-to-HPA memory address translation, where the nested page table for the protected process may impose permission overrides for certain memory pages belonging to any of the other/different protected processes (block 540). Thereafter, the virtualization layer returns control to the virtual machine (block 560), where the virtualization layer awaits another VM Exit.
However, where the newly active guest process is not a protected process, the virtualization layer ensures that a nested page table hierarchy (e.g., one or more nested page tables) associated with the particular (unprotected) process is used by the MMU to complete the GPA-to-HPA memory address translation (block 550). This nested page table hierarchy may impose page permission overrides, which may remove certain page permissions for memory pages associated with one or more protected processes. Thereafter, the virtualization layer returns control to the virtual machine (block 560).
Referring now to
Also, in order to allocate guaranteed execution time to the agent, the guest monitor component is configured with interrupt injection logic to inject a virtual interrupt into the guest OS kernel that, when handled, causes the ISR to perform particular services (block 620). The frequency of the virtual interrupts may be periodic or perhaps aperiodic in nature. Furthermore, the services may vary, as the purpose for the interrupt is to ensure that the protected software component is not hijacked and disabled. The services may include, but are not limited or restricted to checking for the integrity of certain critical OS or data structures for the protected process, or requesting a response message from the protected process to verifying that it is not disabled.
After configuration, a determination is made as to whether an event has occurred to cause a virtual interrupt to be issued (block 630). For example, this operation may occur after a prescribed period of time has elapsed, when the frequency of the virtual interrupts is periodic. Similarly, this operation may occur in response to detection of a particular event (e.g., EPT violation) when the frequency of the virtual interrupts is aperiodic. The determination is iterative, until the virtual interrupt is to be issued. Thereafter, the virtual interrupt is received by the guest OS kernel, where the virtual interrupt will cause execution to vector to the protected ISR within the guest OS kernel at the next suitable point (block 640).
The protected ISR can then perform operations that are selected to cause operations to be conducted by the protected process (blocks 640 and 650). For instance, upon receipt of a first type of virtual interrupt, the ISR may check the integrity of certain critical OS data structures and/or data structures associated with the protected process in order to determine whether any of these data structures has been tampered with. Upon receipt of a second type of virtual interrupt, the ISR may check operability of the protected process and determine whether that process has been disabled. Upon receipt of a third type of virtual interrupt, the ISR may determine whether an entry within a system call table has altered to change an API call.
Thereafter, as shown, the second security scheme continues in an iterative manner to guarantee processing time for the protected software component.
In the foregoing description, the invention is described with reference to specific exemplary embodiments thereof. For instance, the first and second security mechanisms described above may be deployed in a MDS appliance instead of an endpoint device or in another computing device other than the MDS appliance and the endpoint device. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention as set forth in the appended claims.
This application is based upon and claims the benefit of priority from U.S. Provisional Patent Application No. 62/187,108 filed Jun. 30, 2015, the entire contents of which are incorporated herein by reference.
Number | Name | Date | Kind |
---|---|---|---|
4292580 | Ott et al. | Sep 1981 | A |
5175732 | Hendel et al. | Dec 1992 | A |
5319776 | Hile et al. | Jun 1994 | A |
5440723 | Arnold et al. | Aug 1995 | A |
5490249 | Miller | Feb 1996 | A |
5657473 | Killean et al. | Aug 1997 | A |
5802277 | Cowlard | Sep 1998 | A |
5842002 | Schnurer et al. | Nov 1998 | A |
5878560 | Johnson | Mar 1999 | A |
5960170 | Chen et al. | Sep 1999 | A |
5978917 | Chi | Nov 1999 | A |
5983348 | Ji | Nov 1999 | A |
6013455 | Bandman et al. | Jan 2000 | A |
6088803 | Tso et al. | Jul 2000 | A |
6092194 | Touboul | Jul 2000 | A |
6094677 | Capek et al. | Jul 2000 | A |
6108799 | Boulay et al. | Aug 2000 | A |
6154844 | Touboul et al. | Nov 2000 | A |
6269330 | Cidon et al. | Jul 2001 | B1 |
6272641 | Ji | Aug 2001 | B1 |
6279113 | Vaidya | Aug 2001 | B1 |
6298445 | Shostack et al. | Oct 2001 | B1 |
6357008 | Nachenberg | Mar 2002 | B1 |
6424627 | Sorhaug et al. | Jul 2002 | B1 |
6442696 | Wray et al. | Aug 2002 | B1 |
6484315 | Ziese | Nov 2002 | B1 |
6487666 | Shanklin et al. | Nov 2002 | B1 |
6493756 | O'Brien et al. | Dec 2002 | B1 |
6550012 | Villa et al. | Apr 2003 | B1 |
6775657 | Baker | Aug 2004 | B1 |
6831893 | Ben Nun et al. | Dec 2004 | B1 |
6832367 | Choi et al. | Dec 2004 | B1 |
6895550 | Kanchirayappa et al. | May 2005 | B2 |
6898632 | Gordy et al. | May 2005 | B2 |
6907396 | Muttik et al. | Jun 2005 | B1 |
6941348 | Petry et al. | Sep 2005 | B2 |
6971097 | Wallman | Nov 2005 | B1 |
6981279 | Arnold et al. | Dec 2005 | B1 |
7007107 | Ivchenko et al. | Feb 2006 | B1 |
7028179 | Anderson et al. | Apr 2006 | B2 |
7043757 | Hoefelmeyer et al. | May 2006 | B2 |
7058791 | Hughes et al. | Jun 2006 | B1 |
7058822 | Edery et al. | Jun 2006 | B2 |
7069316 | Gryaznov | Jun 2006 | B1 |
7080407 | Zhao et al. | Jul 2006 | B1 |
7080408 | Pak et al. | Jul 2006 | B1 |
7093002 | Wolff et al. | Aug 2006 | B2 |
7093239 | van der Made | Aug 2006 | B1 |
7096498 | Judge | Aug 2006 | B2 |
7100201 | Izatt | Aug 2006 | B2 |
7107617 | Hursey et al. | Sep 2006 | B2 |
7159149 | Spiegel et al. | Jan 2007 | B2 |
7213260 | Judge | May 2007 | B2 |
7231667 | Jordan | Jun 2007 | B2 |
7240364 | Branscomb et al. | Jul 2007 | B1 |
7240368 | Roesch et al. | Jul 2007 | B1 |
7243371 | Kasper et al. | Jul 2007 | B1 |
7249175 | Donaldson | Jul 2007 | B1 |
7287278 | Liang | Oct 2007 | B2 |
7308716 | Danford et al. | Dec 2007 | B2 |
7328453 | Merkle, Jr. et al. | Feb 2008 | B2 |
7346486 | Ivancic et al. | Mar 2008 | B2 |
7356736 | Natvig | Apr 2008 | B2 |
7386888 | Liang et al. | Jun 2008 | B2 |
7392542 | Bucher | Jun 2008 | B2 |
7409719 | Armstrong et al. | Aug 2008 | B2 |
7418729 | Szor | Aug 2008 | B2 |
7424745 | Cheston et al. | Sep 2008 | B2 |
7428300 | Drew et al. | Sep 2008 | B1 |
7441272 | Durham et al. | Oct 2008 | B2 |
7448084 | Apap et al. | Nov 2008 | B1 |
7458098 | Judge et al. | Nov 2008 | B2 |
7464404 | Carpenter et al. | Dec 2008 | B2 |
7464407 | Nakae et al. | Dec 2008 | B2 |
7467408 | O'Toole, Jr. | Dec 2008 | B1 |
7478428 | Thomlinson | Jan 2009 | B1 |
7480773 | Reed | Jan 2009 | B1 |
7487543 | Arnold et al. | Feb 2009 | B2 |
7496960 | Chen et al. | Feb 2009 | B1 |
7496961 | Zimmer et al. | Feb 2009 | B2 |
7519990 | Xie | Apr 2009 | B1 |
7523493 | Liang et al. | Apr 2009 | B2 |
7530104 | Thrower et al. | May 2009 | B1 |
7540025 | Tzadikario | May 2009 | B2 |
7546638 | Anderson et al. | Jun 2009 | B2 |
7565550 | Liang et al. | Jul 2009 | B2 |
7568233 | Szor et al. | Jul 2009 | B1 |
7584455 | Ball | Sep 2009 | B2 |
7603715 | Costa et al. | Oct 2009 | B2 |
7607171 | Marsden et al. | Oct 2009 | B1 |
7639714 | Stolfo et al. | Dec 2009 | B2 |
7644441 | Schmid et al. | Jan 2010 | B2 |
7657419 | van der Made | Feb 2010 | B2 |
7676841 | Sobchuk et al. | Mar 2010 | B2 |
7698548 | Shelest et al. | Apr 2010 | B2 |
7707633 | Danford et al. | Apr 2010 | B2 |
7712136 | Sprosts et al. | May 2010 | B2 |
7730011 | Deninger et al. | Jun 2010 | B1 |
7739740 | Nachenberg et al. | Jun 2010 | B1 |
7779463 | Stolfo et al. | Aug 2010 | B2 |
7784097 | Stolfo et al. | Aug 2010 | B1 |
7832008 | Kraemer | Nov 2010 | B1 |
7836502 | Zhao et al. | Nov 2010 | B1 |
7849506 | Dansey et al. | Dec 2010 | B1 |
7854007 | Sprosts et al. | Dec 2010 | B2 |
7869073 | Oshima | Jan 2011 | B2 |
7877803 | Enstone et al. | Jan 2011 | B2 |
7904959 | Sidiroglou et al. | Mar 2011 | B2 |
7908660 | Bahl | Mar 2011 | B2 |
7930738 | Petersen | Apr 2011 | B1 |
7937387 | Frazier et al. | May 2011 | B2 |
7937761 | Bennett | May 2011 | B1 |
7949849 | Lowe et al. | May 2011 | B2 |
7958558 | Leake et al. | Jun 2011 | B1 |
7996556 | Raghavan et al. | Aug 2011 | B2 |
7996836 | McCorkendale et al. | Aug 2011 | B1 |
7996904 | Chiueh et al. | Aug 2011 | B1 |
7996905 | Arnold et al. | Aug 2011 | B2 |
8006305 | Aziz | Aug 2011 | B2 |
8010667 | Zhang et al. | Aug 2011 | B2 |
8020206 | Hubbard et al. | Sep 2011 | B2 |
8028338 | Schneider et al. | Sep 2011 | B1 |
8042184 | Batenin | Oct 2011 | B1 |
8045094 | Teragawa | Oct 2011 | B2 |
8045458 | Alperovitch et al. | Oct 2011 | B2 |
8069484 | McMillan et al. | Nov 2011 | B2 |
8087086 | Lai et al. | Dec 2011 | B1 |
8151263 | Venkitachalam et al. | Apr 2012 | B1 |
8171553 | Aziz et al. | May 2012 | B2 |
8176049 | Deninger et al. | May 2012 | B2 |
8176480 | Spertus | May 2012 | B1 |
8201169 | Venkitachalam et al. | Jun 2012 | B2 |
8201246 | Wu et al. | Jun 2012 | B1 |
8204984 | Aziz et al. | Jun 2012 | B1 |
8214905 | Doukhvalov et al. | Jul 2012 | B1 |
8220055 | Kennedy | Jul 2012 | B1 |
8225288 | Miller et al. | Jul 2012 | B2 |
8225373 | Kraemer | Jul 2012 | B2 |
8233882 | Rogel | Jul 2012 | B2 |
8234640 | Fitzgerald et al. | Jul 2012 | B1 |
8234709 | Viljoen et al. | Jul 2012 | B2 |
8239944 | Nachenberg et al. | Aug 2012 | B1 |
8260914 | Ranjan | Sep 2012 | B1 |
8266091 | Gubin et al. | Sep 2012 | B1 |
8266395 | Li | Sep 2012 | B2 |
8271978 | Bennett et al. | Sep 2012 | B2 |
8286251 | Eker et al. | Oct 2012 | B2 |
8290912 | Searls et al. | Oct 2012 | B1 |
8291499 | Aziz et al. | Oct 2012 | B2 |
8307435 | Mann et al. | Nov 2012 | B1 |
8307443 | Wang et al. | Nov 2012 | B2 |
8312545 | Tuvell et al. | Nov 2012 | B2 |
8321936 | Green et al. | Nov 2012 | B1 |
8321941 | Tuvell et al. | Nov 2012 | B2 |
8332571 | Edwards, Sr. | Dec 2012 | B1 |
8347380 | Satish et al. | Jan 2013 | B1 |
8353031 | Rajan et al. | Jan 2013 | B1 |
8365286 | Poston | Jan 2013 | B2 |
8365297 | Parshin et al. | Jan 2013 | B1 |
8370938 | Daswani et al. | Feb 2013 | B1 |
8370939 | Zaitsev et al. | Feb 2013 | B2 |
8375444 | Aziz et al. | Feb 2013 | B2 |
8381299 | Stolfo et al. | Feb 2013 | B2 |
8387046 | Montague et al. | Feb 2013 | B1 |
8397306 | Tormasov | Mar 2013 | B1 |
8402529 | Green et al. | Mar 2013 | B1 |
8418230 | Cornelius et al. | Apr 2013 | B1 |
8464340 | Ahn et al. | Jun 2013 | B2 |
8479174 | Chiriac | Jul 2013 | B2 |
8479276 | Vaystikh et al. | Jul 2013 | B1 |
8479291 | Bodke | Jul 2013 | B1 |
8479294 | Li et al. | Jul 2013 | B1 |
8510827 | Leake et al. | Aug 2013 | B1 |
8510828 | Guo et al. | Aug 2013 | B1 |
8510842 | Amit et al. | Aug 2013 | B2 |
8516478 | Edwards et al. | Aug 2013 | B1 |
8516590 | Ranadive et al. | Aug 2013 | B1 |
8516593 | Aziz | Aug 2013 | B2 |
8522236 | Zimmer et al. | Aug 2013 | B2 |
8522348 | Chen et al. | Aug 2013 | B2 |
8528086 | Aziz | Sep 2013 | B1 |
8533824 | Hutton et al. | Sep 2013 | B2 |
8539582 | Aziz et al. | Sep 2013 | B1 |
8549638 | Aziz | Oct 2013 | B2 |
8555391 | Demir et al. | Oct 2013 | B1 |
8561177 | Aziz et al. | Oct 2013 | B1 |
8566476 | Shiffer et al. | Oct 2013 | B2 |
8566946 | Aziz et al. | Oct 2013 | B1 |
8584094 | Dadhia et al. | Nov 2013 | B2 |
8584234 | Sobel et al. | Nov 2013 | B1 |
8584239 | Aziz et al. | Nov 2013 | B2 |
8595834 | Xie et al. | Nov 2013 | B2 |
8612659 | Serebrin et al. | Dec 2013 | B1 |
8627476 | Satish et al. | Jan 2014 | B1 |
8635696 | Aziz | Jan 2014 | B1 |
8682054 | Xue et al. | Mar 2014 | B2 |
8682812 | Ranjan | Mar 2014 | B1 |
8689333 | Aziz | Apr 2014 | B2 |
8695096 | Zhang | Apr 2014 | B1 |
8713631 | Pavlyushchik | Apr 2014 | B1 |
8713681 | Silberman et al. | Apr 2014 | B2 |
8726392 | McCorkendale et al. | May 2014 | B1 |
8739280 | Chess et al. | May 2014 | B2 |
8756696 | Miller | Jun 2014 | B1 |
8775715 | Tsirkin et al. | Jul 2014 | B2 |
8776180 | Kumar et al. | Jul 2014 | B2 |
8776229 | Aziz | Jul 2014 | B1 |
8782792 | Bodke | Jul 2014 | B1 |
8789172 | Stolfo et al. | Jul 2014 | B2 |
8789178 | Kejriwal et al. | Jul 2014 | B2 |
8793278 | Frazier et al. | Jul 2014 | B2 |
8793787 | Ismael et al. | Jul 2014 | B2 |
8799997 | Spiers et al. | Aug 2014 | B2 |
8805947 | Kuzkin et al. | Aug 2014 | B1 |
8806647 | Daswani et al. | Aug 2014 | B1 |
8832352 | Tsirkin et al. | Sep 2014 | B2 |
8832829 | Manni et al. | Sep 2014 | B2 |
8839245 | Khajuria et al. | Sep 2014 | B1 |
8850060 | Beloussov et al. | Sep 2014 | B1 |
8850570 | Ramzan | Sep 2014 | B1 |
8850571 | Staniford et al. | Sep 2014 | B2 |
8863279 | McDougal et al. | Oct 2014 | B2 |
8875295 | Lutas et al. | Oct 2014 | B2 |
8881234 | Narasimhan et al. | Nov 2014 | B2 |
8881271 | Butler, II | Nov 2014 | B2 |
8881282 | Aziz et al. | Nov 2014 | B1 |
8898788 | Aziz et al. | Nov 2014 | B1 |
8910238 | Lukacs et al. | Dec 2014 | B2 |
8935779 | Manni et al. | Jan 2015 | B2 |
8949257 | Shiffer et al. | Feb 2015 | B2 |
8984478 | Epstein | Mar 2015 | B2 |
8984638 | Aziz et al. | Mar 2015 | B1 |
8990939 | Staniford et al. | Mar 2015 | B2 |
8990944 | Singh et al. | Mar 2015 | B1 |
8997219 | Staniford et al. | Mar 2015 | B2 |
9003402 | Carbone et al. | Apr 2015 | B1 |
9009822 | Ismael et al. | Apr 2015 | B1 |
9009823 | Ismael et al. | Apr 2015 | B1 |
9027125 | Kumar et al. | May 2015 | B2 |
9027135 | Aziz | May 2015 | B1 |
9071638 | Aziz et al. | Jun 2015 | B1 |
9087199 | Sallam | Jul 2015 | B2 |
9092616 | Kumar et al. | Jul 2015 | B2 |
9092625 | Kashyap et al. | Jul 2015 | B1 |
9104867 | Thioux et al. | Aug 2015 | B1 |
9106630 | Frazier et al. | Aug 2015 | B2 |
9106694 | Aziz et al. | Aug 2015 | B2 |
9117079 | Huang et al. | Aug 2015 | B1 |
9118715 | Staniford et al. | Aug 2015 | B2 |
9159035 | Ismael et al. | Oct 2015 | B1 |
9171160 | Vincent et al. | Oct 2015 | B2 |
9176843 | Ismael et al. | Nov 2015 | B1 |
9189627 | Islam | Nov 2015 | B1 |
9195829 | Goradia et al. | Nov 2015 | B1 |
9197664 | Aziz et al. | Nov 2015 | B1 |
9213651 | Malyugin et al. | Dec 2015 | B2 |
9223972 | Vincent et al. | Dec 2015 | B1 |
9225740 | Ismael et al. | Dec 2015 | B1 |
9241010 | Bennett et al. | Jan 2016 | B1 |
9251343 | Vincent et al. | Feb 2016 | B1 |
9262635 | Paithane et al. | Feb 2016 | B2 |
9268936 | Butler | Feb 2016 | B2 |
9275229 | LeMasters | Mar 2016 | B2 |
9282109 | Aziz et al. | Mar 2016 | B1 |
9292686 | Ismael et al. | Mar 2016 | B2 |
9294501 | Mesdaq et al. | Mar 2016 | B2 |
9300686 | Pidathala et al. | Mar 2016 | B2 |
9306960 | Aziz | Apr 2016 | B1 |
9306974 | Aziz et al. | Apr 2016 | B1 |
9311479 | Manni et al. | Apr 2016 | B1 |
9355247 | Thioux et al. | May 2016 | B1 |
9356944 | Aziz | May 2016 | B1 |
9363280 | Rivlin et al. | Jun 2016 | B1 |
9367681 | Ismael et al. | Jun 2016 | B1 |
9398028 | Karandikar et al. | Jul 2016 | B1 |
9413781 | Cunningham et al. | Aug 2016 | B2 |
9426071 | Caldejon et al. | Aug 2016 | B1 |
9430646 | Mushtaq et al. | Aug 2016 | B1 |
9432389 | Khalid et al. | Aug 2016 | B1 |
9438613 | Paithane et al. | Sep 2016 | B1 |
9438622 | Staniford et al. | Sep 2016 | B1 |
9438623 | Thioux et al. | Sep 2016 | B1 |
9459901 | Jung et al. | Oct 2016 | B2 |
9467460 | Otvagin et al. | Oct 2016 | B1 |
9483644 | Paithane et al. | Nov 2016 | B1 |
9495180 | Ismael | Nov 2016 | B2 |
9497213 | Thompson et al. | Nov 2016 | B2 |
9507935 | Ismael et al. | Nov 2016 | B2 |
9516057 | Aziz | Dec 2016 | B2 |
9519782 | Aziz et al. | Dec 2016 | B2 |
9536091 | Paithane et al. | Jan 2017 | B2 |
9537972 | Edwards et al. | Jan 2017 | B1 |
9560059 | Islam | Jan 2017 | B1 |
9563488 | Fadel et al. | Feb 2017 | B2 |
9565202 | Kindlund et al. | Feb 2017 | B1 |
9591015 | Amin et al. | Mar 2017 | B1 |
9591020 | Aziz | Mar 2017 | B1 |
9594904 | Jain et al. | Mar 2017 | B1 |
9594905 | Ismael et al. | Mar 2017 | B1 |
9594912 | Thioux et al. | Mar 2017 | B1 |
9609007 | Rivlin et al. | Mar 2017 | B1 |
9626509 | Khalid et al. | Apr 2017 | B1 |
9628498 | Aziz et al. | Apr 2017 | B1 |
9628507 | Haq et al. | Apr 2017 | B2 |
9633134 | Ross | Apr 2017 | B2 |
9635039 | Islam et al. | Apr 2017 | B1 |
9641546 | Manni et al. | May 2017 | B1 |
9654485 | Neumann | May 2017 | B1 |
9661009 | Karandikar et al. | May 2017 | B1 |
9661018 | Aziz | May 2017 | B1 |
9674298 | Edwards et al. | Jun 2017 | B1 |
9680862 | Ismael et al. | Jun 2017 | B2 |
9690606 | Ha et al. | Jun 2017 | B1 |
9690933 | Singh et al. | Jun 2017 | B1 |
9690935 | Shiffer et al. | Jun 2017 | B2 |
9690936 | Malik et al. | Jun 2017 | B1 |
9736179 | Ismael | Aug 2017 | B2 |
9740857 | Ismael et al. | Aug 2017 | B2 |
9747446 | Pidathala et al. | Aug 2017 | B1 |
9756074 | Aziz et al. | Sep 2017 | B2 |
9773112 | Rathor et al. | Sep 2017 | B1 |
9781144 | Otvagin et al. | Oct 2017 | B1 |
9787700 | Amin et al. | Oct 2017 | B1 |
9787706 | Otvagin et al. | Oct 2017 | B1 |
9792196 | Ismael et al. | Oct 2017 | B1 |
9824209 | Ismael et al. | Nov 2017 | B1 |
9824211 | Wilson | Nov 2017 | B2 |
9824216 | Khalid et al. | Nov 2017 | B1 |
9825976 | Gomez et al. | Nov 2017 | B1 |
9825989 | Mehra et al. | Nov 2017 | B1 |
9838408 | Karandikar et al. | Dec 2017 | B1 |
9838411 | Aziz | Dec 2017 | B1 |
9838416 | Aziz | Dec 2017 | B1 |
9838417 | Khalid et al. | Dec 2017 | B1 |
9846776 | Paithane et al. | Dec 2017 | B1 |
9876701 | Caldejon et al. | Jan 2018 | B1 |
9888016 | Amin et al. | Feb 2018 | B1 |
9888019 | Pidathala et al. | Feb 2018 | B1 |
9910988 | Vincent et al. | Mar 2018 | B1 |
9912644 | Cunningham | Mar 2018 | B2 |
9912681 | Ismael et al. | Mar 2018 | B1 |
9912684 | Aziz et al. | Mar 2018 | B1 |
9912691 | Mesdaq et al. | Mar 2018 | B2 |
9912698 | Thioux et al. | Mar 2018 | B1 |
9916440 | Paithane et al. | Mar 2018 | B1 |
9921978 | Chan et al. | Mar 2018 | B1 |
9934376 | Ismael | Apr 2018 | B1 |
9934381 | Kindlund et al. | Apr 2018 | B1 |
9946568 | Ismael et al. | Apr 2018 | B1 |
9954890 | Staniford et al. | Apr 2018 | B1 |
9973531 | Thioux | May 2018 | B1 |
10002252 | Ismael et al. | Jun 2018 | B2 |
10019338 | Goradia et al. | Jul 2018 | B1 |
10019573 | Silberman et al. | Jul 2018 | B2 |
10025691 | Ismael et al. | Jul 2018 | B1 |
10025927 | Khalid et al. | Jul 2018 | B1 |
10027689 | Rathor et al. | Jul 2018 | B1 |
10027690 | Aziz et al. | Jul 2018 | B2 |
10027696 | Rivlin et al. | Jul 2018 | B1 |
10033747 | Paithane et al. | Jul 2018 | B1 |
10033748 | Cunningham et al. | Jul 2018 | B1 |
10033753 | Islam et al. | Jul 2018 | B1 |
10033759 | Kabra et al. | Jul 2018 | B1 |
10050998 | Singh | Aug 2018 | B1 |
10068091 | Aziz et al. | Sep 2018 | B1 |
10075455 | Zafar et al. | Sep 2018 | B2 |
10083302 | Paithane et al. | Sep 2018 | B1 |
10084813 | Eyada | Sep 2018 | B2 |
10089461 | Ha et al. | Oct 2018 | B1 |
10097573 | Aziz | Oct 2018 | B1 |
10104102 | Neumann | Oct 2018 | B1 |
10108446 | Steinberg et al. | Oct 2018 | B1 |
10121000 | Rivlin et al. | Nov 2018 | B1 |
10122746 | Manni et al. | Nov 2018 | B1 |
10133863 | Bu et al. | Nov 2018 | B2 |
10133866 | Kumar et al. | Nov 2018 | B1 |
10146810 | Shiffer et al. | Dec 2018 | B2 |
10148693 | Singh et al. | Dec 2018 | B2 |
10165000 | Aziz et al. | Dec 2018 | B1 |
10169585 | Pilipenko et al. | Jan 2019 | B1 |
10176095 | Ferguson et al. | Jan 2019 | B2 |
10176321 | Abbasi et al. | Jan 2019 | B2 |
10181029 | Ismael et al. | Jan 2019 | B1 |
10191858 | Tsirkin | Jan 2019 | B2 |
10191861 | Steinberg et al. | Jan 2019 | B1 |
10192052 | Singh et al. | Jan 2019 | B1 |
10198574 | Thioux et al. | Feb 2019 | B1 |
10200384 | Mushtaq et al. | Feb 2019 | B1 |
10210329 | Malik et al. | Feb 2019 | B1 |
10216927 | Steinberg | Feb 2019 | B1 |
10218740 | Mesdaq et al. | Feb 2019 | B1 |
10242185 | Goradia | Mar 2019 | B1 |
20010005889 | Albrecht | Jun 2001 | A1 |
20010047326 | Broadbent et al. | Nov 2001 | A1 |
20020013802 | Mori et al. | Jan 2002 | A1 |
20020018903 | Kokubo et al. | Feb 2002 | A1 |
20020038430 | Edwards et al. | Mar 2002 | A1 |
20020091819 | Melchione et al. | Jul 2002 | A1 |
20020095607 | Lin-Hendel | Jul 2002 | A1 |
20020116627 | Tarbotton et al. | Aug 2002 | A1 |
20020144156 | Copeland | Oct 2002 | A1 |
20020162015 | Tang | Oct 2002 | A1 |
20020166063 | Lachman et al. | Nov 2002 | A1 |
20020169952 | DiSanto et al. | Nov 2002 | A1 |
20020184528 | Shevenell et al. | Dec 2002 | A1 |
20020188887 | Largman et al. | Dec 2002 | A1 |
20020194490 | Halperin et al. | Dec 2002 | A1 |
20030021728 | Sharpe et al. | Jan 2003 | A1 |
20030074578 | Ford et al. | Apr 2003 | A1 |
20030084318 | Schertz | May 2003 | A1 |
20030101381 | Mateev et al. | May 2003 | A1 |
20030115483 | Liang | Jun 2003 | A1 |
20030188190 | Aaron et al. | Oct 2003 | A1 |
20030191957 | Hypponen et al. | Oct 2003 | A1 |
20030200460 | Morota et al. | Oct 2003 | A1 |
20030212902 | van der Made | Nov 2003 | A1 |
20030229801 | Kouznetsov et al. | Dec 2003 | A1 |
20030237000 | Denton et al. | Dec 2003 | A1 |
20040003323 | Bennett et al. | Jan 2004 | A1 |
20040006473 | Mills et al. | Jan 2004 | A1 |
20040015712 | Szor | Jan 2004 | A1 |
20040019832 | Arnold et al. | Jan 2004 | A1 |
20040025016 | Focke et al. | Feb 2004 | A1 |
20040047356 | Bauer | Mar 2004 | A1 |
20040083408 | Spiegel et al. | Apr 2004 | A1 |
20040088581 | Brawn et al. | May 2004 | A1 |
20040093513 | Cantrell et al. | May 2004 | A1 |
20040111531 | Staniford et al. | Jun 2004 | A1 |
20040117478 | Triulzi et al. | Jun 2004 | A1 |
20040117624 | Brandt et al. | Jun 2004 | A1 |
20040128355 | Chao et al. | Jul 2004 | A1 |
20040165588 | Pandya | Aug 2004 | A1 |
20040236963 | Danford et al. | Nov 2004 | A1 |
20040243349 | Greifeneder et al. | Dec 2004 | A1 |
20040249911 | Alkhatib et al. | Dec 2004 | A1 |
20040255161 | Cavanaugh | Dec 2004 | A1 |
20040268147 | Wiederin et al. | Dec 2004 | A1 |
20050005159 | Oliphant | Jan 2005 | A1 |
20050021740 | Bar et al. | Jan 2005 | A1 |
20050033960 | Vialen et al. | Feb 2005 | A1 |
20050033989 | Poletto et al. | Feb 2005 | A1 |
20050050148 | Mohammadioun et al. | Mar 2005 | A1 |
20050086523 | Zimmer et al. | Apr 2005 | A1 |
20050091513 | Mitomo et al. | Apr 2005 | A1 |
20050091533 | Omote et al. | Apr 2005 | A1 |
20050091652 | Ross et al. | Apr 2005 | A1 |
20050108562 | Khazan et al. | May 2005 | A1 |
20050114663 | Cornell et al. | May 2005 | A1 |
20050125195 | Brendel | Jun 2005 | A1 |
20050149726 | Joshi et al. | Jul 2005 | A1 |
20050157662 | Bingham et al. | Jul 2005 | A1 |
20050183143 | Anderholm et al. | Aug 2005 | A1 |
20050201297 | Peikari | Sep 2005 | A1 |
20050210533 | Copeland et al. | Sep 2005 | A1 |
20050238005 | Chen et al. | Oct 2005 | A1 |
20050240781 | Gassoway | Oct 2005 | A1 |
20050262562 | Gassoway | Nov 2005 | A1 |
20050265331 | Stolfo | Dec 2005 | A1 |
20050283839 | Cowburn | Dec 2005 | A1 |
20060010495 | Cohen et al. | Jan 2006 | A1 |
20060015416 | Hoffman et al. | Jan 2006 | A1 |
20060015715 | Anderson | Jan 2006 | A1 |
20060015747 | Van de Ven | Jan 2006 | A1 |
20060021029 | Brickell et al. | Jan 2006 | A1 |
20060021054 | Costa et al. | Jan 2006 | A1 |
20060031476 | Mathes et al. | Feb 2006 | A1 |
20060047665 | Neil | Mar 2006 | A1 |
20060070130 | Costea et al. | Mar 2006 | A1 |
20060075252 | Kallahalla et al. | Apr 2006 | A1 |
20060075496 | Carpenter et al. | Apr 2006 | A1 |
20060095968 | Portolani et al. | May 2006 | A1 |
20060101516 | Sudaharan et al. | May 2006 | A1 |
20060101517 | Banzhof et al. | May 2006 | A1 |
20060112416 | Ohta et al. | May 2006 | A1 |
20060117385 | Mester et al. | Jun 2006 | A1 |
20060123477 | Raghavan et al. | Jun 2006 | A1 |
20060130060 | Anderson et al. | Jun 2006 | A1 |
20060143709 | Brooks et al. | Jun 2006 | A1 |
20060150249 | Gassen et al. | Jul 2006 | A1 |
20060161983 | Cothrell et al. | Jul 2006 | A1 |
20060161987 | Levy-Yurista | Jul 2006 | A1 |
20060161989 | Reshef et al. | Jul 2006 | A1 |
20060164199 | Gilde et al. | Jul 2006 | A1 |
20060173992 | Weber et al. | Aug 2006 | A1 |
20060179147 | Tran et al. | Aug 2006 | A1 |
20060184632 | Marino et al. | Aug 2006 | A1 |
20060191010 | Benjamin | Aug 2006 | A1 |
20060221956 | Narayan et al. | Oct 2006 | A1 |
20060236127 | Kurien et al. | Oct 2006 | A1 |
20060236393 | Kramer et al. | Oct 2006 | A1 |
20060242709 | Seinfeld et al. | Oct 2006 | A1 |
20060248519 | Jaeger et al. | Nov 2006 | A1 |
20060248528 | Oney et al. | Nov 2006 | A1 |
20060248582 | Panjwani et al. | Nov 2006 | A1 |
20060251104 | Koga | Nov 2006 | A1 |
20060288417 | Bookbinder et al. | Dec 2006 | A1 |
20070006226 | Hendel | Jan 2007 | A1 |
20070006288 | Mayfield et al. | Jan 2007 | A1 |
20070006313 | Porras et al. | Jan 2007 | A1 |
20070011174 | Takaragi et al. | Jan 2007 | A1 |
20070016951 | Piccard et al. | Jan 2007 | A1 |
20070019286 | Kikuchi | Jan 2007 | A1 |
20070033645 | Jones | Feb 2007 | A1 |
20070038943 | FitzGerald et al. | Feb 2007 | A1 |
20070055837 | Rajagopal et al. | Mar 2007 | A1 |
20070064689 | Shin et al. | Mar 2007 | A1 |
20070074169 | Chess et al. | Mar 2007 | A1 |
20070094676 | Fresko et al. | Apr 2007 | A1 |
20070094730 | Bhikkaji et al. | Apr 2007 | A1 |
20070101435 | Konanka et al. | May 2007 | A1 |
20070128855 | Cho et al. | Jun 2007 | A1 |
20070142030 | Sinha et al. | Jun 2007 | A1 |
20070143565 | Corrigan et al. | Jun 2007 | A1 |
20070143827 | Nicodemus et al. | Jun 2007 | A1 |
20070156895 | Vuong | Jul 2007 | A1 |
20070157180 | Tillmann et al. | Jul 2007 | A1 |
20070157306 | Elrod et al. | Jul 2007 | A1 |
20070168988 | Eisner et al. | Jul 2007 | A1 |
20070171824 | Ruello et al. | Jul 2007 | A1 |
20070174915 | Gribble et al. | Jul 2007 | A1 |
20070192500 | Lum | Aug 2007 | A1 |
20070192858 | Lum | Aug 2007 | A1 |
20070198275 | Malden et al. | Aug 2007 | A1 |
20070208822 | Wang et al. | Sep 2007 | A1 |
20070220607 | Sprosts et al. | Sep 2007 | A1 |
20070240218 | Tuvell et al. | Oct 2007 | A1 |
20070240219 | Tuvell et al. | Oct 2007 | A1 |
20070240220 | Tuvell et al. | Oct 2007 | A1 |
20070240222 | Tuvell et al. | Oct 2007 | A1 |
20070250930 | Aziz et al. | Oct 2007 | A1 |
20070256132 | Oliphant | Nov 2007 | A2 |
20070271446 | Nakamura | Nov 2007 | A1 |
20070300227 | Mall et al. | Dec 2007 | A1 |
20080005782 | Aziz | Jan 2008 | A1 |
20080018122 | Zierler et al. | Jan 2008 | A1 |
20080028463 | Dagon et al. | Jan 2008 | A1 |
20080040710 | Chiriac | Feb 2008 | A1 |
20080046781 | Childs et al. | Feb 2008 | A1 |
20080065854 | Schoenberg et al. | Mar 2008 | A1 |
20080066179 | Liu | Mar 2008 | A1 |
20080072326 | Danford et al. | Mar 2008 | A1 |
20080077793 | Tan et al. | Mar 2008 | A1 |
20080080518 | Hoeflin et al. | Apr 2008 | A1 |
20080086720 | Lekel | Apr 2008 | A1 |
20080098476 | Syversen | Apr 2008 | A1 |
20080120722 | Sima et al. | May 2008 | A1 |
20080123676 | Cummings et al. | May 2008 | A1 |
20080127348 | Largman et al. | May 2008 | A1 |
20080134178 | Fitzgerald et al. | Jun 2008 | A1 |
20080134334 | Kim et al. | Jun 2008 | A1 |
20080141376 | Clausen et al. | Jun 2008 | A1 |
20080184367 | McMillan et al. | Jul 2008 | A1 |
20080184373 | Traut et al. | Jul 2008 | A1 |
20080189787 | Arnold et al. | Aug 2008 | A1 |
20080201778 | Guo et al. | Aug 2008 | A1 |
20080209557 | Herley et al. | Aug 2008 | A1 |
20080215742 | Goldszmidt et al. | Sep 2008 | A1 |
20080222729 | Chen et al. | Sep 2008 | A1 |
20080235793 | Schunter et al. | Sep 2008 | A1 |
20080244569 | Challener et al. | Oct 2008 | A1 |
20080263665 | Ma et al. | Oct 2008 | A1 |
20080294808 | Mahalingam et al. | Nov 2008 | A1 |
20080295172 | Bohacek | Nov 2008 | A1 |
20080301810 | Lehane et al. | Dec 2008 | A1 |
20080307524 | Singh et al. | Dec 2008 | A1 |
20080313738 | Enderby | Dec 2008 | A1 |
20080320594 | Jiang | Dec 2008 | A1 |
20090003317 | Kasralikar et al. | Jan 2009 | A1 |
20090007100 | Field et al. | Jan 2009 | A1 |
20090013408 | Schipka | Jan 2009 | A1 |
20090031423 | Liu et al. | Jan 2009 | A1 |
20090036111 | Danford et al. | Feb 2009 | A1 |
20090037835 | Goldman | Feb 2009 | A1 |
20090044024 | Oberheide et al. | Feb 2009 | A1 |
20090044274 | Budko et al. | Feb 2009 | A1 |
20090064332 | Porras et al. | Mar 2009 | A1 |
20090077666 | Chen et al. | Mar 2009 | A1 |
20090083369 | Marmor | Mar 2009 | A1 |
20090083855 | Apap et al. | Mar 2009 | A1 |
20090089860 | Forrester et al. | Apr 2009 | A1 |
20090089879 | Wang et al. | Apr 2009 | A1 |
20090094697 | Provos et al. | Apr 2009 | A1 |
20090106754 | Liu et al. | Apr 2009 | A1 |
20090113425 | Ports et al. | Apr 2009 | A1 |
20090125976 | Wassermann et al. | May 2009 | A1 |
20090126015 | Monastyrsky et al. | May 2009 | A1 |
20090126016 | Sobko et al. | May 2009 | A1 |
20090133125 | Choi et al. | May 2009 | A1 |
20090144823 | Lamastra et al. | Jun 2009 | A1 |
20090158430 | Borders | Jun 2009 | A1 |
20090158432 | Zheng et al. | Jun 2009 | A1 |
20090172661 | Zimmer et al. | Jul 2009 | A1 |
20090172815 | Gu et al. | Jul 2009 | A1 |
20090187992 | Poston | Jul 2009 | A1 |
20090193293 | Stolfo et al. | Jul 2009 | A1 |
20090198651 | Shiffer et al. | Aug 2009 | A1 |
20090198670 | Shiffer et al. | Aug 2009 | A1 |
20090198689 | Frazier et al. | Aug 2009 | A1 |
20090199274 | Frazier et al. | Aug 2009 | A1 |
20090199296 | Xie et al. | Aug 2009 | A1 |
20090204964 | Foley et al. | Aug 2009 | A1 |
20090228233 | Anderson et al. | Sep 2009 | A1 |
20090241187 | Troyansky | Sep 2009 | A1 |
20090241190 | Todd et al. | Sep 2009 | A1 |
20090254990 | McGee | Oct 2009 | A1 |
20090265692 | Godefroid et al. | Oct 2009 | A1 |
20090271867 | Zhang | Oct 2009 | A1 |
20090276771 | Nickolov et al. | Nov 2009 | A1 |
20090300415 | Zhang et al. | Dec 2009 | A1 |
20090300761 | Park et al. | Dec 2009 | A1 |
20090320011 | Chow et al. | Dec 2009 | A1 |
20090328185 | Berg et al. | Dec 2009 | A1 |
20090328221 | Blumfield et al. | Dec 2009 | A1 |
20100005146 | Drako et al. | Jan 2010 | A1 |
20100011205 | McKenna | Jan 2010 | A1 |
20100017546 | Poo et al. | Jan 2010 | A1 |
20100030996 | Butler, II | Feb 2010 | A1 |
20100031353 | Thomas et al. | Feb 2010 | A1 |
20100031360 | Seshadri et al. | Feb 2010 | A1 |
20100037314 | Perdisci et al. | Feb 2010 | A1 |
20100043073 | Kuwamura | Feb 2010 | A1 |
20100054278 | Stolfo et al. | Mar 2010 | A1 |
20100058474 | Hicks | Mar 2010 | A1 |
20100064044 | Nonoyama | Mar 2010 | A1 |
20100077481 | Polyakov et al. | Mar 2010 | A1 |
20100083376 | Pereira et al. | Apr 2010 | A1 |
20100100718 | Srinivasan | Apr 2010 | A1 |
20100115621 | Staniford et al. | May 2010 | A1 |
20100132038 | Zaitsev | May 2010 | A1 |
20100154056 | Smith et al. | Jun 2010 | A1 |
20100180344 | Malyshev et al. | Jul 2010 | A1 |
20100191888 | Serebrin et al. | Jul 2010 | A1 |
20100192223 | Ismael et al. | Jul 2010 | A1 |
20100220863 | Dupaquis et al. | Sep 2010 | A1 |
20100235647 | Buer | Sep 2010 | A1 |
20100235831 | Dittmer | Sep 2010 | A1 |
20100251104 | Massand | Sep 2010 | A1 |
20100254622 | Kamay et al. | Oct 2010 | A1 |
20100281102 | Chinta et al. | Nov 2010 | A1 |
20100281541 | Stolfo et al. | Nov 2010 | A1 |
20100281542 | Stolfo et al. | Nov 2010 | A1 |
20100287260 | Peterson et al. | Nov 2010 | A1 |
20100299665 | Adams | Nov 2010 | A1 |
20100299754 | Amit et al. | Nov 2010 | A1 |
20100306173 | Frank | Dec 2010 | A1 |
20100306560 | Bozek et al. | Dec 2010 | A1 |
20110004737 | Greenebaum | Jan 2011 | A1 |
20110004935 | Moffie et al. | Jan 2011 | A1 |
20110022695 | Dalal et al. | Jan 2011 | A1 |
20110025504 | Lyon et al. | Feb 2011 | A1 |
20110041179 | St Hlberg | Feb 2011 | A1 |
20110047542 | Dang et al. | Feb 2011 | A1 |
20110047544 | Yehuda et al. | Feb 2011 | A1 |
20110047594 | Mahaffey et al. | Feb 2011 | A1 |
20110047620 | Mahaffey et al. | Feb 2011 | A1 |
20110055907 | Narasimhan et al. | Mar 2011 | A1 |
20110060947 | Song et al. | Mar 2011 | A1 |
20110078794 | Manni et al. | Mar 2011 | A1 |
20110078797 | Beachem et al. | Mar 2011 | A1 |
20110082962 | Horovitz et al. | Apr 2011 | A1 |
20110093951 | Aziz | Apr 2011 | A1 |
20110099620 | Stavrou et al. | Apr 2011 | A1 |
20110099633 | Aziz | Apr 2011 | A1 |
20110099635 | Silberman et al. | Apr 2011 | A1 |
20110113231 | Kaminsky | May 2011 | A1 |
20110145918 | Jung et al. | Jun 2011 | A1 |
20110145920 | Mahaffey et al. | Jun 2011 | A1 |
20110145934 | Abramovici et al. | Jun 2011 | A1 |
20110153909 | Dong | Jun 2011 | A1 |
20110167422 | Eom et al. | Jul 2011 | A1 |
20110167493 | Song et al. | Jul 2011 | A1 |
20110167494 | Bowen et al. | Jul 2011 | A1 |
20110173213 | Frazier et al. | Jul 2011 | A1 |
20110173460 | Ito et al. | Jul 2011 | A1 |
20110219449 | St. Neitzel et al. | Sep 2011 | A1 |
20110219450 | McDougal et al. | Sep 2011 | A1 |
20110225624 | Sawhney et al. | Sep 2011 | A1 |
20110225655 | Niemela et al. | Sep 2011 | A1 |
20110247072 | Staniford et al. | Oct 2011 | A1 |
20110265182 | Peinado et al. | Oct 2011 | A1 |
20110289582 | Kejriwal et al. | Nov 2011 | A1 |
20110296412 | Banga et al. | Dec 2011 | A1 |
20110296440 | Laurich et al. | Dec 2011 | A1 |
20110299413 | Chatwani et al. | Dec 2011 | A1 |
20110302587 | Nishikawa et al. | Dec 2011 | A1 |
20110307954 | Melnik et al. | Dec 2011 | A1 |
20110307955 | Kaplan et al. | Dec 2011 | A1 |
20110307956 | Yermakov et al. | Dec 2011 | A1 |
20110314546 | Aziz et al. | Dec 2011 | A1 |
20110321040 | Sobel et al. | Dec 2011 | A1 |
20110321165 | Capalik et al. | Dec 2011 | A1 |
20110321166 | Capalik et al. | Dec 2011 | A1 |
20120011508 | Ahmad | Jan 2012 | A1 |
20120023593 | Puder et al. | Jan 2012 | A1 |
20120047576 | Do et al. | Feb 2012 | A1 |
20120054869 | Yen et al. | Mar 2012 | A1 |
20120066698 | Yanoo | Mar 2012 | A1 |
20120079596 | Thomas et al. | Mar 2012 | A1 |
20120084859 | Radinsky et al. | Apr 2012 | A1 |
20120096553 | Srivastava et al. | Apr 2012 | A1 |
20120110667 | Zubrilin et al. | May 2012 | A1 |
20120117652 | Manni et al. | May 2012 | A1 |
20120121154 | Xue et al. | May 2012 | A1 |
20120124426 | Maybee et al. | May 2012 | A1 |
20120131156 | Brandt et al. | May 2012 | A1 |
20120144489 | Jarrett et al. | Jun 2012 | A1 |
20120159454 | Barham et al. | Jun 2012 | A1 |
20120174186 | Aziz et al. | Jul 2012 | A1 |
20120174196 | Bhogavilli et al. | Jul 2012 | A1 |
20120174218 | McCoy et al. | Jul 2012 | A1 |
20120198279 | Schroeder | Aug 2012 | A1 |
20120198514 | McCune et al. | Aug 2012 | A1 |
20120210423 | Friedrichs et al. | Aug 2012 | A1 |
20120216046 | McDougal et al. | Aug 2012 | A1 |
20120216069 | Bensinger | Aug 2012 | A1 |
20120222114 | Shanbhogue | Aug 2012 | A1 |
20120222121 | Staniford et al. | Aug 2012 | A1 |
20120254993 | Sallam | Oct 2012 | A1 |
20120254995 | Sallam | Oct 2012 | A1 |
20120255002 | Sallam | Oct 2012 | A1 |
20120255003 | Sallam | Oct 2012 | A1 |
20120255012 | Sallam | Oct 2012 | A1 |
20120255015 | Sahita et al. | Oct 2012 | A1 |
20120255016 | Sallam | Oct 2012 | A1 |
20120255017 | Sallam | Oct 2012 | A1 |
20120255021 | Sallam | Oct 2012 | A1 |
20120260304 | Morris et al. | Oct 2012 | A1 |
20120260342 | Dube et al. | Oct 2012 | A1 |
20120260345 | Quinn et al. | Oct 2012 | A1 |
20120265976 | Spiers et al. | Oct 2012 | A1 |
20120266244 | Green et al. | Oct 2012 | A1 |
20120278886 | Luna | Nov 2012 | A1 |
20120291029 | Kidambi et al. | Nov 2012 | A1 |
20120297057 | Ghosh et al. | Nov 2012 | A1 |
20120297489 | Dequevy | Nov 2012 | A1 |
20120311708 | Agarwal et al. | Dec 2012 | A1 |
20120317566 | Santos et al. | Dec 2012 | A1 |
20120330801 | McDougal et al. | Dec 2012 | A1 |
20120331553 | Aziz et al. | Dec 2012 | A1 |
20130007325 | Sahita et al. | Jan 2013 | A1 |
20130014259 | Gribble et al. | Jan 2013 | A1 |
20130036470 | Zhu et al. | Feb 2013 | A1 |
20130036472 | Aziz | Feb 2013 | A1 |
20130047257 | Aziz | Feb 2013 | A1 |
20130055256 | Banga et al. | Feb 2013 | A1 |
20130074185 | McDougal et al. | Mar 2013 | A1 |
20130086235 | Ferris | Apr 2013 | A1 |
20130086299 | Epstein | Apr 2013 | A1 |
20130086684 | Mohler | Apr 2013 | A1 |
20130091571 | Lu | Apr 2013 | A1 |
20130097699 | Balupari et al. | Apr 2013 | A1 |
20130097706 | Titonis et al. | Apr 2013 | A1 |
20130111587 | Goel et al. | May 2013 | A1 |
20130111593 | Shankar et al. | May 2013 | A1 |
20130117741 | Prabhakaran et al. | May 2013 | A1 |
20130117848 | Golshan et al. | May 2013 | A1 |
20130117849 | Golshan et al. | May 2013 | A1 |
20130117852 | Stute | May 2013 | A1 |
20130117855 | Kim et al. | May 2013 | A1 |
20130139264 | Brinkley et al. | May 2013 | A1 |
20130159662 | Iyigun et al. | Jun 2013 | A1 |
20130160125 | Likhachev et al. | Jun 2013 | A1 |
20130160127 | Jeong et al. | Jun 2013 | A1 |
20130160130 | Mendelev et al. | Jun 2013 | A1 |
20130160131 | Madou et al. | Jun 2013 | A1 |
20130167236 | Sick | Jun 2013 | A1 |
20130174214 | Duncan | Jul 2013 | A1 |
20130179971 | Harrison | Jul 2013 | A1 |
20130185789 | Hagiwara et al. | Jul 2013 | A1 |
20130185795 | Winn et al. | Jul 2013 | A1 |
20130185798 | Saunders et al. | Jul 2013 | A1 |
20130191915 | Antonakakis et al. | Jul 2013 | A1 |
20130191924 | Tedesco et al. | Jul 2013 | A1 |
20130196649 | Paddon et al. | Aug 2013 | A1 |
20130227680 | Pavlyushchik | Aug 2013 | A1 |
20130227691 | Aziz et al. | Aug 2013 | A1 |
20130246370 | Bartram et al. | Sep 2013 | A1 |
20130247186 | LeMasters | Sep 2013 | A1 |
20130263260 | Mahaffey et al. | Oct 2013 | A1 |
20130282776 | Durrant et al. | Oct 2013 | A1 |
20130283370 | Vipat et al. | Oct 2013 | A1 |
20130291109 | Staniford et al. | Oct 2013 | A1 |
20130298243 | Kumar et al. | Nov 2013 | A1 |
20130298244 | Kumar et al. | Nov 2013 | A1 |
20130312098 | Kapoor et al. | Nov 2013 | A1 |
20130312099 | Edwards et al. | Nov 2013 | A1 |
20130318038 | Shiffer et al. | Nov 2013 | A1 |
20130318073 | Shiffer et al. | Nov 2013 | A1 |
20130325791 | Shiffer et al. | Dec 2013 | A1 |
20130325792 | Shiffer et al. | Dec 2013 | A1 |
20130325871 | Shiffer et al. | Dec 2013 | A1 |
20130325872 | Shiffer et al. | Dec 2013 | A1 |
20130326625 | Anderson et al. | Dec 2013 | A1 |
20130333033 | Khesin | Dec 2013 | A1 |
20130333040 | Diehl et al. | Dec 2013 | A1 |
20130347131 | Mooring et al. | Dec 2013 | A1 |
20140006734 | Li et al. | Jan 2014 | A1 |
20140019963 | Deng et al. | Jan 2014 | A1 |
20140032875 | Butler | Jan 2014 | A1 |
20140053260 | Gupta et al. | Feb 2014 | A1 |
20140053261 | Gupta et al. | Feb 2014 | A1 |
20140075522 | Paris et al. | Mar 2014 | A1 |
20140089266 | Une et al. | Mar 2014 | A1 |
20140096134 | Barak et al. | Apr 2014 | A1 |
20140115578 | Cooper et al. | Apr 2014 | A1 |
20140115652 | Kapoor et al. | Apr 2014 | A1 |
20140130158 | Wang et al. | May 2014 | A1 |
20140137180 | Lukacs et al. | May 2014 | A1 |
20140157407 | Krishnan et al. | Jun 2014 | A1 |
20140169762 | Ryu | Jun 2014 | A1 |
20140179360 | Jackson et al. | Jun 2014 | A1 |
20140181131 | Ross | Jun 2014 | A1 |
20140189687 | Jung et al. | Jul 2014 | A1 |
20140189866 | Shiffer et al. | Jul 2014 | A1 |
20140189882 | Jung et al. | Jul 2014 | A1 |
20140208123 | Roth et al. | Jul 2014 | A1 |
20140230024 | Uehara et al. | Aug 2014 | A1 |
20140237600 | Silberman et al. | Aug 2014 | A1 |
20140245423 | Lee | Aug 2014 | A1 |
20140259169 | Harrison | Sep 2014 | A1 |
20140280245 | Wilson | Sep 2014 | A1 |
20140283037 | Sikorski et al. | Sep 2014 | A1 |
20140283063 | Thompson et al. | Sep 2014 | A1 |
20140289105 | Sirota et al. | Sep 2014 | A1 |
20140304819 | Ignatchenko et al. | Oct 2014 | A1 |
20140310810 | Brueckner et al. | Oct 2014 | A1 |
20140325644 | Oberg et al. | Oct 2014 | A1 |
20140328204 | Klotsche et al. | Nov 2014 | A1 |
20140337836 | Ismael | Nov 2014 | A1 |
20140344926 | Cunningham et al. | Nov 2014 | A1 |
20140351810 | Pratt et al. | Nov 2014 | A1 |
20140351935 | Shao et al. | Nov 2014 | A1 |
20140359239 | Hiremane et al. | Dec 2014 | A1 |
20140380473 | Bu et al. | Dec 2014 | A1 |
20140380474 | Paithane et al. | Dec 2014 | A1 |
20150007312 | Pidathala et al. | Jan 2015 | A1 |
20150013008 | Lukacs | Jan 2015 | A1 |
20150095661 | Sell et al. | Apr 2015 | A1 |
20150096022 | Vincent et al. | Apr 2015 | A1 |
20150096023 | Mesdaq et al. | Apr 2015 | A1 |
20150096024 | Haq et al. | Apr 2015 | A1 |
20150096025 | Ismael | Apr 2015 | A1 |
20150121135 | Pape | Apr 2015 | A1 |
20150128266 | Tosa | May 2015 | A1 |
20150172300 | Cochenour | Jun 2015 | A1 |
20150180886 | Staniford et al. | Jun 2015 | A1 |
20150186645 | Aziz et al. | Jul 2015 | A1 |
20150199513 | Ismael et al. | Jul 2015 | A1 |
20150199514 | Tosa et al. | Jul 2015 | A1 |
20150199531 | Ismael et al. | Jul 2015 | A1 |
20150199532 | Ismael et al. | Jul 2015 | A1 |
20150220735 | Paithane et al. | Aug 2015 | A1 |
20150244732 | Golshan et al. | Aug 2015 | A1 |
20150304716 | Sanchez-Leighton | Oct 2015 | A1 |
20150317495 | Rodgers et al. | Nov 2015 | A1 |
20150318986 | Novak et al. | Nov 2015 | A1 |
20150372980 | Eyada | Dec 2015 | A1 |
20160004869 | Ismael et al. | Jan 2016 | A1 |
20160006756 | Ismael et al. | Jan 2016 | A1 |
20160044000 | Cunningham | Feb 2016 | A1 |
20160048680 | Lutas et al. | Feb 2016 | A1 |
20160057123 | Jiang et al. | Feb 2016 | A1 |
20160103698 | Yang et al. | Apr 2016 | A1 |
20160127393 | Aziz et al. | May 2016 | A1 |
20160191547 | Zafar et al. | Jun 2016 | A1 |
20160191550 | Ismael et al. | Jun 2016 | A1 |
20160261612 | Mesdaq et al. | Sep 2016 | A1 |
20160285914 | Singh et al. | Sep 2016 | A1 |
20160301703 | Aziz | Oct 2016 | A1 |
20160335110 | Paithane et al. | Nov 2016 | A1 |
20160371105 | Sieffert et al. | Dec 2016 | A1 |
20170083703 | Abbasi et al. | Mar 2017 | A1 |
20170124326 | Wailly et al. | May 2017 | A1 |
20170213030 | Mooring et al. | Jul 2017 | A1 |
20170344496 | Chen et al. | Nov 2017 | A1 |
20170364677 | Soman et al. | Dec 2017 | A1 |
20180013770 | Ismael | Jan 2018 | A1 |
20180048660 | Paithane et al. | Feb 2018 | A1 |
20180121316 | Ismael et al. | May 2018 | A1 |
20180288077 | Siddiqui et al. | Oct 2018 | A1 |
Number | Date | Country |
---|---|---|
2439806 | Jan 2008 | GB |
2490431 | Oct 2012 | GB |
02006928 | Jan 2002 | WO |
0223805 | Mar 2002 | WO |
2007117636 | Oct 2007 | WO |
2008041950 | Apr 2008 | WO |
2011084431 | Jul 2011 | WO |
2011112348 | Sep 2011 | WO |
2012075336 | Jun 2012 | WO |
2012145066 | Oct 2012 | WO |
2012135192 | Oct 2012 | WO |
2012154664 | Nov 2012 | WO |
2012177464 | Dec 2012 | WO |
2013067505 | May 2013 | WO |
2013091221 | Jun 2013 | WO |
2014004747 | Jan 2014 | WO |
Entry |
---|
U.S. Appl. No. 15/199,873, filed Jun. 30, 2016 Non-Final Office Action dated Feb. 9, 2018. |
“Mining Specification of Malicious Behavior”—Jha et al, UCSB, Sep. 2007 https://www.cs.ucsb.edu/.about.chris/research/doc/esec07.sub.--mining.pdf- . |
“Network Security: NetDetector—Network Intrusion Forensic System (NIFS) Whitepaper”, (“NetDetector Whitepaper”), (2003). |
“When Virtual is Better Than Real”, IEEEXplore Digital Library, available at, http://ieeexplore.ieee.org/xpl/articleDetails.isp?reload=true&arnumbe- r=990073, (Dec. 7, 2013). |
Abdullah, et al., Visualizing Network Data for Intrusion Detection, 2005 IEEE Workshop on Information Assurance and Security, pp. 100-108. |
Adetoye, Adedayo , et al., “Network Intrusion Detection & Response System”, (“Adetoye”), (Sep. 2003). |
Apostolopoulos, George; hassapis, Constantinos; “V-eM: A cluster of Virtual Machines for Robust, Detailed, and High-Performance Network Emulation”, 14th IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems, Sep. 11-14, 2006, pp. 117-126. |
Aura, Tuomas, “Scanning electronic documents for personally identifiable information”, Proceedings of the 5th ACM workshop on Privacy in electronic society. ACM, 2006. |
Baecher, “The Nepenthes Platform: An Efficient Approach to collect Malware”, Springer-verlag Berlin Heidelberg, (2006), pp. 165-184. |
Bayer, et al., “Dynamic Analysis of Malicious Code”, J Comput Virol, Springer-Verlag, France., (2006), pp. 67-77. |
Boubalos, Chris , “Extracting syslog data out of raw pcap dumps, seclists.org, Honeypots mailing list archives”, available at http://seclists.org/honeypots/2003/q2/319 (“Boubalos”), (Jun. 5, 2003). |
Chaudet, C. , et al., “Optimal Positioning of Active and Passive Monitoring Devices”, International Conference on Emerging Networking Experiments and Technologies, Proceedings of the 2005 ACM Conference on Emerging Network Experiment and Technology, CoNEXT '05, Toulousse, France, (Oct. 2005), pp. 71-82. |
Chen, P. M. and Noble, B. D., “When Virtual is Better Than Real, Department of Electrical Engineering and Computer Science”, University of Michigan (“Chen”) (2001). |
Cisco “Intrusion Prevention for the Cisco ASA 5500-x Series” Data Sheet (2012). |
Cohen, M.I. , “PyFlag—An advanced network forensic framework”, Digital investigation 5, Elsevier, (2008), pp. S112-S120. |
Costa, M. , et al., “Vigilante: End-to-End Containment of Internet Worms”, SOSP '05, Association for Computing Machinery, Inc., Brighton U.K., (Oct. 23-26, 2005). |
Didier Stevens, “Malicious PDF Documents Explained”, Security & Privacy, IEEE, IEEE Service Center, Los Alamitos, CA, US, vol. 9, No. 1, Jan. 1, 2011, pp. 80-82, XP011329453, ISSN: 1540-7993, DOI: 10.1109/MSP.2011.14. |
Distler, “Malware Analysis: An Introduction”, SANS Institute InfoSec Reading Room, SANS Institute, (2007). |
Dunlap, George W. , et al., “ReVirt: Enabling Intrusion Analysis through Virtual-Machine Logging and Replay”, Proceeding of the 5th Symposium on Operating Systems Design and Implementation, USENIX Association, (“Dunlap”), (Dec. 9, 2002). |
FireEye Malware Analysis & Exchange Network, Malware Protection System, FireEye Inc., 2010. |
FireEye Malware Analysis, Modern Malware Forensics, FireEye Inc., 2010. |
FireEye v.6.0 Security Target, pp. 1-35, Version 1.1, FireEye Inc., May 2011. |
Goel, et al., Reconstructing System State for Intrusion Analysis, Apr. 2008 SIGOPS Operating Systems Review, vol. 42 Issue 3, pp. 21-28. |
Gregg Keizer: “Microsoft's HoneyMonkeys Show Patching Windows Works”, Aug. 8, 2005, XP055143386, Retrieved from the Internet: URL:http://www.informationweek.com/microsofts-honeymonkeys-show-patching-windows-works/d/d-id/1035069? [retrieved on Jun. 1, 2016]. |
Heng Yin et al, Panorama: Capturing System-Wide Information Flow for Malware Detection and Analysis, Research Showcase @ CMU, Carnegie Mellon University, 2007. |
Hiroshi Shinotsuka, Malware Authors Using New Techniques to Evade Automated Threat Analysis Systems, Oct. 26, 2012, http://www.symantec.com/connect/blogs/, pp. 1-4. |
Idika et al., A-Survey-of-Malware-Detection-Techniques, Feb. 2, 2007, Department of Computer Science, Purdue University. |
Isohara, Takamasa, Keisuke Takemori, and Ayumu Kubota. “Kernel-based behavior analysis for android malware detection.” Computational intelligence and Security (CIS), 2011 Seventh International Conference on. IEEE, 2011. |
Kaeo, Merike , “Designing Network Security”, (“Kaeo”), (Nov. 2003). |
Kevin A Roundy et al: “Hybrid Analysis and Control of Malware”, Sep. 15, 2010, Recent Advances in Intrusion Detection, Springer Berlin Heidelberg, Berlin, Heidelberg, pp. 317-338, XP019150454 ISBN:978-3-642-15511-6. |
Khaled Salah et al: “Using Cloud Computing to Implement a Security Overlay Network”, Security & Privacy, IEEE, IEEE Service Center, Los Alamitos, CA, US, vol. 11, No. 1, Jan. 1, 2013 (Jan. 1, 2013). |
Kim, H. , et al., “Autograph: Toward Automated, Distributed Worm Signature Detection”, Proceedings of the 13th Usenix Security Symposium (Security 2004), San Diego, (Aug. 2004), pp. 271-286. |
King, Samuel T., et al., “Operating System Support for Virtual Machines”, (“King”), (2003). |
Kreibich, C. , et al., “Honeycomb-Creating Intrusion Detection Signatures Using Honeypots”, 2nd Workshop on Hot Topics in Networks (HotNets-11), Boston, USA, (2003). |
Kristoff, J. , “Botnets, Detection and Mitigation: DNS-Based Techniques”, NU Security Day, (2005), 23 pages. |
Lastline Labs, The Threat of Evasive Malware, Feb. 25, 2013, Lastline Labs, pp. 1-8. |
Li et al., A VMM-Based System Call Interposition Framework for Program Monitoring, Dec. 2010, IEEE 16th International Conference on Parallel and Distributed Systems, pp. 706-711. |
Lindorfer, Martina, Clemens Kolbitsch, and Paolo Milani Comparetti. “Detecting environment-sensitive malware.” Recent Advances in Intrusion Detection. Springer Berlin Heidelberg, 2011. |
Marchette, David J., “Computer Intrusion Detection and Network Monitoring: A Statistical Viewpoint”, (“Marchette”), (2001). |
Moore, D. , et al., “Internet Quarantine: Requirements for Containing Self-Propagating Code”, INFOCOM, vol. 3, (Mar. 30-Apr. 3, 2003), pp. 1901-1910. |
Morales, Jose A., et al., ““Analyzing and exploiting network behaviors of malware.””, Security and Privacy in Communication Networks. Springer Berlin Heidelberg, 2010. 20-34. |
Mori, Detecting Unknown Computer Viruses, 2004, Springer-Verlag Berlin Heidelberg. |
Natvig, Kurt , “SANDBOXII: Internet”, Virus Bulletin Conference, (“Natvig”), (Sep. 2002). |
NetBIOS Working Group. Protocol Standard for a NetBIOS Service on a TCP/UDP transport: Concepts and Methods. STD 19, RFC 1001, Mar. 1987. |
Newsome, J. , et al., “Dynamic Taint Analysis for Automatic Detection, Analysis, and Signature Generation of Exploits on Commodity Software”, In Proceedings of the 12th Annual Network and Distributed System Security, Symposium (NDSS '05), (Feb. 2005). |
Nojiri, D. , et al., “Cooperation Response Strategies for Large Scale Attack Mitigation”, DARPA Information Survivability Conference and Exposition, vol. 1, (Apr. 22-24, 2003), pp. 293-302. |
Oberheide et al., CloudAV.sub.-N-Version Antivirus in the Network Cloud, 17th USENIX Security Symposium USENIX Security '08 Jul. 28-Aug. 1, 2008 San Jose, CA. |
Reiner Sailer, Enriquillo Valdez, Trent Jaeger, Roonald Perez, Leendert van Doorn, John Linwood Griffin, Stefan Berger., sHype: Secure Hypervisor Appraoch to Trusted Virtualized Systems (Feb. 2, 2005) (“Sailer”). |
Silicon Defense, “Worm Containment in the Internal Network”, (Mar. 2003), pp. 1-25. |
Singh, S. , et al., “Automated Worm Fingerprinting”, Proceedings of the ACM/USENIX Symposium on Operating System Design and Implementation, San Francisco, California, (Dec. 2004). |
Thomas H. Ptacek, and Timothy N. Newsham , “Insertion, Evasion, and Denial of Service: Eluding Network Intrusion Detection”, Secure Networks, (“Ptacek”), (Jan. 1998). |
U.S. Appl. No. 15/199,873, filed Jun. 30, 2016 Non-Final Office Action dated Jun. 26, 2019. |
U.S. Appl. No. 15/199,879, filed Jun. 30, 2016 Non-Final Office Action dated Apr. 27, 2018. |
U.S. Appl. No. 15/199,879, filed Jun. 30, 2016 Notice of Allowance dated Oct. 4, 2018. |
U.S. Appl. No. 15/199,882, filed Jun. 30, 2016 Final Office Action dated Jun. 11, 2019. |
U.S. Appl. No. 15/199,871, filed Jun. 30, 2016 Advisory Action dated Nov. 8, 2018. |
U.S. Appl. No. 15/199,871, filed Jun. 30, 2016 Notice of Allowance dated Mar. 20, 2019. |
U.S. Appl. No. 15/199,873, filed Jun. 30, 2016 Advisory Action dated Dec. 3, 2018. |
Venezia, Paul , “NetDetector Captures Intrusions”, InfoWorld Issue 27, (“Venezia”), (Jul. 14, 2003). |
Vladimir Getov: “Security as a Service in Smart Clouds—Opportunities and Concerns”, Computer Software and Applications Conference (COMPSAC), 2012 IEEE 36th Annual, IEEE, Jul. 16, 2012 (Jul. 16, 2012). |
Wahid et al., Characterising the Evolution in Scanning Activity of Suspicious Hosts, Oct. 2009, Third International Conference on Network and System Security, pp. 344-350. |
Whyte, et al., “DNS-Based Detection of Scanning Works in an Enterprise Network”, Proceedings of the 12th Annual Network and Distributed System Security Symposium, (Feb. 2005), 15 pages. |
Williamson, Matthew M., “Throttling Viruses: Restricting Propagation to Defeat Malicious Mobile Code”, ACSAC Conference, Las Vegas, NV, USA, (Dec. 2002), pp. 1-9. |
Yuhei Kawakoya et al: “Memory behavior-based automatic malware unpacking in stealth debugging environment”, Malicious and Unwanted Software (Malware), 2010 5th International Conference on, IEEE, Piscataway, NJ, USA, Oct. 19, 2010, pp. 39-46, XP031833827, ISBN:978-1-4244-8-9353-1. |
Zhang et al., The Effects of Threading, Infection Time, and Multiple-Attacker Collaboration on Malware Propagation, Sep. 2009, IEEE 28th International Symposium on Reliable Distributed Systems, pp. 73-82. |
U.S. Appl. No. 15/197,634, filed Jun. 29, 2016 Notice of Allowance dated Apr. 18, 2018. |
U.S. Appl. No. 15/199,871, filed Jun. 30, 2016 Final Office Action dated Aug. 16, 2018. |
U.S. Appl. No. 15/199,871, filed Jun. 30, 2016 Non-Final Office Action dated Apr. 9, 2018. |
U.S. Appl. No. 15/199,873, filed Jun. 30, 2016 Final Office Action dated Sep. 10, 2018. |
U.S. Appl. No. 15/199,882, filed Jun. 30, 2016 Advisory Action dated Nov. 8, 2018. |
U.S. Appl. No. 15/199,882, filed Jun. 30, 2016 Final Office Action dated Aug. 31, 2018. |
U.S. Appl. No. 15/199,882, filed Jun. 30, 2016 Non-Final Office Action dated Apr. 5, 2018. |
U.S. Appl. No. 15/199,882, filed Jun. 30, 2016 Non-Final Office Action dated Dec. 20, 2018. |
U.S. Appl. No. 15/199,871, filed Jun. 30, 2016. |
U.S. Appl. No. 15/199,873, filed Jun. 30, 2016. |
U.S. Appl. No. 15/199,876, filed Jun. 30, 2016. |
U.S. Appl. No. 15/199,882, filed Jun. 30, 2016. |
Number | Date | Country | |
---|---|---|---|
62187108 | Jun 2015 | US |