Embodiments of the present invention relate generally to network security. More particularly, embodiments of the invention relate to changing security session forwarding when detecting network event of virtual machine migration.
Virtual machine live migration is an important feature that it allows a network administrator to move virtual machines to another physical host without interrupting operation during the live migration. However, if there is a security device, such as gateway or firewall, protecting the virtual machine in front of the original physical host, once the virtual machine is moved to another physical host, the data connections to the virtual machine would be disconnected. This would interrupt data traffic and stop current operation of the virtual machine.
Traditionally, virtual machine live migration is detected by querying a virtual machine management system and retrieving the operation information. This is done by using the API (Application Programming Interface) being provided by the virtual machine management system. However, each virtual machine management system provides a different access method and the API may change overtime. This method also requires network security devices constantly connect to the virtual machine management system, which is not efficient and sometime it is not practical.
This summary is provided to introduce a selection of concepts in a simplified form that are further described in the Detailed Description below. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
A method and apparatus is disclosed herein for using a security gateway. In one embodiment, the method comprises receiving information from a virtual machine after the virtual machine has migrated from a first network access device to a second network access device, where the information identifies the virtual machine as one associated with a privilege level; determining that access to the virtual machine at the first network access device was permitted by the privilege level; assigning the virtual machine at the second network access device to the privilege level; and applying a set of rules associated with the privilege level to communications between the network and the virtual machine at the second network access device.
Embodiments of the invention are illustrated by way of example and not limitation in the figures of the accompanying drawings in which like references indicate similar elements.
Various embodiments and aspects of the inventions will be described with reference to details discussed below, and the accompanying drawings will illustrate the various embodiments. The following description and drawings are illustrative of the invention and are not to be construed as limiting the invention. Numerous specific details are described to provide a thorough understanding of various embodiments of the present invention. However, in certain instances, well-known or conventional details are not described in order to provide a concise discussion of embodiments of the present inventions.
Reference in the specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in conjunction with the embodiment can be included in at least one embodiment of the invention. The appearances of the phrase “in one embodiment” in various places in the specification do not necessarily all refer to the same embodiment.
According to some embodiments, a mechanism of detecting network event of virtual machine live migration and modified security session forwarding of a network security device is utilized to maintain the data connection running without interruption during the virtual machine (VM) migration. In one embodiment, the mechanism is to detect network events of virtual machine live migration, thus there is no dependency of the virtual machine management system. Once a network security device (e.g., network access device or firewall) detects these packets, it can change the security session forwarding of all connections of the virtual machine from a first physical location to a second or new location. The security sessions will forward all related packet traffic to the new location of the virtual machine, under the same privilege level or security policy assigned to the virtual machine, thus data operation is not interrupted. The network event detection and the security session forwarding changes could happen on different network security devices. There could be a network security device detecting the virtual machine live migration happening, and notifying another network security device which is handling the security operation in the original physical host of the virtual machine.
According to one embodiment, network access device 204 is associated with a security gateway 212 that includes various firewall processing modules, for example, each being executed within a virtual machine (VM). In one embodiment, each firewall module is responsible for performing one or more firewall functions, but it does not include all of the firewall functions of a firewall. Examples of the firewall functions include, but are not limited to, network address translation (NAT), virtual private network (VPN), deep packet inspection (DPI), and/or anti-virus, etc. In one embodiment, some of the firewall processing modules are located within network access device 204 (e.g., firewall modules 209) and some are located external to network access device 204 (e.g., firewall modules 210 maintained by firewall processing node(s) 211, which may be a dedicated firewall processing machine. All of the firewall modules 209-210 are managed by firewall controller 208, which may be located within network access device 204, or external to network access device 204, such as, for example, in a public cloud associated with network 203, or in a private cloud associated with network 205. Firewall controller 208 and firewall processing modules 209-210 collectively are referred to herein as security gateway 212.
A virtual machine represents a completely isolated operating environment with a dedicated set of resources associated therewith. A virtual machine may be installed or launched as a guest operating system (OS) hosted by a host OS. In one embodiment, a host OS represents a virtual machine monitor (VMM) (also referred to as a hypervisor) for managing the hosted virtual machines. A guest OS may be of the same or different types with respect to the host OS. For example, a guest OS may be a Windows™ operating system and a host OS may be a LINUX® operating system. In addition, the guest operating systems (OSes) running on a host can be of the same or different types. A virtual machine can be any type of virtual machine, such as, for example, hardware emulation, full virtualization, para-virtualization, and operating system-level virtualization virtual machines. Different virtual machines hosted by a server may have the same or different privilege levels or security policies for accessing different resources. Each privilege level or security policy may be associated with a set of rules that governs communications between the network and the virtual machine and determines the resources to which the virtual machine has access. The virtual machines, the set of privilege levels or security policies, and the sets of rules may be stored within a data structure within the memory of at least one of the server or host. The data structure may be in the form of a table.
According to one embodiment, when VM 215 of node 206 is migrated to VM 216 of node 207, network access device 204 detects certain network events or messages that typically occur during or after the VM migration. In this example, security gateway 212 operates as a security device to nodes 206-207. In response to the detection, certain routing or forwarding tables (e.g., flow table or session table) of network access device 204 are modified, such that subsequent incoming packets can be properly routed to VM 216 of node 207 without having to terminate the current connection session during the virtual machine (VM) migration. The network event detection and the security session forwarding modifications can be performed on different network security devices. There could be a network security device detecting the occurrence of the virtual machine live migration, and notifying another network security device which is handling the security operation in the original physical host of the virtual machine. The original physical host and the new physical host of the virtual machine may be separate physical hosts with different Internet Protocol (IP) addresses.
According to one embodiment, each of network access devices 204A-204C maintains a persistent connection such as secure connections or tunnels 260 with a firewall controller or management entity 208 for exchanging management messages and configurations, or distributing routing information to network access devices 204A-204C, etc. In one embodiment, firewall controller 208 communicates with each of the network access devices 204A-204C using a management protocol such as the OpenFlow™ protocol. OpenFlow™ is a Layer 2 communications protocol (e.g., media access control or MAC layer) that gives access to the forwarding plane of a network switch or router over the network. In simpler terms, OpenFlow™ allows the path of network packets through the network of switches to be determined by software running on multiple routers (minimum two of them, primary and secondary, having a role of observers). This separation of the control from the forwarding allows for more sophisticated traffic management than is feasible using access control lists (ACLs) and routing protocols.
The OpenFlow™ technology consists of three parts: flow tables installed on switches, a controller and an OpenFlow™ protocol for the controller to talk securely with switches. Flow tables are set up on switches or routers. Controllers talk to the switches via the OpenFlow™ Protocol, which is secure, and impose policies on flows. For example, a simple flow might be defined as any traffic from a given IP address. The rule governing it might be to route the flow through a given switch port. With its knowledge of the network, the controller could set up paths through the network optimized for speed, fewest number of hops or reduced latency, among other characteristics. Using OpenFlow™ can take control of how traffic flows through the network out of the hands of the infrastructure, the switches and routers, and puts it in the hands of the network owner (such as a corporation), individual users or individual applications.
Referring back to
Firewall modules 209A-209C may be part of a security gateway described above. For example, firewall modules 209A-209C may be the Input/Output (I/O) functions of a firewall while nodes 211A-211B may be firewall processing nodes. That is, nodes 211A-211B may be dedicated firewall processing devices that perform some firewall processing operations such as DPI, content inspection, antivirus, etc., while firewall modules 209A-209C are responsible for routing data packets. For example, when firewall module 209B receives a packet from node 206, it may forward the packet to firewall processing node 211A for content inspection and/or forward the packet to firewall controller 208 for routing information. In response, firewall processing node 211A analyzes the received packet and/or may further communicate with firewall controller 208. Firewall controller 208 may provide further routing information back to network access device 204B regarding how to route the packet. Each of the firewall processing nodes 211A-211B may further maintains a persistent connection or tunnel with firewall controller 208, for example, using the OpenFlow™ communication protocol.
According to one embodiment, when VM migration is performed from VM 215 of node 206 to VM 216 of node 207 via path 271, typically, node 207 may broadcast certain events in the associated local network. Such events may be captured or received via path 272 by firewall module 209C of network access device 204C. Based on the captured events, firewall module 209C may recognize or determine that the VM migration has occurred. In response, in addition to updating flow table 251C, firewall module 209C notifies, via paths 273-274, firewall modules 209A-209B to allow the network access devices 204A-204B to update their respective flow tables 251A-251B, such that the current session to VM 215 can be seamlessly routed to VM 216 without interruption or terminating the session. An example of a flow table is shown in
In general, virtual machine live migrations generate certain network events after the live migration. For VMware vCenter™, the VM manager such as, for example, a hypervisor, sends out a RARP (Reverse Address Resolution Protocol) packet after VM migration. For Xen™ or Microsoft Hyper-V™, it sends out an Address Resolution Protocol (ARP) or broadcast ARP reply packet to the network. These packets can be detected as a network event after live migration. Once a network security device detects these packets, it can change the security session forwarding of all connections of the virtual machine to the new location. The network security device may also receive information from the virtual machine, including but not limited to, the original physical host's identity, IP address, current login user and security patch level. The security sessions will forward all related packet traffic to the new location of the virtual machine, thereby ensures that data operations are not interrupted. To reduce a false positive with respect to the detection of virtual machine live migration network events, according to one embodiment, one can combine the detection of RARP or ARP reply packets (or other similar network event after virtual machine live migration) with the traffic volume of virtual machine live migration, as the indication of virtual machine live migration. In one embodiment, the detection mechanism is triggered only when a certain amount of traffic is transferred to the new physical host through the live migration protocol (e.g., amount of incoming packets exceeds a predetermined threshold), before the network event packet is sent out. The combination of these two events would reduce the occurrence of a false positive with respect to detecting virtual machine live migration detection. The network event detection and the security session forwarding changes could happen on different network security devices. There could be a network security device detecting the occurrence of virtual machine live migration, and notifying another network security device which is handling the security operation in the original physical host of the virtual machine. The separation of event detection and security enforcement could enable distributed network security inside the data center.
An I/O module running within a virtual machine is referred to herein as a virtual I/O module. Each of virtual I/O modules 301-304 receives packets from any of servers 321-324 of LAN 320 and sends packets to external network 315 outside of the firewall. In one embodiment, each of virtual I/O modules 301-304 keeps a local cache (e.g., caches 305-308) storing location(s) of a security processing module(s) (e.g., security processing modules 309-311) for each connection session. A cache maintained by each virtual I/O module 301-304 contains a forwarding table mapping certain connection sessions to any of security processing modules 309-311. An example of a forwarding table is shown in
In one embodiment, each of security processing modules 309-311 performs major security processing functions, such as, for example, NAT, VPN, DPI, and/or anti-virus, etc. A security processing module receives packets and runs the packets through one or more various security functions in the module for security processing. There could be several security modules and each handles the same or different security functions. If the packets need to go through another security or service processing, the module sends the packets to the other modules. Optionally, it can run the packets through a load balancing mechanism to distribute the load to multiple modules. If a module is the last processing module in the chain to process the packets, it can forward the packets back to the virtual I/O module to send out, or send the packet out directly to its destination if its configured to do so.
In one embodiment, each of service processing modules 312-313 performs one or more of the functions of security processing module, such as, for example, NAT, VPN, DPI, and/or anti-virus, etc. However, it is different from the security processing module in that it only receives and sends packets to the same security processing module. If the tasks cannot be done in a security processing module, for example, due to a resource limitation, system load, or the requirement of a different operation system, the packets can be forwarded to one or more of service processing modules 312-313 for further processing. The packets then are sent back to the same security processing module for the next security function processing. To further share the system load, any of security processing modules 309-311 can load balance the computational-intensive services to multiple service processing modules.
In one embodiment, central controller 208 is the central place to control forwarding of the packets are forwarded amongst virtual I/O modules 301-304, security processing modules 309-311, and service processing modules 312-313. When a virtual I/O module 301-304 receives a packet, according to one embodiment, it forwards the packet to central controller 208 if it cannot find an existing connection in its local cache, as shown in
By dividing a firewall into different modules, it allows putting virtual I/O and security processing functions at the best locations to protect the network entrance, while keeping the central control and monitoring functionality at the central controller. It also enhances the scalability of the system since all modules can be expanded to multiple instances to share the system load. Note that a service processing module is optional in the architecture, as it is only required when there are needs to use additional resources to handle the security functions.
In one embodiment, firewall modules 300A and 300B can run on virtual machines or physical hosts. Running on virtual machines provides additional benefit that a firewall module can be added dynamically. Initially the security gateway may have only one virtual I/O module, one security processing module, and a central controller. When there is more traffic coming, it can add more virtual I/O modules to support increasing connections. If it needs more CPU resources to handle the security processing, it may add more security processing modules and/or add more service processing modules, to support the increasing load. This provides lots of flexibility to support various network conditions.
In one embodiment, firewall modules 300A-300B could be distributed in different networks, even on different locations, as long as the modules can reach the module that is next in terms of processing and the central controller. In one embodiment, virtual I/O modules and corresponding security processing modules are in a public cloud and the central controller is in a private cloud. This configuration may provide the flexibility to secure and control packets coming from the public cloud, and allow central controller having overall view of traffic from Internet as well as from internal network.
One of the advantages of embodiments of the present invention includes, but not limited to, that the security gateway can employ a significantly large amount of CPU and memory resources for service processing and protect the networks at multiple geometric locations. The central controller decides which security processing module capable of processing particular connection, and is able to start a new security processing at the place deemed best for packet processing.
As a result, the location of the packet I/O is not limited on a single appliance. The I/O modules can be placed anywhere as virtual machines. The security processing power is significantly higher as packets and connections can be load balanced to any number of the security processing modules, and the modules could be added or deleted dynamically. Using such modules in a firewall cloud provides a security design that is best-fit for the emerging cloud computing, and provides great scalability and system availability.
Bus 412 allows data communication between central processor 414 and system memory 417. System memory 417 (e.g., RAM) may be generally the main memory, having memory 460, into which the operating system and application programs are loaded. The ROM or flash memory can contain, among other code, the Basic Input-Output System (BIOS) which controls basic hardware operation such as the interaction with peripheral components. Applications resident with computer system 410 are generally stored on and accessed via a computer readable medium, such as a hard disk drive (e.g., fixed disk 444), an optical drive (e.g., optical disk drive 440), a floppy disk unit 437, or other storage medium.
Storage interface 434, as with the other storage interfaces of computer system 410, can connect to a standard computer readable medium for storage and/or retrieval of information, such as a fixed disk drive 444. Fixed disk drive 444 may be a part of computer system 410 or may be separate and accessed through other interface systems.
Modem 447 may provide a direct connection to a remote server via a telephone link or to the Internet via an internet service provider (ISP). Network interface 448 may provide a direct connection to a remote server. Network interface 448 may provide a direct connection to a remote server via a direct network link to the Internet via a POP (point of presence). Network interface 448 may provide such connection using wireless techniques, including digital cellular telephone connection, a packet connection, digital satellite data connection or the like.
Many other devices or subsystems (not shown) may be connected in a similar manner (e.g., document scanners, digital cameras and so on). Conversely, all of the devices shown in
Code to implement the gateway operations described herein can be stored in computer-readable storage media such as one or more of system memory 417, fixed disk 444, optical disk 442, or floppy disk. The operating system provided on computer system 410 may be MS-DOS®, MS-WINDOWS®, OS/2®, UNIX®, Linux®, or another known operating system.
Referring to
As described above, the servers in
According to one embodiment, node 1006 may have multiple virtual machines, including virtual machine 1015, and node 1007 may have multiple virtual machines. As an exemplary virtual machine live migration, virtual machine 1015 of node 1006 may migrate to node 1007. That is, if network access devices 1004B-1004C are physical hosts or locations, the virtual machine 1015 migrates from a first physical host or location to a second physical host or location. When virtual machine 1015 migrates, security gateway 1004A receives information via path 1073 from the virtual machine 1015. The information may be in the form of certain virtual machine migration events, network events or messages, and/or include the virtual machine's identity, original physical host, privilege level, routing or forwarding information, user information, etc.
According to one embodiment, when VM migration is performed from VM 1015 of node 1006 to node 1007 via path 1071, typically, node 1007 may broadcast certain events in the associated local network. Such events may be captured or received via path 1072 by firewall module 1009C of network access device 1004C. Based on the captured events, firewall module 1009C may recognize or determine that the VM migration has occurred. In response, in addition to updating flow table 1051C, firewall module 1009C notifies, via paths 1073-4074, firewall modules 1009A-1009B to allow the security gateway 1004A. and the network access device 1004B to update their respective flow tables 1051A-1051B, such that the current session to VM 1015 in the first physical location at network access device 1004B can be seamlessly routed to the second physical location at network access device 1004C without interruption or terminating the session.
The security gateway 1004A may provide protection to network access devices 1004B-1004C and communicate with the virtual machines. Once the security gateway 1004A detects the information via path 1073, it can change the security session forwarding of all connections of the virtual machine to the new location. The security sessions will forward all related packet traffic to the new location of the virtual machine 1015, thereby ensuring that data operations are not interrupted.
As previously described, the virtual machines may have same or different privilege levels or security policies for accessing different resources. The privilege levels or security policies may define a set of rules governing flows between the other networks and the virtual machines. By way of example, if the virtual or physical machines protected by one security policy communicate with virtual or physical machines protected by another security policy, the security gateway 1004A may permit or deny those communications depending on the set of rules governing those flows. One or more virtual machines may be protected by the same security policy even though they may reside on different nodes or network access devices.
The security gateway 1004A includes memory that includes information such as the privilege levels and imposes those privilege levels or security policies on flows, as previously described herein. As such, when the virtual machine 1015 migrates from node 1006 to node 1007, the information may identify the virtual machine 1015 as having a first privilege level (PL) 1050. In response to detection of the virtual machine live migration, certain routing or forwarding tables of network access devices 1004B-1004C may be modified, such that subsequent incoming packets can be properly routed to virtual machine 1015 now of node 1007 without having to terminate the current connection session during the migration.
In one embodiment, the security gateway 1004A of
Next, at block 1102, processing logic receives information from the virtual machine running on a network access device after the virtual machine has migrated from a first network access device to a second network access device. The information identifies that a virtual machine live migration has occurred, and identifies the virtual machine as the one previously assigned to a privilege level at the first network access device. In response, at block 1103, processing logic uses the information to determine that access to the virtual machine at the first network access device was permitted by the privilege level. At block 1104, processing logic assigns the virtual machine at the second network access device to the privilege level, and at block 1105 applies a set of rules associated with the privilege level to communications between the network and the virtual machine at the second network access device.
Some portions of the preceding detailed descriptions have been presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the ways used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of operations leading to a desired result. The operations are those requiring physical manipulations of physical quantities.
It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise as apparent from the above discussion, it is appreciated that throughout the description, discussions utilizing terms such as those set forth in the claims below, refer to the action and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.
The techniques shown in the figures can be implemented using code and data stored and executed on one or more electronic devices. Such electronic devices store and communicate (internally and/or with other electronic devices over a network) code and data using computer-readable media, such as non-transitory computer-readable storage media (e.g., magnetic disks; optical disks; random access memory; read only memory; flash memory devices; phase-change memory) and transitory computer-readable transmission media (e.g., electrical, optical, acoustical or other form of propagated signals—such as carrier waves, infrared signals, digital signals).
The processes or methods depicted in the preceding figures may be performed by processing logic that comprises hardware (e.g., circuitry, dedicated logic, etc.), firmware, software (e.g., embodied on a non-transitory computer readable medium), or a combination of both. Although the processes or methods are described above in terms of some sequential operations, it should be appreciated that some of the operations described may be performed in a different order. Moreover, some operations may be performed in parallel rather than sequentially.
In the foregoing specification, embodiments of the invention have been described with reference to specific exemplary embodiments thereof. It will be evident that various modifications may be made thereto without departing from the broader spirit and scope of the invention as set forth in the following claims. The specification and drawings are, accordingly, to be regarded in an illustrative sense rather than a restrictive sense.
This application is a continuation-in-part of U.S. patent application Ser. No. 13/860,404, filed Apr. 10, 2013 and entitled “Adaptive Session Forwarding Following Virtual Machine Migration Detection,” which in turn claims the benefit of U.S. Provisional Patent Application No. 61/686,837, filed Apr. 11, 2012, which are incorporated by reference in their entirety herein.
Number | Date | Country | |
---|---|---|---|
61686837 | Apr 2012 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 13860404 | Apr 2013 | US |
Child | 15209275 | US |