Cloud service providers have multiple datacenters in the same or multiple geographical locations. A tenant's workload might span across these datacenters in multiple locations managed by multiple compute and network management entities. In addition, it is desirable to transition applications from one zone to another. For instance, customers like to move applications from staging to production, where staging servers and the production servers are located in different datacenters and are managed by different compute and network management entities.
It is desirable to provide enhanced security solutions via a distributed firewall where rules can be propagated to and enforced across all the datacenters. The current solutions can only make use of static definition of grouping constructs (or containers) or security groups that include Internet protocol (IP) addresses and media access control (MAC) addresses.
A translation service translates the objects consumed in services such as distributed firewall into a set of IP addresses and MAC addresses based on the firewall rule field it is used. The translation process resolves the objects used in the firewall rules locally based on the objects stored in the datacenter's local object inventory and not based on objects across multiple datacenter environments. This approach only allows enforcing firewall rules that are based on static addresses across multiple datacenters.
Other firewall management solutions provide firewall protection in a single datacenter. Different datacenter sites typically have different management solutions for site-local firewall management. As such, there are no solutions available for centralized management of firewall across datacenters where one site can request or set firewall policies on another sites.
In some embodiments, the virtual machines (VMs) of a tenant are spread across multiple datacenters that are managed by different compute manager servers and network manager servers. The firewall rules typically include several tuples and an action. The tuples refer to different objects that may or may not be recognizable by each network manager server. Each network manager server recognizes the objects that are locally managed.
Some embodiments provide a method for creating global objects and translating firewall rules. The method, for each VM in the datacenter, creates a lightweight VM object that includes a subset of the VM's properties stored in the local object data store. The method stores each lightweight VM object in a global object data store in a distributed cache that is accessible by the network managers of participating datacenters.
The lightweight VM objects in the distributed cache are used to translate distributed firewall rules that include dynamically defined objects. Dynamically defined objects use object identifiers instead of static objects such as IP addresses or MAC addresses. When the identifier of an object in a firewall rule is found in the local object data store of the datacenter, the object identifier is translated to a corresponding globally recognized static address specified in the local object data store. For instance, a VM identifier is translated into a corresponding IP address or MAC address.
When the object is not found in the local object data store, the method searches for the object in the global data store. If the object is found, the object identifier is translated to a corresponding globally recognized static address specified in the global object data store. For instance, a VM identifier is translated into a corresponding IP address or MAC address. The method then distributes the translated firewall rules to the firewall enforcement points.
Some embodiments provide a method for defining universal security tags and using the tags to define distributed firewall rules that can be enforced across multiple datacenters. The method receives the definition of one or more universal security tag on a master network manager. For instance, the method receives definitions of one or more security tags from an administrator console coupled to the master network manager.
The master network manager then replicates the universal security tags to the slave network managers. One or more VMs are then associated to each security tag. For instance, a user associates workload VMs to each security tag using the master or slave network managers. The method then receives definition of one or more universal security groups and includes one or more security tags to each universal security group. The method then receives definition of firewall rules based on the universal security groups. For instance, the method receives definitions of the firewall rules from an administrator console coupled to the master network manager.
The method then receives definition of one or more universal firewall sections and adds the firewall rules to the universal firewall sections. The master network manager then replicates then universal firewall sections to the slave network managers. Each network manager then provisions the firewall rules on all hosts and enforces security for the VMs on the hosts in the corresponding datacenter.
The preceding Summary is intended to serve as a brief introduction to some embodiments of the invention. It is not meant to be an introduction or overview of all inventive subject matter disclosed in this document. The Detailed Description that follows and the Drawings that are referred to in the Detailed Description will further describe the embodiments described in the Summary as well as other embodiments. Accordingly, to understand all the embodiments described by this document, a full review of the Summary, Detailed Description and the Drawings is needed. Moreover, the claimed subject matters are not to be limited by the illustrative details in the Summary, Detailed Description and the Drawing.
The novel features of the invention are set forth in the appended claims. However, for purposes of explanation, several embodiments of the invention are set forth in the following figures.
In the following detailed description of the invention, numerous details, examples, and embodiments of the invention are set forth and described. However, it will be clear and apparent to one skilled in the art that the invention is not limited to the embodiments set forth and that the invention may be practiced without some of the specific details and examples discussed.
Some embodiments provide a method for creating global objects and translating firewall rules using the global objects. These embodiments, for each VM in the datacenter, create a lightweight VM object that includes a subset of the VM's properties stored in the local object data store. The lightweight VM objects are stored in a global object data store in a distributed cache that is accessible to network managers of all participating datacenters.
The lightweight VM objects in the distributed cache are used to translate distributed firewall rules that include dynamically defined objects. Dynamically defined objects use object identifiers instead of static objects such as IP addresses or MAC addresses. For instance, any identifier that uniquely identifies an object across the multiple datacenters can be used as the dynamically defined identifier of the object.
I. Dynamically Defined Global Objects that are Recognizable Across Multiple Datacenters
Some embodiments provide global objects for consumption in distributed firewall rules. A traditional firewall implements a set of security rules to enforce access control between two networks to protect the resources located inside one of the networks (e.g., an enterprise's private network) from the other network (e.g., the Internet). The traditional firewalls rely on topology restrictions and enforcement of firewall rules at the network entry points.
In distributed firewalls, the security policy is defined centrally but the firewall rules are enforces at the individual endpoints such as the hosts and forwarding elements inside a network. When a tenant's workload spans across these datacenters, it is desirable to provide enhanced security solutions via a distributed firewall where the security rules can be propagated to and enforced across all datacenters. Accordingly, objects used in firewall rules have to be translated into addresses that are recognized at different enforcement points across multiple datacenters.
A. Issues with Translation of Objects Consumed in Services Across Multiple Datacenters
A translation process translates objects that are consumed in network services, such as a distributed firewall, into a set of IP addresses, MAC addresses, and virtual network interface card (VNIC) universally unique identifiers (UUIDs). A translation process that resolves the objects in firewall rules based on the local object inventory in a datacenter faces limitations translating firewall rules when the workload spans across multiple datacenters that are managed by different compute and network manager servers. For instance, a network manager server can only translate grouping constructs based on the entities that are imported from the compute manager to which the network manager is connected. In other words, the network manager only recognizes the entities that are managed locally by the network manager and the compute manager.
As a result, when a tenant defines a security group to match all VMs satisfying a certain condition (e.g., belonging to the finance department with VMs spanning across multiple datacenters), the security group will only contain the IP addresses that are locally resolvable. The tenant has to use static groups like IPSets and MACSets in rules defined in the global firewall section. An IPSet in some embodiments includes a set of IP addresses. A MACSet in some embodiments includes a set of MAC addresses.
The following example illustrates the shortcoming of a translation process that resolves the objects in firewall rules based on the local object inventory. A typical translation service translates the objects consumed in firewall rules into a set of IP addresses, MAC addresses, and VNIC UUIDs based on whether the object is used in source, destination or AppliedTo field in the firewall rule. The translation process resolves the objects used in firewall rules locally based on the objects present in the local inventory and not based on objects across multiple network managers in a multi datacenter environment. The same applies with the security groups (with static definitions) and virtual center objects.
Each datacenter 101-103 also includes a network manager 151-153, respectively. The network manager is a virtualized server that is installed on one or more physical servers and is used to define and/or manage the networking resources of a datacenter.
Datacenters 101-103 also include one or more hosts 171-0173. For simplicity, only one host 171-173 is shown on each datacenter 101-103. The hosts in some embodiments are physical servers that include virtualization software (or hypervisor). VMs and guest operating systems are installed on the hosts. Each host hosts one or more VMs 161-165.
In some embodiments, a virtual machine is an emulation of a particular computer system. In some embodiments, the virtualization software includes a physical forwarding element (PFE) such as a virtual switch. In the virtualization field, some refer to software switches as virtual switches as these are software elements. However, in this specification, the software forwarding elements are referred to as physical forwarding elements (PFEs), in order to distinguish them from logical forwarding elements (LFEs), which are logical constructs that are not tied to the physical world. In other words, the software forwarding elements are referred to as PFEs because they exist and operate in the physical world, whereas LFEs are a logical representation of a forwarding element that is presented to a user when designing a logical network. In some embodiments, several PFEs are distributed throughout the network implement tenant's LFEs, where each PFE is a local instantiation of an LFE that operate across different host machines and can perform Open Systems Interconnection (OSI) model Layer 2 (L2)/Layer 3 (L3) packet forwarding between VMs on the host machine or on different host machines. An LFE is sometimes referred to as a virtual distributed switch (VDS). In some embodiments, an LFE spans across multiple datacenters. The routing is provided by global virtual distributed router (VDR) 190. A VDR provides a one-hop routing that enables a packet to be sent between two LFEs without going through a gateway
Network connections can be provided by different methods. For instance, a stretched L2 network that uses physical routers can be utilized to connect VMs on different hosts. Global (long) logical wires can be defined to stretch an overlay network across multiple datacenters. The global logical wires can be defined, e.g., by using forwarding elements. Other techniques such as defining virtual local area networks (VLANs) can be utilized to provide network connectivity.
As shown, a global logical wire “webwire” 105 stretches to datacenter 101 (DC-1), datacenter 102 (DC-2), and datacenter 103 (DC-3). DC-1 101 is managed by compute manager 1 141 and network manager 1 151 {CM-1, NM-1}, DC-2 102 is managed by compute manager 2 142 and network manager 2 152 {CM-2, NM-2}, and DC-3 103 is managed by compute manager 3 143 and network manager 3 153 {CM-3, NM-3}.
The VMs connected to the global webwire 105 in DC-1 101 are VMs 161-162 {web-vm-1, web-vm-2}. The VM connected to the global webwire 105 in DC-2 102 is VM 163 {web-vm-3}. The VMs connected to the global webwire 105 in DC-3 103 are VMs 164-165 {web-vm4, web-vm-5}.
Now consider the rule defined as webwire→webwire any drop. The intention is that each host should have following rule on the data path:
The rule, however, will be realized as following across multiple datacenters. For DC-1 {CM-1, NM-1}, the rule will be:
For DC-2 {CM-2, NM-2}, the rule will be:
For DC-3 {CM-3, NM-3}, the rule will be:
This means web-vms within the same datacenter cannot talk to each other while they can talk across datacenters, though the intention was that no web-vm on this global logical switch should be able to talk to each other. This also holds true for a security group with dynamic criteria such as {web-vm*}, tags, etc.
In order to overcome the above-mentioned shortcomings, the prior art systems use statically defined objects such as IPSets, MacSets, IP addresses, or MAC addresses in the distributed firewall rules. However, firewall rules that are written based on objects such as IPSets, MacSets, IP addresses, or MAC addresses are hard to read and maintain. The user has to manually add and update IP and MAC addresses used in the statically defined sets/grouping constructs/objects.
B. Dynamically Defined Global Objects that are Recognizable Across Multiple Datacenters
To prevent forcing a tenant to use static IP addresses or MAC addresses for firewall rules, some embodiments provide a method that allows a user to define dynamic objects and define security policies based on these objects. The membership changes in these dynamic objects are automatically handled and the objects are automatically translated to objects such as IP addresses and MAC addresses by the underlying system.
Some embodiments provide a distributed cache across multiple datacenters. The distributed cache is used as a global object data store
As shown, datacenters 201-203 share a distributed cache 270 that acts as (or is used to store) a global object data store. The distributed cache is accessible to the network managers in each datacenter. The cache used in some embodiments is volatile memory (as opposed to non-volatile storage such as disk).
In some embodiments, the global object data store in distributed cache 270 operates as a key-value data store (or database). Objects or records are stored and retrieved by using a key that uniquely identifies the record. Each record can have one or more fields that contain data.
In some embodiments, the distributed cache is implemented as a cluster where each network manager server operates as a node of the distributed cache cluster. When a new network manager server is added, a distributed cache node is instantiated on the new network manager server and the node is added to the cluster. Once a node is added to the cluster, all the objects stored in the distributed cache cluster are replicated on the new node.
Some embodiments define lightweight VM objects that persists in the distributed cache. These lightweight VM objects 281-286 are derived from their counterpart VM information stored in the local object data stores 251-253. These lightweight VM objects include the bare minimum properties of each VM that are required for the translation process (i.e., the information required to translate a dynamically defined identifier of VM to a corresponding statically defined identifier such as an IP address or MAC address).
Although the example of
Each distributed firewall publisher 221-223 receives a set of firewall rules (e.g., firewall rules that are defined using an administrator console (not shown) connected to the master network manager 211 and distributed by the master network manager to the slave network managers 212-213). Typically, firewall rule definitions include the following five tuples: source, source port, destination, destination port, and service (or application), in addition to an action value.
In some embodiments, the firewall rules include an additional tuple (referred to herein as the AppliedTo tuple). The AppliedTo tuple lists a set of enforcement points (network nodes) at which the firewall rule has to be applied. In some embodiments, the enforcement points can be defined in terms of (1) VNICs, VMs, hosts, or other compute constructs (e.g., compute clusters, datacenters, etc.), (2) network elements, such as PFEs, LFEs, other managed appliances, unmanaged third-party appliances (e.g., third party firewalls), and/or combination of such elements, and/or (3) security groups that are formed by a set of one or more VNICs, VMs, hosts, compute constructs and/or network constructs.
For instance, an AppliedTo firewall rule can be limited (by the AppliedTo tuple) to a security group that is specified in terms of (e.g., by using the identifier of) a particular compute cluster and a particular logical network that connects a particular tenant's VMs that execute on the cluster's hosts. Security groups can be specified by users (e.g., network administrators) in some embodiments. Conjunctively, or alternatively, security groups can be specified by automated process in some embodiments. Each grouping construct has an identifier. The grouping constructs identifiers are, for example, used in the AppliedTo tuple of firewall rules.
The local translation provider component 241-243 of each network manager 211-213 resolves local objects by searching the corresponding local objects data store to resolve the rules. For instance, local translation provider 241 receives firewall rules from distributed firewall publisher 221 and searches local object data store 251 to resolve the objects in the rules to a globally recognized static object such as an IP address or a MAC address. The local object data store 251 in some embodiments provides the mapping of the locally defined objects to the corresponding globally recognized IP address, MAC address, etc. If the object is not in the local object data store 251, the distributed firewall publisher 221 uses the global translation provider 231 to search the global object data store maintained in the distributed cache 270 to resolve the rules.
The lifecycle (i.e., add, update, delete) of the lightweight objects 305 (e.g., lightweight objects 281-286 in the global object data store 270 in
In the example of
In this example, CachedVnic 310 class includes the UUID 331 of the VNIC, the MAC address 332 of the VNIC, the IP address 333 of the VNIC, the port group identification 334 of the VNIC, a flag 335 to indicate whether the spoof guard is enabled for the VNIC, and the published IP address 336 of the VNIC (i.e., the IP address used for the spoof guard).
CachedVirtualMachine 305 class (in which CachedVnic 310 class is embedded) is stored in the global object data store. The class CachedManagedEntity 315 is an abstract entity whose properties are inherited by CachedVirtualMachine 305 and CachedVnic 310 classes. CachedManagedEntity 315 includes the following attributes: identification 341, revision 342, the parent identification 343, the parent managed entity 344, name 345, and a flag 346 to indicate whether the object is a system resource.
C. Translation of Dynamically Defined Global Objects
Referring back to
The global translation provider 231 uses the relationships among different objects such as universal security groups, universal security tags, and VMs to derive a list of instance identifiers of the VMs. Further details for universal security groups and universal security tags are provided below.
The global translation provider 231 then translates this list into IP addresses or Mac addresses from objects (e.g., CachedVirtualMachine object 305 in
The process then stores (at 410) each lightweight VM object in a global object data store in a distributed cache that is accessible by the network managers of participating datacenters. For instance, the process stores VM object 281 in global object data store in distributed cache 270 shown in
The process then receives (at 415) a set of firewall rules that include dynamically defined objects. Dynamically defined objects use object identifiers instead of static objects such as IP addresses or MAC addresses.
The process then identifies (at 420) an object in the firewall rules to translate. The process then determines (at 425) whether the object identifier is found in the local object data store of the datacenter. For instance, the local translation provider 242 searches local object data store 252 of datacenter 202 to determine whether the object exists in the local object data store 252.
If not, the process proceeds to 435, which is described below. Otherwise, the process translates (at 430) the object identifier to a corresponding globally recognized identifier specified in the local object data store. For instance, the process translates a VM identifier into a corresponding IP address or MAC address. The process then proceeds to 450, which is described below.
When the object is not found in the local object data store, the process determines (at 435) whether the object is found in the global data store. For instance, the process searches objects such as “CachedVirtualMachine” object 305 in
Otherwise, the process translates (at 445) the object identifier to a corresponding globally recognized identifier specified in the global object data store. For instance, the process uses the mapping of a VM UUID to an IP address or MAC address provided by a CachedVirtualMachine object 305 and the corresponding embedded CachedVnic object 310.
The process then determines (at 450) whether all objects in the received set of rules are resolved. If not, the process proceeds to 420, which was described above. Otherwise, the process distributes (at 455) the translated firewall rules to the firewall enforcement points. The process then exits.
II. Universal Security Tags
Some embodiments define universal security tags that are used to define firewall rules. Security tags are user-defined objects to which one or more VM objects can be associated. The security tags give more flexibility to the end user in deployments where the VMs do not follow a naming pattern or the name may be changed. The security tags are exposed to the tenant (the end user) and the tenant can freely attach and detach VMs to security tags. The tenant can also handle the lifecycle and membership management of the security tags.
The security tags are used to classify workloads in the datacenters. One aspect of such classification is compliance. For instance, all workloads that need to follow certain kind of compliance can be associated with a particular compliance tag. Users can then create security policies using security groups that are based on these tags to enforce compliance for these workloads. Security groups are grouping constructs that group datacenter elements such as security tags, IPSets, and MACSets.
Some embodiments provide universal security tags that are replicated across multiple network managers in different datacenters. When new objects are created in a datacenter (e.g. adding a host or creating a VM), an identifier that is unique in the datacenter (but not necessarily unique across multiple datacenters) is assigned to the new object. This locally unique identifier is used by different services in the datacenter to reference the objects created within the datacenter (i.e., the local objects).
Local security tags can be defined to associate with a list of the VMs. The relationship among the local security tags and the VMs are mapped using the VMs' identifier that is unique in the datacenter. This approach, however, has certain limitations and does work well in cases where a VM re-registers itself, e.g., in case of storage motion within the datacenter. Storage motion occurs when a VM's disk files are relocated from one shared storage location to another shared storage location.
As the identifier that is unique in the datacenter changes in this process, the security tag to VM association is lost and the user has to associate the VM to the security tag again using a newly assigned identifier. To overcome this shortcoming, some embodiments utilize a different VM identifier for associating the VM with a universal security tag. In these embodiments, the VM to universal tag association is based on the VM UUID or instance ID, which does change during the storage motion or during motion of a VM from one datacenter to another datacenter. Changes in VM to tag associations are replicated by the master network manager to slave network managers.
Universal security groups in some embodiments support universal security tags as members. In addition, universal security groups support IPSets and MACSets as members.
In some embodiments, firewall rules are placed in universal (global) sections, which are entities that are replicated across the datacenters. The universal firewall sections allow multiple network managers to share, replicate and remain in sync for distributed firewall rules or part of the rule configuration. The universal firewall sections contain the rules that are to be replicated across multiple network managers. Some embodiments place firewall rules in a grouping construct that can include any type of network, compute, or security constructs. In some embodiments, one or more of the compute constructs, network constructs, and security constructs can be specified as dynamic grouping constructs that can have members (e.g., forwarding elements, VMs, hosts, VNICs, etc.) dynamically added and/or removed from them. In order to provide support for a multi datacenter work environment, some embodiments provide a set of universal (or global) grouping constructs as well as local grouping constructs. In some embodiments, a member is added to a grouping construct by adding an identifier of the member in the grouping construct.
The members of the universal grouping constructs are defined such that each member's identification is unique across all datacenters that are participating in the multi datacenter solution. On the other hand, the members in a local grouping construct only need to be uniquely identified in the datacenter where the members are located. For instance, two VMs with the same identifier can exist in local grouping constructs in two different datacenters. In contrast, every VM in a universal grouping construct has to be uniquely identifiable across all participating datacenters.
The process replicates (at 610) the universal security tags from the primary network manager to slave network managers. For instance, master network manager 211 replicates the security tags to the slave network manager 212-213 in
The process associates (at 615) one or more VMs to each security tag. For instance, the process associates a set of workload VMs of a tenant in each datacenter to each security tag using master or slave network managers.
The process then receives (at 620) definition of one or more universal security groups and includes one or more security tags to each universal security group. For instance, the process receives definitions of one or more universal security group and the identification of the associated universal security tags from an administrator console coupled to the master network manager. The process then receives (at 625) definition of firewall rules based on the universal security groups. For instance, the process receives definitions of the firewall rules from an administrator console coupled to the master network manager.
Each universal security group 701-704 is a grouping construct (or container) that includes (or associates with) one or more security tags 711-713. Each security tag is also a grouping construct that includes (or associates) with one or more VM objects 721-725. Each VM object 721-725 is a lightweight grouping construct (such as CachedVirtualMachine 305 in
Referring back to
The process then replicates (at 635) the universal firewall section by the master network manager to slave network managers. The process then provisions (at 640) the firewall rules on all hosts and enforces security for the VMs on the hosts. For instance, each network manager provisions the replicated firewall rules to the hosts in the corresponding datacenter. The process then ends.
The process identifies (at 810) an object associated with the universal security tag. For instance, the process identifies an identifier of a VM (e.g., a VM UUID) that is associated with the universal security tag. The process then finds (at 815) the identifier of the object in the global data store. For instance, the process finds the VM UUID 321 in CachedVirtualMachine object 305 of
The process then finds (at 820) the corresponding statically defined identifier of the object in the global object data store. For instance, the process maps the VM UUID 321 to VNIC IP address 333 or VNIC MAC address 332 shown in
Each universal security tag 911-913 is a construct that includes (or associates with) one or more VM objects 921-923. Each VM object 921-923 is a lightweight construct (such as CachedVirtualMachine 305 in
Each VM object 921-923 is associated with one or more VNIC objects 931-934. Each VNIC object is a grouping construct (such as CachedVnic 310 in
Each datacenter includes a compute manager 1041-1043. Each datacenter also includes a network manager 1051-1053. The hosts are physical servers that include virtualization software (or hypervisor). VMs and guest operating systems are installed on the hosts. The compute managers 1041-1043 operate in a linked mode that allows the servers to share information. Each server can connect to the other server's system and view and/or manage the inventories of the other server.
Different embodiments support different network connections. Some embodiments utilize a stretched L2 network using physical routers. Other embodiments utilize VLANs. Other embodiments define logical wires (or logical switches) that stretch an overlay network across multiple datacenters.
In the embodiment shown in
In some embodiments, a system administrator creates and connects VMs across multiple datacenters 1001-1003 using the global logical wires (or global logical switches) 1061-1063. As shown, VMs 1022, 1023, 1025, and 1029 are connected together along the logical line 1061. VMs 1024 and 1026 are connected together along the logical line 1062. VMs 1027, 1028, and 1030 are connected together along the logical line 1063. In addition, the figure shows several VMs 1021, 1031, and 1032, which are not connected to global logical lines. These VMs only require connectivity within a single datacenter and are connected to local logical wires 1055 or 1056. Although the network in
As described above, firewall rules in some embodiments are placed in global sections that are replicated across the datacenters. The global firewall sections allow multiple network managers to share, replicate and remain in sync for distributed firewall rules or part of the rule configuration. The members of the global grouping constructs are defined such that each member's identification is unique across all datacenters that are participating in the multi datacenter solution.
In the example of
The global sections are replicated across multiple network managers 1051-1053. The global sections in some embodiments are read only on the slave network managers and cannot be edited on slave network manager from an API or user interface (UI).
The rule precedence across the global sections does not interfere with local sections on slave network managers. The overall precedence and ordering of the global sections are maintained. On the slave network manager, the global sections are inserted at the top of the existing sections (e.g., the default section). The administrator in some embodiments is allowed to add sections above and below a global section but that order is overridden in the next update of global sections. The grouping constructs used in the global sections are replicated on the network managers. The object definitions are not edited on the slave network managers.
III. Electronic System
Many of the above-described features and applications are implemented as software processes that are specified as a set of instructions recorded on a computer readable storage medium (also referred to as computer readable medium). When these instructions are executed by one or more processing unit(s) (e.g., one or more processors, cores of processors, or other processing units), they cause the processing unit(s) to perform the actions indicated in the instructions. Examples of computer readable media include, but are not limited to, CD-ROMs, flash drives, RAM chips, hard drives, EPROMs, etc. The computer readable media does not include carrier waves and electronic signals passing wirelessly or over wired connections.
In this specification, the term “software” is meant to include firmware residing in read-only memory or applications stored in magnetic storage, which can be read into memory for processing by a processor. Also, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program while remaining distinct software inventions. In some embodiments, multiple software inventions can also be implemented as separate programs. Finally, any combination of separate programs that together implement a software invention described here is within the scope of the invention. In some embodiments, the software programs, when installed to operate on one or more electronic systems, define one or more specific machine implementations that execute and perform the operations of the software programs.
The bus 1105 collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of the electronic system 1100. For instance, the bus 1105 communicatively connects the processing unit(s) 1110 with the read-only memory 1130, the system memory 1120, and the permanent storage device 1135.
From these various memory units, the processing unit(s) 1110 retrieve instructions to execute and data to process in order to execute the processes of the invention. The processing unit(s) may be a single processor or a multi-core processor in different embodiments.
The read-only-memory 1130 stores static data and instructions that are needed by the processing unit(s) 1110 and other modules of the electronic system. The permanent storage device 1135, on the other hand, is a read-and-write memory device. This device is a non-volatile memory unit that stores instructions and data even when the electronic system 1100 is off. Some embodiments of the invention use a mass-storage device (such as a magnetic or optical disk and its corresponding disk drive) as the permanent storage device 1135.
Other embodiments use a removable storage device (such as a floppy disk, flash drive, etc.) as the permanent storage device. Like the permanent storage device 1135, the system memory 1120 is a read-and-write memory device. However, unlike storage device 1135, the system memory is a volatile read-and-write memory, such as random access memory. The system memory stores some of the instructions and data that the processor needs at runtime. In some embodiments, the invention's processes are stored in the system memory 1120, the permanent storage device 1135, and/or the read-only memory 1130. From these various memory units, the processing unit(s) 1110 retrieve instructions to execute and data to process in order to execute the processes of some embodiments.
The bus 1105 also connects to the input and output devices 1140 and 1145. The input devices enable the user to communicate information and select commands to the electronic system. The input devices 1140 include alphanumeric keyboards and pointing devices (also called “cursor control devices”). The output devices 1145 display images generated by the electronic system. The output devices include printers and display devices, such as cathode ray tubes (CRT) or liquid crystal displays (LCD). Some embodiments include devices such as a touchscreen that function as both input and output devices.
Finally, as shown in
Some embodiments include electronic components, such as microprocessors, storage and memory that store computer program instructions in a machine-readable or computer-readable medium (alternatively referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, read-only compact discs (CD-ROM), recordable compact discs (CD-R), rewritable compact discs (CD-RW), read-only digital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), a variety of recordable/rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD cards, mini-SD cards, micro-SD cards, etc.), magnetic and/or solid state hard drives, read-only and recordable Blu-Ray® discs, ultra density optical discs, any other optical or magnetic media, and floppy disks. The computer-readable media may store a computer program that is executable by at least one processing unit and includes sets of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as is produced by a compiler, and files including higher-level code that are executed by a computer, an electronic component, or a microprocessor using an interpreter.
While the above discussion primarily refers to microprocessor or multi-core processors that execute software, some embodiments are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions that are stored on the circuit itself.
As used in this specification, the terms “computer”, “server”, “processor”, and “memory” all refer to electronic or other technological devices. These terms exclude people or groups of people. For the purposes of the specification, the terms display or displaying means displaying on an electronic device. As used in this specification, the terms “computer readable medium,” “computer readable media,” and “machine readable medium” are entirely restricted to tangible, physical objects that store information in a form that is readable by a computer. These terms exclude any wireless signals, wired download signals, and any other ephemeral or transitory signals.
While the invention has been described with reference to numerous specific details, one of ordinary skill in the art will recognize that the invention can be embodied in other specific forms without departing from the spirit of the invention. In addition, a number of the figures (including
This specification refers throughout to computational and network environments that include virtual machines (VMs). However, virtual machines are merely one example of data compute nodes (DCNs) or data compute end nodes, also referred to as addressable nodes. DCNs may include non-virtualized physical hosts, virtual machines, containers that run on top of a host operating system without the need for a hypervisor or separate operating system, and hypervisor kernel network interface modules.
VMs, in some embodiments, operate with their own guest operating systems on a host using resources of the host virtualized by virtualization software (e.g., a hypervisor, virtual machine monitor, etc.). The tenant (i.e., the owner of the VM) can choose which applications to operate on top of the guest operating system. Some containers, on the other hand, are constructs that run on top of a host operating system without the need for a hypervisor or separate guest operating system. In some embodiments, the host operating system uses name spaces to isolate the containers from each other and therefore provides operating-system level segregation of the different groups of applications that operate within different containers. This segregation is akin to the VM segregation that is offered in hypervisor-virtualized environments that virtualize system hardware, and thus can be viewed as a form of virtualization that isolates different groups of applications that operate in different containers. Such containers are more lightweight than VMs.
Hypervisor kernel network interface module, in some embodiments, is a non-VM DCN that includes a network stack with a hypervisor kernel network interface and receive/transmit threads. One example of a hypervisor kernel network interface module is the vmknic module that is part of the ESXi™ hypervisor of VMware, Inc.
One of ordinary skill in the art will recognize that while the specification refers to VMs, the examples given could be any type of DCNs, including physical hosts, VMs, non-VM containers, and hypervisor kernel network interface modules. In fact, the example networks could include combinations of different types of DCNs in some embodiments.
In view of the foregoing, one of ordinary skill in the art would understand that the invention is not to be limited by the foregoing illustrative details, but rather is to be defined by the appended claims.
This application is a continuation application of U.S. patent application Ser. No. 15/197,652, filed Jun. 29, 2016, now published as U.S. Patent Publication 2017/0005988. U.S. patent application Ser. No. 15/197,652 claims the benefit of U.S. Provisional Patent Application 62/187,123, filed Jun. 30, 2015. U.S. patent application Ser. No. 15/197,652, now published as U.S. Patent Publication 2017/0005988, and U.S. Provisional Patent Application 62/187,123 are incorporated herein by reference.
Number | Name | Date | Kind |
---|---|---|---|
5581722 | Welland | Dec 1996 | A |
5968176 | Nessett et al. | Oct 1999 | A |
6154775 | Coss et al. | Nov 2000 | A |
6219786 | Cunningham et al. | Apr 2001 | B1 |
6496935 | Fink et al. | Dec 2002 | B1 |
6600744 | Carr et al. | Jul 2003 | B1 |
6708187 | Shanumgam et al. | Mar 2004 | B1 |
6880089 | Bommareddy et al. | Apr 2005 | B1 |
7032022 | Shanumgam et al. | Apr 2006 | B1 |
7055173 | Chaganty et al. | May 2006 | B1 |
7106756 | Donovan et al. | Sep 2006 | B1 |
7227842 | Ji et al. | Jun 2007 | B1 |
7349382 | Marimuthu et al. | Mar 2008 | B2 |
7657887 | Kothandaraman et al. | Feb 2010 | B2 |
7676836 | Prigent et al. | Mar 2010 | B2 |
7724740 | Wang et al. | May 2010 | B1 |
7818452 | Matthews et al. | Oct 2010 | B2 |
7831826 | Koti et al. | Nov 2010 | B2 |
7894480 | Wang et al. | Feb 2011 | B1 |
7948986 | Ghosh et al. | May 2011 | B1 |
7954143 | Aaron | May 2011 | B2 |
8032933 | Turley et al. | Oct 2011 | B2 |
8065725 | Zheng et al. | Nov 2011 | B2 |
8190767 | Maufer et al. | May 2012 | B1 |
8365294 | Ross | Jan 2013 | B2 |
8578500 | Long | Nov 2013 | B2 |
8621552 | Lotem et al. | Dec 2013 | B1 |
8660129 | Brendel et al. | Feb 2014 | B1 |
8813209 | Bhattacharya et al. | Aug 2014 | B2 |
8904511 | O'neill et al. | Dec 2014 | B1 |
9015823 | Koponen et al. | Apr 2015 | B2 |
9047109 | Wang et al. | Jun 2015 | B1 |
9130901 | Lee | Sep 2015 | B2 |
9154462 | Grimes et al. | Oct 2015 | B2 |
9215210 | Raman et al. | Dec 2015 | B2 |
9215213 | Bansal et al. | Dec 2015 | B2 |
9215214 | Bansal et al. | Dec 2015 | B2 |
9276904 | Bansal et al. | Mar 2016 | B2 |
9367257 | Hamilton et al. | Jun 2016 | B2 |
9369431 | Kirby et al. | Jun 2016 | B1 |
9479464 | Wang et al. | Oct 2016 | B1 |
9553806 | Anand | Jan 2017 | B2 |
9614748 | Battersby et al. | Apr 2017 | B1 |
9680706 | Masurekar et al. | Jun 2017 | B2 |
9755903 | Masurekar et al. | Sep 2017 | B2 |
9774537 | Jain et al. | Sep 2017 | B2 |
9774707 | Parthasarathy et al. | Sep 2017 | B2 |
9806948 | Masurekar et al. | Oct 2017 | B2 |
9860279 | Jain et al. | Jan 2018 | B2 |
9894103 | Kwok et al. | Feb 2018 | B2 |
9906560 | Jain et al. | Feb 2018 | B2 |
9906561 | Jain et al. | Feb 2018 | B2 |
9906562 | Jain et al. | Feb 2018 | B2 |
10135727 | Gude et al. | Nov 2018 | B2 |
20020078370 | Tahan | Jun 2002 | A1 |
20030041266 | Ke et al. | Feb 2003 | A1 |
20030093481 | Mitchell et al. | May 2003 | A1 |
20030120955 | Bartal et al. | Jun 2003 | A1 |
20030126468 | Markham | Jul 2003 | A1 |
20030226027 | Marquet et al. | Dec 2003 | A1 |
20040049701 | Le Pennec et al. | Mar 2004 | A1 |
20040177276 | Mackinnon et al. | Sep 2004 | A1 |
20040223495 | Pachl | Nov 2004 | A1 |
20040243835 | Terzis et al. | Dec 2004 | A1 |
20050190909 | Yoneyama et al. | Sep 2005 | A1 |
20050198125 | Macleod Beck et al. | Sep 2005 | A1 |
20050210291 | Miyawaki et al. | Sep 2005 | A1 |
20050276262 | Schuba et al. | Dec 2005 | A1 |
20050278431 | Goldschmidt et al. | Dec 2005 | A1 |
20060013136 | Goldschmidt et al. | Jan 2006 | A1 |
20060129808 | Koti et al. | Jun 2006 | A1 |
20060168213 | Richardson et al. | Jul 2006 | A1 |
20060195896 | Fulp et al. | Aug 2006 | A1 |
20070028291 | Brennan et al. | Feb 2007 | A1 |
20070061492 | Van Riel | Mar 2007 | A1 |
20070118893 | Crawford | May 2007 | A1 |
20070136813 | Wong | Jun 2007 | A1 |
20080072305 | Casado et al. | Mar 2008 | A1 |
20080082977 | Araujo et al. | Apr 2008 | A1 |
20080115190 | Aaron | May 2008 | A1 |
20080148382 | Bartholomy et al. | Jun 2008 | A1 |
20080189769 | Casado et al. | Aug 2008 | A1 |
20080215518 | Matsuda | Sep 2008 | A1 |
20080267177 | Johnson et al. | Oct 2008 | A1 |
20080289028 | Jansen et al. | Nov 2008 | A1 |
20080298274 | Takashige et al. | Dec 2008 | A1 |
20090007219 | Abzarian et al. | Jan 2009 | A1 |
20090007251 | Abzarian et al. | Jan 2009 | A1 |
20090083727 | Fu et al. | Mar 2009 | A1 |
20090129271 | Ramankutty et al. | May 2009 | A1 |
20090150521 | Tripathi | Jun 2009 | A1 |
20090228972 | Bandi et al. | Sep 2009 | A1 |
20090235325 | Dimitrakos et al. | Sep 2009 | A1 |
20090249436 | Coles et al. | Oct 2009 | A1 |
20090249438 | Litvin et al. | Oct 2009 | A1 |
20090249470 | Litvin et al. | Oct 2009 | A1 |
20090249472 | Litvin | Oct 2009 | A1 |
20090276771 | Nickolov et al. | Nov 2009 | A1 |
20090327198 | Farah | Dec 2009 | A1 |
20090327781 | Tripathi | Dec 2009 | A1 |
20100037310 | Turley et al. | Feb 2010 | A1 |
20100037311 | He et al. | Feb 2010 | A1 |
20100037331 | Blake et al. | Feb 2010 | A1 |
20100043067 | Varadhan et al. | Feb 2010 | A1 |
20100100616 | Bryson et al. | Apr 2010 | A1 |
20100100949 | Sonwane et al. | Apr 2010 | A1 |
20100125667 | Soundararajan | May 2010 | A1 |
20100180331 | Murakami et al. | Jul 2010 | A1 |
20100192215 | Yaxuan et al. | Jul 2010 | A1 |
20110016467 | Kane | Jan 2011 | A1 |
20110022695 | Dalal et al. | Jan 2011 | A1 |
20110055916 | Ahn | Mar 2011 | A1 |
20110072486 | Hadar et al. | Mar 2011 | A1 |
20110103259 | Aybay et al. | May 2011 | A1 |
20110113467 | Agarwal et al. | May 2011 | A1 |
20110154470 | Grimes et al. | Jun 2011 | A1 |
20110213875 | Ferris et al. | Sep 2011 | A1 |
20110246637 | Murakami | Oct 2011 | A1 |
20110302647 | Bhattacharya et al. | Dec 2011 | A1 |
20120042033 | Ayala, Jr. et al. | Feb 2012 | A1 |
20120137199 | Liu | May 2012 | A1 |
20120180104 | Gronich et al. | Jul 2012 | A1 |
20120240182 | Narayanaswamy et al. | Sep 2012 | A1 |
20120263049 | Venkatachalapathy et al. | Oct 2012 | A1 |
20120291024 | Barabash et al. | Nov 2012 | A1 |
20120314617 | Erichsen et al. | Dec 2012 | A1 |
20130007740 | Kikuchi et al. | Jan 2013 | A1 |
20130019277 | Chang et al. | Jan 2013 | A1 |
20130031544 | Sridharan et al. | Jan 2013 | A1 |
20130047151 | Sridharan et al. | Feb 2013 | A1 |
20130073743 | Ramasamy et al. | Mar 2013 | A1 |
20130074066 | Sanzgiri et al. | Mar 2013 | A1 |
20130125230 | Koponen et al. | May 2013 | A1 |
20130163594 | Sharma et al. | Jun 2013 | A1 |
20130198355 | Kalyanaraman et al. | Aug 2013 | A1 |
20130219384 | Srinivasan et al. | Aug 2013 | A1 |
20130227097 | Yasuda et al. | Aug 2013 | A1 |
20130227550 | Weinstein et al. | Aug 2013 | A1 |
20130311358 | Sethi et al. | Nov 2013 | A1 |
20130311612 | Dickinson | Nov 2013 | A1 |
20130332983 | Koorevaar et al. | Dec 2013 | A1 |
20140068602 | Gember et al. | Mar 2014 | A1 |
20140108319 | Klauser et al. | Apr 2014 | A1 |
20140115578 | Cooper et al. | Apr 2014 | A1 |
20140149794 | Shetty et al. | May 2014 | A1 |
20140195666 | Dumitriu et al. | Jul 2014 | A1 |
20140245423 | Lee et al. | Aug 2014 | A1 |
20140281030 | Cui et al. | Sep 2014 | A1 |
20140282855 | Clark et al. | Sep 2014 | A1 |
20140282889 | Ishaya et al. | Sep 2014 | A1 |
20140297780 | Zhou et al. | Oct 2014 | A1 |
20140304387 | Bansal et al. | Oct 2014 | A1 |
20140325037 | Elisha | Oct 2014 | A1 |
20150052521 | Raghu | Feb 2015 | A1 |
20150112933 | Satapathy | Apr 2015 | A1 |
20150193466 | Luft et al. | Jul 2015 | A1 |
20150200816 | Yung et al. | Jul 2015 | A1 |
20150229641 | Sun et al. | Aug 2015 | A1 |
20150237013 | Bansal et al. | Aug 2015 | A1 |
20150237014 | Bansal et al. | Aug 2015 | A1 |
20150237015 | Bansal et al. | Aug 2015 | A1 |
20150277949 | Loh | Oct 2015 | A1 |
20160050117 | Voellmy et al. | Feb 2016 | A1 |
20160050141 | Wu et al. | Feb 2016 | A1 |
20160094456 | Jain et al. | Mar 2016 | A1 |
20160094643 | Jain et al. | Mar 2016 | A1 |
20160105333 | Lenglet et al. | Apr 2016 | A1 |
20160112374 | Branca | Apr 2016 | A1 |
20160149863 | Walker et al. | May 2016 | A1 |
20160156591 | Zhou et al. | Jun 2016 | A1 |
20160182454 | Phonsa et al. | Jun 2016 | A1 |
20160191466 | Pernicha | Jun 2016 | A1 |
20160191570 | Bansal et al. | Jun 2016 | A1 |
20160241702 | Gorajala Chandra et al. | Aug 2016 | A1 |
20170004192 | Masurekar et al. | Jan 2017 | A1 |
20170005867 | Masurekar et al. | Jan 2017 | A1 |
20170005987 | Masurekar et al. | Jan 2017 | A1 |
20170005988 | Bansal et al. | Jan 2017 | A1 |
20170026283 | Williams et al. | Jan 2017 | A1 |
20170064749 | Jain | Mar 2017 | A1 |
20170134422 | Shieh et al. | May 2017 | A1 |
20170180319 | Nimmagadda et al. | Jun 2017 | A1 |
20170180320 | Nimmagadda et al. | Jun 2017 | A1 |
20170180321 | Nimmagadda et al. | Jun 2017 | A1 |
20170180423 | Nimmagadda et al. | Jun 2017 | A1 |
20170187679 | Basak et al. | Jun 2017 | A1 |
20170250869 | Voellmy | Aug 2017 | A1 |
20170317928 | Gude et al. | Nov 2017 | A1 |
20170317976 | Chalvadi et al. | Nov 2017 | A1 |
20170317977 | Popuri et al. | Nov 2017 | A1 |
20170317979 | Bansal et al. | Nov 2017 | A1 |
20170318055 | Popuri et al. | Nov 2017 | A1 |
20180007000 | Bansal et al. | Jan 2018 | A1 |
20180007007 | Bansal et al. | Jan 2018 | A1 |
20180007008 | Bansal et al. | Jan 2018 | A1 |
Number | Date | Country |
---|---|---|
2748750 | Jul 2014 | EP |
2003188906 | Jul 2003 | JP |
2006040274 | Feb 2006 | JP |
2009017269 | Jan 2009 | JP |
2013012865 | Jan 2013 | JP |
20080100620 | Nov 2008 | KR |
2008095010 | Aug 2008 | WO |
2013074828 | May 2013 | WO |
Entry |
---|
Author Unknown, “Enabling Service Chaining on Cisco Nexus 1000V Series,” Month Unknown, 2012, 25 pages, Cisco. |
Author Unknown, “Next-Generation Firewalls,” Oct. 1, 2013, 1 page, Palo Alto Networks. |
Basak, Debashis, et al., “Virtualizing Networking and Security in the Cloud,” Month Unknown, 2010, 9 pages, VMware.com. |
Casado, Martin, et al., “SANE: A Protection Architecture for Enterprise Networks,” Proceedings of the 15th USENIX Security Symposium, Jul. 31, 2006, 15 pages. |
Dixon, Colin, et al., “An End to the Middle,” Proceedings of the 12th conference on Hot topics in operating systems USENIX Association, May 2009, 5 pages, Berkeley, CA, USA. |
Dumitriu, Dan Mihai, et al., U.S. Appl. No. 61/514,990, filed Aug. 4, 2011. |
Guichard, J., et al., “Network Service Chaining Problem Statement; draft-quinn-nsc-probiem-statement-00.txt,” Jun. 13, 2013, 14 pages, Cisco Systems, Inc. |
Ioannidis, Sotiris, et al., “Implementing a Distributed Firewall,” CCS'00, Month Unknown, 2000, 10 pages, ACM, Athens, Greece. |
Joseph, Dilip Antony, et al., “A Policy-aware Switching Layer for Data Centers,” Jun. 24, 2008, 26 pages, Electrical Engineering and Computer Sciences, University of California, Berkeley, CA, USA. |
Mayer, Alain, et al., “Offline Firewall Analysis,” Int. L. Inf. Secur. (2006) 5(3), Jun. 16, 2005, 20 pages, Springer-Verlag. |
Scarfone, Karen, et al., “Guidelines on Firewalls and Firewall Policy,” Sep. 2009, 48 pages, NIST, U.S. Department of Commerce. |
Sekar, Vyas, at al., “Design and Implementation of a Consolidated Middlebox Architecture,” In Proc. of NSDI, Month Unknown, 2012, 14 pages. |
Sherry, Justine, et al., “Making Middleboxes Someone Else's Problem: Network Processing as a Cloud Service,” In Proc. of SIGCOMM, Aug. 13-17, 2012, 12 pages, Helsinki, Finland. |
Stojanovski, Nenad, et al., “Architecture of a Identity Based Firewall System,” Jul. 2011, 9 pages. |
Stojanovski, Nenad, et al., “Analysis of Identity Based Firewall Systems,” Jun. 2010, 9 pages. |
Thames, J. Lane, et al., “A Distributed Firewall and Active Response Architecture Providing Preemptive Protection,” ACM-SE08, Mar. 28-29, 2008, 6 pages, Auburn, AL, USA. |
Herndon, Joseph, “FairWarning IP, LLC v. Iatric Systems, Inc. (Fed. Cir. 2016),” Oct. 13, 2016, 3 ppages. |
Number | Date | Country | |
---|---|---|---|
20170104720 A1 | Apr 2017 | US |
Number | Date | Country | |
---|---|---|---|
62187123 | Jun 2015 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 15197652 | Jun 2016 | US |
Child | 15387242 | US |