Patent Application
20230229479
Benefit is claimed under 35 U.S.C. 119(a)-(d) to Foreign Application Serial No. 202241002420 filed in India entitled “APPLICATION TOPOLOGY DERIVATION IN A VIRTUALIZED COMPUTING SYSTEM”, on Jan. 15, 2022, by VMware, Inc., which is herein incorporated in its entirety by reference for all purposes.
Applications today are deployed onto a combination of virtual machines (VMs), containers, application services, physical servers without virtualization, and more within a software-defined datacenter (SDDC). The SDDC includes a server virtualization layer having clusters of physical servers that are virtualized and managed by virtualization management servers. Each host includes a virtualization layer (e.g., a hypervisor) that provides a software abstraction of a physical server (e.g., central processing unit (CPU), random access memory (RAM), storage, network interface card (NIC), etc.) to the VMs. A user, or automated software on behalf of an Infrastructure as a Service (IaaS), interacts with a virtualization management server to create server clusters (“host clusters”), add/remove servers (“hosts”) from host clusters, deploy/move/remove VMs on the hosts, deploy/configure networking and storage virtualized infrastructure, and the like. The virtualization management server sits on top of the server virtualization layer of the SDDC and treats host clusters as pools of compute capacity for use by applications.
Applications executing in a virtualized computing system can include many software components. A user’s view of the applications via virtualization management tools can drift from the actual state of the applications as the virtualized computing system and applications evolve over time. A user may be unaware of the application software executing in the VMs. It is desirable to provide an application discovery process that is automated and provides a more accurate view of applications, their components, relationships, dependencies, and interdependencies.
Embodiments include a method of determining application topology in a virtualized computing system having a cluster of hosts, the hosts including hypervisors supporting virtual machines (VMs). The method includes: executing agents on the VMs to obtain process metadata describing processes executing in the VMs; receiving, at an application analysis system, the process metadata; receiving, at the application analysis system, network flow metadata from the agents on the VMs, from a network analyzer in the virtualized computing system, or from both the agents and the network analyzer; parsing, by the application analysis system, the network flow metadata to identify a source VM and a destination VM of the VMs; relating, by the application analysis system, the network flow metadata to portions of the process metadata associated with the source and the destination VMs to identify a source process and a destination process; and generating, by the application analysis software, a topology of a source component connected to a destination component, the source component identifying the source VM and the source process, the destination component identifying the destination VM and the destination process.
Further embodiments include a non-transitory computer-readable storage medium comprising instructions that cause a computer system to carry out the above methods, as well as a computer system configured to carry out the above methods.
In the embodiment illustrated in
A software platform 124 of each host 120 provides a virtualization layer, referred to herein as a hypervisor 150, which directly executes on hardware platform 122. In an embodiment, there is no intervening software, such as a host operating system (OS), between hypervisor 150 and hardware platform 122. Thus, hypervisor 150 is a Type-1 hypervisor (also known as a “bare-metal” hypervisor). As a result, the virtualization layer in host cluster 118 (collectively hypervisors 150) is a bare-metal virtualization layer executing directly on host hardware platforms. Hypervisor 150 abstracts processor, memory, storage, and network resources of hardware platform 122 to provide a virtual machine execution space within which multiple virtual machines (VM) 140 may be concurrently instantiated and executed. One example of hypervisor 150 that may be configured and used in embodiments described herein is a VMware ESXi™ hypervisor provided as part of the VMware vSphere® solution made commercially available by VMware, Inc. of Palo Alto, CA.
Software 148 executes in VMs 140 (e.g.. guest operating systems, applications, etc.) and includes processes 149. A process 149 is an instance of a computer program. A process includes a portion of the computer’s (e.g., a VM 140) virtual memory, which is occupied by the computer program’s executable code and data, and a data structure maintained by the computer’s operating system. For example, the Linux® operating system maintains a data structure for each process known as a Process Control Block (PCB). The data structure includes information such as the process running state, the process scheduling state, memory management information, interprocess communication (IPC) information, open file descriptors held by the process, and the like. Other commercial operating systems include similar data structures for each process.
Virtualized computing system 100 is configured with a software-defined (SD) network layer 175. SD network layer 175 includes logical network services executing on virtualized infrastructure of hosts 120. The virtualized infrastructure that supports the logical network services includes hypervisor-based components, such as resource pools, distributed switches, distributed switch port groups and uplinks, etc., as well as VM-based components, such as router control VMs, load balancer VMs, edge service VMs, etc. Logical network services include logical switches and logical routers, as well as logical firewalls, logical virtual private networks (VPNs), logical load balancers, and the like, implemented on top of the virtualized infrastructure. In embodiments, virtualized computing system 100 includes edge transport nodes 178 that provide an interface of host cluster 118 to wide area network (WAN) (e.g., a corporate network, the public Internet, etc.). Edge transport nodes 178 can include a gateway (e.g., implemented by a router) between the internal logical networking of host cluster 118 and the external network. Edge transport nodes 178 can be physical servers or VMs. Virtualized computing system 100 also includes physical network devices (e.g., physical routers/switches) as part of physical network 180. which are not explicitly shown.
Virtualization management server 116 is a physical or virtual server that manages hosts 120 and the hypervisors therein. Virtualization management server 116 installs agent(s) in hypervisor 150 to add a host 120 as a managed entity. Virtualization management server 116 can logically group hosts 120 into host cluster 118 to provide cluster-level functions to hosts 120, such as VM migration between hosts 120 (e.g.. for load balancing), distributed power management, dynamic VM placement according to affinity and anti-affinity rules, and high-availability. The number of hosts 120 in host cluster 118 may be one or many. Virtualization management server 116 can manage more than one host cluster 118. While only one virtualization management server 116 is shown, virtualized computing system 100 can include multiple virtualization management servers each managing one or more host clusters.
In an embodiment, virtualized computing system 100 further includes a network manager 112. Network manager 112 is a physical or virtual server that orchestrates SD network layer 175. In an embodiment, network manager 112 comprises one or more virtual servers deployed as VMs. Network manager 112 installs additional agents in hypervisor 150 to add a host 120 as a managed entity, referred to as a transport node. One example of an SD networking platform that can be configured and used in embodiments described herein as network manager 112 and SD network layer 175 is a VMware NSX® platform made commercially available by VMware, Inc. of Palo Alto, CA. In other embodiments. SD network layer 175 is orchestrated and managed by virtualization management server 116 without the presence of network manager 112.
Virtualization management server 116 can include various virtual infrastructure (VI) services 108. VI services 108 can include various services, such as a management daemon, distributed resource scheduler (DRS), high-availability (HA) service, single sign-on (SSO) service, and the like. VI services 108 persist data in a database 115, which stores an inventory of objects, such as clusters, hosts, VMs, resource pools, datastores, and the like. Users interact with VI services 108 through user interfaces, application programming interfaces (APIs), and the like to issue commands, such as forming a host cluster 118, configuring resource pools, define resource allocation policies, configure storage and networking, and the like.
In embodiments, software 148 can also execute in containers 130. In embodiments, hypervisor 150 can support containers 130 executing directly thereon. In other embodiments, containers 130 are deployed in VMs 140 or in specialized VMs referred to as “pod VMs 131.” A pod VM 131 is a VM that includes a kernel and container engine that supports execution of containers, as well as an agent (referred to as a pod VM agent) that cooperates with a controller executing in hypervisor 150. In embodiments, virtualized computing system 100 can include a container orchestrator 177. Container orchestrator 177 implements an orchestration control plane, such as Kubernetes®, to deploy and manage applications or services thereof in pods on hosts 120 using containers 130. Container orchestrator 177 can include one or more master servers configured to command and configure controllers in hypervisors 150. Master server(s) can be physical computers attached to network 180 or implemented by VMs 140/131 in a host cluster 118.
In embodiments, virtualized computing system includes application analysis system 132. Application analysis system 132 can execute on one or more servers, which can be physical servers, VMs, containers, or some combination thereof. Application analysis system can execute in host cluster 118, execute external to host cluster 118, execute external to the SDDC, or some combination thereof.
Application analysis system 132 is configured to discover, collect, and store metadata about applications executing on host cluster 118. The collected metadata is useful for discovering the nature of constituent components of target applications and their topologies. The collected metadata can be used for various purposes, such as re-platforming a traditional application executing on operating systems to a containerized application executing in a container-based environment (e.g., Kubernetes®). In embodiments, application analysis software 132 installs agents in VMs 131/140 to collect information about executing processes during metadata collection. The term installation, as used herein, encompasses various forms of having agents be executed in VMs 131/140, such as a conventional installation process, adding agents to a template from which VMs 131/140 are provisioned, attaching virtual disks to VMs 131/140 having executable code of agents, instructing an interpreter in VMs 131/140 to execute a sequence of commands as agents, and the like. In general, application analysis system 132 configures VMs 131/140 to execute agents using any or a combination of such techniques. A user can access application analysis system 132 through user interfaces and/or APIs.
In embodiments, virtualized computing system 100 can include network analyzer 113. Network analyzer 113 is configured to perform various network analyses on SD network layer 175 and VMs 131/140 connected thereto. For example, network analyzer 113 can collect network flow information from virtualization management server 116 and/or network manager 112. The network flow information describes the network traffic flows between VMs 131/140. Network analyzer 113 can also detect communications with external services, such as domain name service (DNS), network time protocol (NTP), and the like as part of the network flow information. In embodiments, network analyzer 113 can be implemented using VMware vRealize® Network Insight™ commercially available from VMware, Inc. of Palo Alto, CA. Application analysis system 132 can leverage network flow information collected by network analyzer 113 to detect traffic flows between VMs and map such traffic flows to the identified application components to determine the application topology, as described further herein.
Software 148 executes in VMs 131/140 on hosts 120 in cluster 118 alongside virtualization management server 116 and network analyzer 113 (if present). Application analysis system 132 communicates with VMs 131/140 in cluster 118, virtualization management server 116, and network analyzer 113 (if present) to collect and store metadata for components 208. Application analysis system 132 can process the collected metadata to identify applications 210 and determine topologies 212. The metadata collected by application analysis system 132 includes OS-level process metadata 214 and network flow data 216. Application analysis system 132 obtains OS-level process metadata 214 from VMs 131/140. In embodiments, application analysis system 132 obtains some or all of network flow data 216 from VMs 131/140 (e.g., derived from OS-level process metadata 214 or collected independently). In embodiments, application analysis system 132 obtains some or all of network flow data 216 from network analyzer 113. Network analyzer 113 can collect network flow records from traffic flow data source(s) 214 in cluster 118. Example network flow records include NetFlow records and Internet Protocol Flow Information Export (IPFIX) records. NetFlow is a network protocol developed for collecting IP traffic information and monitoring network flow. IPFIX is a universal solution to collect and analyze useful network data.
At step 504, application analysis software 132 receives network flow data 216. In embodiments, application analysis software 132 receives VM-level network event data from application analysis software agent 314 (506). Application analysis software agent 314 can obtain network event data from guest OS 312 using a network event library (e.g., libnetfilter). A network event can include, for example, the protocol, lifetime, state, original direction, reply direction, and the like. Original direction information can include source IP, destination IP, source port, and destination port (source tuple). Reply direction information can include source IP, destination IP, source port, and destination port (reply tuple). Network flow data 216 can include a plurality of such network events obtained from VMs 131/140.
In embodiments, application analysis software 132 receives SDDC-level network flow data from network analyzer 113 (508). SDDC-level network flow data includes network flow records collected from sources in host cluster 118 (e.g., DVS 308). A network flow record can include protocol, timestamp, traffic type, firewall action, source identifiers, destination identifiers. Source and destination identifiers can include virtualization management server ID, VM ID, port, source IP, destination IP, cluster ID, host ID, SDDC ID, datastore folder, and the like.
At step 510. application analysis software 132 receives inventory data from virtualization management server 116 of the target SDDC. Such inventory data can include, for example, VM IDs matching input IP addresses. For example, application analysis software 132 can obtain an IP address from a network event and query virtualization management server 116 for a VM ID associated with that IP address.
At step 512, parse logic 418 of application analysis software 132 parses the collected metadata to generate process information (info) objects 402 and network info objects 410. Process info objects 402 include process metadata 404, open socket list 406, and previously open socket list 408. Process metadata 404 includes fields for storing the various process metadata collected in OS-level process metadata 214 (e.g., process name, process identifier (PID), executable path, executable version, command line parameters, working directory, environment variables, start time, owner/user, and the like). Open socket list 406 includes currently open sockets for the process (e.g., local port, local address, remote port, remote address, connection type, socket state, etc.). Previously open socket list 408 includes a list of sockets once used by the process (e.g., local port, local address, remote port, remote address, connection type, socket state, etc.). Network info object 410 includes source identity 412, destination identity 414, and flow data 416. Source identity 412 can include virtualization management server ID, VM ID, source port, source IP, cluster ID, host ID, SDDC ID, datastore folder, and the like. Destination identity 414 can include virtualization management server ID, VM ID, destination port, destination IP, cluster ID, host ID, SDDC ID, datastore folder, and the like. Flow data 416 can include protocol, timestamp, traffic type, firewall action, and the like.
At step 514, matching logic 420 of application analysis system 132 performs metadata matching using process info objects 402 and network info objects 410 as parametric input. In embodiments, at step 516, matching logic 420 relates metadata from network flows and inventory data to determine source and destination VMs. At step 518. matching logic 420 scans process info objects 420 related to source and destination VMs to determine a component-flow topology. At step 520, application analysis system 132 generates topology objects 422 representing discovered component-flow topologies. A topology object 422 includes source component info 424, destination component info 426, and connection data (e.g., protocol, timestamp, etc.). Source component info 424 can include virtualization management server ID, VM ID, process ID, and port. Destination component info 426 can include virtualization management server ID, VM ID, process ID, and port.
At step 614, matching logic 420 selects a destination IP from the network info object. At step 616, matching logic 420 compares the destination IP against inventory data obtained from virtualization management server 116 to identify a destination VM. At step 618, matching logic 420 identifies a destination process in the destination VM. For example, at step 620, matching logic 420 identifies all processes in destination VM from process info objects 402. At step 622, matching logic 420 compares network info from the network info object against socket lists in the process info objects to identify a matching process. At step 624, matching logic 420 outputs a destination component.
In the example, assume VM 702 has a VM ID of 1 and an IP address 172.0.0.1. VM 706 has a VM ID of 2 and an IP address of 172.0.0.2. Process 704 has a PID 20 and process 708 has a PID 30. Process 704 establishes a UDP connection through its port 3259 with process 708 through its port 32098. Application analysis system 132 can identify the UDP connection from a network event. Application analysis system 132 queries virtualization management server 116 with the IP addresses 172.0.0.1 and 172.0.0.2 to identify VMs 702 and 706 having VM IDs 1 and 2, respectively. Application analysis system 132 the generates a component topology where the source is identified by virtualization management server ID 000-000-001, VM ID 1, process ID 20, and port 3259. and the destination is identified by virtualization management server ID 000-000-001, VM ID 2, process ID 30, and port 32098. The connection is identified as a UDP connection.
In the example, assume VM 802 has a VM ID of 3 and VM 806 has a VM ID of 4. VM 802 has an IP address 172.0.0.1 and VM 806 has an IP address 192.168.0.1. Process 804 has a PID 3001 and connects through port 1506. Process 808 has a PID 4001 and receives connection on port 80. Application analysis system 132 obtains the following network flow record:
The example network flow record shows TCP traffic originating on VM 802 (VM ID 3) destined for VM 806 (VM ID 4) at port 80. The only missing data from the network flow record are the process IDs of the communicating processes. Application analysis system 132 correlates the port numbers and timestamps against socket lists in process info objects 402 for VMs 802 and 806 to identify processes 804 and 808. For example, OS-level metadata from VM 802 can include:
OS-level metadata from VM 806 can include:
By comparing the port numbers against the socket lists, application analysis system 132 identifies the processes 804 and 808 and their respective PIDs. Application analysis system 132 then generates the final component topology where the source is identified by virtualization management server ID 000-000-0001, VM ID 3, process ID 3001, and port 1506, and the destination is identified by virtualization management server ID 000-000-002, VM ID 4, process ID 4001, and port 80. This example assumes two separate virtualization management servers managing each VM 802 and 806. The connection between source and destination components is identified as a TCP connection having the associated timestamp obtained from the metadata.
One or more embodiments of the invention also relate to a device or an apparatus for performing these operations. The apparatus may be specially constructed for required purposes, or the apparatus may be a general-purpose computer selectively activated or configured by a computer program stored in the computer. Various general-purpose machines may be used with computer programs written in accordance with the teachings herein, or it may be more convenient to construct a more specialized apparatus to perform the required operations.
The embodiments described herein may be practiced with other computer system configurations including hand-held devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, etc.
One or more embodiments of the present invention may be implemented as one or more computer programs or as one or more computer program modules embodied in computer readable media. The term computer readable medium refers to any data storage device that can store data which can thereafter be input to a computer system. Computer readable media may be based on any existing or subsequently developed technology that embodies computer programs in a manner that enables a computer to read the programs. Examples of computer readable media are hard drives, NAS systems, read-only memory (ROM), RAM, compact disks (CDs), digital versatile disks (DVDs), magnetic tapes, and other optical and non-optical data storage devices. A computer readable medium can also be distributed over a network-coupled computer system so that the computer readable code is stored and executed in a distributed fashion.
Although one or more embodiments of the present invention have been described in some detail for clarity of understanding, certain changes may be made within the scope of the claims. Accordingly, the described embodiments are to be considered as illustrative and not restrictive, and the scope of the claims is not to be limited to details given herein but may be modified within the scope and equivalents of the claims. In the claims, elements and/or steps do not imply any particular order of operation unless explicitly stated in the claims.
Virtualization systems in accordance with the various embodiments may be implemented as hosted embodiments, non-hosted embodiments, or as embodiments that blur distinctions between the two. Furthermore, various virtualization operations may be wholly or partially implemented in hardware. For example, a hardware implementation may employ a look-up table for modification of storage access requests to secure non-disk data.
Many variations, additions, and improvements are possible, regardless of the degree of virtualization. The virtualization software can therefore include components of a host, console, or guest OS that perform virtualization functions.
Plural instances may be provided for components, operations, or structures described herein as a single instance. Boundaries between components, operations, and data stores are somewhat arbitrary, and particular operations are illustrated in the context of specific illustrative configurations. Other allocations of functionality are envisioned and may fall within the scope of the invention. In general, structures and functionalities presented as separate components in exemplary configurations may be implemented as a combined structure or component. Similarly, structures and functionalities presented as a single component may be implemented as separate components. These and other variations, additions, and improvements may fall within the scope of the appended claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202241002420 | Jan 2022 | IN | national |
