Patent Application
20240007463
In a software-defined data center (SDDC), virtual infrastructure, which includes virtual machines (VMs) and virtualized storage and networking resources, is provisioned from hardware infrastructure that includes a plurality of host computers (hereinafter also referred to simply as “hosts”), storage devices, and networking devices. The provisioning of the virtual infrastructure is carried out by SDDC management software that is deployed on management appliances, such as a VMware vCenter Server® appliance and a VMware NSX® appliance, from VMware, Inc. The SDDC management software communicates with virtualization software (e.g., a hypervisor) installed in the hosts to manage the virtual infrastructure.
It has become common for multiple SDDCs to be deployed across multiple clusters of hosts. Each cluster is a group of hosts that are managed together by the management software to provide cluster-level functions, such as load balancing across the cluster through VM migration between the hosts, distributed power management, dynamic VM placement according to affinity and anti-affinity rules, and high availability (HA). The management software also manages a shared storage device to provision storage resources for the cluster from the shared storage device, and a software-defined network through which the VMs communicate with each other. For some customers, their SDDCs are deployed across different geographical regions, and may even be deployed in a hybrid manner, e.g., on-premise, in a public cloud, and/or as a service. “SDDCs deployed on-premise” means that the SDDCs are provisioned in a private data center that is controlled by a particular organization. “SDDCs deployed in a public cloud” means that SDDCs of a particular organization are provisioned in a public data center along with SDDCs of other organizations. “SDDCs deployed as a service” means that the SDDCs are provided to the organization as a service on a subscription basis. As a result, the organization does not have to carry out management operations on the SDDC, such as configuration, upgrading, and patching, and the availability of the SDDCs is provided according to the service level agreement of the subscription.
With a large number of SDDCs, monitoring and performing operations on the SDDCs through interfaces, e.g., application programming interfaces (APIs), provided by the management software, and managing the lifecycle of the management software, have proven to be challenging. Conventional techniques for managing the SDDCs and the management software of the SDDCs are not practicable when there is a large number of SDDCs, especially when they are spread out across multiple geographical locations and in a hybrid manner.
One or more embodiments provide a cloud platform from which various services, referred to herein as “cloud services” are delivered to the SDDCs through agents of the cloud services that are running in an appliance (referred to herein as a “agent platform appliance”). As used herein, an SDDC is a virtual computing environment provisioned from a plurality of host computers, storage devices, and networking devices by management software for the virtual computing environment that communicates with hypervisors running in the host computers. The cloud platform is a computing platform that hosts containers or virtual machines corresponding to the cloud services that are delivered from the cloud platform. The agent platform appliance is deployed in the same customer environment, e.g., a private data center, as the management appliances of the SDDCs. In one embodiment, the cloud platform is provisioned in a public cloud and the agent platform appliance is provisioned as a virtual machine, and the two are connected over a public network, such as the Internet. In addition, the agent platform appliance and the management appliances are connected to each other over a private physical network, e.g., a local area network. Examples of cloud services that are delivered include an SDDC configuration service, an SDDC upgrade service, an SDDC monitoring service, an SDDC inventory service, a message broker service, and a service for modifying the inventory of virtual objects deployed in the SDDC (hereinafter referred to as the “mutation service”). As used herein, virtual objects are virtualized resources (e.g., virtual machines and virtual disks) that have been provisioned in the SDDC. Each of these cloud services has a corresponding agent deployed on the agent platform appliance. All communication between the cloud services and the management software of the SDDCs is carried out through the respective agents of the cloud services.
In one embodiment, a command is issued to a management appliance of an SDDC to modify an inventory of virtual objects deployed in the SDDC. A method of issuing such a command includes the steps of: retrieving a message generated by a cloud service running in a cloud platform, the message including a task to modify the inventory of virtual objects deployed in the SDDC, a first token identifying a user who requested the task, and a second token containing information about the management appliance and a role assigned to the user; exchanging the first and second tokens with the management appliance for an authentication token for accessing the management appliance; and transmitting the command to the management appliance along with the authentication token.
Further embodiments include a non-transitory computer-readable storage medium comprising instructions that cause a computer system to carry out the above method, as well as a computer system configured to carry out the above method.
One or more embodiments enable commands input through a multi-tenant cloud platform to be delivered to a targeted management appliance. A number of different tokens are issued in the process of delivering the commands from the cloud platform to the targeted management appliance. These tokens include ID and access tokens that are generated upon user login, an access token that is scoped for the targeted management appliance, a bearer token that is generated by the targeted management appliance in exchange for the scoped access token, and a holder-of-key token that is generated by the targeted management appliance in exchange for the bearer token. The bearer token allows any holder of the token to access the targeted management appliance but is valid only for a limited period of time after it is issued. The holder-of-key token does not expire but it can be used only by the holder of a key associated with the token.
In the embodiments, an agent of the mutation service (hereinafter referred to as the “mutation agent”) is issued the holder-of-key token, and the mutation agent issues the command to modify the inventory of virtual objects, e.g., a command to create a VM, to the targeted management appliance. In addition, the mutation agent checks on the status of the command execution by periodically polling the targeted management appliance for the status. In both cases, the mutation agent transmits the holder-of-key token to the targeted management appliance to authenticate its access to the targeted management appliance.
A plurality of SDDCs is depicted in
The VIM appliances in each customer environment communicate with an agent platform (AP) appliance, which hosts agents (not shown in
As used herein, a “customer environment” means one or more private data centers managed by the customer, which is commonly referred to as “on-prem,” a private cloud managed by the customer, a public cloud managed for the customer by another organization, or any combination of these. In addition, the SDDCs of any one customer may be deployed in a hybrid manner, e.g., on-premise, in a public cloud, or as a service, and across different geographical regions.
In the embodiments, each of the agent platform appliances and the management appliances is a VM instantiated on one or more physical host computers (not shown in
The components of cloud platform 12 include a number of different cloud services that enable each of a plurality of tenants that have registered with cloud platform 12 to manage its SDDCs through cloud platform 12. During registration for each tenant, the tenant's profile information, such as the URLs of the management appliances of its SDDCs and the URL of the tenant's AAA (authentication, authorization and accounting) server 201, is collected, and user IDs and passwords for accessing (i.e., logging into) cloud platform 12 through UI 11 are set up for the tenant. The user IDs and passwords are associated with various users of the tenant's organization who are assigned different roles. The tenant profile information is stored in tenant dbase 111, and login credentials for the tenants are managed according to conventional techniques, e.g., Active Directory® or LDAP (Lightweight Directory Access Protocol).
In one embodiment, each of the cloud services is a microservice that is implemented as one or more container images executed on a virtual infrastructure of public cloud 10. The cloud services include a cloud service provider (CSP) ID service 110, a mutation service 120, a task service 130, a scheduler service 140, and a message broker (MB) service 150. Similarly, each of the agents deployed in the AP appliances is a microservice that is implemented as one or more container images executing in the agent platform appliances.
CSP ID service 110 manages authentication of access to cloud platform 12 through UI 11 or through an API call made to one of the cloud services via API gateway 15. Access through UI 11 is authenticated if login credentials entered by the user are valid. API calls made to the cloud services via API gateway 15 are authenticated if they contain CSP access tokens issued by CSP ID service 110. Such CSP access tokens are issued by CSP ID service 110 in response to a request from identity agent 210 if the request contains valid credentials.
Mutation service 120 is the endpoint service of cloud platform 12 to which all commands to modify an inventory of virtual objects deployed in the SDDCs are routed. These commands are hereinafter referred to as “mutation commands.” In response to a mutation command, mutation service 120 creates a task corresponding to the mutation command and makes an API call to task service 130 to perform the task. Task service 130 then schedules the task to be performed with scheduler service 140, which then creates a message containing the task to be performed and inserts the message in a message queue managed by MB service 150. After scheduling the task to be performed with scheduler service 140, task service 130 periodically polls scheduler service 140 for status of the scheduled task.
At predetermined time intervals, MB agent 220, which is deployed in AP appliance 31, makes an API call to MB service 150 to exchange messages that are queued in their respective queues (not shown), i.e., to transmit to MB service 150 messages MB agent 220 has in its queue and to receive from MB service 150 messages MB service 150 has in its queue. Messages from MB service 150 are routed to coordinator agent 230, and if a message contains a task to modify the inventory of virtual objects deployed in SDDC 41, coordinator agent 230 instantiates a mutation agent 240 to manage the performance of the task. Mutation agent 240 thereafter issues a command to a management appliance that is targeted in the task (e.g., by invoking APIs of the management appliance) to perform the task and to check on the status of the task performed by the management appliance. When the task is completed by the management appliance, mutation agent 240 invokes an API of scheduler service 140 to report the completion of the task.
Discovery agent 250 communicates with the management appliances of SDDC 41 to obtain authentication tokens for accessing the management appliances. In the embodiments, mutation agent 240 acquires the authentication token for accessing the management appliance from discovery agent 250 prior to issuing commands to the management appliance, and includes the authentication token in any commands issued to the management appliance.
When the user who input the mutation command inquires about the status of the task corresponding to the mutation command, through UI 11, mutation service 120 invokes an API of task service 130 to check on the status of the task, and task service 130 returns the status of the task to the user through UI 11.
The steps depicted in
CSP ID service 110 receives the login credentials of the user at step 408. Then, at step 410, CSP ID service 110 accesses an authentication server of cloud platform 12, e.g., Active Directory® server or LDAP server, to authenticate the user, i.e., that the user provided valid login credentials. If the login credentials are not valid (step 412, No), a login error message is returned to the user through UI 11 at step 414. If the login credentials are valid (step 412, Yes), CSP ID service 110 at step 416 accesses the AAA server of the user's organization to determine the role assigned to the user. At step 418, CSP ID service 110 issues an ID token that identifies the user and an access token for accessing resources of the user's organization, in particular management appliances of the SDDCs of the user's organization. The access token also contains information about the role assigned to the user.
Returning to
In response to this create VM API, mutation service 120 carries out a method to exchange the access token with a limited access token, as illustrated in
Returning to
At step S8, MB agent 220 delivers the message to coordinator agent 230. In the example illustrated herein, because the message contains a task to modify the inventory of virtual objects deployed in the SDDC, i.e., to create a VM, coordinator agent 230 at step S9 instantiates mutation agent 240 to manage the performance of the task.
In the embodiments, access tokens issued by cloud platform 12 provide only discovery agent 250 access to the management appliances of SDDC 41. Any of the other agents may not access the management appliances of SDDC 41 using such access tokens. For any of the other agents to access the management appliances of SDDC 41, the agents must first acquire an authentication token, specifically, a bearer token, issued by the management appliances through discovery agent 250, to be able to access the management appliances. Therefore, prior to issuing commands to management appliance 51A, mutation agent 240 requests discovery agent at step S10 to acquire the bearer token. The request includes the ID and limited access tokens included in the message delivered by MB agent 220 at step S8.
At step S11, discovery agent 250 communicates with a token exchange service (TES) 261 of VM management appliance 51A to acquire a bearer token in exchange for the limited access token. TES 261 first confirms that the limited access token permits access to VM management appliance 51A before issuing the bearer token and returning it to discovery agent 250 at step S12. The bearer token has a time-to-live period that is set upon issuance and is no longer valid upon expiration of the time-to-live period. The bearer token also inherits the role defined in the limited access token. If TES 261 cannot confirm that the limited access token permits access to VM management appliance 51A, the request for the bearer token is denied. Upon receipt of the bearer token from TES 261, discovery agent 250 at step S13 passes the bearer token to mutation agent 240.
At step S14, mutation agent 240 communicates with a stats service 262 of VM management appliance 51A to request that a holder-of-key (hok) token be issued in exchange for the bearer token. The token request includes certificate data of mutation agent 240 and a digital signature that is signed with a private key of mutation agent 240. Prior to issuing the holder-of-key token, stats service 262 first validates the certificate of mutation agent 240 and then validates the digital signature using the public key contained in the certificate. If both validations are successful, the holder-of-key token is issued and returned to mutation agent 240 at step S15. If either validation fails, the request for the holder-of-key token is denied.
Both the bearer token and the holder-of-key token are examples of authentication tokens. Unlike the bearer token, the holder-of-key token does not have a time-to-live period associated therewith and so it does not expire. However, it can only be used by the holder of the key, in this example, mutation agent 240. The holder-of-key token also inherits the role defined in the bearer token.
After issuing the command to management appliance 51A to instantiate the VM in SDDC 41, mutation agent 240 periodically accesses VM management appliance 51A, e.g., at steps S18 and S21, to get the status of the command execution. Each time mutation agent 240 accesses VM management appliance 51A to get the status of the command execution, mutation agent 240 transmits the holder-of-key token to authenticates its access to VM management appliance 51A. Step S18 represents a status check prior to completion of the command, so the status returned at step S19 is “pending.” When vpxd 263 completes execution of the command, it reports the completion of the command to stats service 262 at step S20. Step S21 represents a status check after completion of the command, so the status returned at step S22 is “completed.”
Upon being notified that the command has completed at step S22, mutation agent 240 acquires a CSP access token through identity agent 210 at steps S23 and S24. This token allows mutation agent 240 to report the status of the task created by mutation service 120 directly to scheduler service 140 without going through the message exchange between MB agent 220 and MB service 150. At step S25, mutation agent 240 calls an API of scheduler service 140 to report the completion status of the task created by mutation service 120.
Step S26 represents mutation service 120 acquiring the task completion status from scheduler service 140 through task service 130. Step S27 represents a periodic polling of UI 11 to obtain the status of the command input at step S3 of
The embodiments described herein may employ various computer-implemented operations involving data stored in computer systems. For example, these operations may require physical manipulation of physical quantities. Usually, though not necessarily, these quantities may take the form of electrical or magnetic signals, where the quantities or representations of the quantities can be stored, transferred, combined, compared, or otherwise manipulated. Such manipulations are often referred to in terms such as producing, identifying, determining, or comparing. Any operations described herein that form part of one or more embodiments may be useful machine operations.
One or more embodiments of the invention also relate to a device or an apparatus for performing these operations. The apparatus may be specially constructed for required purposes, or the apparatus may be a general-purpose computer selectively activated or configured by a computer program stored in the computer. Various general-purpose machines may be used with computer programs written in accordance with the teachings herein, or it may be more convenient to construct a more specialized apparatus to perform the required operations.
The embodiments described herein may be practiced with other computer system configurations including hand-held devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, etc.
One or more embodiments of the present invention may be implemented as one or more computer programs or as one or more computer program modules embodied in computer readable media. The term computer readable medium refers to any data storage device that can store data which can thereafter be input to a computer system. Computer readable media may be based on any existing or subsequently developed technology that embodies computer programs in a manner that enables a computer to read the programs. Examples of computer readable media are hard drives, NAS systems, read-only memory (ROM), RAM, compact disks (CDs), digital versatile disks (DVDs), magnetic tapes, and other optical and non-optical data storage devices. A computer readable medium can also be distributed over a network-coupled computer system so that the computer readable code is stored and executed in a distributed fashion.
Although one or more embodiments of the present invention have been described in some detail for clarity of understanding, certain changes may be made within the scope of the claims. Accordingly, the described embodiments are to be considered as illustrative and not restrictive, and the scope of the claims is not to be limited to details given herein but may be modified within the scope and equivalents of the claims. In the claims, elements and/or steps do not imply any particular order of operation unless explicitly stated in the claims.
Virtualization systems in accordance with the various embodiments may be implemented as hosted embodiments, non-hosted embodiments, or as embodiments that blur distinctions between the two. Furthermore, various virtualization operations may be wholly or partially implemented in hardware. For example, a hardware implementation may employ a look-up table for modification of storage access requests to secure non-disk data.
Many variations, additions, and improvements are possible, regardless of the degree of virtualization. The virtualization software can therefore include components of a host, console, or guest OS that perform virtualization functions.
Plural instances may be provided for components, operations, or structures described herein as a single instance. Boundaries between components, operations, and data stores are somewhat arbitrary, and particular operations are illustrated in the context of specific illustrative configurations. Other allocations of functionality are envisioned and may fall within the scope of the invention. In general, structures and functionalities presented as separate components in exemplary configurations may be implemented as a combined structure or component. Similarly, structures and functionalities presented as a single component may be implemented as separate components. These and other variations, additions, and improvements may fall within the scope of the appended claims.
