Due to the maturity, robustness, flexibility and simplicity of cloud computing architecture, the cloud is now ubiquitous and customers may obtain infrastructure services through various public cloud vendors. For a variety of reasons, however, such as security, cost, latency, and the like, customers may be reluctant to put every workload in the public cloud. As such, there remains a need for on-premises hosted cloud (referred to as “private cloud”) as well as a cloud computing environment that uses a mix of private cloud and public cloud services (referred to as “hybrid cloud”) for scenarios in which customers would like to make use of both.
Embodiments described here are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like reference numerals refer to similar elements.
Embodiments described herein are generally directed to a creation of a High-Availability (HA) private cloud gateway based on a two-node Hyperconverged Infrastructure (HCI) cluster with a self-hosted Hypervisor Management System (HMS). In the following description, numerous specific details are set forth in order to provide a thorough understanding of example embodiments. It will be apparent, however, to one skilled in the art that embodiments described herein may be practiced without some of these specific details.
Cloud customers currently have the ability to manage the private cloud like a service through web-based Software-as-a-Service (SaaS) portals. A typical architecture for performing SaaS-based private cloud management includes three core components: the SaaS portal, a private cloud gateway and a managed private cloud (e.g., in the form of an Infrastructure-as-a-Service (IaaS) cloud). SaaS-based private cloud management relies on the existence of the private cloud gateway that acts as an intermediary for communications between the SaaS portal and the private cloud, for example, for cloud operations, such as provisioning, configuration, monitoring, and management. A variety of approaches exist to facilitate creation of a private cloud gateway on behalf of cloud customers; however, none are entirely satisfactory as they all suffer from one or more of the following limitations:
A representative example of existing approaches for creating a private cloud gateway involves taking inventory of the hardware available within an on-premises hosted cloud data center. Then, an administrator creates the private cloud gateway using the available hardware with a minimalistic hardware footprint, but with desired high-availability. The private cloud gateway is then configured by the administrator to run a set of infrastructure agents to manage the private cloud, which helps in private cloud management as well as addresses end users' desire to create an IaaS cloud to which user workloads can be deployed. As it is desirable for the IaaS to be capable of being managed 24×7, 365 days in year without interruption, the private cloud gateway should support high availability of the HMS as well as the various services deployed for private cloud management that leverage native HMS HA solutions provided therein.
Most existing approaches for creation of a private cloud gateway use independent infrastructure to host the HMS in HA mode, thereby resulting in a sub-optimal hardware footprint (e.g., a solution involving three-nodes or more) as noted above. These sub-optimal HA private cloud gateway architectures require at least three nodes, one running a Hypervisor Management System (HMS) and a two-node HCI cluster. If the HA private cloud gateway could be reduced to two nodes, enormous cost and space efficiencies are achievable across the relevant set of current and future cloud customers. It is theoretically possible to have the HMS run within the HCI cluster, which would allow elimination of the separate infrastructure to host the HMS; however, existence of the HMS is a precondition to creation of the HCI cluster. This situation raises the age-old conundrum of the “chicken or egg” paradox. As such, one challenge to be overcome in connection with reducing the hardware footprint of an HA private cloud gateway is identifying how to migrate the HMS, which is running on a separate host (or cluster), to the HCI cluster that it is going to manage. Additionally, in order to make the solution attractive to cloud consumers, the entire process (e.g., creation of the HA private cloud gateway, deployment of the HCI cluster, and configuration of the HMS in HA mode) should be as automated as possible.
Embodiments described herein seek to achieve the goal of programmatically creating an HA private cloud gateway that may be limited to a two-node HCI cluster by, among other things, making use of resources external to the private cloud and external to the private cloud gateway to bootstrap the hardware that will make up the private cloud gateway. As described further below, with reference to
According to one embodiment, a centralized SaaS portal facilitates (i) creation of a private cloud gateway based on HCI servers that are automatically and remotely discovered by the SaaS portal with zero or minimal human intervention by leveraging a geographically distributed network of base stations and a temporary HSM running on a deployment node to seed a two-server HCI cluster; and (ii) replacement of the temporary HMS by deploying a new, self-hosted HMS within the same two-server HCI cluster in HA mode, which the self-hosted HMS is to manage using re-registration of the HCI cluster to the self-hosted HMS.
Terminology
The terms “connected” or “coupled” and related terms are used in an operational sense and are not necessarily limited to a direct connection or coupling. Thus, for example, two devices may be coupled directly, or via one or more intermediary media or devices. As another example, devices may be coupled in such a way that information can be passed there between, while not sharing any physical connection with one another. Based on the disclosure provided herein, one of ordinary skill in the art will appreciate a variety of ways in which connection or coupling exists in accordance with the aforementioned definition.
If the specification states a component or feature “may”, “can”, “could”, or “might” be included or have a characteristic, that particular component or feature is not required to be included or have the characteristic.
As used in the description herein and throughout the claims that follow, the meaning of “a,” “an,” and “the” includes plural reference unless the context clearly dictates otherwise. Also, as used in the description herein, the meaning of “in” includes “in” and “on” unless the context clearly dictates otherwise.
Reference in the specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment.
As used herein, a “Hypervisor Management System” or an “HMS” generally refers to a set of nodes running virtual software forming a cluster. Non-limiting examples of an HMS include a HyperV cluster, a VMware cluster, a Microsoft System Center cluster and a Nutanix cluster.
As used herein, a “self-hosted Hypervisor Management System” or a “self-hosted HMS” generally refers to an HMS that runs on a node of a cluster that is comprised of multiple nodes that is managed by the HMS.
As used herein, a “Hyperconverged infrastructure node” or an “HCI node” generally refers to a software-defined information technology infrastructure node that virtualizes various elements of conventional hardware-defined systems. An HCI node may include a combination of one or more of virtualized computing (e.g., a hypervisor), software-defined storage, and virtualized networking (software-defined networking). HCI solutions may run on commercial off-the-shelf servers. Some HCI solutions may also include a full set of software-defined services for compute, backup storage, networking, security and cloud management.
As used herein, a “base station” generally refers to a node of a distributed set of nodes that is used to bootstrap and prepare physical infrastructure for pre-day zero activities. As described further below, a base station may be managed by and/or directed by a centralized SaaS portal to perform various tasks on behalf of the SaaS portal in connection with creation of a private cloud gateway for a private cloud. A base station may be used to provide a secure launching pad from which to, among other things, install an operating system on an on-premises host, connect to the host via the secure shell (SSH) protocol, and run various scripts.
As used herein, a “private cloud gateway” generally refers to one or more servers that provides a set of services for managing various aspects of a private cloud, for example, under the direction of a SaaS portal. In various embodiments described herein, an HA private cloud gateway comprises a two-node HCI cluster.
In one embodiment, base stations 111 may be deployed in different geographical regions, for example, to reduce communication latency with respective private clouds 112a-d (which may be referred to herein collectively or generally as 112). Those skilled in the art will appreciate the distribution illustrated herein is merely used as an example and that numerous other geographical distributions of base stations 111 may be employed based on various factors (e.g., customer demand in a particular region, network communication latencies in a particular region, concentration of private clouds in different geographies, etc.). For example, base station density in the US and Europe would likely be greater than that in Antarctica and/or other virtually uninhabited regions. In one embodiment, the base stations may be virtual machines (VMs) running in different availability zones of a public cloud. Alternatively, the base stations may be located in a private data center and exposed via a public address point.
After a private cloud gateway is established within a private cloud 112, SaaS portal 110 may provide remote accessibility to the private clouds 112 via the private cloud gateway. The SaaS portal 110 may represent a web-based portal through which cloud customers have the ability to manage on-premises infrastructure, for example, networking equipment, load balancers, storage arrays, backup devices, and servers within an on-premises hosted cloud data center. For example, SaaS portal 110 may provide a set of services which make use of a customer's private cloud gateway to perform various cloud management operations (e.g. discovery of nodes and deployment, updating, and upgrading of the customer's private cloud 112).
In embodiments described herein, the SaaS portal 110 manages and directs a base station 111 (e.g., one determined to be in geographic proximity to the private cloud 112 at issue) in connection with carrying out various automated tasks associated with setting up an HA private cloud gateway (e.g., creation of the HA private cloud gateway, deployment of an HCI cluster, and configuration of an HMS in HA mode within the HCI cluster) within the private cloud 112. For example, for an enterprise located in California seeking to establish SaaS-based private cloud management of private cloud 112a, SaaS portal 110 may select base station 111a to perform tasks associated with setting up a private cloud gateway (not shown) within private cloud 112a on behalf of the enterprise. Similarly, for a customer located in India desiring SaaS-based private cloud management of private cloud 112c, SaaS portal 110 may select base station 111c to perform tasks associated with setting up a private cloud gateway (not shown) within private cloud 112c on behalf of the customer.
While the SaaS portal 110 may be used to perform various on-premises cloud operations, including provisioning, configuration, monitoring, and management, embodiments described herein relate primarily to an intelligent mechanism for deployment of private cloud gateways within private clouds 112 from the SaaS portal 110 that, among other things, minimizes human intervention during the deployment process.
While in the above-described example, a set of four geographically distributed base stations 111 are shown, those skilled in the art will appreciate more or fewer base stations may be used. Additionally, while deploying base stations 111 in different geographical regions reduces latency for communications with proximate private clouds, the systems and methods herein may be use without such geographical distribution of base stations, albeit, with the associated trade-off of increased time for communications between the base stations and the respective private clouds. Those skilled in the art will also appreciate more or fewer private clouds 112 may be serviced via the SaaS portal 110 than shown in the present example.
SaaS portal 210 may be a web-based SaaS portal as described above with reference to
According to one embodiment, as part of the registration and discovery workflow, the private cloud registry manager 214 may be responsible for establishing a connection with the base station 211 and directing the base station to configure a private network environment using the enterprise's on-premises infrastructure. As noted above, the base station 211 may be specific to a geographical region in which the private cloud environment 250 resides. Responsive to the request from the private cloud registry manager 214, the base station 211 may dynamically establish the private network environment and use it to discover servers in the connect to physical infrastructure of the private cloud. In this manner, the use of base stations 111 enhances security as the internal Internet Protocol (IP) addresses of the private cloud environment need not be publicly exposed. Further details regarding an example of private cloud gateway creation processing are described below with reference to
Responsive to a request from the operation administrator 201, the private cloud gateway client 218 may be responsible for performing a private cloud creation workflow, including deploying and configuring infrastructure for the private cloud gateway 240. According to one embodiment, the private cloud gateway client 218 may direct the base station 211 to run various scripts. For example, a first script may be run by the base station 211 to create a deployment node 241 hosting an HMS 245 (e.g., a seed or temporary HMS for creation of cluster 246) and an arbiter VM 243 (e.g., for installing management software associated with HCI nodes 248a-b), a second script may be run by the base station 211 to create the cluster 246 comprising the two HCI nodes 248a-b, and a third script may be run by the base station 211 to replace the temporary HMS with a new, self-hosted HMS (not shown) within the cluster 246 by re-registering the cluster 246 to the self-hosted HMS. During the private cloud creation workflow, the base station 211 may pull (e.g., download), and locally store within local storage associated with the base station 211 (not shown), appropriate artifacts (e.g., operating system ISO image files, an HMS image, installer package files, etc.) as needed from artifact repository 215. According to one embodiment a URI may be provided by the private cloud gateway client 218 that can be used by the operation administrator 201 to poll for successful completion of the assigned operation. Further details regarding an example of private cloud gateway creation processing are described below with reference to
After the private cloud gateway creation workflow has been completed, the private cloud gateway 240 may participate in creating an IaaS stack within the managed private cloud 251 and the private cloud gateway 240 may be used to manage the physical infrastructure associated with the managed private cloud 251, for example, using tools, such as Ops Ramp, ServiceNow, etc.
According to one embodiment, the newly-created managed private cloud 251 may run various services that provide virtual infrastructure as a service using on-premises infrastructure, for example, physical hardware infrastructure within an on-premises hosted cloud data center of the enterprise. The services may be provided in the form of one or more turnkey appliances. Non-limiting examples of the services include OpenStack and Azure Stack.
While not directly related to the workflows described above, for purposes of completeness, a private cloud client 220 is shown through which users (e.g., users 202) of the managed private cloud 251 may make use of various compute, networking, and/or storage services (e.g., OpenStack or Azure Stack) that may be provided by the managed private cloud 251 based on the type of the private cloud created.
The SaaS portal 210 and the various services (e.g., the private cloud registry manager 214 and the private cloud gateway client 218) provided by the SaaS portal 210 described above with reference to
While in the context of various examples described herein, reference is made to various scripts, those skilled in the art will appreciate many of the operations performed by the scripts may alternatively be orchestrated through a well-defined Application Programming Interface (API) (e.g., a Representational State Transfer (REST) API). As such, depending upon the particular implementation, one or more of the various scripts could be deployed as services.
At block 310, a request is received to register a private cloud. According to one embodiment, an administrator (e.g., operation administrator 201) of an enterprise that desires to enable SaaS-based private cloud management via the SaaS portal interacts with a user interface presented by the SaaS portal to provide various details to facilitate creation of private cloud gateway 240. According to one embodiment, the administrator may provide information (e.g., a Uniform Resource Locator (URL)) associated with an artifact repository (e.g., artifact repository 215) from which to pull various artifacts (e.g., operating system ISO image files, an HMS image, installer package files, etc.) for use during process of creating the private cloud gateway. The administrator may also input information for use in connection with identifying an appropriate base station (e.g., base station 111 or 211) that will be used during the process of creating the private cloud gateway. For example, the administrator may specify a particular base station by providing its IP address or the administrator may provide information regarding a geographical region in which the private cloud is located. Alternatively, an appropriate base station may be programmatically identified based on an IP address associated with the private cloud environment. The administrator may also input information (e.g., a private cloud name, an IP address of a server participating in the gateway, types of nodes and number of nodes) regarding the private cloud (e.g., private cloud 111 or private cloud environment 250) in which the managed private cloud (e.g., managed private cloud 251) is to be created.
At block 320, a base station is instructed to discover on-premises servers. According to one embodiment, a private network environment is dynamically created by the base station to facilitate a cloud connection using the on-premises physical infrastructure. For example, the base station may create a Virtual Private Cloud (VPC) within the public cloud in which the base station resides using the enterprise's on-premises infrastructure by running one or more scripts. Upon establishment of the private network environment, the base station may join the secure private network, use it to connect to the physical infrastructure of the private cloud, and discover the available servers in the private cloud environment. Upon completion of the discovery process, the base station may relay information regarding the discovered servers back to the private cloud registry manager 214, which may persist the information in the private cloud configuration database 225. For example, the base station may provide SaaS portal with the cluster name, cluster IP address and it elements (e.g., a set of nodes). Information regarding the discovered servers may also include server types and models, data about the hardware components and configuration of the servers (e.g., number of drives, size of the drives, Network Interface Controller (NIC) adapters, etc.). This information may also be used in connection with identifying the right servers to pair together in a cluster relationship in the event there are multiple servers available.
At block 330, the base station is instructed to prepare a deployment node for creation of an HA private cloud gateway in the form of a cluster of two HCI nodes. According to one embodiment, the SaaS portal directs the base station to run a script to create the deployment node (e.g., deployment node 241) on a bare metal server discovered during the discovery process. Creation of the deployment node may involve deploying and configuring a hypervisor node on a discovered server to host a seed or temporary HMS (e.g., HMS 245) and a VM that will host an arbiter and potentially other management tools for the HCI nodes (e.g., HCI nodes 284a-b).
Alternatively, as noted above, all or some portion of the deployment node preparation process may be provided in the form of a server deployment and configuration service. For example, a program can be invoked via a REST API to perform pre-defined configuration operations (e.g., update the server firmware, Install an operating system (hypervisor), deploy and configure the HMS, etc.).
At block 340, the base station is instructed to cause the seed HMS running on the deployment node to create the cluster. According to one embodiment, the SaaS portal directs the base station to run a script to make use of the seed HMS to create the cluster (e.g., cluster 246), which may be strictly limited to two HCI nodes (e.g., HCI nodes 284a-b).
At block 350, the base station is instructed to cause the deployment node to install a new HMS within the cluster. According to one embodiment, the new HMS will replace the seed HMS by un-registering the cluster from the seed HMS and re-registering the cluster with the new HMS. One or more scripts may be run by the base station to effectuate the installation of the new HMS and the re-registration, including one or more reconfiguration scripts to cause the cluster to properly reconnect to the new HMS. After successful transition of management of the cluster from the seed HMS to the new HMS, the seed HMS may be discarded. Further details regarding an example of a self-hosted HMS bootstrap process on a two-node HCI system are provided with reference to
In
In
In
In
In
In
Embodiments described herein include various steps, examples of which have been described above. As described further below, these steps may be performed by hardware components or may be embodied in machine-executable instructions, which may be used to cause a general-purpose or special-purpose processor programmed with the instructions to perform the steps. Alternatively, at least some steps may be performed by a combination of hardware, software, and/or firmware.
Embodiments described herein may be provided as a computer program product, which may include a machine-readable storage medium tangibly embodying thereon instructions, which may be used to program a computer (or other electronic devices) to perform a process. The machine-readable medium may include, but is not limited to, fixed (hard) drives, magnetic tape, floppy diskettes, optical disks, compact disc read-only memories (CD-ROMs), and magneto-optical disks, semiconductor memories, such as ROMs, PROMs, random access memories (RAMs), programmable read-only memories (PROMs), erasable PROMs (EPROMs), electrically erasable PROMs (EEPROMs), flash memory, magnetic or optical cards, or other type of media/machine-readable medium suitable for storing electronic instructions (e.g., computer programming code, such as software or firmware).
Various methods described herein may be practiced by combining one or more machine-readable storage media containing the code according to example embodiments described herein with appropriate standard computer hardware to execute the code contained therein. An apparatus for practicing various example embodiments described herein may involve one or more computing elements or computers (or one or more processors within a single computer) and storage systems containing or having network access to computer program(s) coded in accordance with various methods described herein, and the method steps of various example embodiments described herein may be accomplished by modules, routines, subroutines, or subparts of a computer program product.
The machine readable medium 520 may be any medium suitable for storing executable instructions. Non-limiting examples of machine readable medium 520 include RAM, ROM, EEPROM, flash memory, a hard disk drive, an optical disc, or the like. The machine readable medium 520 may be disposed within the computer system 500, as shown in
In the context of the present example, the machine readable medium 520 is encoded with a set of executable instructions 530-570. It should be understood that part or all of the executable instructions and/or electronic circuits included within one block may, in alternate implementations, be included in a different block shown in the figures or in a different block not shown.
Instructions 530, upon execution, cause the processing resource 510 to receive a request to register a private cloud. In one embodiment, instructions 530 may correspond generally to instructions for performing block 310 of
Instructions 540, upon execution, cause the processing resource 510 to instruct a base station to discover on-premises servers. In one embodiment, instructions 540 may correspond generally to instructions for performing block 320 of
Instructions 550, upon execution, cause the processing resource 510 to instruct the base station to prepare a deployment node. In one embodiment, instructions 550 may correspond generally to instructions for performing the block 330 of
Instructions 560, upon execution, cause the processing resource 510 to instruct the base station to cause a seed/temporary HMS to create a cluster. In one embodiment, instructions 560 may correspond generally to instructions for performing block 340 of
Instructions 570, upon execution, cause the processing resource 510 to instruct the base station to cause the deployment node to install a new, self-hosted HMS within the cluster. In one embodiment, instructions 560 may correspond generally to instructions for performing block 350 of
In the foregoing description, numerous details are set forth to provide an understanding of the subject matter disclosed herein. However, implementation may be practiced without some or all of these details. Other implementations may include modifications and variations from the details discussed above. It is intended that the following claims cover such modifications and variations.
Number | Name | Date | Kind |
---|---|---|---|
9612926 | Cao | Apr 2017 | B2 |
9882969 | Reddy et al. | Jan 2018 | B2 |
10067780 | Chang et al. | Sep 2018 | B2 |
10097620 | Reddy et al. | Oct 2018 | B2 |
10148493 | Ennis, Jr. et al. | Dec 2018 | B1 |
10389586 | Hockett et al. | Aug 2019 | B2 |
10447538 | Maes | Oct 2019 | B2 |
10498837 | Bondalapati et al. | Dec 2019 | B1 |
10534629 | St. Pierre et al. | Jan 2020 | B1 |
10686755 | Nirwal et al. | Jun 2020 | B2 |
10848379 | Sharma et al. | Nov 2020 | B2 |
10860362 | Lal | Dec 2020 | B2 |
10909009 | Ali et al. | Feb 2021 | B2 |
20150331763 | Cao | Nov 2015 | A1 |
20170279692 | Llagostera et al. | Sep 2017 | A1 |
20170353531 | Conn | Dec 2017 | A1 |
20180077007 | Olson | Mar 2018 | A1 |
20180115468 | Bildhauer et al. | Apr 2018 | A1 |
20180287864 | Hockett et al. | Oct 2018 | A1 |
20190253311 | Hockett et al. | Aug 2019 | A1 |
20200004570 | Glade | Jan 2020 | A1 |
20200007408 | Siddappa | Jan 2020 | A1 |
20200218561 | Lal et al. | Jul 2020 | A1 |
20200344325 | Sarisky | Oct 2020 | A1 |
20200396127 | Lochhead et al. | Dec 2020 | A1 |
20200396179 | Lochhead et al. | Dec 2020 | A1 |
20210036889 | Jain et al. | Feb 2021 | A1 |
20210173695 | Dai | Jun 2021 | A1 |
20210216234 | Singler, Jr. | Jul 2021 | A1 |
20210255885 | Purohit | Aug 2021 | A1 |
20210334178 | Yang | Oct 2021 | A1 |
Number | Date | Country |
---|---|---|
WO-2014184800 | Nov 2014 | WO |
WO-2016018680 | Feb 2016 | WO |
WO-2019094522 | May 2019 | WO |
Entry |
---|
https://www.nakivo.com/blog/hyper-v-high-availability-works/ (Year: 2019). |
Kristopher Jon Turner, “Building a Microsoft Hyper-Converged Private Cloud Solution,” Apr. 7, 2017, pp. 1-2, Retrieved from the Internet on Feb. 10, 2020 at URL: <kristopherjturner.com/2017/04/07/building-a-microsoft-hyper-converged-private-cloud-solution/>. |
Microsoft, “Hyper-V Network Virtualization Gateway Architectural Guide.” Aug. 31, 2016, pp. 1-14, Retrieved from the Internet on Feb. 10, 2020 at URL: <docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/jj618319(v%3Dws.11)>. |
Number | Date | Country | |
---|---|---|---|
20210392041 A1 | Dec 2021 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 16902423 | Jun 2020 | US |
Child | 17304073 | US |