Patent Grant
11671489
Cloud computing allows dynamically scalable virtualized resources to host applications and services. Cloud computing enables appropriate levels of resources to power software applications when and where the resources are needed in response to demand. As a result, cloud computing allows entities to respond quickly, efficiently, and in an automated fashion to rapidly changing business environments.
Paramount to any computing platform is availability. Availability is typically measured as a percentage of time during which a given computing system is operational. While optimal availability is 100%, this is often not achievable. The current standard of “high availability” is often referred to as “five 9s” or 99.999% availability. Over a period of 1 year, a computing system operating at five 9s availability will experience only 5 minutes and 36 seconds of downtime.
Five 9s availability has long been the goal of system administrators, whether the target system is a telecom or computing system. With the recent trend of cloud computing being used as a replacement for hardware-based solutions, the concept of five 9s availability has taken a backseat to the ease of simply instantiating new instances of an application or service to counteract lower-than-average availability. However, as cloud computing becomes more ubiquitous, cloud providers will endeavor once again to reach five 9s availability.
Next generation applications, such as Internet of Things (“IoT”), connected cars, remote surgery, augmented reality, virtual reality, video streaming, 5G voice and data applications, and others, require real-time sensitive applications running on cloud resources to provide such services. These applications, in addition to stringent real-time performance requirements (e.g., latency, jitter, etc.), demand five 9s availability to provide such services.
In addition to availability and real-time performance requirements, a primary indicator of system performance is utilization. Utilization generally refers to the energy efficiency of a system. Many existing cloud computing models operate with low utilization. In other words, cloud computing resources are often left idle consuming electricity but not performing any tasks. As more and more applications and services are moved to the cloud, utilization will need to increase while maintaining high availability and real-time performance requirements.
The concepts and technologies disclosed herein provide a high availability and high utilization cloud data center architecture for supporting telecommunications services. The high availability and high utilization cloud data center architecture can include a plurality of sites (also referred to as geo-sites or geo-regions) representing specific geographical locations in which computing resources are located. Each site can include one or more availability zones (“AZs”), each of which represents an isolated physical location within a site. Site resources of a given site are available to any AZ within that site. Zonal resources are available only in the associated AZ. Machines in different AZs have no single point of failure. Availability regions (“AR”) (also referred to as cloud instances) are isolated instances of a cloud controller and associated cloud resources within an AZ. AR resources are available only in the corresponding AR. In some embodiments, AZ resources also can be shared across multiple ARs within the corresponding AZ.
An instance of a machine, a virtual machine (“VM”), an application, a container POD, a container instance, or a container cluster can be instantiated in any site, AZ, or AR. A collection of VMs together can provide a service such as, for example, connected car, 5G voice and data service, and others. A local redundancy model for a service can spread VMs locally in a site across AZs and ARs to manage AZ and/or AR failures. Another level of redundancy, referred to herein as geo-redundancy, can spread a service across sites to manage site failures. In spreading VMs across AZs in different ARs, the real-time performance requirements (e.g., latency, jitter, etc.) of these services still need to be met. In general, an AR provides for resiliency within an AZ and enables high availability and higher cloud resource utilization while providing capabilities to meet the stringent real-time requirements of the services.
According to one aspect of the concepts and technologies disclosed herein, a 4-site local and geo-redundancy model for application placement within a cloud computing environment provides 37.5% cloud resource utilization with site availability of five 9s (99.999%) and virtual machine availability of five 9s. In particular, a plurality of sites operating as part of the cloud computing environment can include, for example, a first site, a second site, a third site, and a fourth site. The first site can include a first availability zone (“AZ”) that, in turn, includes a first availability region (“AR”) and a second AR. The first AR can include a first server and the second AR includes a second server. The first server can include a first virtual machine, and the second server can include a second virtual machine. The second site can include a second AZ that, in turn, includes a first duplicate of the first AR and a first duplicate of the second AR. The first duplicate of the first AR can include a first duplicate of the first server and the first duplicate of the second AR can include a first duplicate of the second server. The first duplicate of the first server can include a first duplicate of the first virtual machine. The first duplicate of the second server can include a first duplicate of the second virtual machine. The third site can include a third AZ that, in turn, includes a second duplicate of the first AR and a second duplicate of the second AR. The second duplicate of the first AR can include a second duplicate of the first server and the second duplicate of the second AR can include a second duplicate of the second server. The second duplicate of the first server can include a second duplicate of the first virtual machine. The second duplicate of the second server can include a second duplicate of the second virtual machine. The fourth site can include a fourth AZ that, in turn, includes a third duplicate of the first AR and a third duplicate of the second AR. The third duplicate of the first AR can include a third duplicate of the first server and the third duplicate of the second AR can include a third duplicate of the second server. The third duplicate of the first server can include a third duplicate of the first virtual machine. The third duplicate of the second server can include a third duplicate of the second virtual machine. The VMs across ARs and within an AZ can be connected via a layer 3 or layer 2 network. The plurality of sites also can be connected via a layer 3 or layer 2 network. The first server and the second server can be connected via a first layer 2 connection within the first AZ. The first duplicate of the first server and the first duplicate of the second server can be connected via a second layer 2 connection within the second AZ. The second duplicate of the first server and the second duplicate of the second server can be connected via a third layer 2 connection within the third AZ. The third duplicate of the first server and the third duplicate of the second server can be connected via a fourth layer 2 connection within the fourth AZ.
According to another aspect of the concepts and technologies disclosed herein, a 3-site local and geo-redundancy model for application placement within the cloud computing environment provides 66% cloud resource utilization with site availability of five 9s and virtual machine availability of five 9s. In particular, a plurality of sites operating as part of the cloud computing environment can include, for example, a first site, a second site, and a third site. The first site can include a first AZ that, in turn, includes a first AR, a second AR, and a third AR. The first AR can include a first server, the second AR can include a second server, and the third AR can include a third server. The first server can include a first virtual machine, the second server can include a second virtual machine, and the third server can include a third virtual machine. The second site can include a second AZ that, in turn, includes a first duplicate of the first AR, a first duplicate of the second AR, and a first duplicate of the third AR. The first duplicate of the first AR can include a first duplicate of the first server, the first duplicate of the second AR can include a first duplicate of the second server, and the first duplicate of the third AR can include a first duplicate of the third server. The first duplicate of the first server can include a first duplicate of the first virtual machine, the first duplicate of the second server can include a first duplicate of the second virtual machine, and the first duplicate of the third server can include a first duplicate of the third virtual machine. The third site can include a third AZ that, in turn, includes a second duplicate of the first AR, a second duplicate of the second AR, and a second duplicate of the third AR. The second duplicate of the first AR can include a second duplicate of the first server, the second duplicate of the second AR can include a second duplicate of the second server, and the second duplicate of the third AR can include a second duplicate of the third server. The second duplicate of the first server can include a second duplicate of the first virtual machine. The second duplicate of the second server can include a second duplicate of the second virtual machine, and the second duplicate of the third server can include a second duplicate of the third virtual machine. The VMs across ARs and within an AZ can be connected via a layer 3 or layer 2 network. The plurality of sites also can be connected via a layer 3 or layer 2 network. The first server, the second server, and the third server can be connected via a first layer 2 connection within the first AZ. The first duplicate of the first server, the first duplicate of the second server, and the first duplicate of the third server can be connected via a second layer 2 connection within the second AZ. The second duplicate of the first server, the second duplicate of the second server, and the second duplicate of the third server can be connected via a third layer 2 connection within the third AZ.
According to another aspect of the concepts and technologies disclosed herein, a 4-site local and geo-redundancy model for application placement within the cloud computing environment provides 75% cloud resource utilization with site availability of five 9s and virtual machine availability of five 9s. In particular, a plurality of sites operating as part of a cloud computing environment can include, for example, a first site, a second site, a third site, and a fourth site. The first site can include a first AZ that, in turn, includes a first AR, a second AR, a third AR, and a fourth AR. The first AR can include a first server, the second AR can include a second server, the third AR can include a third server, and the fourth AR can include a fourth server. The first server can include a first virtual machine, the second server can include a second virtual machine, the third server can include a third virtual machine, and the fourth server can include a fourth virtual machine. The second site can include a second AZ that, in turn, includes a first duplicate of the first AR, a first duplicate of the second AR, a first duplicate of the third AR, and a first duplicate of the fourth AR. The first duplicate of the first AR includes a first duplicate of the first server, the first duplicate of the second AR includes a first duplicate of the second server, the first duplicate of the third AR includes a first duplicate of the third server, and the first duplicate of the fourth AR includes a first duplicate of the fourth server. The first duplicate of the first server can include a first duplicate of the first virtual machine, the first duplicate of the second server can include a first duplicate of the second virtual machine, the first duplicate of the third server can include a first duplicate of the third virtual machine, and the first duplicate of the fourth server can include a first duplicate of the fourth virtual machine. The third site can include a third AZ that, in turn, includes a second duplicate of the first AR, a second duplicate of the second AR, a second duplicate of the third AR, and a second duplicate of the fourth AR. The second duplicate of the first AR can include a second duplicate of the first server, the second duplicate of the second AR can include a second duplicate of the second server, the second duplicate of the third AR can include a second duplicate of the third server, and the second duplicate of the fourth AR can include a second duplicate of the fourth server. The second duplicate of the first server can include a second duplicate of the first virtual machine, the second duplicate of the second server can include a second duplicate of the second virtual machine, the second duplicate of the third server can include a second duplicate of the third virtual machine, and the second duplicate of the fourth server can include a second duplicate of the fourth virtual machine. The fourth site can include a fourth AZ that, in turn, includes a third duplicate of the first AR, a third duplicate of the second AR, a third duplicate of the third AR, and a third duplicate of the fourth AR. The third duplicate of the first AR can include a third duplicate of the first server, the third duplicate of the second AR can include a third duplicate of the second server, the third duplicate of the third AR can include a third duplicate of the third server, and the third duplicate of the fourth AR can include a third duplicate of the fourth server. The third duplicate of the first server can include a third duplicate of the first virtual machine, the third duplicate of the second server can include a third duplicate of the second virtual machine, the third duplicate of the third server can include a third duplicate of the third virtual machine, and the third duplicate of the fourth server can include a third duplicate of the fourth virtual machine. The VMs across ARs within an AZ can be connected via a layer 3 or layer 2 network. The plurality of sites also can be connected via a layer 3 or layer 2 network. The first server, the second server, the third server, and the fourth server can be connected via a first layer 2 connection within the first AZ. The first duplicate of the first server, the first duplicate of the second server, the first duplicate of the third server, and the first duplicate of the fourth server can be connected via a second layer 2 connection within the second AZ. The second duplicate of the first server, the second duplicate of the second server, the second duplicate of the third server, and the second duplicate of the fourth server can be connected via a third layer 2 connection within the third AZ. The third duplicate of the first server, the third duplicate of the second server, the third duplicate of the third server, and the third duplicate of the fourth server can be connected via a fourth layer 2 connection within the fourth AZ.
In some embodiments, the cloud computing environment can detect an event within one of the plurality of sites. The event can include a planned event or an unplanned event. A planned event can include an upgrade to at least a portion of one of the plurality of sites. An unplanned event can include a failure of a least a portion of one of the plurality of sites. In response to the event, the cloud computing environment can redirect traffic from a first portion of the plurality of sites to a second portion of the plurality of sites.
In some embodiments, each of the virtual machines can provide, at least in part, a real-time service. Each of the virtual machines can be an instance of a virtual network function (“VNF”) that provides traditional or evolving mobility networking functions, such as access network elements, core network elements, transport network elements, and others, from purpose-built hardware to commercial-off-the-shelf (“COTS”) server-based platforms, such as those operating within the aforementioned servers. The real-time service can include, particularly, a voice service that can benefit greatly from the high availability and high utilization characteristics provided by the aforementioned models.
It should be appreciated that the above-described subject matter may be implemented as a computer-controlled apparatus, a computer process, a computing system, or as an article of manufacture such as a computer-readable storage medium. These and various other features will be apparent from a reading of the following Detailed Description and a review of the associated drawings.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended that this Summary be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all disadvantages noted in any part of this disclosure.
The practice standard for deployment of information technology (“IT”) applications in a cloud computing environment is to use availability zones to achieve resiliency. Many cloud service providers, such as AWS by Amazon Web Services, Inc., rely on availability zones for application deployment. These service providers can cope with increases in latency and jitter common with such practices. Telecommunications service providers may use cloud computing environments to deploy virtual network functions (“VNFs”) that provide various network functionality in support of real-time services, such as voice and data services. Most telecommunications VNFs cannot exploit availability zones for resiliency due to high latency and high jitter. Currently, the use of availability zones in a cloud computing environment results in very low cloud resource utilization (e.g., on the order of peak utilization of around 25% and average utilization of around 13%) to achieve high availability (i.e., five 9s of availability). This creates issues for telecommunications service providers, since deployment and maintenance of a cloud computing environment with low utilization and high availability significantly increases capital expenditures (“capex”) and operational expenditures (“opex”) to meet high availability requirements.
While the subject matter described herein may be presented, at times, in the general context of program modules that execute in conjunction with the execution of an operating system and application programs on a computer system, those skilled in the art will recognize that other implementations may be performed in combination with other types of program modules. Generally, program modules include routines, programs, components, data structures, computer-executable instructions, and/or other types of structures that perform particular tasks or implement particular abstract data types. Moreover, those skilled in the art will appreciate that the subject matter described herein may be practiced with other computer systems, including hand-held devices, mobile devices, wireless devices, multiprocessor systems, distributed computing systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, routers, switches, other computing devices described herein, and the like.
Referring now to
The site 102 can be identified, for example, by the city, such as San Diego, Houston, or New York, in which hardware resources operating in one or more data centers of the cloud computing environment 100 are located. The site 102 is not intended to encompass the entirety of a named location (e.g., San Diego), but instead a general area in which the hardware resources are located. Alternatively, the site 102 can identify general areas, such as, for example, North-United States, South-United States, East-United States, or West-United States. Although only a single site 102 is illustrated, the cloud computing environment 100 can include any number of sites 102. An example multi-site configuration of the cloud computing environment 100 is shown in
A given site 102 can include any number of AZs 104, each of which represents an isolated physical location within the site 102, and each of which can be defined by a provider edge/gateway (“PE/GW”) 106 that designates a service demarcation for connectivity between resources in an AZ 104 and a backbone network (i.e., layer 3 “L3” network) 108. The illustrated site 102 includes a first AZ (“AZ1”) 104A and a second AZ (“AZ2”) 104B defined by a first PE/GW (“PE/GW1”) 106A and a second PE/GW (“PE/GW2”) 106B, respectively. A site 102 identified as “San Diego” might include multiple AZs, such as “San Diego 1,” “San Diego 2,” and so on.
In accordance with the concepts and technologies disclosed herein, and different from current availability zone distribution models used to achieve resiliency in real-time applications deployed in a cloud, a given AZ 104 can include any number of ARs 110 (alternatively referred to as “cloud instances”), each having a local control plane (“LCP”) that includes a server (not shown in
In the illustrated embodiment, the AZ1 104A includes a first AR (“AR1”) 110A that, in turn, includes a first CC (“CC1”) 112A that manages a first pool of tenant servers (“tenant1”) 114A; a second AR (“AR2”) 110B that, in turn, includes a second CC (“CC2”) 112B that manages a second pool of tenant servers (“tenant2”) 114B; and a third AR (“AR3”) 110C that, in turn, includes a third CC (“CC3”) 112C that manages a third pool of tenant servers (“tenant3”) 114C. The AZ2 104B duplicates the configuration of the AZ1 104A for high availability, and as such, the elements in each AZ 104 of the site 102 are identified using the same numbering scheme. This numbering scheme is used throughout the description of the remaining FIGURES. Moreover, references to the AZs 104, the PE/GWs 106, the ARs 110, the CCs 112, and the tenants 114 hereinafter can be interpreted as an instance thereof. For example, both the AZ1 104A and the AZ2 104B shown in the configuration of the cloud computing environment 100 in
Each of the CCs 112 provides a set of cloud controller services 116, such as compute services, networking services, storage services, orchestration services, and other services. In the illustrated embodiment, the cloud controller services 116 are OPENSTACK services, including Nova 118, Neutron 120, Cinder 122, Swift 124, Glance 126, Heat 127, and other services (not shown), each accessible via application programming interfaces (“APIs”; not shown) exposed by OPENSTACK. Nova 118 is an OPENSTACK service that allows the provisioning of compute instances (e.g., virtual machines, bare metal servers, and containers). Neutron 120 is an OPENSTACK service that provides network connectivity as-a-service between interface devices (e.g., virtual network interface controllers) managed by other OPENSTACK services, such as Nova 118. Cinder 122 is an OPENSTACK service that provides tools to manage storage resources consumed by compute instances created by Nova 118. Swift 124 is an OPENSTACK service that provides tools for data storage in the cloud. Glance 126 is an OPENSTACK service that facilitates discovery, registration, and retrieval of virtual machine images and associated metadata. Heat 127 is an OPENSTACK service that orchestrates cloud applications using a declarative template format through an OPENSTACK REST API. An example portion of a Heat 127 template for use in accordance with the concepts and technologies disclosed herein is described herein below with reference to
Network functions virtualization (“NFV”) is a new technology initiative that aims to move traditional and evolving mobility networking functions, such as access network elements, core network elements, transport network elements, and others, from purpose-built hardware to commercial-off-the-shelf (“COTS”) server-based platforms, such as those operating within servers disclosed herein. This is achieved through virtualization of mobility networking functions to create VNFs that operate on COTS hardware. The VNFs can perform any telecommunications function in support of one or more real-time services, including, particularly, voice services that benefit greatly from high availability.
In some embodiments, the cloud computing environment 100 is or includes a software-defined network (“SDN”). SDN is an architectural framework that provides a software-centric cloud environment for creating intelligent networks that are programmable, application aware, and more open. SDN provides an agile and cost-effective communications platform for handling the dramatic increase in data traffic on carrier networks by providing a high degree of scalability, security, and flexibility. SDN provides several benefits over traditional networks. SDN allows for the creation of multiple virtual network control planes on common hardware. SDN helps extend service virtualization and software control into many existing network elements. SDN enables applications to request and manipulate services provided by the network and to allow the network to expose network states back to the applications. SDN exposes network capabilities through application programming interfaces (“APIs”), making the control of network equipment remotely accessible and modifiable via third-party software clients using open protocols such as OpenFlow, available from Open Network Forum (“ONF”).
Combining SDN and NFV functionality, such as in Domain 2.0, available from AT&T, provides a highly complex and dynamic set of relationships between virtual, logical, and physical resources. Networks, such as embodied in Domain 2.0 deployments, provide intelligent software systems and applications operating on general purpose commodity hardware (e.g., COTS). This not only drives down capital expenditures, ongoing operational costs, and helps to configure networks with less human intervention, but also creates significant opportunities to scale and monetize existing and new intelligent services.
Within service providers, such as AT&T, orchestration systems like control, orchestration, management, and policy (“ECOMP”) have been created to dramatically reduce monotonous tasks and monitoring required by human operators through data-based analytics. Current orchestration systems often incite frustration among operators due to over-complicated network status readouts, non-specific network manipulations automatically performed by the orchestration system, and the inability to quickly “revert” changes caused by such manipulations. AT&T's ECOMP has been combined with the Open Orchestrator Project (“OPEN-O”) to create the Open Network Automation Platform (“ONAP”) project supported by the Linux Foundation. ONAP is an open source software platform that delivers capabilities for the design, creation, orchestration, monitoring, and life cycle management of SDNs and the VNFs operating therein, as well as higher-level services that utilize the functionality of SDN/VNF. ONAP provides automatic, policy-driven interaction of these functions and services in a dynamic, real-time cloud environment, such as the cloud computing environment 100.
In some embodiments, the cloud computing environment 100 provides, at least in part, Infrastructure-as-a-Service (“IaaS”), through which the tenants(s) 114 can interact with a front end (not shown) to provision processing, storage, networks, and other computing resources, whereby the tenants(s) 114 is/are able to deploy and run software, which can include, for example, VNFs to provide, at least in part, one or more telecommunications service(s) for the tenants 114 and/or others such as users or subscribers to the service(s). The tenant(s) 114 do not manage or control the underlying cloud infrastructure of the cloud computing environment 100, but have control over operating systems, storage, and deployed applications, and in some implementations, limited control of select networking components (e.g., host firewalls and/or other security components).
In some embodiments, the cloud computing environment 100 is provided as part of a private cloud infrastructure. A private cloud infrastructure is a cloud infrastructure that is provisioned for exclusive use by a single organization, which can include multiple users. A private cloud infrastructure might be owned, managed, and operated by the organization, a third party, or some combination thereof. A private cloud infrastructure can exist on or off premises. The tenant 114 can access a private cloud infrastructure provided, at least in part, by the cloud computing environment 100 via a front end, which can be provided by and/or accessed through a client, such as a web client application, or a native client application, for example.
In some embodiments, the cloud computing environment 100 is provided as part of a community cloud infrastructure. A community cloud infrastructure is a cloud infrastructure that is provisioned for exclusive use by a specific community of users from organizations that have shared concerns (e.g., mission, security requirements, policy, and compliance considerations). A community cloud infrastructure might be owned, managed, and operated by one or more organizations in the community, a third party, or some combination thereof. A community cloud infrastructure may exist on or off premises. The tenant 114 can access a community cloud infrastructure provided, at least in part, by the cloud computing environment 100 via a front end, which can be provided by and/or accessed through a client, such as a web client application, or a native client application, for example.
In some embodiments, the cloud computing environment 100 is provided as part of a public cloud infrastructure. A public cloud infrastructure is a cloud infrastructure that is provisioned for open use by the general public. A public cloud infrastructure might be owned, managed, and operated by a business, academic, or government organization, or some combination thereof. A public cloud infrastructure exists on the premises of the cloud service provider. The tenants 114 can access a public cloud infrastructure provided, at least in part, by the cloud computing environment 100 via a front end, which can be provided by and/or accessed through a client, such as a web client application, or a native client application, for example.
In some embodiments, the cloud computing environment 100 is provided as part of a hybrid cloud infrastructure. A hybrid cloud infrastructure is a cloud infrastructure that is a composition of two or more distinct cloud infrastructures—private, community, or public—that remain unique entities, but are bound together by standardized or proprietary technology that enables data and application portability. The tenants 114 can access a hybrid cloud infrastructure provided, at least in part, by the cloud computing environment 100 via a front end, which can be provided by and/or accessed through a client, such as a web client application, or a native client application, for example.
Referring now to
Referring now to
Referring now to
Table 1 below shows the availability and latency achieved with the new cloud architecture 200 and the conventional cloud architecture 202. The new cloud architecture 200 is capable of offering five 9s availability within the sites 102A-102B, the AZs 104A-104B, and the ARs 110A-110C. The conventional cloud architecture 202 also is capable of offering five 9s availability within the regions 204A-204B (as compared to the sites 102), but fails to provide such high availability in the zones 206A-206B (as compared to the AZs 104). Moreover, the conventional cloud architecture 202 does not offer the additional distribution granularity provided by the ARs 110A-110C in the new cloud architecture 200, which also are capable of offering five 9s availability. Latency remains the same (>2 ms) for communications between AZs 104A-104B in the new cloud architecture 200 and between zones 206A-206B in the conventional cloud architecture 202. Latency less than 2 ms (i.e., low latency) is achievable for communications between the
ARs 110A-110C. Since the conventional cloud architecture 202 fails to provide a demarcation similar such as provided by the ARs 110A-110C in the new cloud architecture 200, latency values associated with such a demarcation are not available for the conventional cloud architecture 202.
Referring now to
L2 networks within the AR1 110A and the AR2 110B are represented as the AR1 L2 network 306A and the AR2 L2 network 306B, respectively. The VNF L2 stretch network 300 utilizes Ethernet virtual private network (“EVPN”) routes to stretch the AR1 L2 network 306A and the AR2 L2 network 306B between the ARs 110. East-to-west traffic between the ARs 110A-110B (i.e., via vR/vS1 132A to vR/vS2 132B) traverses the VNF L2 stretch network 300 through spine switches 310 (to achieve latency<2 ms) without the traffic being routed through the PE/GW 106.
The vR/vS1 132A and the vR/vS2 132B are communicatively connected to the SDN controller1 134A and the SDN controllers 134B, respectively. The SDN controllers 134A-134B communicate, via a peer-to-peer connection, with a virtual route reflector (“vRR”) 312. The vRR 312 advertises, to the SDN controllers 134A-134B, IP addresses/routes and/or MAC addresses across the ARs 110A-110B. The SDN controllers 134A-134B instruct the vR/vSs 132A-132B to forward tenant traffic through the routes/addresses/MACs advertised by the vRR 312. Though the East-West communications between VMs 130 across ARs 110 is typically layer 2, and hence stretch L2 networks, it could be layer 3 as well. To meet the stringent real-time requirements, the L2/L3 traffic can be switched/routed within an AZ 104.
Referring now to
Referring now to
Turning now to
Turning now to
Referring now to
Most telecommunications carrier grade physical network functions have 1+1 redundancy for high availability. The concepts and technologies disclosed herein for the cloud computing environment 100 are able to achieve high availability and high utilization on par with these physical network functions using VNFs 302/304. In the example illustrated in
The example cluster distribution model shown in
The concepts and technologies disclosed herein for the cloud computing environment 100 are able to achieve high availability and high utilization on par with physical network functions with cluster redundancy for high availability. In the example illustrated in
Referring now to
The orchestrator 402 generates and sends a VNF homing request 416 to a conductor service 410 provided by the central placement decision system 404. The conductor service 410 performs a capacity check for each candidate site of the sites 102. The site 102 having capacity needed to accommodate the VNF homing request 416 is selected by the conductor service 410 as the target site 408. The conductor service 410 responds to the orchestrator 402 by identifying the target site 408 to which VNFs should be homed.
The orchestrator 402 then generates and sends a VNF placement request 418, including an OPENSTACK Heat orchestration template (“HOT”) (example shown in
The orchestrator 402 then instantiates VNF placements (generally shown as placement instantiation 422). In particular, the orchestrator 402 communicates with OPENSTACK Heat 127 to instantiate VMs (e.g., the VMs 130 best shown in
The orchestrator 402 creates new valet group declarations (i.e., valet affinity, diversity, and exclusivity groups), and updates the metadata associated therewith. The valet service 412 listens to OPENSTACK events (shown as VM and host update event 428) from the inventory 406. The valet service 412 also performs periodic resource status checks 426 of the target site 408 resources and caches via Nova REST APIs.
Referring now to
Referring now to
Turning to
Referring now to
Turning now to
Turning now to
Turning now to
Turning now to
Referring now to
The illustrated 4-site model 900A includes sites 102A-102D, each communicatively connected via the backbone network 108. Each of the sites 102 includes one AZ 104. In particular, the SITE1 102A includes the AZ1 104A; the SITE2 102B includes the AZ2 104B; the SITE3 102C includes the AZ3 104C; and the site4 102D includes the AZ4 104D.
Each of the AZs 104 in the 4-site model 900A includes one AR 110. In particular, the AZ1 104A includes the AR1 110A; the AZ2 104B includes the AR2 110B; the AZ3 104C includes the AR3 110C; and the AZ4 104D includes the AR4 110D. Each of the ARs 110 can include a pool of tenant servers 114 (shown as “tenant 114”; single tenant server configurations are also contemplated) hosting one or more applications. In particular, the AR1 110A includes the tenant1 114A; the AR2 110B includes the tenant2 114B; the AR3 110C includes tenant3 114C; and the AR4 110D includes the tenant4 114D.
Each of the tenants 114 can host one or more of the VMs 130. In particular, the tenant1 114A hosts the VM1 130A and the VM2 130B; the tenant2 114B hosts the VM3 130C and the VM4 130D; the tenant3 114C hosts the VM5 130E and the VM6 130F; and the tenant4 114D hosts the VM7 130G and the VM8 130H. Each pair of VMs 130 (e.g., the VM1 130A and the VM2 130B) can be implemented in an active-passive configuration.
The 4-site model 900A provides a total 8 million (“8M”) quota in the four sites 102A-102D, with each site 102 providing a 2M quota with 1M each for active and passive (i.e., stand-by) VMs 130 to achieve 25% utilization. Site availability for the 4-site model 900A is three 9s (99.9%); AR 110 (LCP) availability also is three 9s; VM availability for a given active-passive VM pair is three 9s; site availability is five 9s; and the storage design of the 4-site model 900A provides a single point of failure. Each site 102 in the 4-site model 900A has only one AR 110. Each of the sites 102 carries 500 thousand (“500K”) active traffic for a total traffic of 2M in the four sites 102A-102D. An upgrade or failure within any of the sites 102A-102D results in the upgraded/failed site going out-of-service. Thus, when one of the sites 102A-102D (e.g., SITE1 102A is upgraded during a planned event) and another one of the sites 102A-102D (e.g., SITE2 102B) fails as a result of an unplanned event (e.g., a dramatic increase in traffic), the remaining two sites 102C, 102D are required to manage the 2M traffic.
Turning now to
Each of the AZs 104A-104D in the illustrated novel 4-site model 900B includes two ARs 110. In particular, the AZ1 104A includes the AR1 110A and the AR2 110B; the AZ2 104B includes duplicate instances of the AR1 110A and the AR2 110B; the AZ3 104C includes duplicate instances of the AR1 110A and the AR2 110B; and the AZ4 104D includes duplicate instances of the AR1 110A and the AR2 110B. Each of the ARs 110 in the illustrated novel 4-site model 900B includes one CC 112 and one tenant 114. In particular, the AR1 110A includes the CC1 112A and the tenant1 114A, and the AR2 110B includes the CC2 112B and the tenant2 114B. Each of the tenants 114 in the illustrated novel 4-site model 900B includes one VM 130. In particular, the tenant1 114A includes the VM1 130A, and the tenant2 114B includes the VM2 130B. Each pair of the tenants 114A, 114B can communicate via an L2 connection.
The illustrated novel 4-site model 900B provides a total 8M quota in the four sites 102A-102D, with each site 102 providing a 2M quota with 1M quota for active VMs 130 in one AR 110 (e.g., the AR1 110A) and 1M quota for standby VMs 130 in the other AR 110 (e.g., the AR2 110B) to achieve 50% utilization (a 25% utilization improvement over the 4-site model 900A described above with reference
Turning now to
Each of the AZs 104A-104C in the illustrated novel 3-site model 900C includes three ARs 110A-110C. In particular, the AZ1 104A includes the AR1 110A, the AR2 110B, and the AR3 110C; the AZ2 104B includes a duplicate of the AR1 110A, the AR2 110B, and the AR3 110C; and the AZ3 104C includes a duplicate of the AR1 110A, the AR2 110B, and the AR3 110C. Each of the ARs 110 in the illustrated novel 3-site model 900C includes one CC 112 and one tenant 114. In particular, the AR1 110A includes the CC1 112A and the tenant1 114A, the AR2 110B includes the CC2 112B and the tenant2 114B, and the AR3 110C includes the CC3 112C and the tenant3 114C. Each of the tenants 114 in the illustrated novel 3-site model 900C includes one VM 130. In particular, the tenant1 114A includes the VM1 130A, the tenant2 114B includes the VM2 130B, and the tenant3 114C includes the VM3 130C. The tenants 114A-114C can communicate via an L2 connection.
The illustrated novel 3-site model 900C provides a total 3M quota in the three sites 102A-102C, with each site 102 providing a 330K quota for each AR 110 for a total quota of 1M per site 102. Also, each site 102 carries only 666K traffic, thus providing 66% utilization (a 41% improvement over the 4-site model 900A; see
Referring now to
Each of the AZs 104A-104D in the illustrated novel 4-site model 900D includes four ARs 110A-110D. In particular, the AZ1 104A includes the AR1 110A, the AR2 110B, the AR3 110C, and the AR4 110D; the AZ2 104B includes a duplicate of the AR1 110A, the AR2 110B, the AR3 110C, and the AR4 110D; the AZ3 104C includes a duplicate of the AR1 110A, the AR2 110B, the AR3 110C, and the AR4 110D; and the AZ4 104D includes a duplicate of the AR1 110A, the AR2 110B, the AR3 110C, and the AR4 110D. Each of the ARs 110 in the illustrated novel 4-site model 900D includes one CC 112 and one tenant 114. In particular, the AR1 110A includes the CC1 112A and the tenant1 114A, the AR2 110B includes the CC2 112B and the tenant2 114B, the AR3 110C includes the CC3 112C and the tenant3 114C, and the AR4 110D includes the CC4 112D and the tenant4 114D. Each of the tenants 114 in the illustrated novel 4-site model 900D includes one VM 130. In particular, the tenant1 114A includes the VM1 130A, the tenant2 114B includes the VM2 130B, the tenant3 114C includes the VM3 130C, and the tenant4 114D includes the VM4 130D. The tenants 114A-114D can communicate via an L2 connection.
The illustrated novel 4-site model 900D provides a total 3M quota in four sites 102A-102D, with each site 102 providing a 250K quota for each AR 110 for a total quota of 1M per site 102. Also, each site 102 carries only 750K traffic, thus providing 75% utilization (a 50% improvement over the 4-site model 900A shown in
Turning now to
Turning now to
Turning now to
It also should be understood that the methods disclosed herein can be ended at any time and need not be performed in its entirety. Some or all operations of the methods, and/or substantially equivalent operations, can be performed by execution of computer-readable instructions included on a computer storage media, as defined herein. The term “computer-readable instructions,” and variants thereof, as used herein, is used expansively to include routines, applications, application modules, program modules, programs, components, data structures, algorithms, and the like. Computer-readable instructions can be implemented on various system configurations including single-processor or multiprocessor systems, minicomputers, mainframe computers, personal computers, hand-held computing devices, microprocessor-based, programmable consumer electronics, combinations thereof, and the like.
Thus, it should be appreciated that the logical operations described herein are implemented (1) as a sequence of computer implemented acts or program modules running on a computing system and/or (2) as interconnected machine logic circuits or circuit modules within the computing system. The implementation is a matter of choice dependent on the performance and other requirements of the computing system. Accordingly, the logical operations described herein are referred to variously as states, operations, structural devices, acts, or modules. These states, operations, structural devices, acts, and modules may be implemented in software, in firmware, in special purpose digital logic, and any combination thereof. As used herein, the phrase “cause a processor to perform operations” and variants thereof is used to refer to causing one or more processors to perform operations.
For purposes of illustrating and describing some of the concepts of the present disclosure, the methods disclosed herein are described as being performed, at least in part, by a VM placement system, such as the central placement decision system 404 (see
The method 1100 begins and proceeds to operation 1102, where the central placement decision system 404 receives an application placement request from a request queue. From operation 1102, the method 1100 proceeds to operation 1104, where the central placement decision system 404 determines the availability and utilization requirements for application placement.
In some embodiments, the application placement request specifies availability and/or utilization requirements to be met for placement of the requested application. In this manner, the application placement request can identify any of the novel high availability and high utilization models disclosed herein to be used for application placement.
In other embodiments, the central placement decision system 404 determines the availability and utilization requirements under which the application is to be placed. The central placement decision system 404 can make such determinations based upon one or more policies created by or for the provider of at least a portion of the cloud computing environment.
The central placement decision system 404 also can consider the status of one or more cloud resources in this determination. The status can include current utilization metrics for one or more of the cloud resources available from the cloud computing environment. The status can identify any cloud resource failures based upon output received from one or more monitoring systems of one or more servers (e.g., server 128). The status can include information regarding any planned event, including, for example, any planned upgrades to any of the sites 102, the AZs 104, the ARs 110, the servers 128 associated with at least a portion of the cloud computing environment. From operation 1104, the method 1100 proceeds to operation 1106, where the central placement decision system 404 places the requested application in the cloud computing environment in accordance with the availability and utilization requirements determined at operation 1104.
From operation 1106, the method 1100 proceeds to operation 1108, where the cloud computing environment detects a failure or a planned event. A failure can be detected via one or more monitoring systems that are deployed within the cloud computing environment at any level—that is, the site 102, AZ 104, AR 110, server 128, or VM 130 level. The planned event can be an upgrade or other modification to any hardware and/or software associated with at least a portion of the cloud computing environment in which the application was placed at operation 1106.
From operation 1108, the method 1100 proceeds to operation 1110, where the cloud computing environment, in response to the failure or planned event detected at operation 1106, redirects traffic associated with application from the portion of the cloud computing environment affected by the failure or planned event to one or more spare VMs operating elsewhere in the cloud computing environment. For example, the cloud computing environment can redirect traffic from one of the sites 102 to one or more other sites 102 that have available spare VMs 130. From operation 1110, the method 1100 proceeds to operation 1112, where the method 1100 ends.
Turning now to
The hardware resource layer 1202 provides hardware resources. In the illustrated embodiment, the hardware resource layer 1202 includes one or more compute resources 1212, one or more memory resources 1214, and one or more other resources 1215. The compute resource(s) 1212 can include one or more hardware components that perform computations to process data and/or to execute computer-executable instructions of one or more application programs, one or more operating systems, and/or other software. In particular, the compute resources 1212 can include one or more central processing units (“CPUs”) configured with one or more processing cores. The compute resources 1212 can include one or more graphics processing unit (“GPU”) configured to accelerate operations performed by one or more CPUs, and/or to perform computations to process data, and/or to execute computer-executable instructions of one or more application programs, one or more operating systems, and/or other software that may or may not include instructions particular to graphics computations. In some embodiments, the compute resources 1212 can include one or more discrete GPUs. In some other embodiments, the compute resources 1212 can include CPU and GPU components that are configured in accordance with a co-processing CPU/GPU computing model, wherein the sequential part of an application executes on the CPU and the computationally-intensive part is accelerated by the GPU processing capabilities. The compute resources 1212 can include one or more system-on-chip (“SoC”) components along with one or more other components, including, for example, one or more of the memory resources 1214, and/or one or more of the other resources 1215. In some embodiments, the compute resources 1212 can be or can include one or more SNAPDRAGON SoCs, available from QUALCOMM of San Diego, Calif.; one or more TEGRA SoCs, available from NVIDIA of Santa Clara, Calif.; one or more HUMMINGBIRD SoCs, available from SAMSUNG of Seoul, South Korea; one or more Open Multimedia Application Platform (“OMAP”) SoCs, available from TEXAS INSTRUMENTS of Dallas, Tex.; one or more customized versions of any of the above SoCs; and/or one or more proprietary SoCs. The compute resources 1212 can be or can include one or more hardware components architected in accordance with an ARM architecture, available for license from ARM HOLDINGS of Cambridge, United Kingdom. Alternatively, the compute resources 1212 can be or can include one or more hardware components architected in accordance with an x86 architecture, such an architecture available from INTEL CORPORATION of Mountain View, Calif., and others. Those skilled in the art will appreciate the implementation of the compute resources 1212 can utilize various computation architectures, and as such, the compute resources 1212 should not be construed as being limited to any particular computation architecture or combination of computation architectures, including those explicitly disclosed herein.
The memory resource(s) 1214 can include one or more hardware components that perform storage/memory operations, including temporary or permanent storage operations. In some embodiments, the memory resource(s) 1214 include volatile and/or non-volatile memory implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data disclosed herein. Computer storage media includes, but is not limited to, random access memory (“RAM”), read-only memory (“ROM”), Erasable Programmable ROM (“EPROM”), Electrically Erasable Programmable ROM (“EEPROM”), flash memory or other solid state memory technology, CD-ROM, digital versatile disks (“DVD”), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store data and which can be accessed by the compute resources 1212.
The other resource(s) 1215 can include any other hardware resources that can be utilized by the compute resources(s) 1212 and/or the memory resource(s) 1214 to perform operations described herein. The other resource(s) 1215 can include one or more input and/or output processors (e.g., network interface controller or wireless radio), one or more modems, one or more codec chipset, one or more pipeline processors, one or more fast Fourier transform (“FFT”) processors, one or more digital signal processors (“DSPs”), one or more speech synthesizers, and/or the like.
The hardware resources operating within the hardware resource layer 1202 can be virtualized by one or more hypervisors 1216A-1216N (also known as “virtual machine monitors”) operating within the hypervisor layer 1204 to create virtual resources that reside in the virtual resource layer 1206. The hypervisors 1216A-1216N can be or can include software, firmware, and/or hardware that alone or in combination with other software, firmware, and/or hardware, creates and manages virtual resources 1218A-1218N operating within the virtual resource layer 1206.
The virtual resources 1218A-1218N operating within the virtual resource layer 1206 can include abstractions of at least a portion of the compute resources 1212, the memory resources 1214, and/or the other resources 1215, or any combination thereof. In some embodiments, the abstractions can include one or more VMs, virtual volumes, virtual networks, and/or other virtualizes resources upon which one or more VNFs 1219A-1219N can be executed. The VNFs 1219A-1219N in the virtual function layer 1208 are constructed out of the virtual resources 1218A-1218N in the virtual resource layer 1206. In the illustrated example, the VNFs 1219A-1219N can provide, at least in part, one or more services 1220A-1220N in the service layer 1210.
The computer system 1300 includes a processing unit 1302, a memory 1304, one or more user interface devices 1306, one or more input/output (“I/O”) devices 1308, and one or more network devices 1310, each of which is operatively connected to a system bus 1312. The bus 1312 enables bi-directional communication between the processing unit 1302, the memory 1304, the user interface devices 1306, the I/O devices 1308, and the network devices 1310.
The processing unit 1302 may be a standard central processor that performs arithmetic and logical operations, a more specific purpose programmable logic controller (“PLC”), a programmable gate array, or other type of processor known to those skilled in the art and suitable for controlling the operation of the server computer. Processing units are generally known, and therefore are not described in further detail herein.
The memory 1304 communicates with the processing unit 1302 via the system bus 1312. In some embodiments, the memory 1304 is operatively connected to a memory controller (not shown) that enables communication with the processing unit 1302 via the system bus 1312. The illustrated memory 1304 includes an operating system 1314 and one or more program modules 1316. The operating system 1314 can include, but is not limited to, members of the WINDOWS, WINDOWS CE, and/or WINDOWS MOBILE families of operating systems from MICROSOFT CORPORATION, the LINUX family of operating systems, the SYMBIAN family of operating systems from SYMBIAN LIMITED, the BREW family of operating systems from QUALCOMM CORPORATION, the MAC OS, OS X, and/or iOS families of operating systems from APPLE CORPORATION, the FREEBSD family of operating systems, the SOLARIS family of operating systems from ORACLE CORPORATION, other operating systems, and the like.
The program modules 1316 may include various software and/or program modules to perform the various operations described herein. The program modules 1316 and/or other programs can be embodied in computer-readable media containing instructions that, when executed by the processing unit 1302, perform various operations such as those described herein. According to embodiments, the program modules 1316 may be embodied in hardware, software, firmware, or any combination thereof.
By way of example, and not limitation, computer-readable media may include any available computer storage media or communication media that can be accessed by the computer system 1300. Communication media includes computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics changed or set in a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of the any of the above should also be included within the scope of computer-readable media.
Computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, Erasable Programmable ROM (“EPROM”), Electrically Erasable Programmable ROM (“EEPROM”), flash memory or other solid state memory technology, CD-ROM, digital versatile disks (“DVD”), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by the computer system 1300. In the claims, the phrase “computer storage medium” and variations thereof does not include waves or signals per se and/or communication media.
The user interface devices 1306 may include one or more devices with which a user accesses the computer system 1300. The user interface devices 1306 may include, but are not limited to, computers, servers, PDAs, cellular phones, or any suitable computing devices. The I/O devices 1308 enable a user to interface with the program modules 1316. In one embodiment, the I/O devices 1308 are operatively connected to an I/O controller (not shown) that enables communication with the processing unit 1302 via the system bus 1312. The I/O devices 1308 may include one or more input devices, such as, but not limited to, a keyboard, a mouse, or an electronic stylus. Further, the I/O devices 1308 may include one or more output devices, such as, but not limited to, a display screen or a printer.
The network devices 1310 enable the computer system 1300 to communicate with other networks or remote systems via a network 1318. Examples of the network devices 1310 include, but are not limited to, a modem, a radio frequency (“RF”) or infrared (“IR”) transceiver, a telephonic interface, a bridge, a router, or a network card. The network 1318 may include a wireless network such as, but not limited to, a Wireless Local Area Network (“WLAN”), a Wireless Wide Area Network (“WWAN”), a Wireless Personal Area Network (“WPAN”) such as provided via BLUETOOTH technology, a Wireless Metropolitan Area Network (“WMAN”) such as a WiMAX network or metropolitan cellular network. Alternatively, the network 1318 may be a wired network such as, but not limited to, a Wide Area Network (“WAN”), a wired Personal Area Network (“PAN”), or a wired Metropolitan Area Network (“MAN”). The network 1318 can be or can include the backbone network 108 and/or, one or more networks operating within the cloud computing environment 100.
Turning now to
The cellular network 1402 includes various components such as, but not limited to, base transceiver stations (“BTSs”), Node-B's or e-Node-B's, base station controllers (“BSCs”), radio network controllers (“RNCs”), mobile switching centers (“MSCs”), mobile management entities (“MMEs”), short message service centers (“SMSCs”), multimedia messaging service centers (“MMSCs”), home location registers (“HLRs”), home subscriber servers (“HSSs”), visitor location registers (“VLRs”), charging platforms, billing platforms, voicemail platforms, GPRS core network components, location service nodes, an IP Multimedia Subsystem (“IMS”), and the like. The cellular network 1402 also includes radios and nodes for receiving and transmitting voice, data, and combinations thereof to and from radio transceivers, networks, the packet data network 1404, and the circuit switched network 1406.
A mobile communications device 1408, such as, for example, a cellular telephone, a user equipment, a mobile terminal, a PDA, a laptop computer, a handheld computer, and combinations thereof, can be operatively connected to the cellular network 1402. The cellular network 1402 can be configured as a 2G Global System for Mobile communications (“GSM”) network and can provide data communications via General Packet Radio Service (“GPRS”) and/or Enhanced Data rates for Global Evolution (“EDGE”). Additionally, or alternatively, the cellular network 1402 can be configured as a 3G Universal Mobile Telecommunications System (“UMTS”) network and can provide data communications via the High-Speed Packet Access (“HSPA”) protocol family, for example, High-Speed Downlink Packet Access (“HSDPA”), Enhanced Uplink (“EUL”) (also referred to as High-Speed Uplink Packet Access “HSUPA”), and HSPA+. The cellular network 1402 also is compatible with 4G mobile communications standards such as Long-Term Evolution (“LTE”), or the like, as well as evolved and future mobile standards.
The packet data network 1404 includes various devices, for example, servers, computers, databases, and other devices in communication with one another, as is generally known. The packet data network 1404 devices are accessible via one or more network links. The servers often store various files that are provided to a requesting device such as, for example, a computer, a terminal, a smartphone, or the like. Typically, the requesting device includes software (a “browser”) for executing a web page in a format readable by the browser or other software. Other files and/or data may be accessible via “links” in the retrieved files, as is generally known. The circuit switched network 1406 includes various hardware and software for providing circuit switched communications. The circuit switched network 1406 may include, or may be, what is often referred to as a plain old telephone system (“POTS”). The functionality of a circuit switched network 1406 or other circuit-switched network are generally known and will not be described herein in detail.
The illustrated cellular network 1402 is shown in communication with the packet data network 1404 and a circuit switched network 1406, though it should be appreciated that this is not necessarily the case. One or more Internet-capable devices 1410, a PC, a laptop, a portable device, or another suitable device, can communicate with one or more cellular networks 1402, and devices connected thereto, through the packet data network 1404. It also should be appreciated that the Internet-capable device 1410 can communicate with the packet data network 1404 through the circuit switched network 1406, the cellular network 1402, and/or via other networks (not illustrated).
As illustrated, a communications device 1412, for example, a telephone, facsimile machine, modem, computer, or the like, can be in communication with the circuit switched network 1406, and therethrough to the packet data network 1404 and/or the cellular network 1402. It should be appreciated that the communications device 1412 can be an Internet-capable device, and can be substantially similar to the Internet-capable device 1410. In the specification, the network is used to refer broadly to any combination of the networks 1402, 1404, 1406 shown in
Turning now to
The virtual network topology layer 1506 includes the VMs 1514, one or more hypervisors 1516, and one or more server modules (“blades”) 1518. Each blade 1518 can support one hypervisor 1516 that, in turn, can manage one or more of the VMs 1514. The blades 1518 provide computing capacity to support the VMs 1514 carrying the VNFs 1512. The hypervisors 1516 provide resource management among the VMs 1514 supported thereby. A logical server cluster 1520 is created for resource allocation and reallocation purpose, which includes the blades 1518 in the same server host 1522. Each server host 1522 includes one or more of the server clusters 1520.
The physical network topology layer 1508 includes an Ethernet switch (“ESwitch”) group 1524 and a router group 1526. The ESwitch group 1524 provides traffic switching function among the blades 1518. The router group 1526 provides connectivity for traffic routing between the data center cloud 1502 and virtualized IP network(s) 1528. The router group 1526 may or may not provide multiplexing functions, depending upon network design.
The virtual network topology 1506 is dynamic by nature, and as such, the VMs 1514 can be moved among the blades 1518 as needed. The physical network topology 1508 is more static, and as such, no dynamic resource allocation is involved in this layer. Through such a network topology configuration, the association among application VNFs 1510, the VM 1514 supporting the application VNFs 1510, and the blades 1518 that host the VM 1514 can be determined.
In the illustrated example, a first VNF is divided into two sub-VNFs, VNF1-11512A and VNF1-21512C, which is executed by VM 1-1-11514A and VM 1-N-11514C, respectively. The VM 1-1-11514A is hosted by the blade 1-11518A and managed by the hypervisor 1-11516A in the server cluster 11520 of the server host 1522. Traffic switching between the blade 1-11518A and the blade 1-N 1518N is performed via ESwitch-11524A. Traffic communications between the ESwitch group 1524 and the virtualized IP network(s) 1528 are performed via the router group 1526. In this example, the VM 1-1-11514A can be moved from the blade 1-11518A to the blade 1-N 1518N for VM live migration if the blade 1-11518A is detected to have difficulty to support the VNF 1-11512A performance requirements and the blade 1-N 1518N has sufficient capacity and is available to support the VNF1-11512A performance requirements. The virtual network topology 1506 is dynamic by nature due to real-time resource allocation/reallocation capability of cloud SDN. The association of application, VM, and blade host in this example is the VNF1-11512A is executed on the VM 1-1-11514A hosted by the blade 1-11518A in the server cluster 11520A.
Based on the foregoing, it should be appreciated that concepts and technologies directed to a high availability and high utilization cloud computing environment for supporting telecommunications services have been disclosed herein. Although the subject matter presented herein has been described in language specific to computer structural features, methodological and transformative acts, specific computing machinery, and computer-readable media, it is to be understood that the concepts and technologies disclosed herein are not necessarily limited to the specific features, acts, or media described herein. Rather, the specific features, acts and mediums are disclosed as example forms of implementing the concepts and technologies disclosed herein.
The subject matter described above is provided by way of illustration only and should not be construed as limiting. Various modifications and changes may be made to the subject matter described herein without following the example embodiments and applications illustrated and described, and without departing from the true spirit and scope of the embodiments of the concepts and technologies disclosed herein.
This application is a continuation of and claims priority to U.S. patent application Ser. No. 16/225,050, entitled “High Availability and High Utilization Cloud Data Center Architecture for Supporting Telecommunications Services,” filed Dec. 19, 2018, now U.S. Pat. No. 10,855,757, which is incorporated herein by reference in its entirety.
| Number | Name | Date | Kind |
|---|---|---|---|
| 7450498 | Golia et al. | Nov 2008 | B2 |
| 7464151 | Drennen et al. | Dec 2008 | B1 |
| 7909880 | Grant | Mar 2011 | B1 |
| 7933987 | Aidun et al. | Apr 2011 | B2 |
| 8160063 | Maltz et al. | Apr 2012 | B2 |
| 8394132 | Lewis et al. | Mar 2013 | B2 |
| 8438277 | Chepuri et al. | May 2013 | B1 |
| 8510625 | Back et al. | Aug 2013 | B1 |
| 9246994 | Julien et al. | Jan 2016 | B2 |
| 9420422 | Rice | Aug 2016 | B1 |
| 9483258 | Labat et al. | Nov 2016 | B1 |
| 9529883 | Bourbonnais et al. | Dec 2016 | B2 |
| 9582559 | Faibish et al. | Feb 2017 | B1 |
| 9679008 | Vosshall et al. | Jun 2017 | B2 |
| 9852027 | Simoncelli | Dec 2017 | B2 |
| 9877752 | Tyber et al. | Jan 2018 | B2 |
| 9998862 | Rice | Jun 2018 | B1 |
| 10264586 | Beattie, Jr. | Apr 2019 | B2 |
| 10353603 | Baruch | Jul 2019 | B1 |
| 10397280 | Sood | Aug 2019 | B2 |
| 10423634 | Shemer | Sep 2019 | B1 |
| 10432537 | Zhang | Oct 2019 | B2 |
| 10432552 | Zhang | Oct 2019 | B2 |
| 10521223 | Bogushefsky, III | Dec 2019 | B1 |
| 10530740 | Jana | Jan 2020 | B2 |
| 10587465 | Sandham | Mar 2020 | B1 |
| 10855757 | Thyagarajan | Dec 2020 | B2 |
| 10880180 | Zhang | Dec 2020 | B2 |
| 11184233 | Neelakantam | Nov 2021 | B1 |
| 11228497 | Maes | Jan 2022 | B2 |
| 11340837 | Vohra | May 2022 | B1 |
| 11403000 | Barker, Jr. | Aug 2022 | B1 |
| 11416298 | Barker, Jr. | Aug 2022 | B1 |
| 20020174050 | Eynard | Nov 2002 | A1 |
| 20030154236 | Dar et al. | Aug 2003 | A1 |
| 20060153068 | Dally et al. | Jul 2006 | A1 |
| 20090245137 | Hares et al. | Oct 2009 | A1 |
| 20100228819 | Wei | Sep 2010 | A1 |
| 20130268672 | Justafort et al. | Oct 2013 | A1 |
| 20140207918 | Kowalski | Jul 2014 | A1 |
| 20150082302 | Cheng et al. | Mar 2015 | A1 |
| 20160218939 | Tiwari | Jul 2016 | A1 |
| 20160246648 | Karnath et al. | Aug 2016 | A1 |
| 20160255139 | Rathod | Sep 2016 | A1 |
| 20160291876 | Alatorre | Oct 2016 | A1 |
| 20170031700 | Yasrebi | Feb 2017 | A1 |
| 20170034117 | Yasrebi | Feb 2017 | A1 |
| 20170034318 | Gong et al. | Feb 2017 | A1 |
| 20170034346 | Yasrebi | Feb 2017 | A1 |
| 20170034359 | Yasrebi | Feb 2017 | A1 |
| 20170041296 | Ford | Feb 2017 | A1 |
| 20170094377 | Herdrich | Mar 2017 | A1 |
| 20170332421 | Sternberg | Nov 2017 | A1 |
| 20180039580 | Schneider | Feb 2018 | A1 |
| 20180070100 | Chen | Mar 2018 | A1 |
| 20180077024 | Zhang | Mar 2018 | A1 |
| 20180191859 | Sharma | Jul 2018 | A1 |
| 20180260745 | Jana | Sep 2018 | A1 |
| 20180295067 | Xia | Oct 2018 | A1 |
| 20180337942 | Htay | Nov 2018 | A1 |
| 20180349202 | Sharma | Dec 2018 | A1 |
| 20180367412 | Sethi | Dec 2018 | A1 |
| 20190036869 | Jana | Jan 2019 | A1 |
| 20190171492 | Yousaf | Jun 2019 | A1 |
| 20190223055 | Bor Yaliniz | Jul 2019 | A1 |
| 20190243547 | Duggal | Aug 2019 | A1 |
| 20190243688 | Karmarkar | Aug 2019 | A1 |
| 20190243702 | Shilane | Aug 2019 | A1 |
| 20190245918 | Xu | Aug 2019 | A1 |
| 20190379731 | Johnsen | Dec 2019 | A1 |
| 20200065086 | Woodmansee | Feb 2020 | A1 |
| 20200067776 | Williams | Feb 2020 | A1 |
| 20200136825 | Gupta | Apr 2020 | A1 |
| 20200160208 | Tan | May 2020 | A1 |
| 20200160227 | Liu | May 2020 | A1 |
| 20200162323 | Abu Lebdeh | May 2020 | A1 |
| 20200244551 | Tidemann | Jul 2020 | A1 |
| 20200274375 | Griffiths | Aug 2020 | A1 |
| 20200314126 | Schmugar | Oct 2020 | A1 |
| 20200366493 | Sood | Nov 2020 | A1 |
| 20210019063 | Lee | Jan 2021 | A1 |
| 20210021494 | Yao | Jan 2021 | A1 |
| 20210037400 | Yao | Feb 2021 | A1 |
| 20210112119 | Thyagarajan | Apr 2021 | A1 |
| 20210144517 | Guim Bernat | May 2021 | A1 |
| 20210184925 | Narayanan | Jun 2021 | A1 |
| 20210360083 | Duggal | Nov 2021 | A1 |
| 20220014450 | Srikanteswara | Jan 2022 | A1 |
| 20220019367 | Freilich | Jan 2022 | A1 |
| 20220019385 | Karr | Jan 2022 | A1 |
| 20220035541 | Genereux | Feb 2022 | A1 |
| 20220035714 | Schultz | Feb 2022 | A1 |
| 20220044480 | Zavesky | Feb 2022 | A1 |
| 20220075546 | Potyraj | Mar 2022 | A1 |
| 20220083245 | Kant | Mar 2022 | A1 |
| 20220124560 | Yeh | Apr 2022 | A1 |
| 20220158926 | Wennerström | May 2022 | A1 |
| 20220179685 | Lemoine | Jun 2022 | A1 |
| 20220197773 | Butler | Jun 2022 | A1 |
| 20220231964 | Alam | Jul 2022 | A1 |
| 20220232423 | Thyagaturu | Jul 2022 | A1 |
| 20220247678 | Atwal | Aug 2022 | A1 |
| 20220248296 | Merwaday | Aug 2022 | A1 |
| 20220300418 | Baugh | Sep 2022 | A1 |
| 20220391124 | Darji | Dec 2022 | A1 |
| Entry |
|---|
| Chang et al., “High-Performed Virtualization Services for In-Cloud Enterprise Resource Planning System,” Journal of Information Hiding and Multimedia Signal Processing, Oct. 2014, vol. 5, No. 4, pp. 614-624. |
| Cully et al., “Remus: High Availability via Asynchronous Virtual Machine Replication,” Proceedings of the 5th USENIX Symposium on Networked Systems Design and Implementation, Apr. 16-18, 2008, pp. 161-174. |
| Hawilo, Hassan, “Elastic Highly Available Cloud Computing,” Electronic Thesis and Dissertation Repository, Aug. 2015. |
| Li et al., “Improving Availability of Cloud-Based Applications through Deployment Choices,” 2013 IEEE Sixth International Conference on Cloud Computing, Jun. 28-Jul. 3, 2013, pp. 43-50, IEEE 2013. |
| Almasi et al., “Toward building highly available and scalable OpenStack clouds,” IBM Journal of Research and Development, Mar./May 2016, vol. 60, No. 2/3, Paper 5. |
| U.S. Office Action dated May 15, 2020 in U.S. Appl. No. 16/225,050. |
| U.S. Notice of Allowance dated Sep. 16, 2020 in U.S. Appl. No. 16/225,050. |
| Number | Date | Country | |
|---|---|---|---|
| 20210112119 A1 | Apr 2021 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16225050 | Dec 2018 | US |
| Child | 17107394 | US |
