Estimating model parameters for automatic deployment of scalable micro services

Abstract

One aspect of the disclosure relates to, among other things, a method for optimizing and provisioning a software-as-a-service (SaaS). The method includes determining a graph comprising interconnected stages for the SaaS, wherein each stage has a replication factor and one or more metrics that are associated with one or more service level objectives of the SaaS, determining a first replication factor associated with a first one of the stages which meets a first service level objective of the SaaS, adjusting the first replication factor associated with the first one of the stage based on the determined first replication factor, and provisioning the SaaS onto networked computing resources based on the graph and replication factors associated with each stage.

Description

TECHNICAL FIELD

This disclosure relates in general to the field of computing and, more particularly, to estimating model parameters for automatic deployment of scalable micro services.

BACKGROUND

Cloud computing aggregates physical and virtual compute, storage, and network resources in the “cloud” and offers users many ways to utilize the resources. One kind of product leveraging cloud computing is called Software-as-a-Service (SaaS). Software vendors can acquire resources in the cloud (e.g., renting and/or building their own) to run software applications for their customers. The cloud having physical hardware resources would host the software applications to which customers would have access. Software applications that can be offered using SaaS can include financial services application, gaming application, supply chain application, inventory management application, data management application, talent acquisition application, etc. Users or customers can request an instance of a software application from the software vendor. The software vendor can in turn instantiate the software application on the cloud. The customer would then be able to use and access the software application that he/she requested.

BRIEF DESCRIPTION OF THE DRAWINGS

To provide a more complete understanding of the present disclosure and features and advantages thereof, reference is made to the following description, taken in conjunction with the accompanying figures, wherein like reference numerals represent like parts, in which:

FIG. 1 depicts a system providing Software-as-a-Service (SaaS), according to some embodiments of the disclosure;

FIG. 2 depicts exemplary stages of a SaaS, according to some embodiments of the disclosure;

FIG. 3 depicts a SaaS having three exemplary stages, according to some embodiments of the disclosure;

FIG. 4 illustrates increasing a replication factor for a stage, according to some embodiments of the disclosure;

FIG. 5 is a flow diagram illustrating a method for optimizing and provisioning a SaaS, according to some embodiments of the disclosure; and

FIG. 6 illustrates adding a load balancer to account for replication of stages, according to some embodiments of the disclosure;

FIG. 7 illustrates estimation of queue length as a metric for each one of the stages, according to some embodiments of the disclosure; and

FIG. 8 depicts a block diagram illustrating an exemplary data processing system that may be used to implement optimizing and/or provisioning a SaaS, according to some embodiments of the disclosure.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

Overview

One aspect of the disclosure relates to, among other things, a method for optimizing and provisioning a software-as-a-service (SaaS). The method includes determining a graph comprising interconnected stages for the SaaS, wherein each stage has a replication factor and one or more metrics that are associated with one or more service level objectives of the SaaS, determining a first replication factor associated with a first one of the stages which meets a first service level objective of the SaaS, adjusting the first replication factor associated with the first one of the stage based on the determined first replication factor, and provisioning the SaaS onto networked computing resources based on the graph and replication factors associated with each stage.

In other aspects, apparatuses comprising means for carrying out one or more of the method steps are envisioned by the disclosure. As will be appreciated by one skilled in the art, aspects of the disclosure, in particular the functionality associated with modelling and deploying scalable micro services herein, may be embodied as a system, a method or a computer program product. Accordingly, aspects of the disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Functions described in this disclosure may be implemented as an algorithm executed by a processor, e.g., a microprocessor, of a computer. Furthermore, aspects of the disclosure may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied, e.g., stored, thereon.

EXAMPLE EMBODIMENTS

Understanding Challenges for Providing Service-as-a-Service SaaS

A Software-as-a-Service (SaaS) offering has certain service level objectives (SLO) such as number of subscribers (or customers), number of requests, and volume of traffic. Furthermore, SaaS may have certain service level agreements (SLAs) with the subscribers that defines a commitment for aspects of the service such as quality, availability, etc. Increasingly, a SaaS service is composed of multiple micro-services which are interconnected to each other as different parts of a software application. Micro-services enables smaller, light weight processes to be used, which can offer benefits such as flexibility and scalability than other architectures. Micro-services can be replicated to increase throughput and efficiency.

Auto-scaling of an existing/running SaaS is not uncommon in cloud-based software. In auto-scaling, constant monitoring is required and based on certain monitored metrics (such as processor usage and memory usage), newer instances are automatically launched (and existing instances removed). The problem of being able to initially deploy a micro-services based SaaS optimally with the necessary scale numbers has yet to be addressed.

Modeling a SaaS to Optimize Deployment

When deploying a SaaS onto the cloud, it is advantageous to determine the set of micro services that would meet SLOs and SLAs. Specifically, a solution would intelligently deploy a micro services based SaaS with necessary scale or replication numbers so that the SLAs and SLOs of the overall SaaS offering is met. A method is implemented to determine an initial installation of a SaaS that would meet the SLOs and SLAs, starting from a SaaS model provided to a SaaS manager.

FIG. 1 depicts a system providing Software-as-a-Service (SaaS), according to some embodiments of the disclosure. The system 100 includes a cloud 102 having networked hardware resources, a SaaS Manager 104, and a plurality of users 106_A, 106_B, 106_c, . . . who may access or subscribe to SaaS deployed in cloud 102. The networked hardware resources in cloud 102 can include physical compute, network, storage resources (or virtualized versions thereof implemented on the physical resources). The SaaS Manager 104 has a provisioner 114, which can install or deploy a SaaS in 102. Typically, the SaaS is deployed as a series of interconnected micro services (or smaller light weight software applications) being executed in cloud 102.

A SaaS deployment is modeled as a graph with nodes representing the constituent stages and edges representing the flow of data through the stages. FIG. 2 depicts exemplary stages of a SaaS, according to some embodiments of the disclosure. A SaaS offering model graph 200 can have the following exemplary stages: ingestion stage 202, processing stage 204, and storage stage 206. A UI/Query process 208 may access storage 206. Data is pushed into the SaaS via the ingestion stage 202. Second, the processing stage 204 receives data from ingestion stage 202 and processes the data to implement features of the SaaS. Third, the processed data from the processing stage 204 is pushed to the storage stage 206 where results of the processing, e.g., insights achieved through analysis, are stored for later retrieval by the user (e.g., users 106_A, 106_B, 106_c) via UI/Query process 208. Each of these stages can include one or more micro-services as sub-stages within the stage. A bottleneck in one or more of these stages or sub-stages can cause the stage to be the bottleneck for the entire SaaS, thus leading to suboptimal experience for the users of the SaaS (e.g., users 106_A, 106_B, 106_c).

The SaaS Manager 104 has corresponding SLO's and SLA's 108. Bottlenecks can greatly affect the ability to meet the SLO's and SLA's. For simplicity, embodiments described herein may reference them interchangeably, since the embodiments are equally applicable to meeting both SLOs and SLAs. When a SaaS service is designed, there are some overall service level objectives. Some typical SLOs are:

• • Maximum data throughput: maximum amount of data that the SaaS can process, e.g. per minute,
  • Maximum numbers of users the SaaS can support,
  • Maximum tolerated processing delay through the SaaS pipeline, and
  • Maximum amount of storage: amount of data that can be stored per user for a specific period of time

Because the SaaS is deployed using micro services, the stages may be scalable or replicated easily. However, determining how to scale the stages optimally to meet the SLOs is not trivial. In other words, when a SaaS is deployed, it is not clear how to resource (e.g., replicate) each of its stages (and thus each of the sub-stages of the stages) such that the overall SLOs of the SaaS can be satisfied.

Referring back to FIG. 1, the SaaS Manager 104 further includes optimizer 112, which is configured to install the model graph with appropriate scaling or replication factors for each stage of the model graph so that the SLOs 108 of the overall SaaS offering are met. Advantageously, manual intervention in figuring out the best-case deployment scale need not be a trial and error process.

Example of Modeling and Provisioning for Throughput

FIG. 3 depicts a SaaS having three exemplary stages, according to some embodiments of the disclosure. In this example, the SaaS 300, shown in the form of a model graph, has three stages in sequence: stage 302, stage 304, and stage 306. One exemplary SLO for the SaaS may require the SaaS 300 to be able to process two data payloads per second (a throughput SLO). If stage 304 has a metric which requires 500 milliseconds for each payload, it would be difficult to meet the SLO because stage 306 alone would require one second to process two data payload and stage 302 and stage 306 would need some time to process each payload (still).

However, if the stage 306 can be replicated, the SLO can be met, under the assumption that each replicated copy of stage 306 can independently handle the data payload, the SLO can be maintained. FIG. 4 illustrates increasing a replication factor for a stage, according to some embodiments of the disclosure. The SaaS 400, shown in the form of a model graph, has been adjusted to meet the exemplary SLO. Stage 304 is replicated to have two stages: replicated stage 402 and replicated stage 404. Together, replicated stage 402 and replicated stage 404 can process two payloads (in parallel) in 500 milliseconds, and it is possible that the SaaS 400 can meet the SLO.

Modeling a SaaS

A SaaS is represented as a model. The model has a graph comprising interconnected stages for the SaaS, as the nodes of the graph. Nodes represent the constituent stages and edges can represent the flow of data through the stages. Each node has one or more metrics that can, either directly or in combination with other metrics, translate to one or more SLO of the overall SaaS. In other words, the metrics are associated with one or more service level objectives of the SaaS, and the metrics provide information whether one or more service level objectives can be met. A stage can have sub-stages therein. Each stage also has a replication factor, which specifies the number of copies the stage is replicated. Furthermore, each stage has a maximum replication factor, which specifies the maximum number of copies the stage can be replicated.

Model can be written in a suitable modeling language such as Extensible Markup Language (XML) or YAML or in other specification formats such as Topology and Orchestration Specification for Cloud Applications (TOSCA) (used for web services running in the cloud) or OpenStack Heat. For easier understanding, the following is a model written in YAML.

File: saas.yml
stage: saas
slo:
-  max throughput: 10 mbps
-  max users: 1000
-  max storage: 100GB
-  max delay: 1 sec
-  ...
max_replication_factor: 0 # means this module cannot be replicated.
sub-stages:
-  ingestion
∘  next: processing, backup
-  processing:
∘  next: storage
-  storage:
∘  next: NULL
-  backup:
∘  next: NULL
File: ingestion.yml
stage: ingestion
sla:
-  max throughput: 5 mbps
-  max users: 300
-  max storage: 100GB
max_replication_factor: 10 # 10 instances can be launched
sub-stages: ...
File: processing.yml
...
File: storage.yml
...
File: backup.yml
...

Installation Method: Modeling and Provisioning a SaaS

FIG. 5 is a flow diagram illustrating a method for optimizing and provisioning a SaaS, according to some embodiments of the disclosure. In 502, an optimizer (e.g., optimizer 112) can determine a graph comprising interconnected stages for the SaaS. The interconnected stages are the nodes of the graph. In some cases, the optimizer may be supplied with a definition for the graph which models a SaaS to be installed/deployed, or the optimizer may retrieve a definition for the graph modeling a SaaS to be installed/deployed. For the graph, each stage has a replication factor. The replication factor indicates the number of (parallel) instances of the stage for a given installation being considered. Furthermore, each stage has one or more metrics that are associated with one or more service level objectives of the SaaS.

In 504, to optimize the SaaS installation, the optimizer can determine a first replication factor associated with a first one of the stages which meets a first service level objective of the SaaS. In some embodiments, a replication factor of a given stage is adjusted, and the SLO of the SaaS is evaluated based on the adjusted replication factor to determine whether the SLO of the SaaS can be met. Adjusting the replication factor and evaluating the SLO can include determining whether the first service level is met based on the one or more metrics and the replication factor associated with each stage in the graph. When adjusting the replication factor for a given stage, the SLO can be evaluated based on the whole SaaS, i.e., all of the stages.

The following describes an exemplary method for determining replication factors that would meet an SLO relating to maximum throughput. Most SLOs can be handled in a similar fashion when it comes to calculating the dependency on the constituent stages. Suppose T is the maximum throughput that the SaaS needs to support. The SaaS can have n stages and each stage can support a maximum throughput of t_i. This means that the maximum throughput supported by the SaaS would be:

$T_{m\mathrm{ax}}=\underset{1\le i\le n}{\min }{t}_i$

To meet the SLO:T_max≥T

Put in words, each stage has a metric t_i which specifies the stage's maximum throughput. The maximum throughput of the SaaS would be the minimum of the metrics associated with the stages specifying the maximum throughput of the stages. To meet the overall SLO T (the maximum throughput that the SaaS needs to support), the minimum has to be greater than T.

Suppose there is a stage i such that t_i<T. If that stage is not scalable (as specified in the model as the maximum replication factor), the SaaS SLO for throughput cannot be achieved. However, if the stages are scalable, any one or more of the stages can be replicated to increase throughput. Micro services can be scaled horizontally, i.e., parallel micro services, leading to linear increase in performance (albeit at the cost of higher complexity of managing the multiple instances). To meet the SLA, the method can determine a replication factor of stage i, r_i as the smallest positive integer such that:

$\sum _{j=0}^{r_i}{t}_i^j\ge T.$

t_i^j is the throughput supported by the j^th instance of the stage. Determining replication factor and adjusting the replication factor can help meet the SLO. If identical copies/instances cannot be launched, the capacity of each copy can be considered. However, more often than not, all the instances of the same stage would have equal capability and hence previous equation can reduce to: r_it_i≥T.

In some embodiments, each stage has a maximum replication factor (“max_replication_factor”). The maximum replication factor limits how many instances a stage can be replicated. Determining the first replication factor can include checking whether the first replication factor exceeds the maximum replication factor. If the calculated replication factor is higher than the max_replication_factor specified in the model, it means that the target SLA cannot be achieved.

In 506, the optimizer can adjust the first replication factor associated with the first one of the stage based on the determined first replication factor. In 508, a provisioner (e.g., provisioner 114 of FIG. 1) can provision the SaaS onto networked computing resources based on the graph and replication factors associated with each stage. Phrased differently, the provisioner can create or provision instances of micro services and configuration thereof to implement the SaaS according to the determined replication factors.

The above example illustrates determining and adjusting the replication factor for a stage for a single SLO. In practice, there may be multiple SLO metrics, and the method is performed a number of SLOs, and the maximum of all the determined replication factors of a given stage for all the SLOs is considered as the replication factor for the given stage. In some embodiments, the method would further include determining one or more replication factors associated with the first one of the stages which meets one or more further service level objectives. Adjusting the first replication factor associated with the first one of the stages comprises adjusting the first one of the stages based on a maximum of the determined replication factors. For example, there might be 2 instances needed (determined replication factor of 2) to support the number of users (a first SLO) but 3 instances required (determined replication factor of 3) to support the required throughput (a second SLO). In such a scenario, the replication factor should be considered as 3, the maximum of the two determined replication factors (for meeting the first SLO and the second SLO respectively).

Besides computing the replication factor for one stage for a given SLO, the method is to be performed on every other stage to determine the appropriate replication factor to meet the SLO. In other words, besides determining a first replication factor for a given stage, the method would iterate through other stages in the graph to determine further replication factors for a given SLO. In some embodiments, the method would further include determining a second replication factor associated with a second one of the stages which meets the first service level objective of the SaaS, and adjusting the second replication factor associated with the second one of the stage based on the determined second replication factor. For example, in a pipeline with 3 stages and 2 SLO metrics, a total of 6 replication factors may have to be computed, i.e., 2 replication factors at each stage. The maximum replication factor from each stage would be selected as the final replication factor (3 replication factors, one for each stage).

Whenever stages have sub-stages, the computation of the replication factor for sub-stages may have similar computation to figure out the replication factor for each of the individual sub-stages so that the desired SLO can be achieved for the stage.

The following is an example of a SaaS model and the installation method implemented in pseudocode, which illustrates how to determine replication factors for all the stages and all the SLOs.

# Following is the replication count for each stage
replication = { }
#
# The following is data that is generated from parsing the YAML model
files
#
SaaS = {
‘slo’: {
‘throughput’: 100, # mbps
‘users’:      1000,
‘storage’:    100, # GB
‘delay’:      1, # sec
},
‘max_replication_factor’: 0 # there will be just one copy of the
entire SaaS
‘sub-stages’: {
‘ingestion’: {
‘next’: [“processing”]
‘sla’: {
‘throughput’: 10, # mbps
‘users’:      300,
‘storage’:    100, # GB
}
‘max_replication_factor’: 10 # max 10 copies
‘sub-stages’: {...}
},
‘processing’: {
‘next’: [“storage”]
‘slo’: {...},
‘max_replication_factor’: {...},
‘sub-stages’: {...},
},
‘storage’: {
‘next’: None
},
‘backup’: {
‘next’: None
}
}
}
def main( ):
for metric in SaaS[‘slo’].keys( ):
if type(metric) == “minmax”:
for stage in SaaS[‘sub-stages’].keys( ):
(rep, success) = get_replication_required(stage,
metric)
if (success):
replication[‘stage’] = max(replication[‘stage’],
rep)
else:
report error
abort or choose to continue
else: # type(metric) == “cumulative”
# Keep increasing replication till either we are out of
replication
# or metric is satisfied.
while (replications not exhausted):
for stage in SaaS[‘substages’].keys( ):
cumulative_metric += get_metric_value (stage,
metric)
if cumulative_metric is within required metric:
report success
break
else:
# replication can be increased by following either
# - “breadth first” policy (increase replication
once for
# all stages before increasing replication
again for
# first stage) or
# - “depth first” policy (max out replication for
first
# stage before increasing replication of next
stage).
increase_replication_of_a_stage( )

The installation method and its implementation are not trivial. First, there are certain metrics that cannot be handled in a straightforward manner. One example of such metric is delay. End-to-end delay keeps increasing with every stage. In order to keep delay within SLA bounds, either a single stage can be replicated or multiple stages can be replicated. It is not clear which one should be preferred. A maximum replication factor being set for each stage can prevent a single stage from replicating without limit to bring down delay. Rather, having maximum replication factors for each stage would enable the method to consider other stages for replication.

In some cases, the data flow deviates from an ideal pipeline (seen in many examples herein) and a single stage can forward data to two different stages, each stage performing different processing. For example, a processing stage can forward logs data to one stage and metrics data to another stage. One way to handle this is to include a split in the model itself (this split can be empirically determined or can be hard coded into the underlying model). The method can still be applied to a SaaS model with this property.

When the target SLO cannot be achieved, the installer method may stop with an error message. In some cases the installer may continue to install the SaaS according to the determined replication factors which can achieve only portion of the target SLO.

Load Balancer for Replicated Stages

Whenever, any instance is replicated, it may be beneficial to launch a load balancer to evenly distribute the incoming load among the multiple instances. In other words, the method of optimizing and provisioning the SaaS may include provisioning a load balancer in front of a stage whose replication factor was increased to meet service level objectives. FIG. 6 illustrates adding a load balancer to account for replication of stages, according to some embodiments of the disclosure. As seen in FIGS. 3 and 4, stage 304 is replicated into replicated stage 402 and replicated stage 404. To ensure that the replicated stages can process data in parallel in a balanced manner, the SaaS 600 includes a load balancer 602 between stage 302 and in front of replicated stage 402 and replicated stage 404 to distribute the data to the replicated stage 402 and replicated stage 404 evenly. It is assumed that the load balancer is itself not a bottleneck in the system. In practice, even with a single instance of a stage, it may be prudent to use a load balancer even if there is a remote possibility of increasing the number of instances. This also resolves domain name service (DNS) issues that would otherwise have to be handled.

Estimating Queue Length to Meet SLO

In some cases, metrics for a stage within a graph may not be available. For instance, some metrics may depend on the overall graph and dynamics of the stages. The method for installation may further include determining an estimated metric if the metric is not predefined. One example of such metric is queue length. FIG. 7 illustrates estimation of queue length as a metric for each one of the stages, according to some embodiments of the disclosure. The method for installation may further include determining a first metric associated with each stage by modeling each stage as a first in first out (FIFO) queue with exponentially distributed service time, wherein the first metric is queue length. This can be done for sub-stages as well. As seen in FIG. 7, various stages are modeled with respective queues (e.g., queue_302, queue_402, queue_404, and queue_308). The FIFO queue can be a M/M/1 queue, which represents the queue length in a system having a single server, where arrivals are determined by a Poisson process and job service times have an exponential distribution. In some embodiments, the service time for each stage can be calculated empirically from performance testing. Similarly, in the absence of knowledge of actual split of traffic from one stage to multiple stages (if any), the split ratio can also be determined empirically.

The entire SaaS having the various queues can then modeled as a Jackson Network, i.e., a network of M/M/1 FIFO queues where jobs enter and exit the network (as opposed to looping in the network). Properties of Jackson Networks can be used to estimate the queue lengths in each stage (and sub-stage(s)) of the SaaS, i.e., metrics for these stages. Based on the estimated metric, it is possible to determine replication factors which can meet a given SLO associated with that metric. For instance, the installation method can then calculate the replication factor for each stage to keep the queue length of each stage under some predefined number. For example, if the desired queue length for a stage is 50 (as an example of an SLO) whereas the calculated/estimated queue length (as an example of an estimated metric) is 100 (or is unstable, e.g., the arrival rate at the queue is higher than the service rate of the queue), a replication factor of 2 can ensure an expected queue length of 50 in each of the two instances of the stage.

Data Processing System

FIG. 8 depicts a block diagram illustrating an exemplary data processing system 800 (sometimes referred herein as a “node”) that may be used to implement the functionality associated with a SaaS, according to some embodiments of the disclosure. For instance, a SaaS manager, users (user machines) of the SaaS, the cloud (as seen in FIG. 1), and any networked hardware resources having one or more parts of a SaaS implemented thereon, may have one or more of the components of the system 800. As shown in FIG. 8, the data processing system 800 may include at least one processor 802 coupled to memory elements 804 through a system bus 806. As such, the data processing system may store program code within memory elements 804. Further, the processor 802 may execute the program code accessed from the memory elements 804 via a system bus 806. In one aspect, the data processing system may be implemented as a computer that is suitable for storing and/or executing program code. It should be appreciated, however, that the data processing system 800 may be implemented in the form of any system including a processor and a memory that is capable of performing the functions described within this specification.

The memory elements 804 may include one or more physical memory devices such as, for example, local memory 808 and one or more bulk storage devices 810. The local memory may refer to random access memory or other non-persistent memory device(s) generally used during actual execution of the program code. A bulk storage device may be implemented as a hard drive or other persistent data storage device. The processing system 800 may also include one or more cache memories (not shown) that provide temporary storage of at least some program code in order to reduce the number of times program code must be retrieved from the bulk storage device 810 during execution.

Input/output (I/O) devices depicted as an input device 812 and an output device 814 optionally can be coupled to the data processing system. User (machines) accessing the application implemented with the SaaS would typically have such I/O devices. Examples of input devices may include, but are not limited to, a keyboard, a pointing device such as a mouse, or the like. Examples of output devices may include, but are not limited to, a monitor or a display, speakers, or the like. Input and/or output devices may be coupled to the data processing system either directly or through intervening I/O controllers. In an embodiment, the input and the output devices may be implemented as a combined input/output device (illustrated in FIG. 8 with a dashed line surrounding the input device 812 and the output device 814). An example of such a combined device is a touch sensitive display, also sometimes referred to as a “touch screen display” or simply “touch screen”. In such an embodiment, input to the device may be provided by a movement of a physical object, such as e.g. a stylus or a finger of a user, on or near the touch screen display.

A network adapter 816 may also be coupled to the data processing system to enable it to become coupled to other systems, computer systems, remote network devices, and/or remote storage devices through intervening private or public networks. The network adapter may comprise a data receiver for receiving data that is transmitted by said systems, devices and/or networks to the data processing system 800, and a data transmitter for transmitting data from the data processing system 800 to said systems, devices and/or networks. Modems, cable modems, and Ethernet cards are examples of different types of network adapter that may be used with the data processing system 800.

As pictured in FIG. 8, the memory elements 804 may store an application 818. In various embodiments, the application 818 may be stored in the local memory 808, the one or more bulk storage devices 810, or apart from the local memory and the bulk storage devices. It should be appreciated that the data processing system 800 may further execute an operating system (not shown in FIG. 8) that can facilitate execution of the application 818. The application 818, being implemented in the form of executable program code, can be executed by the data processing system 800, e.g., by the processor 802. Responsive to executing the application, the data processing system 800 may be configured to perform one or more operations or method steps described herein.

Persons skilled in the art will recognize that while the elements 802-818 are shown in FIG. 8 as separate elements, in other embodiments their functionality could be implemented in lesser number of individual elements or distributed over a larger number of components.

Examples

Example 1 is a method for optimizing and provisioning a software-as-a-service (SaaS), the method comprising: determining a graph comprising interconnected stages for the SaaS, wherein each stage has a replication factor and one or more metrics that are associated with one or more service level objectives of the SaaS; determining a first replication factor associated with a first one of the stages which meets a first service level objective of the SaaS; adjusting the first replication factor associated with the first one of the stage based on the determined first replication factor; and provisioning the SaaS onto networked computing resources based on the graph and replication factors associated with each stage.

In Example 2, the method of Example 1 can further include: each stage having a maximum replication factor; and determining the first replication factor comprising checking whether the first replication factor exceeds the maximum replication factor.

In Example 3, the method of Example 1 or 2 can further include: determining a second replication factor associated with a second one of the stages which meets the first service level objective of the SaaS; and adjusting the second replication factor associated with the second one of the stage based on the determined second replication factor.

In Example 4, the method of any one of Examples 1-3 can further include determining the first replication factor associated with the first one of the stages which meets the first service level objective comprising determining whether the first service level is met based on the one or more metrics and the replication factor associated with each stage in the graph.

In Example 5, the method of any one of Examples 1-4 can further include: determining one or more replication factors associated with the first one of the stages which meets one or more further service level objectives; and wherein adjusting the first replication factor associated with the first one of the stages comprises adjusting the first one of the stages based on a maximum of the determined replication factors.

In Example 6, the method of any one of Examples 1-5 can further include provisioning the SaaS onto networked computing resources based on the graph and replication factors associated with each stage comprising provisioning a load balancer in front of a stage whose replication factor was increased to meet a service level objective.

In Example 7, the method of any one of Examples 1-6 can further include determining a first metric associated with each stage by modeling each stage as a first in first out queue with exponentially distributed service time, wherein the first metric is queue length.

Example 8 is a system comprising: at least one memory element; at least one processor coupled to the at least one memory element; and a software-as-a-service (SaaS) optimizer that when executed by the at least one processor is configured to: determine a graph comprising interconnected stages for a SaaS, wherein each stage has a replication factor and one or more metrics that are associated with one or more service level objectives of the SaaS; determine a first replication factor associated with a first one of the stages which meets a first service level objective of the SaaS; and adjust the first replication factor associated with the first one of the stage based on the determined first replication factor; and a SaaS provisioner that when executed by the at least one processor is configured to provision the SaaS onto networked computing resources based on the graph and replication factors associated with each stage.

In Example 9, the system of Example 8 can further include: each stage having a maximum replication factor; and determining the first replication factor comprising checking whether the first replication factor exceeds the maximum replication factor.

In Example 10, the system of Example 8 or 9 can further include the SaaS optimizer being further configured to: determine a second replication factor associated with a second one of the stages which meets the first service level objective of the SaaS; and adjust the second replication factor associated with the second one of the stage based on the determined second replication factor.

In Example 11, the system of any one of Examples 8-10 can further include determining the first replication factor associated with the first one of the stages which meets the first service level objective comprising determining whether the first service level is met based on the one or more metrics and the replication factor associated with each stage in the graph.

In Example 12, the system of any one of Examples 8-11 can further include the SaaS optimizer being further configured to: determine one or more replication factors associated with the first one of the stages which meets one or more further service level objectives; and wherein adjusting the first replication factor associated with the first one of the stages comprises adjusting the first one of the stages based on a maximum of the determined replication factors.

In Example 13, the system of any one of Examples 8-12 can further include provisioning the SaaS onto networked computing resources based on the graph and replication factors associated with each stage comprising provisioning a load balancer in front of a stage whose replication factor was increased to meet a service level objective.

In Example 14, the system of any one of Examples 8-13 can further include the SaaS optimizer being further configured to: determine a first metric associated with each stage by modeling each stage as a first in first out queue with exponentially distributed service time, wherein the first metric is queue length.

Example 15 is a computer-readable non-transitory medium comprising one or more instructions, for optimizing and provisioning a software-as-a-service (SaaS), that when executed on a processor configure the processor to perform one or more operations comprising: determining a graph comprising interconnected stages for the SaaS, wherein each stage has a replication factor and one or more metrics that are associated with one or more service level objectives of the SaaS; determining a first replication factor associated with a first one of the stages which meets a first service level objective of the SaaS; adjusting the first replication factor associated with the first one of the stage based on the determined first replication factor; and provisioning the SaaS onto networked computing resources based on the graph and replication factors associated with each stage.

In Example 16, the computer-readable non-transitory medium of Example 15 can further include: each stage having a maximum replication factor; and determining the first replication factor comprising checking whether the first replication factor exceeds the maximum replication factor.

In Example 17, the computer-readable non-transitory medium of Example 15 or 16 can further include: determining a second replication factor associated with a second one of the stages which meets the first service level objective of the SaaS; and adjusting the second replication factor associated with the second one of the stage based on the determined second replication factor.

In Example 18, the computer-readable non-transitory medium of any one of Examples 15-17 can further include determining the first replication factor associated with the first one of the stages which meets the first service level objective comprising determining whether the first service level is met based on the one or more metrics and the replication factor associated with each stage in the graph.

In Example 19, the computer-readable non-transitory medium of any one of Examples 15-18 can further include the operations further comprising: determining one or more replication factors associated with the first one of the stages which meets one or more further service level objectives; and wherein adjusting the first replication factor associated with the first one of the stages comprises adjusting the first one of the stages based on a maximum of the determined replication factors.

In Example 20, the computer-readable non-transitory medium of any one of Examples 15-19 can further include provisioning the SaaS onto networked computing resources based on the graph and replication factors associated with each stage comprising provisioning a load balancer in front of a stage whose replication factor was increased to meet a service level objective.

In Example 21, the computer-readable non-transitory medium of any one of Examples 15-20 can further include the operations further comprising determining a first metric associated with each stage by modeling each stage as a first in first out queue with exponentially distributed service time, wherein the first metric is queue length.

Example 20 is an apparatus comprising means for implementing and/or carrying out any one of the methods in Examples 1-7.

Variations and Implementations

Within the context of the disclosure, the cloud includes a network used herein represents a series of points, nodes, or network elements of interconnected communication paths for receiving and transmitting packets of information that propagate through a communication system. A network offers communicative interface between sources and/or hosts, and may be any local area network (LAN), wireless local area network (WLAN), metropolitan area network (MAN), Intranet, Extranet, Internet, WAN, virtual private network (VPN), or any other appropriate architecture or system that facilitates communications in a network environment depending on the network topology. A network can comprise any number of hardware or software elements coupled to (and in communication with) each other through a communications medium.

As used herein in this Specification, the term ‘network element’ or ‘node’ in the cloud is meant to encompass any of the aforementioned elements, as well as servers (physical or virtually implemented on physical hardware), machines (physical or virtually implemented on physical hardware), end user devices, routers, switches, cable boxes, gateways, bridges, loadbalancers, firewalls, inline service nodes, proxies, processors, modules, or any other suitable device, component, element, proprietary appliance, or object operable to exchange, receive, and transmit information in a network environment. These network elements may include any suitable hardware, software, components, modules, interfaces, or objects that facilitate the disclosed operations. This may be inclusive of appropriate algorithms and communication protocols that allow for the effective exchange of data or information.

In one implementation, SaaS manager described herein may include software to achieve (or to foster) the functions discussed herein for optimizing and provisioning a SaaS (also referenced herein as installation) where the software is executed on one or more processors to carry out the functions. This could include the implementation of instances of an optimizer, provisioner, and/or any other suitable element that would foster the activities discussed herein. Additionally, each of these elements can have an internal structure (e.g., a processor, a memory element, etc.) to facilitate some of the operations described herein. Exemplary internal structure includes elements shown in data processing system in FIG. 8. In other embodiments, these functions for optimizing and provisioning a SaaS may be executed externally to these elements, or included in some other network element to achieve the intended functionality. Alternatively, the SaaS manager may include software (or reciprocating software) that can coordinate with other network elements in order to achieve the SaaS optimization and provisioning functions described herein. In still other embodiments, one or several devices may include any suitable algorithms, hardware, software, components, modules, interfaces, or objects that facilitate the operations thereof.

In certain example implementations, the functions outlined herein may be implemented by logic encoded in one or more non-transitory, tangible media (e.g., embedded logic provided in an application specific integrated circuit [ASIC], digital signal processor [DSP] instructions, software [potentially inclusive of object code and source code] to be executed by one or more processors, or other similar machine, etc.). In some of these instances, one or more memory elements can store data used for the operations described herein. This includes the memory element being able to store instructions (e.g., software, code, etc.) that are executed to carry out the activities described in this Specification. The memory element is further configured to store information such as graph definitions, metrics, SLOs/SLAs, and replication factors disclosed herein. The processor can execute any type of instructions associated with the data to achieve the operations detailed herein in this Specification. In one example, the processor could transform an element or an article (e.g., data) from one state or thing to another state or thing. In another example, the activities outlined herein may be implemented with fixed logic or programmable logic (e.g., software/computer instructions executed by the processor) and the elements identified herein could be some type of a programmable processor, programmable digital logic (e.g., a field programmable gate array [FPGA], an erasable programmable read only memory (EPROM), an electrically erasable programmable ROM (EEPROM)) or an ASIC that includes digital logic, software, code, electronic instructions, or any suitable combination thereof.

Any of these elements (e.g., the network elements, etc.) can include memory elements for storing information to be used in achieving the optimization functions, as outlined herein. Additionally, each of these devices may include a processor that can execute software or an algorithm to perform the optimization activities as discussed in this Specification. These devices may further keep information in any suitable memory element [random access memory (RAM), ROM, EPROM, EEPROM, ASIC, etc.], software, hardware, or in any other suitable component, device, element, or object where appropriate and based on particular needs. Any of the memory items discussed herein should be construed as being encompassed within the broad term ‘memory element.’ Similarly, any of the potential processing elements, modules, and machines described in this Specification should be construed as being encompassed within the broad term ‘processor.’ Each of the network elements can also include suitable interfaces for receiving, transmitting, and/or otherwise communicating data or information in a network environment.

Additionally, it should be noted that with the examples provided above, interaction may be described in terms of two, three, or four network elements. However, this has been done for purposes of clarity and example only. In certain cases, it may be easier to describe one or more of the functionalities of a given set of flows by only referencing a limited number of network elements. It should be appreciated that the systems described herein are readily scalable and, further, can accommodate a large number of components, as well as more complicated/sophisticated arrangements and configurations. Accordingly, the examples provided should not limit the scope or inhibit the broad techniques of SaaS optimization and provisioning, as potentially applied to a myriad of other architectures.

It is also important to note that the parts of the flow diagram in the FIG. 5 illustrate only some of the possible scenarios that may be executed by, or within, the components shown (e.g., in FIGS. 1 and 8) and described herein. Some of these steps may be deleted or removed where appropriate, or these steps may be modified or changed considerably without departing from the scope of the present disclosure. In addition, a number of these operations have been described as being executed concurrently with, or in parallel to, one or more additional operations. However, the timing of these operations may be altered considerably. The preceding operational flows have been offered for purposes of example and discussion. Substantial flexibility is provided by the components shown and described herein, in that any suitable arrangements, chronologies, configurations, and timing mechanisms may be provided without departing from the teachings of the present disclosure.

The term “system” is used generically herein to describe any number of components, elements, sub-systems, devices, packet switch elements, packet switches, routers, networks, computer and/or communication devices or mechanisms, or combinations of components thereof. The term “computer” is used generically herein to describe any number of computers, including, but not limited to personal computers, embedded processing elements and systems, control logic, ASICs, chips, workstations, mainframes, etc. The term “processing element” is used generically herein to describe any type of processing mechanism or device, such as a processor, ASIC, field programmable gate array, computer, etc. The term “device” is used generically herein to describe any type of mechanism, including a computer or system or component thereof. The terms “task” and “process” are used generically herein to describe any type of running program, including, but not limited to a computer process, task, thread, executing application, operating system, user process, device driver, native code, machine or other language, etc., and can be interactive and/or non-interactive, executing locally and/or remotely, executing in foreground and/or background, executing in the user and/or operating system address spaces, a routine of a library and/or standalone application, and is not limited to any particular memory partitioning technique. The steps, connections, and processing of signals and information illustrated in the FIGURES, including, but not limited to any block and flow diagrams and message sequence charts, may typically be performed in the same or in a different serial or parallel ordering and/or by different components and/or processes, threads, etc., and/or over different connections and be combined with other functions in other embodiments, unless this disables the embodiment or a sequence is explicitly or implicitly required (e.g., for a sequence of read the value, process the value—the value must be obtained prior to processing it, although some of the associated processing may be performed prior to, concurrently with, and/or after the read operation). Furthermore, the term “identify” is used generically to describe any manner or mechanism for directly or indirectly ascertaining something, which may include, but is not limited to receiving, retrieving from memory, determining, defining, calculating, generating, etc.

Moreover, the terms “network” and “communications mechanism” are used generically herein to describe one or more networks, communications mediums or communications systems, including, but not limited to the Internet, private or public telephone, cellular, wireless, satellite, cable, local area, metropolitan area and/or wide area networks, a cable, electrical connection, bus, etc., and internal communications mechanisms such as message passing, interprocess communications, shared memory, etc. The term “message” is used generically herein to describe a piece of information which may or may not be, but is typically communicated via one or more communication mechanisms of any type.

Numerous other changes, substitutions, variations, alterations, and modifications may be ascertained to one skilled in the art and it is intended that the present disclosure encompass all such changes, substitutions, variations, alterations, and modifications as falling within the scope of the appended claims. In order to assist the United States Patent and Trademark Office (USPTO) and, additionally, any readers of any patent issued on this application in interpreting the claims appended hereto, Applicant wishes to note that the Applicant: (a) does not intend any of the appended claims to invoke paragraph six (6) of 35 U.S.C. section 112 as it exists on the date of the filing hereof unless the words “means for” or “step for” are specifically used in the particular claims; and (b) does not intend, by any statement in the specification, to limit this disclosure in any way that is not otherwise reflected in the appended claims.

One or more advantages mentioned herein does not in any way suggest that any one of the embodiments necessarily provides all the described advantages or that all the embodiments of the present disclosure necessarily provide any one of the described advantages.

Claims

1. A method for optimizing and provisioning a software-as-a-service (SaaS), the method comprising: determining a graph with interconnected stages for the SaaS, each of the stages including one or more metrics associated with one or more service level objectives of the SaaS;determining a plurality of replication factors, each of the plurality of replication factors corresponding to one of the stages and one of the service level objectives of the SaaS;increasing a replication factor of one of the stages so that a respective metric corresponding to each one of multiple replicated stages of the one of the stages meets a threshold;enabling parallel processing of data via the multiple replicated stages of the one of the stages by adding a load balancer in front of the multiple replicated stages of the one of the stages, the load balancer configured to distribute the data between the multiple replicated stages of the one of the stages for the parallel processing of the data; andprovisioning the SaaS onto networked computing resources based on the graph and the plurality of replication factors.

2. The method of claim 1, wherein, each of the stages has one of a plurality of maximum replication factors; andthe determining of the plurality of replication factors includes checking whether each of the plurality of replication factors exceeds a corresponding one of the maximum replication factors.

3. The method of claim 1, wherein the determining of the plurality of replication factors includes determining whether a first service level is met.

4. The method of claim 1, wherein the one or more of the plurality of replication factors is increased based on a maximum replication factor.

5. The method of claim 1, further comprising: provisioning another load balancer in front of a stage whose replication factor was not increased, but can be increased.

6. The method of claim 1, further comprising: determining a queue length metric associated with each of the stages by modeling each of the stages as a first in first out queue with exponentially distributed service time.

7. A system comprising: at least one memory element;at least one processor coupled to the at least one memory element; anda software-as-a-service (SaaS) optimizer that when executed by the at least one processor is configured to: determine a graph with interconnected stages for a SaaS, each of the stages including one or more metrics associated with one or more service level objectives of the SaaS;increase a replication factor of one of the stages so that a respective metric corresponding to each one of multiple replicated stages of the one of the stages meets a threshold;determine a plurality of replication factors, each of the plurality of replication factors corresponding to one of the stages and one of the service level objectives of the SaaS; andenable parallel processing of data via the multiple replicated stages of the one of the stages by adding a load balancer in front of the multiple replicated stages of the one of the stages, the load balancer configured to distribute the data between the multiple replicated stages of the one of the stages for the parallel processing of the data; anda SaaS provisioner that when executed by the at least one processor is configured to provision the SaaS onto networked computing resources based on the graph and the plurality of replication factors.

8. The system of claim 7, wherein: each of the stages has one of a plurality of maximum replication factors; anddetermining the plurality of replication factors includes checking whether each of the plurality of replication factors exceeds a corresponding one of the plurality of maximum replication factors.

9. The system of claim 7, wherein determining the plurality of replication factors includes determining whether the service level is met.

10. The system of claim 7, wherein increasing the one or more of the plurality of replication factors is based on a maximum replication factor.

11. The system of claim 7, wherein the SaaS optimizer is further configured to: provision another load balancer in front of a stage whose replication factor was not increased, but can be increased.

12. The system of claim 7, wherein the SaaS optimizer is further configured to: determine a queue length metric associated with each of the stages by modeling each of the stages as a first in first out queue with exponentially distributed service time.

13. A computer-readable non-transitory medium comprising one or more instructions, for optimizing and provisioning a software-as-a-service (SaaS), that when executed on a processor configure the processor to perform one or more operations comprising: determining a graph with interconnected stages for the SaaS, each of the stages including one or more metrics associated with one or more service level objectives of the SaaS;determining a plurality of replication factors, each of the plurality of replication factors corresponding to one of the stages and one of the service level objectives of the SaaS;increasing a replication factor of one of the stages so that a respective metric corresponding to each one of multiple replicated stages of the one of the stages meets a threshold;enabling parallel processing of data via the multiple replicated stages of the one of the stages by adding a load balancer in front of the multiple replicated stages of the one of the stages, the load balancer configured to distribute the data between the multiple replicated stages of the one of the stages for the parallel processing of the data; andprovisioning the SaaS onto networked computing resources based on the graph and the plurality of replication factors.

14. The computer-readable non-transitory medium of claim 13, wherein: each of the stages has one of a plurality of maximum replication factors; andthe determining of the plurality of replication factors includes checking whether each one of the plurality of replication factors exceeds a corresponding one of the plurality of maximum replication factors.

15. The computer-readable non-transitory medium of claim 13, wherein determining the plurality of replication factors includes determining whether the service level is met.

16. The computer-readable non-transitory medium of claim 13, wherein increasing of the one or more of the plurality of replication factors is increased based on a maximum replication factor.

17. The computer-readable non-transitory medium of claim 13, wherein the operations further comprise: determining a queue length metric associated with each of the stages by modeling each of the stages as a first in first out queue with exponentially distributed service time.