Patent Application
20120221373
Queueing models are used to mathematically predict response times of business services in capacity planning scenarios, which are used to understand the attributes of service as well as meet any service level agreement (SLA) performance targets. In general, queueing models require cumbersome predictive modeling validation steps and only predict mean response time values. Additionally, queuing models make simplifying assumptions which may not hold in actual practice. Further, many queueing models may be required for compatibility with trace-based planning methods that consider time varying system behavior.
In order to eliminate the predictive modeling validation steps, empirical models may be used to mathematically predict response times of business services in capacity planning scenarios. However, most empirical models use average performance metrics, such as mean response time, for capacity planning scenarios. Average performance guarantees typically are not sufficient to express SLA requirements for many applications, particularly interactive applications. SLAs often reference percentile performance guarantees, such that end users receive a percentage of response times below an agreed upon threshold. Thus, empirical models using average performance guarantees are not effective in capacity planning scenarios because they may not satisfy typical SLA requirements.
Certain exemplary embodiments are described in the following detailed description and in reference to the drawings, in which:
Embodiments of the invention provide an estimate of business service responsiveness based on historical measures using empirical data. Additionally, embodiments of the present invention operate with minimal domain knowledge. Further, an embodiment of the present invention can use a trace-based capacity planning methodology to estimate the impact of planning alternatives on business service responsiveness. Planning alternatives may include utilizations of allocation to achieve certain response time objectives or predicting the impact of different consolidation scenarios on response times using a particular model. Predicting may include, but is not limited to, rendering the model or otherwise reporting the model data distribution.
The ability to accurately estimate resources required to service a particular workload while meeting performance targets, which may be specified in SLAs, helps to provide effective resource utilization. Accurate estimates of the resources required to service a particular workload may minimize servicing costs. Overestimating the resources required may result in an over-provisioned system with low utilization and idle resources. Conversely, underestimation of the resources needed may result in poor performance leading to possible violation of the SLA.
A quantile modeling approach to building a model that relates an application performance metric, such as response time, to resource utilization using historical traces of resource usage metrics, such as the CPU or memory accurately estimates the resources required to service a particular workload while meeting performance targets. Particularly, modeling the probability distribution of an application performance metric conditioned on one or more input variables that are measured or controlled, such as system resource utilization and allocation metrics, allows for a more accurate model. With a probabilistic model, the probability of satisfying a percentile performance requirement can be calculated given knowledge of the input variables during a particular time interval. The model may be referred to as a Utilization of Allocation to Response time (UA2R) model.
In an embodiment, a model that relates observed allocation usage (or utilizations of allocations) and business service response times is built. A model for the distribution of response time values is maintained for a business service, its workloads, and a range of values of utilization of allocations. The model is then used to provide insights to a capacity planner when selecting target utilization of allocation values. Additionally, the model can also be used by planning tools to report on the expected response time of a business service for some planning scenario.
Consider the use of the UA2R model for a virtual machine (VM) in the following three scenarios. First, suppose an allocation for a VM always stays the same and the VM never gets more or less than its allocation. If the future trace of utilization of allocations is the same as previous utilization of allocations, then the response time distribution may not change. Further, assume the demands and, as a result, utilizations increase by some percentage due to an expected or planned uniform increase in workload. In this scenario, the UA2R model predicts the impact on the response time distribution. Second, suppose that the allocation for a VM is to be changed. The utilizations then increase or decrease proportionally, and the UA2R model predicts the impact on the response time distribution. Third, suppose allocations are not enforced and that servers may be oversubscribed. For this scenario, the historical allocations change depending on contention for the shared server. Calibration of the UA2R model may use “effective” allocation as input. Effective allocation is the time varying capacity that a VM has access to as a result of the observed competition incurred for shared resources. The utilization is the utilization of the effective allocation and the response time is as observed. In this third scenario, the UA2R model may still track the relationship between response times and utilization of allocation.
A UA2R model tracks response time distribution. In particular, instead of modeling the mean value, quantiles of response time are modeled. Since SLAs typically specify performance guarantees in percentiles, such as a performance guarantee that some percentage (for example, ninety-five percent) of end users receive response times below an agreed upon threshold, percentiles may be a more useful metric to model than the mean values. These percentiles may be modeled through quantile regression. As with conventional regression analysis, quantile regression optimizes the parameters or coefficients for a specified functional form such that the function models a certain characteristic of the data. While conventional regression analysis minimizes the sum of squared residuals (or errors) to generate a model of the mean conditioned on a set of variables, an alternative may be to minimize the sum of absolute residuals to yield a model of the median, for example, the 0.5 quantile (50th percentile). Likewise, to obtain models for other quantiles, asymmetric weights may be applied to the absolute value of positive and negative residuals. For example, weighting positive residuals three times as much as negative residuals produces a model for the 0.75 quantile (75th percentile). This optimization problem can be solved using linear programming methods like Simplex.
A model quantile q of the response time as a function of utilization of allocation is mathematically described below:
RT
q
=F(u) (1)
q=P(t<RTq) (2)
In other words, the probability that the predicted response time is less than the modeled response time is q. Note that the utilization of allocation u can be a vector if the response time depends on utilization of multiple resources.
To fit the experimental data, three parametric functional forms may be evaluated: linear (3), exponential (4), and a combination of linear and exponential (5).
F(u)=a+b*u (3)
F(u)=exp(a+b*u) (4)
F(u)=a+b*u+exp(c+d*u) (5)
Variables a and b may be determined with training data during the model generation. If the relationship between utilization and response time is linear, then the linear equation (3) may be used for the model. Likewise, if the relationship between utilization and response time is exponential, then the exponential equation (4) may be used for the model. The combination of equations (5) is able to model both relationships, however, determining the values of a, b, c, and d may be more difficult. Also, note that test data may also be used to determine the parametric functional form selected. The relationship between utilization and response time does not need to be defined prior to selecting a functional form to use for the model. Further, several models with different functional forms may be trained, and the model that performs the best on the unseen test data may be selected.
The function used may take any form, however, these exemplary forms were chosen since response time versus utilization tends to show linear behavior at low utilizations and exponential at high utilizations. However, a non-linear model such as equation (5) requires solution of a more complex optimization problem to determine the coefficients, and in some cases it may be hard to reach convergence. Note that equation (4) is effectively a linear model if the data is transformed by taking logarithm of the response time.
Another performance metric, such as request throughput, could be modeled with one or more utilization values as input data. Additional metrics may include dropped or cancelled requests. These additional metrics are like response times because the relationship is expected to be non-linear. For some systems, throughput may also be non-linear with respect to utilizations, and as a result, throughput may also a reasonable metric. For example, SAP systems have demands per request that tend to decrease as utilization increases. Modeling throughput may show this non-linear relationship. Additionally, such a model may be used to efficiently provision for SLA request throughput targets, if any.
At block 106, the input data is partitioned. Partitioning the input data may include splitting the input data into a training set and a test set. The training set is used for determining the parameters of the model, while the test set is used for evaluating the model's performance. At block 108, a model that predicts responsiveness is generated based on the training data. The model may be generated by fitting the training data set to a suitable parametric form and using quantile regression to build the model. The model generated may be based on include a linear model, an exponential model, and or a combination of linear and exponential models.
At block 110, the model is evaluated using the test set. Metrics such as absolute mean error are used for quantifying the performance of a model. Once the model is prepared, it can be used to support resource planning exercises, such as predicting a business service response time distribution. Additionally, a user of the model may relate the desired business service responsiveness to the service's historical utilization of its resource allocations. That information can then be used to establish a requirement for utilization of allocation that meets the desired response time goals. The requirement can be used for planning purposes in accordance with SLAs. Alternatively, a particular resource management plan may lead to a particular behavior for utilization of allocation. The model can then be used to transform the plan's utilization of allocations to a prediction for business service responsiveness.
The 100% measured data case at 202 is virtually the same as the 100% modeled data case at 204. Likewise, the 25% measured data case at 206 is virtually the same as the 25% modeled data case at 208. However, the 25% consolidated measured data case at 210 is distinguishable from the 25% consolidated modeled data case at 212. Although the allocations in the consolidated measured data case 210 and consolidated modeled data case 212 are the same, their response time distribution differs due to contention. The consolidated measured data case 210 and consolidated modeled data case 212 are not identical, but the predicted behavior is sufficient for capacity planning.
The system 300 may include a server 302, and one or more client computers 304, in communication over a network 306. As illustrated in
The network 306 may be a local area network (LAN), a wide area network (WAN), or another network configuration. The network 306 may include routers, switches, modems, or any other kind of interface device used for interconnection. The network 306 may connect to several client computers 304. Through the network 306, several client computers 304 may connect to the server 302. The client computers 304 may be similarly structured as the server 302.
The server 302 may have other units operatively coupled to the processor 308 through the bus 310. These units may include tangible, machine-readable storage media, such as storage 322. The storage 322 may include any combinations of hard drives, read-only memory (ROM), random access memory (RAM), RAM drives, flash drives, optical drives, cache memory, and the like. The storage 322 may include the software used in an embodiment of the present techniques. In an embodiment, the model generated may reside in storage 322. The database management system (DBMS) 324 may be used to store historical data according to an embodiment of the present techniques. Although the DBMS 324 is shown to reside on server 302, a person of ordinary skill in the art would appreciate that the DBMS 324 may reside on the server 302 or any of the client computers 304.
The non-transitory, computer-readable medium 400 may correspond to any typical storage device that stores computer-implemented instructions, such as programming code or the like. For example, the non-transitory, computer-readable medium 400 may include one or more of a non-volatile memory, a volatile memory, and/or one or more storage devices.
Examples of non-volatile memory include, but are not limited to, electrically erasable programmable read only memory (EEPROM) and read only memory (ROM). Examples of volatile memory include, but are not limited to, static random access memory (SRAM), and dynamic random access memory (DRAM). Examples of storage devices include, but are not limited to, hard disk drives, compact disc drives, digital versatile disc drives, and flash memory devices.
A processor 402 generally retrieves and executes the computer-implemented instructions stored in the non-transitory, computer-readable medium 400 to estimating business service responsiveness. Input data may be gathered. The data may be partitioned into a plurality of data sets. A model is generated based on at least one data set, and the model is evaluated based on another data set. A business service response time may be predicted.
