The present disclosure generally relates to managing service level agreements for Web services.
Initially, content published on the World Wide Web was in the form of static pages that were downloaded to a user's browser. The browser interpreted the page for display, as well as handling user input to objects such as forms or buttons. Recently, “Web services” have been used to extend the Web's capability to provide dynamic content that is accessible by other programs besides browsers.
Web services are network-based (particularly Internet-based) applications that perform a specific task and conform to a specific technical format. Web services are represented by a stack of emerging standards that describe a service-oriented, application architecture, collectively providing a distributed computing paradigm having a particular focus on delivering services across the Internet.
Generally, Web services are implemented. as self-contained modular applications that can be published in a ready-to-use format, located, and invoked across the World Wide Web. When a Web service is deployed, other applications and Web services can locate and invoke the deployed service. They can perform a variety of functions, ranging from simple requests to complicated business processes.
Web services are typically configured to use standard Web protocols such as Hypertext Transfer Protocol (HTTP), Hypertext Markup Language (HTML), Extensible Markup Language (XML) and Simplified Object Access Protocol (SOAP). HTTP is an application-level protocol commonly used to transport data on the Web. HTML and XML are formatting protocols typically used to handle user input, encapsulate user data, and format output for display. SOAP is a remote procedure call (RPC) and document exchange protocol often used for requesting and replying to messages between Web services.
The use of Web services has made the browser a much more powerful tool. Far from being simple static Web pages, Web services can handle tasks as complex as any computer program, yet can be accessed and run most anywhere due to the ubiquity of browsers and the Internet.
A composite Web service is composed of multiple Web services. For purposes of further discussion herein, a composite Web service is referred to as a Web service, and the constituent Web services of the composite Web service are referred to as Web service components or stages. The composite Web service entails the overall work that is to be performed by the collection of stages. For example, a composite Web service may support the purchase of a piece of equipment, and the stages may handle the submission of a purchase order, parts management, assembly management, delivery management, and payment management.
One or more service level agreements (SLAs) may be associated with a Web service. An SLA defines the quality of service offered by a provider to a customer under a given set of circumstances. For example, an SLA may require that 90% of operations executed between the hours of 9:00 a.m. and 5:00 p.m. be completed within three seconds.
It is essential for service providers to satisfy SLAs. Whether a service provider satisfies its SLAs plays a large part in customers' perceptions of the provider. Furthermore, SLAs may be contractual terms, and failure to satisfy an SLA may be a breach of a contract. Thus, not only may failure to satisfy an SLA result in customer defections, but there may be costs direct incurred from failing to meet contract obligations.
The various embodiments of the invention relate to managing service level agreements (SLAs) associated with a composite Web service. An example of a composite Web service is a business process. For each completed process instance, the status data logged in executing the process instance is analyzed to determine whether the process instance satisfied the service level agreement. The violation/satisfaction data and the logged status data are then used to construct a classification model, which in one embodiment may be a decision tree called the explanatory decision tree. Each node in the explanatory decision tree represents at least one attribute of the process instances, each branch from a node represents a subset of attribute values of the attribute of the node, and each leaf node indicates a probability value that process instances having attribute values consistent with the attribute values in nodes on a path to the leaf node fail to satisfy or violate the SLA. Data that represents the decision tree may then be output to explain past violations of the SLA. Classification models may be used to explain patterns of violations of service level agreements in terms of the attributes of Web service process instances and of the attributes of the entities processed by those process instances.
It will be appreciated that various other embodiments are set forth in the Detailed Description and Claims which follow.
Service providers may greatly benefit from the capability of monitoring service level agreement (SLA) violations, understanding why SLAs are not satisfied, and forecasting whether an SLA will be violated. These aspects of SLA management may assist a service provider in correcting or circumventing potential problems. The various embodiments of the present invention provide assistance in managing SLAs. The management techniques described herein provide information on describing what has happened in a Web service relative to compliance to the defined SLAs, information that may be useful in determining why an SLA has been violated, and information that indicates what probably will happen relative to compliance with SLAs.
Generally, status data logged in association with a Web service is examined for violations of SLAs. This SLA violation information may then be used to explain in the aggregate why SLA violations have occurred. In one embodiment, a decision tree is used to explain SLA violations in terms of logged status data of the process instances. The overall processing of a composite Web service from an initial input data set through to completion is referenced herein as a process instance. Each node in the decision tree represents at least one attribute of the process instances, each branch from a node represents a subset of attribute values of the attribute of the node, and each leaf node indicates a probability value that process instances will fail to satisfy the service level agreement for process instances having attribute values consistent with the attribute values in nodes on a path to the leaf node.
In addition to explaining SLA violations, further processing may be performed to predict which SLAs will be violated by active (or “running”) process instances. Each prediction is made in terms of the relative probability that an active process instance will violate an SLA given the process instance's logged status data and how similar process instances have fared, considering only logged status data of the similar process instances up to the current stage of execution of the process instance. The execution of each component of a composite Web service corresponds to a different execution stage of the process instance. In an example embodiment, decision trees are used to formulate and illustrate the predictions.
An interface definition informs a customer how to interact with the buy_ties Web service. An abstract specification of the interface is as follows:
The prefix, in, indicates that the operation is invoked by the customer, and out indicates that the operation is invoked by the provider. The names of the interface operations, for example, quote_request and names of parameters are self-explanatory. The operations of the interface correspond to different execution stages of the process. The stages that have no clear correspondence to operations of the interface definition are stages performed by the provider and beyond the view of the customer.
The process of
Customers' expectations of the quality of service in initiating the buy_ties service may be codified in one or more SLAs. SLAs may be defined by business persons and formalized for use in an SLA management system by a technical manager of the Web service. Alternatively, a user interface may be adapted to remove the need for technical knowledge and allow the business persons to directly codify the SLAs.
In one embodiment, the metrics underlying SLAs may be codified in structured query language (SQL) statements that can be run on status data logged by the Web service.
SLA is sometimes used as the abbreviated term for an SLA clause. A combination of SLA clauses is referred to as a composite SLA. Each SLA clause specifies a Boolean condition relative to Web service logged status data. For example, a first clause may state that the provider guarantees that the time to deliver ties to customer ABC will not exceed 48 hours from the time the order is received by the provider to the time the goods arrive to the customer, assuming that there is sufficient stock available and shipping is available. A second clause may state that the provider guarantees a response time of 5 minutes from the time that a quote is requested by customer ABC to the time that the provider sends the quote.
Status data logged by the Web service management system (not shown) is used to determine whether the Web service has violated the two example SLA clauses (given above) for a process instance. For each clause there is a corresponding metric that is computed from the logged status data. The SLA clause defines a Boolean condition on the metric that must evaluate to true for the SLA clause to be satisfied. While not shown it will be appreciated that various proprietary or commercially available Web service management systems may be used in coordinating the activities and logging status data for a composite Web service.
For both the first and second clauses the metric is the duration, which is computed over different intervals according to the clause. For the first clause, the duration is computed over the interval that begins at the time at which an order is received (receive order request stage 18) and ends at the time at which a shipment confirmation is sent (send confirm shipment stage 32). Each stage has start_time and end_time parameters, and in the present example, the start_time of the receive order request stage 18 and the end_time of the send confirm shipment stage 32 are used to determine the duration. Example SQL statements for this SLA clause may be specified as:
%NODE 1 is the identifier of the node corresponding to stage 18 and %NODE2 is the identifier for stage 32. This SQL may be transformed into a more complex formulation required for optimizing the computation of metrics and referencing any additional tables used in the implementation.
For the second clause, the duration is computed on the interval that begins with the start_time of the receive quote request stage 12 and ends with the end_time of the send quote stage 16. The example SQL for the second clause is similar to that shown above for the first clause, except the values of the parameters %NODE1 and %NODE2 are identifiers of stages 12 and 15, respectively.
Once the SLA clauses have been defined, the logged status data of the Web service may be analyzed to determine whether the SLA was satisfied or violated for each process instance by measuring for each process instance the metric called out in each SLA clause. This allows business persons to monitor the quality of service provided to various customers in terms of the SLAs.
SLA clauses may be classified according to several characteristics, and the classification may be used to facilitate SLA definition and management. One way in which SLA clauses may be classified is by the metric that defines the clause. Example metrics include duration, data value, path, count, or resource. An SLA clause that involves the duration generally requires that the time between two stages of the Web service is equal to, less than, or greater than a certain threshold value as in the two example SLA clauses described above.
An SLA clause that involves a data value is a condition on a variable associated with a process instance. For example, this type of SLA clause may require that at least three quality assurance consultants are named for orders for a quantity that exceeds a selected threshold value. Both the number of consultants and the order quantity are data values that may be determined from the logged status data.
An SLA clause that involves a path requires that a process instance takes a given path to execute specific stage(s) of the Web service. For example, an SLA clause may require that certain types of orders are shipped for overnight delivery. This implies that a stage involving overnight shipment must be executed for qualifying orders.
An SLA clause that involves a count requires that a specified stage is activated a specified number of times for each process instance. For example, a quality assurance stage may need to be repeated a certain number of times for orders from certain customers.
An SLA clause that involves a resource requires that a specified stage of the Web service is executed by a specified resource. For example, an SLA clause may require that projects submitted by certain employees be first reviewed by selected team members.
It will be appreciated that SLA clauses may be members of more than. one class. For example, a clause might state that the delivery time for orders having values. greater than $1000 cannot exceed 20 days. This example SLA clause involves both data and duration metrics.
Yet another classification of SLA clauses is whether the clause is general or specific. A general SLA clause involves analyzing the status data logged for many process instances, and a specific SLA clause involves analyzing the status data logged for a single process instance. The previously presented examples relate to specific SLA clauses. A generic SLA clause might state that 95% of all orders should be delivered within 3 days, and no delivery can exceed 30 days.
It will be appreciated that additional classifications are possible depending on the characteristics of the Web service. In addition, SLA classifications may be constructed based on whether the metrics tend to relate more to business-level issues or relate more to technology-related issues. Example business-related issues include the first and second SLA clauses described above. An example technology-related issue might state that each stage in the Web service require no more than 3 seconds to complete during normal business hours.
The SLA definition tool 102 provides the interface through which a user defines the SLAs against which the status data of process instances is to be evaluated. The definitions may be explicitly entered by a user or defined by parameterizing selected functions from function library 118. SLAs specified by a user may be in the form of SQL statements, for example.
To assist in quickly defining SLAs, the function library consists of predefined, parameterizable functions that enable the computation of many SLAs. The functions may be grouped by the class of SLA to which the functions are applicable, which assists the user in quickly identifying the appropriate function to use for a particular SLA clause. For example, one function might be distanceGreaterThan(S1, S2, T). This function returns a list of process instances for which the time elapsed between the completion stage S1 and stage S2 is greater than the threshold T. S1 may assume the special value start, denoting the start of the process instance, and S2 takes the special value end, denoting the completion of the process instance. Many different SLAs that include conditions on the time elapsed between the execution of certain nodes in a process (or on the duration of the entire process) may be specified using this function.
With the function library 118, a user may select a function and specify the parameters that are to be used by the function in evaluating an SLA. The user may, depending on the function, further specify the particular Web service to which and the customer to whom the SLA is to apply.
The SLA definitions are stored in SLA definition database 104. In an example embodiment, the SLA definition database has tables for the constructors that describe the domain, i.e., entities and relationships, as well as tables for the different abstractions of the metrics model underlying the metrics implementation; there is a metric that underlies each SLA. Specifically, there are tables for the metrics, mappings, meters and contexts. The abstractions support sharing of mappings and the polymorphism of the metrics, which are mechanisms used to make the definition and computation of metrics simple, efficient and flexible.
The SLA computation engine 106 evaluates the Web service status data 108 against the SLAs set forth in the SLA definition database 104. In an example embodiment, the Web service status data 108 may be evaluated in accordance with the methods described in the co-pending patent application entitled, “DISPLAYING METRICS FROM AN ALTERNATIVE REPRESENTATION OF A DATABASE” by Casati et al., which was filed on Jan. 21, 2004, attorney docket number, 200310151-1, and is incorporated herein by reference.
Depending on the Web service, the service status data 108 may be centralized in a single database or distributed amongst several sites in several databases. Correlation of the status data between distributed stages of a composite Web service may be accomplished with the techniques described in the patent application Ser. No. 10/412,497 entitled, “Correlation of Web Service Interactions in Composite Web Services,” by Sayal et al., filed on Apr. 11, 2003, and incorporated herein by reference.
The aggregate data of SLA violations as determined from the Web service status data 108 is stored in a database of SLA violations 110. The database associates information such as the identifier of each process instance, the identifier of the metric underlying the SLA, and the value of the metric for the process instance.
Reporting tool 112 is available to report each individual SLA that has been violated along with the process instances that resulted in the violations. The reporting tool may display statistics such as the number of violations by type of Web service, by customer, by time or by another implementation-specific parameter.
SLA explanation and prediction tool 114 produces explanation and prediction models from the SLA violation information 110, Web service status data 108, and definitions 116 of a composite Web service. Explanation of SLA violations refers to communicating information about patterns found in the status data 108 of complete process instances that have violated SLAs. Prediction of an SLA violation refers to communicating information that indicates the likelihood that an active process instance will violate an SLA.
The explanation and prediction analyses may be performed using the SLA violation data 110, service status data 108, and definitions 116 of the composite Web services. The composite Web service definitions specify the behavior of the composite Web service, for example, the behavior of the composite Web service illustrated in
The composite Web services that are the subject of the explanation and prediction analysis are defined initially (step 202). Various commercially available or proprietary tools may be used to create a model of a Web service. In one tool, a graphical user interface (GUI) is provided. The GUI allows the model to be defined by way of user selection of icons available in the GUI to instantiate boxes and arcs, which represent nodes and data flow, respectively. The composite Web service definitions make visible the constituent Web service components of the composite Web service. The definitions describe the structure of the composite Web service, and status data corresponding to these definitions. For example, status data such as the starting time and the resource assigned for the execution of each node in the process flow may be described. The definitions thereby support generation of the explanation and prediction models from the status data.
The analysis also requires a set of SLAs to be defined (step 204). An example process for defining SLAs is illustrated in
The process instances that violated the defined SLAs are determined based on the status data associated with the process instances and the defined SLAs (step 206). As described above, the SLA violation data includes an association of the identifier of each process instance, an identifier for the metric, and the value of the metric for the process instance.
In an example embodiment, both the explanation model and the prediction models are decision trees. The intuitive structural description provided by a decision tree helps to explain what has been learned about SLA violations (i.e. patterns in the status data of process instances that have led to SLA violations in the past) and provide predictive information about which active process instances might fail the SLAs.
A decision rule (corresponding to a pattern) may be obtained by traversing a branch of the tree from the root node 252 to one of leaves 254, 256, 258, and 260. Nodes 252, 262, and 264 represent process instance variables involved in the Web service, and each branch leading from a node, for example branch 266 represents a set of possible values associated with the variable of the node from which the branch emanates. Each of the leaves has an associated label value, either violation or satisfaction, and an associated probability level ranging from 0.0 to 1.0. An example decision rule obtained from the tree is that if an order is placed on Friday, there is a 0.7 probability that the SLA will be violated. Another example decision rule is that if an order is placed on Saturday through Thursday, for a quantity greater than or equal to 1000 ties, and the type of tie is T12, there is a 0.8 probability that the SLA will be violated.
The decision tree format allows a user to easily identify patterns of SLA violations which may assist in identifying a root cause. For example, a user may find that the reason that orders placed on Friday are likely to violate the SLA is because a majority of employees leave early on Friday. The user may suggest modifying work schedules or providing incentives for employees who stay longer to avoid the SLA violations.
The tree 250 exemplifies the models used for explanation and prediction of compliance with SLAs. For explanation, a tree is built for an SLA using the data generated during the execution of process instances, from beginning to end, along with the SLA violation data that indicates which process instances violated an SLA. For prediction, complete process instances are also used to build a tree because those are the ones whose final outcome (in terms of SLA compliance) is known. However, the prediction tree is constructed based only on status data that existed up to a certain execution stage of one or more process instances because a prediction model is generated for the purpose of predicting SLA outcome of active process instances that have advanced to a certain execution stage. As a process instance advances in its execution, the prediction models corresponding to more advanced execution stages are used to update the prediction. Also as the execution stage is more advanced, the confidence in the prediction grows. Thus, further processing of the status data is performed in preparation for constructing the prediction model(s). More than one prediction model may be created because a composite Web service includes multiple stages, and the present stage of a running process instance may be any of the possible stages.
Returning now to
Status data related to each process instance may be stored in different tables, and the stages through which each process instance flows may differ from one process instance to the next. An example of the different possible stages is illustrated by the decision points in the process flow of
Data mining techniques generally require one record per training instance with each record being of the same length. The preprocessing involves obtaining the status data associated with process instances that have completed. Horizontalization refers to selecting relevant attributes from the obtained status data and storing the attribute values for each process instance in a single record. The selection of relevant attributes may be performed using generally known or proprietary techniques, depending on implementation requirements.
Turning now to generating prediction models, a prediction model is based on a selected prediction stage. A prediction stage corresponds to a stage in the Web service execution in which prediction information may provide a meaningful indication as to whether a running process instance that has reached that prediction stage will violate or satisfy a given SLA. Generally, a prediction stage references a stage that has been completed by a running process instance.
There may be multiple prediction stages with each prediction stage having a corresponding prediction model. Some stages may be less useful as a prediction stage than other stages. For example, in the process flow of
Once the prediction stages are identified, training tables are generated (step 212) from the data obtained in step 210. A training table is created for each identified prediction stage. A training table generated for explanation is assembled from the status data of completed process instances from the beginning to the end of execution (all the data in the training set from step 208). A training table generated for a prediction stage is assembled from data of completed process instances that was generated from the beginning of each process instance (i.e., start stage) up to the last stage (activity) corresponding to that prediction stage (a subset of the training set from step 208) and following a given path.
The effectiveness of the prediction models may depend largely on whether the attributes associated with each process instance are relevant and unique. That is, some attributes may be irrelevant, redundant, or noisy in terms of predicting whether a process instance will violate an SLA. Determining relevant data features (step 214) involves identifying and removing as much of the irrelevant and redundant information as possible as well as deriving new features from relevant, existing ones. Feature selection reduces the dimensionality of the training tables, thereby reducing the size of the hypothesis space and allowing data mining processes to operate faster and more effectively.
In an example embodiment, a correlation-based feature selection technique is used to determine the relevant data features. Correlation-based feature selection handles both discrete and continuous features and discrete and continuous classification problems. Correlation-based feature selection generally rests on the principle that the features in a good set of features are highly correlated with the class and are uncorrelated with each other. The class in this application is the violation/satisfaction of an SLA. Various known processes may be implemented for performing the correlation-based feature selection.
New features may also be derived from the status data. An example may be the number of times each a stage is executed for each process instance. In another example, it may be beneficial to break a timestamp feature into additional features such as day of week, day of the month, week of the month, or month of the year. In another embodiment, features may be manually selected based on experience with the Web service. The correlation-based feature selection process may then be performed on the data with the user defined features.
Once the relevant features have been determined and the training tables appropriately configured, the process continues by generating an explanation model for each SLA (step 216) and one or more prediction models for the identified prediction stages (step 218).
In one embodiment, the explanation model for an SLA is an explanatory decision tree. An example explanatory decision tree of the Web service 10 of
A prediction model is generated for each of the prediction stages (step 218) identified in step 210. For each prediction model, the training table generated for the corresponding prediction stage is used to generate the model. Similar to generating the explanation model, each prediction model is a decision tree and various algorithms are available to generate the decision tree.
A user may then use the explanation model from step 216 and prediction models from step 218 to evaluate the conformance of the Web service to the various SLAs and monitor running process instances.
The decision tree 250 of
If a predefined class is selected (decision step 404), a function associated with the class is selected from a library of functions. Otherwise, the user may codify a new function to add to the library (step 408).
In either case, parameter values are obtained from the user for use by the selected function (step 410). For example, as previously described the parameter values may indicate a time interval, a quantity of an item, a product identifier or other application-specific characteristic. The parameter values are used by the function to evaluate whether a process instance violates or satisfies the SLA implemented by the function.
Because SLAs may generally be viewed as an agreement between a provider and a specific customer, the function may further be parameterized by a customer identifier (step 412). This limits applicability of the SLA definition to only the designated customer. The SLA definition may then be saved for use in determining whether process instances have complied with or violated the SLA.
In explaining the process instances that violated or satisfied an SLA, the process first obtains the explanation model associated with a selected SLA (step 452). The SLA may be selected by a user via a GUI of a reporting tool. The reporting tool may then display the explanation model in a format that illustrates the nodes and edges of the decision tree (step 454). For example, the model may be displayed as a graph image of the nodes, edges, and leaves in the decision tree or as a list of decision rules corresponding to the different paths from the root of the tree to each of the leaves.
Next, the appropriate prediction stage of each running process instance is identified using the information gathered in the single record for the process instance. The prediction stage for a process instance may be determined based on the current state of the process instance. For example in reference to the example Web service of
The prediction model associated with the prediction stage identified for a process instance is then applied to the process instance (step 506). The application of the appropriate prediction model is performed for each active process instance. In applying a process instance to a prediction model (the prediction model is a decision tree), the process traces the decision tree using attribute values associated with the process instance until a leaf node is encountered. The probability value associated with the leaf node indicates the probability that the process instance will violate the SLA.
The prediction information may then be output to a user for each running process instance (step 508). The prediction information may include information such as the identifier of each process instance, all the attribute values of the process instance, and the probability that the process instance will violate the SLA.
Those skilled in the art will appreciate that various alternative computing arrangements would be suitable for hosting the processes of the different embodiments of the present invention. In addition, the processes may be provided via a variety of computer-readable media or delivery channels such as magnetic or optical disks or tapes, electronic storage devices, or as application services over a network.
The present invention is believed to be applicable to a variety of systems for managing SLAs and has been found to be particularly applicable and beneficial in reporting probabilities that SLAs will be violated and to understand in which situations SLAs may be violated. Other aspects and embodiments of the present invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. It is intended that the specification and illustrated embodiments be considered as examples only, with a true scope and spirit of the invention being indicated by the following claims.