Patent Application
20100114621
Over the last few years, information technology (IT) organizations have increasingly adopted standards and best practices to ensure efficient IT service delivery. In this context, the IT Infrastructure Library (ITIL) has been rapidly adopted as the de facto standard. ITIL defines a set of standard processes for the management of IT service delivery organized in processes for Service Delivery (Service Level Management, Capacity Management, Availability Management, IT Continuity Management and Financial Management) and Service Support (Release Management, Configuration Management, Incident Management, Problem Management and Change Management). The Service Support processes, such as Configuration Management, Incident Management, and Configuration Management are some of the more common processes IT organizations have implemented to bring their service to an acceptable level for their businesses.
The implementation of ITIL processes has yielded significant results to IT organizations by defining clear interfaces between service providers and consumers, by clarifying the IT organizational structures, roles, and responsibilities, and by designing internal processes for the management of IT operations. However, once ITIL processes are implemented, the effectiveness of the IT organization depends on the ability of the decision makers involved in those processes to make timely and accurate decisions. Hence, regardless of the quality of the process implementation, if inadequate decisions are made during the execution of those processes, they will result in poor customer experience and low value delivery. For instance, being able to prioritize a large volume of incidents based on their likely impact on the IT organization is key to running an efficient Incident Management process.
The challenge facing IT Service Management is to develop decision support capabilities that will enable decision makers to drive their management processes efficiently. An organizational structure and processes referred to as IT Governance can be used to define the strategy of the IT organization and oversee its execution to achieve the goals of the enterprise. One of the goals of IT Governance is to deliver IT services in alignment with business objectives. The business objectives are used as input to derive goals, objectives, and performance metrics needed to manage IT effectively. Auditing processes are used to measure and analyze the performance of the organization. A concept referred to as the Balanced Scorecard combines financial measures and non-financial measures in a single report and has been used as a strategic and performance measurement tool for IT Governance. However, the development of the balanced scorecard has been focused on the structural aspects (including design processes, metric selection, and ranking) and on the reporting aspects (including metric collection and ranking). The further development and use of the balanced scorecard to drive business decisions within an enterprise to align business strategy with IT operations has been limited.
The teachings of the present disclosure relate to methods, systems, and computer program products for modeling consequences of events on performance indicators. Described herein are several methods, systems, and computer program products for developing a causal model capable of determining the consequences of events on performance indicators. A causality model is provided to define relationships between events and the consequences associated with each one of the events to modify a value of at least one of the performance indicators associated with each one of the events. An algorithm included in the causality model is executable to compute an impact of the consequences on the value. A causality model management system is provided to manage the causality model, including storing, execution of the algorithm, interfacing with other systems, and simulating what-if scenarios of the events and the performance indicators.
Embodiments of the invention relating to both structure and method of operation may best be understood by referring to the following description and accompanying drawings:
Applicants recognize that it would be desirable to provide a causal framework that would include tools and techniques to make business decisions under uncertainty that would leverage the use of balanced scorecard in selecting alternatives, thereby ensuring alignment between business strategy and IT operations, including IT services and IT infrastructure. That is, it would be desired to provide tools and techniques that would evaluate various business alternatives under risk in quantifiable terms and assess their impact on the objectives identified in the balanced scorecard. Applicants further recognize that the selection of a particular alternative from a set of alternatives is a multi criteria decision making (MCDM) problem under uncertainty when criteria are modeled as objectives in balanced scorecards. Therefore, a need exists to provide improved tools and techniques to be used in decision support systems for modeling consequences of events on performance indicators, and hence on balanced scorecards.
The following terminology may be useful in understanding the present disclosure. It is to be understood that the terminology described herein is for the purpose of description and should not be regarded as limiting.
Framework—A basic structure designed to provide one or more functions. A framework used in a computer hardware and software environment may be typically designed to include processes to deliver core functions and extensible functions. The core functions are typically a portion of the framework that may not be modifiable by the framework user. The extensible functions are typically a portion of the framework that has been explicitly designed to be customized and extended by the framework user as a part of the implementation process. For example, Control Objectives for Business Information-related Technology (COBIT) is a type of a framework used for addressing the management's need for control and measurability of IT services. The framework may also include application program interfaces that allow an external program to access the functions of the framework.
Balanced Scorecard—A collection of key measures (financial and non-financial) introduced by Robert Kaplan and David Norton that an enterprise may define to measure its performance in executing a strategy and ‘balance’ the business perspective with the IT service perspective. A typical balanced scorecard may include 4 perspectives (a perspective may be defined as a group or combination of objectives pertaining towards a particular function of the business), including financial, customer, business process, and learning/growth perspectives. A generalized balanced score card may include a set of n perspectives.
Key performance indicator (KPI) (or simply a performance indicator)—KPI is an actionable metric. As defined by COBIT, KPI's are measurable indicators of performance of the enabling factors of IT processes, indicating how well the process enables the goal to be reached.
Objectives—Measurable indicators of the business objectives. The objectives typically express one or more target values over a KPI.
Lottery—A lottery is a type of gaming process that awards prizes based on the element of probability or chance. If Z is a finite set of consequences (or prizes or outcomes), then a lottery is a probability distribution on Z.
System—One or more interdependent elements, components, modules, or devices that co-operate to perform one or more predefined functions.
Configuration—Describes a set up of elements, components, modules, devices, and/or a system, and refers to a process for setting, defining, or selecting hardware and/or software properties, parameters, or attributes associated with the elements, components, modules, devices, and/or the system.
Systems and methods disclosed herein provide an open model for expressing declaratively causal relationships between IT events and performance indicators. The model is said to be open because the framework provides for different styles of consequences to be expressed through the concept of “modifier”. A modifier can be a probability distribution expressed over a range of key performance indicators (KPI's). Therefore, a consequence expresses how events are likely to modify the values of KPI's.
Management of Information Technology Infrastructure Library (ITIL) processes can be based on objectives defined in balanced scorecards to help ensure that IT management decisions and processes are aligned with business objectives. In terms of decision theory, criteria modeled as balanced scorecard objectives presents a multi-criteria decision making (MCDM) problem under uncertainty. Cost functions, utility functions or other means to assess or compare alternatives are defined to achieve efficient, business driven prioritization and scheduling. The functions measure the impact of IT management alternatives against the objectives defined in a balanced scorecard.
Given a set A of IT management alternatives, a preference relation over the element of A is defined based on the relationship between alternatives and their impact on scorecard objectives.
The set of all available performance indicators (also referred to more generally as metrics) K={κi 114, . . . , κn 124} and the set of domain values D={D1, . . . , Dn} where each D1, (1≦i≦n) is a finite set of possible values for performance indicator κi. D will also be used to denote the space of all possible outcomes, which is the Cartesian product of D=D1×D2× . . . ×Dn.
An objective Oi, (1≦i≦n) was defined above as a constraint expressed over an element κj of the set K and an element of its associated domain value Dj. Constraints in scorecards admit two comparators, (“below” or “lower than”) and (“above” or “greater than”).
The generalized scorecard model, with a constraint approach to the definition of the objective is specifically useful for activities such as scorecard performance monitoring and reporting. However, for decision support, the definition of an objective as a constraint is too limited. Accordingly, a utilitarian approach to the definition of scorecard objectives may be taken.
As described in the previous section, a scorecard perspective is a set of objectives over which a preference function is defined. An objective is a constraint expressed between a performance indicator and an element of its domain value. In a utilitarian approach, a scorecard objective can be treated as a utility function expressing a decision maker's preference over the domain values of a performance indicator.
For example, a scorecard objective such as “The overall service level agreement (SLA) penalty cost should be less than $100000” expresses a constraint over the domain values of the performance indicator “overall penalty cost”. Any value above the target value ($100000) would not meet the constraint while any value below would. In other words, the constraint expressed in the scorecard objective partitions the set D of domain values of a performance indicator κ into two subsets, the subset S containing the values that satisfy the constraint and the subset NS containing the values that do not.
PROPOSITION 1. A scorecard objective expresses a decision maker's preference over the domain values of the associated performance indicator.
Because all the values contained in the S subset satisfy the constraint, from a decision maker point of view, they all yield that same degree of satisfaction. As long as the constraint is satisfied, any value of the performance indicator is as good as the other. On the other hand, would a decision maker be equally dissatisfied by the values in NS that do not satisfy the constraint, or would a decision maker would prefer values of the performance indicator that are “close” to the objective than values that are “far” from the objective.
Therefore, there exists a preference relation o (“is preferred with regards to objective o”) over the set D of domain values of a performance indicator κ and that this preference is expressed in reference to a target value in D and the constraint operator (“greater than” or “lower than”) associated with the related objective.
PROPOSITION 2. The preference relation o can be represented by a von Neumann-Morgenstern utility function.
The preference relation o is rational, that is:
Given the properties of the preference relation o, let φo:D→[0,1] be a utility function representing the preference relation
o such that:
∀x, y∈D:xoy
φo(x)≧U0(y) Equation 100
Based on the scorecard structure, mutual utility independence between the utility functions representing the various objectives is assumed.
PROPOSITION 3. The utility function φo(.) representing the preference relationship expressed in the context of an objective o can be derived from the objective's constraint operator, the target value and the risk attitude of the decision maker.
PROPOSITION 4. The utility of a scorecard is calculated as a weighted sum of the objectives utility functions.
In a scorecard, objectives are clustered into perspectives (financial, customer, internal, and others). Relative importance of objectives in a perspective is determined by a weight with sum of the weights equal to one. Similarly, importance of the perspective in a scorecard is also determined by a weight. To capture this, a utility function associated with an objective o in a perspective p is associated with a weight ω0 with:
ω0=weight of objective o in perspective x weight of perspective p in scorecard with
the summation occurring for values of i from 1 to n. The utility of a scorecard s with n objectives is computed by Equation 200 as follows:
To illustrate the causality relationship, a scenario may exist in which a decision maker may desire to prioritize Information Technology (IT) incidents and that Service Level Agreements (SLA's) have been defined on the incident management process specifying specific times for the treatment of such IT incidents. For instance, a gold SLA may specify that incidents be treated within 4 hours of reception. A given alternative would associate a priority to a specific incident resulting in an expected time of treatment. For example, a ‘High’ priority may guarantee a time of treatment of 1 hour, plus or minus 15 minutes. If the expected time of treatment is after the deadline computed form the SLA, the following event may be generated: violation of associated service level agreement (SLA). This event results in a unit increase for the performance indicator measuring customer satisfaction and an increase of the penalty cost associated with the SLA for the performance indicator measuring the total penalty cost.
In this example, there is an uncertainty over the time (e.g., plus or minus 15 minutes) of treatment of the incidents. In other words, the consequences of assigning a given priority to an incident results in uncertain consequences or outcomes. Thus, an alternative is equivalent to a set of simultaneous independent lotteries or prospects defined as probability distributions over the domain values of the set of performance indicators 330. As described earlier, if Z is a finite set of consequences (or prizes or outcomes), then a lottery is a probability distribution on Z.
Given a set of m alternatives included in the set of alternatives 312,
A
i
={L
i
1
, . . . , L
n
i} Equation 300
where Ai, (1<i<m) is the ith alternative and Lij, (1<j<n) is the lottery on the jth performance indicator for the ith alternative. For discrete values of the performance indicator, the probability density function is replaced with the probability masses represented by (p1, x1; . . . ; pn, xn) meaning that probability pj is assigned to outcome xj, for j=1, . . . , n. The probabilities pj are nonnegative and sum to one. Without loss of generality and for simplification, the performance indicators are assumed to be discrete. To determine the optimum alternative, the solution defines a preference structure over the set of alternatives 312.
Therefore, alternative A2420, can be reformulated over the three attributes {x, y, z} by introducing a degenerate lottery on y, which is a simple lottery that assigns probability 1 to the current value of performance indicator y, and 0 to all others. The two alternatives A1410 and A2420 can now be compared after the homogenization process. A decision making situation in which a decision maker or a decision support system can assign mathematical probabilities to the randomness is referred to as a situation under risk or an uncertain situation. In such a case, the behavior of the decision maker is fully defined by its preferences on the probability distributions over the consequences of the alternatives. Therefore, the decision to select an alternative from the set of alternatives 312 is equivalent to defining a preference relation over a set of concurrent lotteries: {{L11, . . . , L1n}, . . . , {Lm1, . . . , Lmn}}.
DEFINITION 1. Let X be a set of variables. A k-concurrent lottery is a set of k (k=card(X)) simultaneous independent simple lotteries over each element of X.
DEFINITION 2. Let X be a set of variables and let Y, Z be two random subsets of X, with Y not equal to 0 and Z not equal to 0. Let Ly be a concurrent lottery defined over Y and Lz be concurrent lottery defined over Z. Ly and Lz are homogeneous if, and only if, Y=Z.
DEFINITION 3. Let X be a set of variables and let x={x1, . . . , xn} be the valuation of the variables in X at time t. Let Lx be an n-concurrent lottery over X at time t. The values in x are said to represent the status quo for Lx.
PROPOSITION 5. Let X be a set of variables and let Y, Z be two random subsets of X, with Y not equal to 0, Z not equal to 0 and Y not equal to Z. Let Ly be a concurrent lottery defined over Y and Lz be concurrent lottery defined over Z. Ly and Lz are homogenized by adding to Ly the degenerated lotteries on the status quo values of Z\Y and by adding to Lz the degenerated lotteries on the status quo values of Y\Z.
Based on the results described in the section entitled MODEL FOR BALANCED SCORECARD, the expected utility (EU) for of a simple lottery L over a performance indicator K is computed by equation 400 as follows:
where φok(.) is the utility function derived from objective o defined over performance indicator k as described in Equation 100. φok(xi) is the utility of consequence or outcome xi and pi is the probability associated with the outcome xi.
PROPOSITION 6. Let X={x1, . . . , xn} be a set of discrete variables. Let L={L1, . . . , Ln} be a set of lotteries defined over X where each Li is a simple lottery over variable xi. The expected utility of L is computed by Equation 500 using Equation 400 as follows:
The EU may also be computed if variables are associated with relative importance as performance indicators in the balanced scorecards.
PROPOSITION 7. Let X={xi, . . . , xn} be a set of discrete variables and let Ω={ω1, . . . , ωn} be a set of weights where each ωi, (1<i<n) is the relative importance of the ith variable in X and sum of ωi=1 (with i varying from 1 to n). Let L={L1, . . . , Ln} be a set of lotteries defined over X where each Li is a simple lottery over variable xi. The expected utility of L is computed by Equation 600 as follows:
THEOREM 1. (Expected Utility Theorem) A preference relation on the space of simple lotteries L satisfying the continuity and independence axiom admits a utility representation of the expected utility form such that for any two lotteries p, q belonging to L, the following holds as described in Equation 700:
That is, a lottery p is preferred to lottery q, if and only if, the expected utility of p is greater than the expected utility of q.
THEOREM 2. Let L be space of simple lotteries defined over a set of discrete variables. A preference relation on the space of homogeneous k-concurrent lotteries Ck defined over L admits a utility representation of the expected utility form such that for any two lotteries k, I belonging to Ck, the following holds as described in Equation 800:
k
I⇄EU(ki) where ki∈ supp(k)>ΣEU(Ii) where Ii∈ supp(l) (Eq. 800)
where ki denotes the ith lottery in the concurrent lottery k and Ii denotes the ith lottery in the concurrent lottery I. Theorem 2 shows that the preference relationship over a set of a k-concurrent lottery can be represented by the expected utility function as determined in Proposition 7. Therefore, the utility functions used to compare alternatives under uncertainty in light of balanced scorecard take the expected utility form expressed in Proposition 7.
The results on concurrent lotteries are integrated into a method for comparing alternatives under risk, as illustrated in the flow chart described with reference to
At step 510, a causality framework that models the relationship between alternatives and consequences is provided. In a particular embodiment, the causality framework defines the relationships between an alternative (Ai) of the set of alternatives 312 and at least one performance indicator of a set of performance indicators 330. Additional detail of the causality framework is described with reference to
Benefits of the tools and techniques for comparing alternatives under risk described herein enable a decision maker to connect alternatives with their impact on a balanced scorecard. For example, if a server goes down and a decision maker needs to assess what alternative to opt for from a set of alternatives, e.g., repair it now, repair it tomorrow, or wait for the next maintenance cycle. The tools and techniques described herein present the decision maker with the three alternatives along with their impact on the various objectives and performance indicators for assessment Repair now may have a positive impact on “up time” objective but may require allocating a specific resource, e.g., engineer being allocated to another task, thereby increasing the cost associated with this alternative, and thus having a negative impact of “cost of delivery” objective. Repair tomorrow may not be favorable to the “up time” objective but may have a positive effect on the “cost of delivery” objective. Lastly, the wait for maintenance window alternative may not be favorable to the “up time” objective but may be favorable to the “cost of delivery” objective. While selecting an alternative, the tools and techniques described herein advantageously provide a declarative and analytical framework to support the analysis and selection by providing clear consequences associated with the alternatives.
The causality model 300 includes an EventInstance 690 module coupled to an EventType 680 module, a Consequence 610 module coupled to the EventType 680 module, a KPIType 620 coupled to the Consequence 610 module, and the KPIType 620 module coupled to a KPlInstance 670 module. Relations between the coupled modules may be 1 to many (indicated by 1 and *) or many to many (indicated by * and *). EventInstance 690 is capable instantiating multiple instances of the event types. Each type of the event that is instantiated is the EventType 680. The EventType 680 may include multiple consequences, including a consequence 610. The KPlInstance 670 is capable of generating an instance KPIType 620. The consequence 610 has a name 612, a description 614 and an IModifier 616, which is an interface that expresses how a consequence modifies the associated performance indicator KPIType 620. Multiple ones of the consequences may affect the same KPIType 620. If the KPIType 620 is of discrete nature, then the IModifier 616 associated with the consequence will be a discrete modifier. Alternatively, if the KPIType 620 is of continuous nature, then the IModifier 616 associated with the consequence will be a continuous modifier. A modifier is essentially a probability distribution expressed over the range of the KPI. Consequences are associated with EventTypes. An EventInstance can have multiple EventTypes. The EventType 680 and the KPIType 620 can have super-type and sub-type relationships.
The various functions, processes, methods, and operations performed or executed by the system 900 can be implemented as the program instructions 930 (also referred to as software or simply programs) that are executable by the processor 910 and various types of computer processors, controllers, central processing units, microprocessors, digital signal processors, state machines, programmable logic arrays, and the like. In an exemplary, non-depicted embodiment, the computer system 900 may be networked (using wired or wireless networks) with other computer systems.
In various embodiments the program instructions 930 may be implemented in various ways, including procedure-based techniques, component-based techniques, object-oriented techniques, rule-based techniques, among others. The program instructions 930 can be stored on the memory 920 or any computer-readable medium for use by or in connection with any computer-related system or method. A computer-readable medium is an electronic, magnetic, optical, or other physical device or means that can contain or store a computer program for use by or in connection with a computer-related system, method, process, or procedure. Programs can be embodied in a computer-readable medium for use by or in connection with an instruction execution system, device, component, element, or apparatus, such as a system based on a computer or processor, or other system that can fetch instructions from an instruction memory or storage of any appropriate type. A computer-readable medium can be any structure, device, component, product, or other means that can store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
The illustrative block diagrams and flow charts depict process steps or blocks that may represent modules, segments, or portions of code that include one or more executable instructions for implementing specific logical functions or steps in the process. Although the particular examples illustrate specific process steps or acts, many alternative implementations are possible and commonly made by simple design choice. Acts and steps may be executed in different order from the specific description herein, based on considerations of function, purpose, conformance to standard, legacy structure, and the like.
Novel features believed characteristic of the present disclosure are set forth in the appended claims. The functionality of various modules, devices or components described herein may be implemented as hardware (including discrete components, integrated circuits and systems-on-a-chip ‘SoC’), firmware (including application specific integrated circuits and programmable chips) and/or software or a combination thereof, depending on the application requirements. The accompanying drawings may not to be drawn to scale and some features of embodiments shown and described herein may be simplified or exaggerated for illustrating the principles, features, and advantages of the disclosure.
While the present disclosure describes various embodiments, these embodiments are to be understood as illustrative and do not limit the claim scope. Many variations, modifications, additions and improvements of the described embodiments are possible. For example, those having ordinary skill in the art will readily implement the steps necessary to provide the structures and methods disclosed herein, and will understand that the process parameters, materials, and dimensions are given by way of example only. The parameters, materials, and dimensions can be varied to achieve the desired structure as well as modifications, which are within the scope of the claims. Variations and modifications of the embodiments disclosed herein may also be made while remaining within the scope of the following claims. For example, a few specific examples of data models are described. The illustrative system for decision making under uncertainty can be used with any suitable data models. The illustrative techniques may be used with any suitable data processing configuration and with any suitable servers, computers, and devices. In the claims, unless otherwise indicated the article “a” is to refer to “one or more than one”.
