The present invention relates to computing systems, and deals more particularly with obtaining feedback on key performance indicators in a rule-based system.
Today, systems that operate using rules are available for a variety of applications, and such systems are commonly referred to as “rule-based systems”. A rule-based system may be provided where business rules operate upon data to calculate outcomes, such as calculating a risk for a mortgage applied for by a bank customer or calculating a recommended price to charge customers for a hotel booking. Writing a good set of business rules for the rule-based system can be a difficult undertaking, and without a good set of rules, the outcomes calculated by the rule-based system may not provide the best result.
The present invention is directed to obtaining feedback on key performance indicators in rule-based systems. In one aspect, rule fitness is evaluated in a rule-based system, which further comprises: responsive to detecting a modification to the rule-based system, executing the rule-based system on an input data set to create at least one decision, each decision having a fitness associated therewith; responsive to executing the rule-based system, executing a fitness computation (preferably, in a background mode) to compute rule fitness for the rule-based system; and responsive to executing the fitness computation, causing display (e.g., in a bar graph) of the computed rule fitness. The detected modification may comprise at least one of: a change to content of the input data set; a change to a definition for the input data set; a change to a rule in the rule-based system; and a change to a function that computes the fitness associated with the at least one created decision.
In one approach, the fitness computation further comprises: summing, for each of a plurality of decisions in the rule-based system, the fitness associated therewith to determine an overall fitness of the rule-based system; and summing, for each rule executed to create the plurality of decisions, the fitness associated with each of the plurality of decisions in which the rule was executed to determine a fitness of the rule. Preferably, the determined overall fitness of the rule-based system and the determined fitness for each of at least one of the rules are then displayed.
Embodiments of these and other aspects of the present invention may be provided as methods, systems, and/or computer program products. It should be noted that the foregoing is a summary and thus contains, by necessity, simplifications, generalizations, and omissions of detail; consequently, those skilled in the art will appreciate that the summary is illustrative only and is not intended to be in any way limiting. Other aspects, inventive features, and advantages of the present invention, as defined by the appended claims, will become apparent in the non-limiting detailed description set forth below.
The present invention will be described with reference to the following drawings, in which like reference numbers denote the same element throughout.
An embodiment of the present invention evaluates fitness of rules in a rule-based system, determining key performance indicator feedback. Fitness evaluations are integrated with rule authoring. The feedback may be used to improve the set of rules with which the rule-based system operates.
One example of a rule-based system for business is the WebSphere® ILOG® Business Rule Management System (“BRMS”) system from International Business Machines Corporation (“IBM”). This BRMS provides test and simulation capabilities. (“WebSphere” and “ILOG” are registered trademarks of IBM in the United States, other countries, or both.)
While known rule-based systems provide a number of advantages, known systems decouple the operation of the rule-based system from rule authoring. This makes it more difficult for a rule author to understand the relative “goodness”, also referred to herein as “fitness”, of a particular business rule. It is also more difficult for the rule author to determine whether changes he or she makes to a rule improves the fitness of the rule.
Consider, by way of example, a travel booking system based on rules. It may be desirable to know how much the profit margin of travel to a particular destination, at a particular time of year, would be impacted if a travel discount was offered, or increased. Historical travel booking data may be used in a rule-based system to simulate purchase transactions under various conditions, and these simulated transactions may then be used to provide feedback on the rules used in the simulations.
An embodiment of the present invention integrates running simulations into the rule authoring process for rules (which are also referred to herein as “business rules”). The rule author can therefore obtain nearly real-time feedback on the relative fitness of an edited rule. More particularly, while a user is editing rules, an embodiment of the present invention runs simulations on the rule set in the background, using a representative input data set to exercise the rule set. Key process indicators, also referred to herein as “KPIs”, are calculated using a fitness function or predictive model. KPI charts depicting these indicators are preferably displayed within the rule authoring environment, giving the user immediate feedback on the likely business impact of the rule changes he or she is making. It should be noted that the simulations discussed herein may also be performed responsive to edits being made to input data records, input data definitions, and/or the fitness function which is applied to an output record, and an embodiment of the present invention therefore more generally provides an interactive rule, data, and modeling environment that allows the user to receive immediate feedback on fitness of the overall rule set and how the modifications being made are impacting that fitness. Accordingly, references herein to rule authoring or rule editing are therefore by way of illustration but not of limitation of the editing that may trigger evaluation of a rule set to generate KPIs.
Components in one embodiment of a system that integrates rule authoring with running simulations to provide key feedback indicators, as disclosed herein, will now be described with reference to
Components shown in
Rules engine 100 is used to compute a decision by firing a rule or rules from rule set 140 during evaluation of an input data record 180 from input dataset 130. A number of decisions may be computed by firing rules, and these decisions are stored in decision warehouse 110.
The sample decision values 300 further comprise execution trace 330, which indicates that a rule named “highMarkupDiscount” 331 was executed on sample input data 310 to create output data 320, and rule set identifier 340 indicates that the set of rules named “MyRuleset” 341 was used to compute this decision.
Referring again to
According to one embodiment of the present invention, the fitness of a rule set is computed as the sum of the fitness for all output data 180, as stored in decisions 120, which were calculated using the rule set 140, and the fitness for a given rule is computed as the sum of the fitness for all output data 180 for which this rule was executed. (Refer to
Using the computed decisions from the decision warehouse, the rule set fitness is computed 430. One or more charts may be presented 440—for example, on a graphical user interface—to represent the computed rule set fitness. The execution trace, which lists the rules which were fired when computing each decision, is obtained 450 from the decision records in the decision warehouse. An iterative process 460 is then performed, according to an embodiment of the present invention, whereby the fitness of each fired rule in the rule set is used in computing 461 the fitness of each rule. One or more charts may be presented 470—for example, on a graphical user interface—to represent the computed fitness of the individual rules.
A rule authoring interface may allow a user to edit various data and rules used by the rule-based system. A sample rule authoring interface is illustrated in
Rather than using historical transactions as input, in another approach, executable code is used to generate sample input data. This is illustrated by the sample code 500 which is depicted in
Notably, the fitness function used by an embodiment of the present invention will typically reference data in the input for a decision as well as data in the output computed with the rule set, in order to assign the fitness values for the rule set and rules. The fitness value may therefore be considered a type of “score” for decisions made with the rule set. Suppose, by way of illustration, that an embodiment of the present invention is used with a loan processing problem domain. The input data may comprise loan application data, where a loan application instance includes a variety of fields that may be used to evaluate whether a lender should underwrite the loan, and on what terms. Suppose that an existing predictive model shows that loans for residents of a particular state, who are over a particular age, have a certain percentage chance of going into default. The lender may wish to limit such loans to a fixed percentage of the lender's business in order to control the lender's financial risk. The rule set can then be used in simulations to see how the rules of the rule set make decisions about granting loans to this population of applicants. If the rule set creates decisions that grant loans to too many of these applicants, then the fitness function preferably “punishes” the rules by assigning a relatively low fitness value thereto. This enables the rule author to quickly see that the rules should be modified to grant fewer of these loans, and the modifying may be an iterative process whereby changes are made until the fitness value reaches a desired goal or threshold.
More particularly, fitness function 710 determines whether the suggested discounted price from a decision record is less than the weekly salary of a sample customer (see 711), and if so, it is presumed that this customer would buy the product. Accordingly, the difference between the discounted price and the product cost is therefore computed and returned to the invoking code (see 712) as the fitness value for this decision. By contrast, if the discounted price is not less than one week's salary for this customer, it is presumed that this customer would not purchase the product, and a value of zero is therefore returned (see 713) as the fitness value for this decision. If the profit as represented by a cumulative sum of the fitness values from a simulation is relatively low, this may indicate that many of the decisions contained discounted prices which were too high (i.e., the code at 713 was executed, indicating that the purchase was not made), and the fitness function will then cause the rules to be punished by assigning low fitness values thereto. In addition, if the discounted price is less than the cost of the product, then a negative value will be returned at 712, which further punishes rules that set the discounted price lower than the product's cost and would therefore allow products to be sold at a net loss. As can be seen by this example fitness function 710, the fitness can be expected to change when the rules change (because the rules compute the product price); when the input data changes (because the input data defines the transactions being processed by the rules); and when the fitness function itself changes.
A variable “D” represents a particular decision for the current input record (see 915), and a variable “FD” represents the fitness of this decision (see 920), as computed using the fitness function. A variable “R” represents the identifier of the rule set used for this decision (see 925), and a variable “FR” represents the fitness value for this rule set (see 930). With reference to
Next, a variable “TD” is set to represent the execution trace for the currently-processed decision “D” (see 940). With reference to
A variable “TR” represents the identifier of the currently-processed rule (see 950), and a variable “FTR” represents the fitness of this rule (see 955). At 960, the fitness “FD” of the currently-processed decision “D” is added to the cumulative fitness “FTR” for the currently-process rule “TR”. In this manner, the fitness “FTR” of each particular rule reflects the fitness of the decisions that were computed by applying that rule.
When all of the executed rules have been processed by the iteration at 945-965, and all of the input records have been processed by the iteration at 910-970, the processing of
One or more fitness charts or graphs are preferably created using the values computed according to
Although
Referring now to
Input/output (“I/O”) devices (including but not limited to keyboards 1218, displays 1224, pointing devices 1220, other interface devices 1222, etc.) can be coupled to the system either directly or through intervening I/O controllers or adapters (1216, 1226).
Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks (as shown generally at 1232). Modems, cable modem attachments, wireless adapters, and Ethernet cards are just a few of the currently-available types of network adapters.
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method, or computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.), or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit”, “module”, or “system”. Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable media having computer readable program code embodied thereon.
Any combination of one or more computer readable media may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (“RAM”), a read-only memory (“ROM”), an erasable programmable read-only memory (“EPROM” or flash memory), a portable compact disc read-only memory (“CD-ROM”), DVD, an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, radio frequency, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++, or the like, and conventional procedural programming languages such as the “C” programming language or similar programming languages. The program code may execute as a stand-alone software package, and may execute partly on a user's computing device and partly on a remote computer. The remote computer may be connected to the user's computing device through any type of network, including a local area network (“LAN”), a wide area network (“WAN”), or through the Internet using an Internet Service Provider.
Aspects of the present invention are described above with reference to flow diagrams and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow or block of the flow diagrams and/or block diagrams, and combinations of flows or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flow diagram flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flow diagram flow or flows and/or block diagram block or blocks.
The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flow diagram flow or flows and/or block diagram block or blocks.
Flow diagrams and/or block diagrams presented in the figures herein illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each flow or block in the flow diagrams or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the flows and/or blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or each flow of the flow diagrams, and combinations of blocks in the block diagrams and/or flows in the flow diagrams, may be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
While embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims shall be construed to include the described embodiments and all such variations and modifications as fall within the spirit and scope of the invention.
Number | Date | Country | Kind |
---|---|---|---|
10306295.6 | Nov 2010 | EP | regional |