Patent Application
20080071716
In the following, the business processes that require the use of multiple alternative plans are described. Next, the interval approach to generating multiple plans is described. Finally, a search-based planning algorithm for generating multiple alternative optimal plans within the interval approach is described.
In the plan-review-before-submission scenario, the user receives a set of alternative plans from the planner, and chooses one of the plans for execution based on cost/quality tradeoffs.
Users of the system can choose the plan to be executed from a limited set of possible alternatives. The users make their choice after the planner has narrowed down the huge set of possibilities, that is, the numerous candidate plans, and has selected only a few resulting plans that represent wide ranges of cost and quality values. The capability of the system to provide this choice is especially valuable in environments where users are responsible for managing resources allocated from limited budgets. In these systems the users must balance resource allocation between multiple tasks they plan and perform, based on relative priority and expected quality of results.
As shown in
The planner 20, uses a domain description from a store 30 of various domain descriptions and the planning request (1) as input data, generates a set of efficient plans and returns the set of plans (2) to the user 10. The planning domain description contains definitions of actions, and possibly other parameters of the planning task. Each of the resulting plans returned to the user 10 is annotated with cost and quality values computed by the planner 20. It is possible, of course, that no plans satisfying the planning request (1) exist, in which case the planner 20 will return an empty set of plans.
The user 10 then inspects the set of returned plans (2). The plans returned by the planner 20 cover a wide range of quality and/or costs and, therefore, the user 10 can study the tradeoffs between the different costs, for example, the monetary costs or the use of critical resources, and the quality of the end results produced by the plan. The user 10 may also at this point request to view the complete or partial sets of actions constituting the plan to gain a better understanding of the implications of the tradeoffs. Finally, the user 10 decides which of the alternative plans she prefers, and submits the preferred plan for execution (3) by the appropriate plan execution system 40. Alternatively, at this point the user 10 may decide not to submit any plans. In that case, the user 10 may revise and re-submit the planning request (1) to the planner 20.
According to an embodiment of the present invention, it is also possible to improve the efficiency of the planning processes as described in connection with
The users 10 are provided with a clear view of tradeoffs between cost and quality, shown by representative plans covering a wide range of cost and quality values as provided by the planner 20.
In addition to the numeric cost and quality metrics, the tradeoffs are illustrated by the different sets of domain-specific actions that are included in the set of resulting plans (2). The users 10 that are familiar with the domain as included in the domain description 30 can gain a better understanding of the tradeoffs by studying the set of resulting plans (2).
In a straightforward alternative to the above-described method, which can be implemented using existing metric planners, is the automatic choice of the plan for execution made based on a user-defined utility function. In practice, however, it has been found that people often feel more comfortable choosing from a number of possible options, rather than describing their preferences by specifying a utility function.
A modification of the system operation of
A fully or partially automated selection of plans can be performed after the resulting set of plans has been produced by the planner. Optionally, the users may reduce the set of plans by removing the plans they view as unacceptable and leaving the final choice of the plan selected for execution to an automatic system. Since at this stage the system has to choose among a smaller number of plans, it can perform extensive analysis for each of the plans in much more detail than the planner, which has to consider a possibly very large set of candidate plans and, therefore, has very limited time and resources available to analyze each plan.
The process consists of the following steps, as shown on
The set of selected plans (3) submitted for final selection is typically much smaller in number than the set of candidate plans considered by the planner. On the other hand, this set represents diverse plans representing various tradeoffs between cost and quality and does not contain obviously inefficient plans due to plan optimality property. Because of the small plan set size, the selection procedure applied at this step can spend more computational resources on the analysis of each plan than the planner spends on the analysis of each candidate plan. Therefore, a more detailed analysis can be performed, although at a coarser granularity of plans, when compared to the analysis of plan candidates performed by the planner. The overall process according to this feature of the present invention achieves the best balance between system performance and quality of the plan chosen for execution.
In an exemplary embodiment, the automatic procedure of selecting one plan from the set of plausible plans can be performed by the scheduler, that is, the component of the system responsible for scheduling. The scheduler allocates resources for plan execution. During plan selection the scheduler performs detailed analysis of resource utilization for each of the candidate plans, and selects the one that is the most appropriate given current resource availability. This separation of functionality between the planner and the scheduler improves planning performance by allowing the use of a coarse-grained resource model during planning, and taking further resource considerations into account when scheduling.
During this process the number of the set of plans is gradually reduced following a sequence, such as listed hereafter, in which a subsequent set is a subset of the set preceding it in the sequence: (1) candidate plans, (2) resulting plans selected by the planner, (3) plans selected by the user, and (4) the plan selected for execution, as shown in connection with
In accordance with an exemplary embodiment of the present invention as represented in
The set of efficient plans provided by the planner 20 may be used for multiple different purposes. In the domains where the plans model the behavior of an agent, which strives to achieve a certain goal, studying the set of efficient plans can help, for example, to test the defense or hedging strategies used against adversary agents. In the same domains, the set of efficient plans can also be used to train systems that predict the behavior of an agent and to detect sequences of actions that can be leading up to a particular goal. As described in the following, studying this set also can help to identify errors in the domain descriptions as stored in the domain description storage unit 30.
To enable the exploration of the set of efficient plans, the planner 20 should still return to the user 10 a relatively small, and easily observable, number of diverse plans. The user 10 should be able to specify a preferred number of plans to be returned by the planner 20. At the same time, the planner 20 should allow planning requests requiring that all returned plans belong to specified ranges of quality and cost, which is needed when the user 10 needs to selectively increase the level of detail.
One application of the exemplary embodiment of the present invention is that the set of efficient plans can be used for attack detection and prevention, with intended applications in intrusion detection. If actions in the planning domain describe the actions of attackers, then multiple efficient plans of attack can be automatically generated using the method of the present invention. Efficient plans use the minimally necessary amount of resources for achieving the goal with corresponding quality, and in many domains, for example, where each action incurs cost, do not contain redundant actions as a consequence. Resulting plans of attack produced by the planner 20 can be used to test resilience of a target to the attacks and to detect ongoing attacks based on observations of partially completed plans in progress.
The set of resulting plans (2) can be used to study the impact of planning domain parameters, such as parameters that play a role in cost and quality computation, on the set of candidate plans considered by the planner. If the number of resulting plans can be controlled, using the planner iteratively, and producing more and more resulting plans will provide more and more details about the set of candidate plans by returning more and more representatives of this set. If the planner implementation allows, an increasing level of detail could be requested specifically for the regions of cost and quality in which problems, such as unexpected choices made by the planner, are occurring. With this approach, a planner producing multiple solutions can be used for identifying and resolving problems in domain definition, as represented in
The process of identifying and correcting errors in the domain descriptions consists of the following steps, as represented in
In some applications of planning it is desirable to produce plans that achieve multiple goals, some of which are optional. Achieving the optional goals contributes to the overall quality of the solution, but it may also increase the cost of the plan.
This situation can be easily represented as a problem of planning with a single or mandatory goal, in which the optional goals are represented by preconditions for special-purpose actions that have the only effect of increasing the overall plan quality value. The efficient plans, by definition, will achieve the optimal subset of optional goals for corresponding cost.
In summary, the method of generating multiple efficient plans can be used for planning with optional goals without modification. Minor changes to the planning domain description may be made in order to represent optional goals via dummy actions. The overall approach of planning with optional goals can be supported in all the situations described above.
The problems of planning with optional goals are most directly related to recent work on oversubscription planning and planning with goal preferences, as described hereinabove. A main difference between previously proposed systems and the exemplary embodiments of the present invention is that multiple plans are created for different ranges of constraints in the present invention. Also, exemplary embodiments of the present invention do not require resource or quality measures to be tied to goals, instead any method of evaluating resources and quality of plans is acceptable and may be used.
The methods of planning for oversubscribed problems can be used to generate efficient candidate plans, as part of the algorithm described in detail herein below. Partial order quality models, as well as more common numeric quality models, can be accommodated by embodiments of the present invention, since component-wise comparison of quality vectors defines a partial order, and the definition of efficient plans for partial order quality, as provided by exemplary embodiments of the present invention.
According to an exemplary embodiment the present invention, a cost interval approach is one of several methods of generating multiple efficient plans for the same planning task. The advantages of the cost interval approach are simplicity of implementation and a natural method of presenting the resulting set of plans, which is especially important when the set must be inspected by human users.
The cost interval approach can be applied under the conditions where the quality of a plan is a real number, and the cost of a plan is a multi-dimensional vector of real numbers.
Planners using the cost interval approach produce efficient plans, which have optimal or near-optimal quality for the corresponding resource range. It is essential for this approach that the quality measure has total ordering, and every pair of plans can be compared.
The cost interval approach consists of the following three steps. (1) Dividing the range of cost values in each dimension of the plan cost vector into a finite number of non-overlapping intervals spanning the entire range. The intervals can have the same length, or can be arbitrarily selected, or specified by the user, or computed using other methods. As a result, this procedure generates a grid in the space of cost vectors. Each cell of this grid is a multidimensional rectangle, the projection of which onto each cost dimension is one of the cost intervals corresponding to that dimension. (2) For each cell of this grid, finding one plan that has the highest or close-to-highest quality among all plans for which the cost vector falls within the cell. This plan is referred to herein as the selected plan for this cell. If there are no plans that have a cost vector within the cell, the cell remains empty. (3) For each non-empty cell, comparing the quality of the plan chosen in that cell to the quality of plans selected in non-empty cells that correspond to the same or lower cost in each dimension. If one of the plans in the lower-cost cells have the same or higher quality as the higher-cost plan, the higher-cost plan is removed from its cell, and the cell becomes empty.
Step (2) above ensures that the selected plans are quality-optimal within their cells. Step (3) above guarantees that after elimination the remaining selected plans are quality-optimal within the entire range of costs. Therefore, the remaining selected plans, or any subset thereof, can be returned by the planner as the set of resulting plans. It should be understood that the Steps (2) and (3) can be combined into one step, and the elimination of inefficient plans in Step (3) can be performed at any point during the search for best plans in the cells, as performed in step (2). In addition, step (2) can be performed separately for each cell, as well as in parallel for multiple cells.
The interval grid in the space of cost vectors can be provided as a parameter to the planning algorithm, which provides a method of controlling the density of the resulting solutions in the cost space. Finer interval grids can be used to request more solutions from the planner, and a higher concentration of intervals in regions of interest can be used to study the properties of the candidate plan set defined by the domain description and the planning problem description. Such analysis can be used for eliminating problems in domain descriptions and planning problem descriptions.
The set of resulting plans can be visualized using the grid structure generated at step (1), above. The visualization method relies on the fact that each cell of the grid contains at most one selected plan. In the case of a two-dimensional cost vector, the plans can be visualized using a table, as shown below:
In such a table, the words “Plan A” and “Plan B” can be replaced by an actual rendering of a plan, or by a hyperlink or a button activation of which will show a detailed rendering of the plan to the user. Visualization methods used in rendering a single plan can be general or application dependent and can be developed separately.
If projection of the grid onto one of the intervals in one of the dimensions does not contain any selected plans, that is, for cost dimension d and interval id with the range of cost values [x(id), x(id+1)] each cell of the grid corresponding to that interval is empty after the elimination step, (3), is completed, and the interval can be merged with a neighboring (higher- or lower-cost) interval. In one case, the intervals [x(id), x(id+1)] and [x(id+1), x(id+2)] are replaced by [x(id), x(id+2)], and selected plans from cells corresponding to the interval [x(id+1), x(id+2)] are assigned to the cells formed by the new interval with the corresponding intervals on other dimensions. This procedure can be repeated until no empty intervals remain, and the most compact representation is obtained.
When the cost vector has more than 2 dimensions, multiple tables or a three-dimensional grid visualization based on the interval grid can be implemented.
In the following, a structure of the planning algorithm based on search is described. It is assumed that the search procedure is implemented, and it allows to enumerate all candidate plans. The outline of the main steps of the planning algorithm is set forth below.
The algorithm relies on grid data structure G that encapsulates the interval grid described above. It is a multidimensional array, which can store at most one plan in each cell. The cells correspond to multidimensional rectangles and can be addressed either by coordinates of any point that lies within the multidimensional rectangle or by cell indices.
This algorithm can be implemented on top of any search procedure that enumerates candidate plans, as described in step (2), above. Various exhaustive search methods, such as a breadth-first search and a depth-first search, have been proposed as general methods for solving combinatorial problems. These methods have also been used for planning in planners based on forward or backward search, for example.
The size of the grid can be used to control the range of the cost and quality, and the granularity of the grid, that is, the cell size, can be used to control the maximum number of plans. The number of the plans in a set returned by the algorithm described above will not exceed the number of cells in the grid. If the candidate plan P has quality or cost outside of grid range at step 4 of the above algorithm, the plan is not registered in the grid, and the iteration continues. The search procedure may use the bounds on resource and quality to perform an efficient pruning and avoid enumerating candidate plans that are outside of grid range.
Depending on the particular implementation at this point, the search may or may not be complete, that is, it may not provide a guarantee that all efficient candidate plans are enumerated. If the search is not complete, the resulting set of plans may be suboptimal, that is, it may contain plans that are not efficient. This solution, however, may still be acceptable for many purposes, especially if the search typically finds a large number of efficient plans. In some planning domains, constructing a suboptimal solution can be the only practical approach, since planning is an undecidable problem in general, and the search must be terminated after the computation time limit is reached.
The procedure of eliminating inefficient plans, EliminatelnefficentPlans(G), which is called out at Step 8 of the above Algorithm Plan( ), eliminates plans that are not efficient. This is done by verifying that the quality of each plan in the grid G dominates the quality of the plans in the lower cost cells of the same grid. All plans violating this efficiency condition are removed from this grid by this procedure. The steps of the algorithm for eliminating inefficient plans are presented below.
It should be noted that the algorithms described herein are encoded at a high level, and mainly serve to show that the desired functionality can be achieved by simple code.
More efficient implementations are possible if the array structure of G is used for indexing directly. For example, grid implementation relies on a multidimensional array and, therefore, cells can be addressed with integer indices of intervals, where intervals are sorted in an increasing order, Index 0=[0,0,0, . . . ,0] will correspond to the lowest-resource cell. The dimensionality of the index vector is the same as the dimensionality of the cost vector. Then, the following more efficient algorithm implementation can be provided by the exemplary embodiment of the present invention.
In the above-described algorithm ji is denoted at the relation between two cell indices i and j that is true if and only if j differs from i in exactly one element of the index vector, and the difference between this element in i and same element in j is exactly 1. In other words, this means that j is the immediate neighbor cell of i with a smaller index in exactly one dimension. The number of operations performed by this modification of the algorithm is linear in the number of cells in the grid, which is less than the quadratic number of operations in the first implementation.
Although exemplary embodiments of the invention have been described herein with reference to the accompanying drawings, it is to be understood that the invention is not limited to these exemplary embodiments, and that various other changes and modifications may be affected therein by one skilled in the art without departing from the spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as defined in the appended claims.
This invention was made with Government support under Contract No. H98230-04-3-001 awarded by the U.S. Department of Defense. The Government has certain rights in this invention.
