The present invention relates to a data processing method and system for knowledge management, and more particularly to a technique for identifying related problem tickets in an information technology environment.
Defect prevention in an information technology environment analyzes historical problem tickets to identify reoccurring problems, explore opportunities for prevention solutions, and track the impact of solution implementation. Identifying sets of tickets that relate to a common system or process problem is a first step towards assessing the root cause and devising actions to eliminate or avoid a reoccurrence of the problem. High accuracy and low costs of identifying related tickets are critical for efficient services delivery.
In first embodiments, the present invention provides a method of identifying related problem tickets in an information technology (IT) environment. The method includes a computer storing user interactions with a computer program. The user interactions include inputs to the computer program to search for problem tickets issued in the IT environment that have the same characteristics. The method further includes the computer recognizing one or more user interaction patterns within the user interactions. The method further includes the computer selecting a user interaction pattern of the one or more user interaction patterns based on an evaluation of effectiveness of each of the one or more user interaction patterns. The method further includes the computer generating a rule, based on the user interaction pattern, for determining which problem tickets in the IT environment share a common characteristic. The method further includes the computer applying the rule to additional problem tickets issued in the IT environment to identify which of the additional problem tickets share the common characteristic.
In second embodiments, the present invention provides a computer system including a central processing unit (CPU), a memory coupled to the CPU, and a computer-readable, tangible storage device coupled to the CPU. The storage device contains instructions that, when carried out by the CPU via the memory, implement a method of identifying related problem tickets in an information technology (IT) environment. The method includes the computer system storing user interactions with a computer program, the user interactions comprising inputs to the computer program to search for problem tickets issued in the IT environment that have the same characteristics. The method further includes the computer system recognizing one or more user interaction patterns within the user interactions. The method further includes the computer system selecting a user interaction pattern of the one or more user interaction patterns based on an evaluation of effectiveness of each of the one or more user interaction patterns. The method further includes the computer system generating a rule, based on the user interaction pattern, for determining which problem tickets in the IT environment share a common characteristic. The method further includes the computer system applying the rule to additional problem tickets issued in the IT environment to identify which of the additional problem tickets share the common characteristic.
In third embodiments, the present invention provides a computer program product including a computer-readable, tangible storage device having computer-readable program instructions stored therein, the computer-readable program instructions, when carried out by a central processing unit (CPU) of a computer system, implement a method of identifying related problem tickets in an information technology (IT) environment. The method includes the computer system storing user interactions with a computer program, the user interactions comprising inputs to the computer program to search for problem tickets issued in the IT environment that have the same characteristics. The method further includes the computer system recognizing one or more user interaction patterns within the user interactions. The method further includes the computer system selecting a user interaction pattern of the one or more user interaction patterns based on an evaluation of effectiveness of each of the one or more user interaction patterns. The method further includes the computer system generating a rule, based on the user interaction pattern, for determining which problem tickets in the IT environment share a common characteristic. The method further includes the computer system applying the rule to additional problem tickets issued in the IT environment to identify which of the additional problem tickets share the common characteristic.
Embodiments of the present invention capture user interactions with existing analysis tools to guide a subsequent automated analysis of defects by a software-based tool, where explicit training of the tool is minimal and the accuracy of the automated analysis is similar to human-based results.
Overview
Embodiments of the present invention recognize that human expert analysis achieves high accuracy in the identification of related incident tickets (a.k.a. problem tickets) in an information technology (IT) environment at a very high data analysis cost. Embodiments of the present invention further recognize that known software-based tools can support human expert analysis, but the tools have diverse degrees of accuracy and overhead. For example, known classification/data mining algorithms have limited accuracy because of a domain specific sublanguage. Embodiments of the present invention further recognize that the aforementioned known tools require extensive human effort to train the tools. For example, for ticket classification into defect categories, a human analyst (e.g., Quality Analyst; a.k.a. QA) must build rule sets for rule-based classification, or manually label a large set of tickets for use with learning-based methods. Embodiments of the present invention further recognize that human action is necessary to determine sets of tickets that identify a problem based on defect category and/or specific ticket details. Embodiments of the present invention further recognize that although automation provided by tools can handle massive volumes of defects, automation on its own cannot adequately deal with the diverse and highly dynamic nature of incoming defects.
Embodiments of the present invention monitor and store user interaction patterns with data analysis tools that detect defects in an IT environment. Rules and/or methods are generated from the user interaction patterns. Effective user interaction patterns are identified. New data is analyzed using the generated rules and/or methods.
System for Identifying Related Problem Tickets
A capture user interaction patterns tool 122 monitors the patterns of user interactions with ticket analysis tool 112. For example, capture user interaction patterns tool 122 monitors a user's drill down pattern (i.e., a prioritization of attributes, which may include selecting the feature of “severity” of an incident ticket first). As another example, capture user interaction patterns tool 122 monitors the filter terms selected by a user, which may include a user selecting Severity 1 as a first filter and then selecting “APPLICATION” as a next filter. As still another example, capture user interaction patterns tool 122 monitors dictionary terms entered by a user, such as monitoring terms based on the regular expression {Process|Hang|Reboot} being used within the type “APPLICATION.” Capture user interaction patterns tool 122 also captures details that describe the steps of a group of patterns of user interactions with ticket analysis tool 112, determines which steps are relevant or not relevant, and determines how results of one step become input for subsequent steps.
Capture user interaction pattern tool 122 stores each of the monitored user interaction patterns in a data repository for usage patterns 124 or in a data repository for grouping patterns 126. Usage patterns 124 may include attribute(s) or column(s) selected by a user and keyword(s) entered by the user. Grouping patterns 126 may include how columns of data are ordered by a user and what filters are selected by the user.
A software-based generate signature tool 128 receives the user interaction patterns monitored and stored by capture user interaction patterns tool 122. Generate signature tool 128 includes a conversion module 130 and a weight assignment module 132. Conversion module 130 converts each group of stored user interaction patterns to a knowledge representation by generating analysis procedures corresponding to respective groups of user interaction patterns (i.e., corresponding to respective group signatures). As used herein, a group signature is defined as a group of user interaction patterns. Each generated analysis procedure includes rule(s) and/or method(s) that describe the relevant steps of a corresponding group of user interaction patterns, and include a specification of how results of one step of a group of user interaction patterns becomes input for one or more subsequent steps. In one embodiment, conversion module 130 transforms each stored group of user interaction patterns into a script that can be run to repeat the steps included in the group of user interaction patterns.
Weight assignment module 132 evaluates each group of user interaction patterns (i.e., each group signature) and assigns a weight to each group signature based on the evaluation. Weight assignment module 132 evaluates the quality of groups of monitored user interaction patterns in order to select the most effective groups of user interaction patterns for use with automated or semi-automated analysis.
In one embodiment, weight assignment module 132 evaluates each group signature based on how many incident tickets are selected by the steps of the group of user interaction patterns associated with the group signature. Weight assignment module 132 may apply clustering algorithms on geographical and pool groups to determine the most prevalent user interaction patterns and may allow for the formation of ticket groups (i.e., inferred ticket categories or classes). For example, weight assignment module 132 may employ principal component analysis (PCA), where the feature set is defined as a subset of user interaction patterns, to determine the most prevalent user interaction patterns.
In one embodiment, generate signature tool 128 provides a semi-automated evaluation of a group signature by requesting user input to confirm the usefulness of components of the group signature. In another embodiment, generate signature tool 128 evaluates a group signature based on automated analyses that determine measures of significance of components of the group signature. For example, generate signature tool 128 may perform frequency analysis of a keyword used in a filter, where the frequency analysis indicates that the keyword is not a significant influence on the filter and may be ignored.
Generate signature tool 128 sends a group signature 134 and a candidate ticket group 136 to a similarity explorer tool 138, which are depicted in portion 100-2 in
An investigation management component 140 of portion 100-2 manages an investigation of root causes of defects in an IT environment to implement proactive prevention of future defects. Investigation management component 140 includes an open investigation module 142 that receives candidate ticket group 136 and group signature 134, and opens an investigation for proactive prevention of defects. Investigation management component 140 further includes a root cause analysis module 144 that determines a root cause of the incidents described in candidate ticket group 136. Investigation management component 140 further includes an implement module 146 that implements a preventive action to prevent future incidents having the same root cause. Investigation management component 140 further includes a validate and close module 148 that validates the effectiveness of the action implemented by implement module 146 and subsequently closes the investigation that had been opened by open investigation module 142. The validate and close module 148 stores the group signature 134 and associated information from the particular investigation being managed by investigation management component 140 (i.e., information about the root cause and preventive action taken to avoid the defect).
A trend analysis tool 150 included in investigation management component 140 receives the root cause determined by root cause analysis module 144 and monitors incident tickets 106 (see
After the validation provided by validate and close module 148, a generate classifiers tool 152 generates a classifier to define how to automatically classify a new incident ticket. The generated classifier is sent to auto classify by interactions pattern tool 108 (see
The functionality of components of system portions 100-1 (see
Process for Identifying Related Problem Tickets
In one embodiment, monitoring user interactions in step 202 includes capturing details that describe steps of user interactions and any combination of:
For example, a QA selects “Severity 1” as the first attribute and “Application” as the next filter by which incident tickets are to be extracted from the incident tickets 106 (see
In one embodiment, capture user interaction patterns tool 122 (see
In step 204, capture user interaction patterns tool 122 (see
In one embodiment, capture user interaction patterns tool 122 (see
In step 206, generate signature tool 128 (see
In another embodiment, generate signature tool 128 (see
Step 206 also includes generate signature tool 128 (see
Further, step 206 includes generate signature tool 128 (see
In step 208, based on the effectiveness evaluated in step 206, generate signature tool 128 (see
In step 210, based on the user interaction pattern selected in step 208, generate classifiers tool 152 (see
In step 212, auto classify interaction patterns tool 108 (see
Step 212 does not cluster the additional incident tickets based on a relevance matrix between query patterns and tickets. Instead, the classification (or topic) of the user interaction patterns is predetermined by the candidate tickets initially clustered in step 204 and are refined iteratively using a classifier newly generated by generate classifiers tool 152 (see
Applying the rule in step 212 may be included in a first use case scenario in which new incident tickets assigned to a pool are automatically classified (see
The process of
Classifying a Problem Ticket
In step 304, ticket analysis tool 112 (see
In step 306, a user interacting with ticket analysis tool 112 (see
In step 308, visualize module 120 (see
In step 310, based on the visualization provided in step 308, ticket analysis tool 112 (see
In step 312, capture user interaction patterns tool 122 (see
Step 312 also includes generate signature tool 128 (see
Generate signature tool 128 (see
In step 318, root cause analysis module 144 (see
In step 320, implement module 146 (see
In step 322, which follows step 320, validate and close module 148 (see
In step 324, which also follows step 320, interaction management database 139 (see
In step 326, generate classifiers tool 152 (see
In step 328, generate classifiers tool 152 (see
In step 330, auto classify by interaction patterns tool 108 (see
Defect Trending
In step 404, open investigation module 142 (see
In step 406, root cause analysis module 144 (see
In step 408, implement module 146 (see
In step 410, validate and close module 148 (see
In step 412, validate and close module 148 (see
In step 416, which follows step 406, trend analysis tool 150 (see
In step 420, trend analysis tool 150 (see
In step 424, trend analysis tool 150 (see
Cross-Pool Defect Reduction Exploration
In step 504, open investigation module 142 (see
In step 506, root cause analysis module 144 (see
In step 508, implement module 146 (see
In step 510, validate and close module 148 (see
In step 514, similarity explorer tool 138 (see
In step 516, similarity explorer tool 138 (see
In step 518, similarity explorer tool 138 (see
In step 520, similarity explorer tool 138 (see
In step 522, if similarity explorer tool 138 (see
Returning to step 522, if similarity explorer tool 138 (see
Following step 524 and step 526, the process of
Computer System
Memory 604 may comprise any known computer-readable storage medium, which is described below. In one embodiment, cache memory elements of memory 604 provide temporary storage of at least some program code (e.g., program code 614) in order to reduce the number of times code must be retrieved from bulk storage while instructions of the program code are carried out. Moreover, similar to CPU 602, memory 604 may reside at a single physical location, comprising one or more types of data storage, or be distributed across a plurality of physical systems in various forms. Further, memory 604 can include data distributed across, for example, a local area network (LAN) or a wide area network (WAN).
I/O interface 606 comprises any system for exchanging information to or from an external source. I/O devices 610 comprise any known type of external device, including a display device (e.g., monitor), keyboard, mouse, printer, speakers, handheld device, facsimile, etc. Bus 608 provides a communication link between each of the components in computer system 102, and may comprise any type of transmission link, including electrical, optical, wireless, etc.
I/O interface 606 also allows computer system 102 to store information (e.g., data or program instructions such as program code 614) on and retrieve the information from computer data storage unit 612 or another computer data storage unit (not shown). Computer data storage unit 612 may comprise any known computer-readable storage medium, which is described below. For example, computer data storage unit 612 may be a non-volatile data storage device, such as a magnetic disk drive (i.e., hard disk drive) or an optical disc drive (e.g., a CD-ROM drive which receives a CD-ROM disk).
Memory 604 and/or storage unit 612 may store computer program code 614 that includes instructions that are carried out by CPU 602 via memory 604 to identify related problem tickets in an IT environment. Although
Further, memory 604 may include other systems not shown in
Storage unit 612 and/or one or more other computer data storage units (not shown) that are coupled to computer system 102 may store incident tickets 106 (see
As will be appreciated by one skilled in the art, the present invention may be embodied as a system, method or computer program product. Accordingly, an aspect of an embodiment of the present invention may take the form of an entirely hardware aspect, an entirely software aspect (including firmware, resident software, micro-code, etc.) or an aspect combining software and hardware aspects that may all generally be referred to herein as a “module”. Furthermore, an embodiment of the present invention may take the form of a computer program product embodied in one or more computer-readable medium(s) (e.g., memory 604 and/or computer data storage unit 612) having computer-readable program code (e.g., program code 614) embodied or stored thereon.
Any combination of one or more computer-readable mediums (e.g., memory 604 and computer data storage unit 612) may be utilized. The computer readable medium may be a computer-readable signal medium or a computer-readable storage medium. In one embodiment, the computer-readable storage medium is a computer-readable storage device or computer-readable storage apparatus. A computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, or semiconductor system, apparatus, device or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer-readable storage medium includes: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer-readable storage medium can contain or store a program (e.g., program 614) for use by or in connection with a system, apparatus, or device for carrying out instructions.
A computer readable signal medium may include a propagated data signal with computer-readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium that is not a computer-readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with a system, apparatus, or device for carrying out instructions.
Program code (e.g., program code 614) embodied on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code (e.g., program code 614) for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java®, Smalltalk, C++ or the like and conventional procedural programming languages, such as the “C” programming language or similar programming languages. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates. Instructions of the program code may be carried out entirely on a user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server, where the aforementioned user's computer, remote computer and server may be, for example, computer system 102 or another computer system (not shown) having components analogous to the components of computer system 102 included in
Aspects of the present invention are described herein with reference to flowchart illustrations (e.g.,
These computer program instructions may also be stored in a computer-readable medium (e.g., memory 604 or computer data storage unit 612) that can direct a computer (e.g., computer system 102), other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions (e.g., program 614) stored in the computer-readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowcharts and/or block diagram block or blocks.
The computer program instructions may also be loaded onto a computer (e.g., computer system 102), other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other devices to produce a computer implemented process such that the instructions (e.g., program 614) which are carried out on the computer, other programmable apparatus, or other devices provide processes for implementing the functions/acts specified in the flowcharts and/or block diagram block or blocks.
Any of the components of an embodiment of the present invention can be deployed, managed, serviced, etc. by a service provider that offers to deploy or integrate computing infrastructure with respect to identifying related problem tickets in an IT environment. Thus, an embodiment of the present invention discloses a process for supporting computer infrastructure, wherein the process comprises providing at least one support service for at least one of integrating, hosting, maintaining and deploying computer-readable code (e.g., program code 614) in a computer system (e.g., computer system 102) comprising one or more processors (e.g., CPU 602), wherein the processor(s) carry out instructions contained in the code causing the computer system to identify related problem tickets in an IT environment.
In another embodiment, the invention provides a method that performs the process steps of the invention on a subscription, advertising and/or fee basis. That is, a service provider, such as a Solution Integrator, can offer to create, maintain, support, etc. a process of identifying related problem tickets in an IT environment. In this case, the service provider can create, maintain, support, etc. a computer infrastructure that performs the process steps of the invention for one or more customers. In return, the service provider can receive payment from the customer(s) under a subscription and/or fee agreement, and/or the service provider can receive payment from the sale of advertising content to one or more third parties.
The flowcharts in
While embodiments of the present invention have been described herein for purposes of illustration, many modifications and changes will become apparent to those skilled in the art. Accordingly, the appended claims are intended to encompass all such modifications and changes as fall within the true spirit and scope of this invention.
This application is a continuation application claiming priority to Ser. No. 13/362,089 filed Jan. 31, 2012, now U.S. Pat. No. 8,898,092, issued Nov. 25, 2014.
Number | Name | Date | Kind |
---|---|---|---|
7406430 | Atefi et al. | Jul 2008 | B2 |
7984007 | Reumann et al. | Jul 2011 | B2 |
8266072 | Grace et al. | Sep 2012 | B2 |
8898092 | Bhamidipaty et al. | Nov 2014 | B2 |
20050027816 | Olney et al. | Feb 2005 | A1 |
20050033464 | Nguyen | Feb 2005 | A1 |
20050081118 | Cheston et al. | Apr 2005 | A1 |
20070118419 | Maga et al. | May 2007 | A1 |
20090216697 | Beaty et al. | Aug 2009 | A1 |
20090276728 | Doan et al. | Nov 2009 | A1 |
20090292742 | Morgan | Nov 2009 | A1 |
20100030768 | Poblete et al. | Feb 2010 | A1 |
20100082620 | Jennings, III et al. | Apr 2010 | A1 |
20100262615 | Oztekin et al. | Oct 2010 | A1 |
20130198116 | Bhamidipaty et al. | Aug 2013 | A1 |
Entry |
---|
Lee et al.; Micro Interaction Metrics for Defect Prediction [online], Jun. 15, 2011. [retrieved on Jun. 23, 2014]. Retrieved from the Internet URL: http://www.cse.ust.hk/˜hunkim/images/c/c2/Mim—fse2011.pdf; 11 pages. |
Method and Apparatus to Manage Layout of User Interfaces Based on User Interaction Patterns, IP.com No. IPCOM000202350D, IP.com Electronic Publication: Dec. 14, 2010, 4 pages. |
Office Action (Mail Date Jan. 16, 2014) for U.S. Appl. No. 13/362,089, filed Oct. 31, 2012, Confirmation No. 4759. |
Amendment filed Apr. 16, 2014 in response to Office Action (Mail Date Jan. 16, 2014) for U.S. Appl. No. 13/362,089, filed Oct. 31, 2012; Confirmation No. 4759. |
Notice of Allowance (Mail Date May 7, 2014) for U.S. Appl. No. 13/362,089, filed Oct. 31, 2012; Confirmation No. 4759. |
Request for Continued Examination filed Jun. 23, 2014 for U.S. Appl. No. 13/362,089, filed Oct. 31, 2012; Confirmation No. 4759. |
Notice of Allowance (Mail Date Jul. 11, 2014) for U.S. Appl. No. 13/362,089, filed Oct. 31, 2012; Confirmation No. 4759. |
Number | Date | Country | |
---|---|---|---|
20150033077 A1 | Jan 2015 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 13362089 | Jan 2012 | US |
Child | 14514029 | US |