VISUALIZATION DISPLAY DEVICE, VISUALIZATION DISPLAY METHOD, AND VISUALIZATION DISPLAY PROGRAM

Abstract

A visualization display device includes visualization display circuitry configured to acquire an operation log, generate an operation sequence in units of operations arrayed in a time series order for each case on the basis of the operation log and align the operation sequence in units of operations, and specify operation types that appear with a threshold or more at the same position in the operation sequence in units of operations of each case as operation types of a main flow on the basis of the aligned operation sequence in units of operations and arrange and visualize nodes corresponding to the operation types of the main flow in a time series order on one axis.

Description

TECHNICAL FIELD

The present invention relates to a visualization display device, a visualization display method, and a visualization display program.

BACKGROUND ART

In order to realize effective business improvement, it is important for business analysts (hereinafter also referred to as “users”) to correctly ascertain an actual business situation. In the past, as a method of accurately and efficiently ascertaining an actual business situation, there has been a method of recording content of operations performed on a terminal by a person in charge of business as an operation log and displaying the content as a flowchart in the order of operations executed for each corresponding case and by superimposing operations of the same type (see, for example, NPL 1). In addition, in order to compare differences in operations for each case, lining up and displaying the sequence of operations in the order of operations executed for each case can also be considered. Here, the term “operation” refers to an event that causes a change on a screen by clicking a graphical user interface (GUI) component or inputting a value.

For example, in a case where a business analyst attempts to find a location to which robotic process automation (hereinafter referred to as “RPA”) can be applied from a flowchart, it is said that it is effective for the business analyst to first find a location where RPA can be applied effectively, for example, a common and repeated operation procedure (hereinafter referred to as a “main flow”) among operations for each case. Therefore, collecting operations performed for business execution as logs and visualizing an operation procedure in the form of a flowchart in which operations of the same type are treated as one node and the operations are connected by edges is said to be an effective visualization method for finding the main flow.

CITATION LIST

Non Patent Literature

• [NPL 1] Urabe, Yagi, Tsuchikawa, Masuda: “Study of business process visualization method using operation log as input,” IEICE Technical Report, vol. 118, No. 483, ICM2018-64, pp. 83-88, 2019.

SUMMARY OF INVENTION

Technical Problem

However, in the related art described above, it is not possible to easily specify the main flow in business analysis. This is because there are a wide variety of operation procedures performed by a person in charge of business, and thus in a case where all the operation procedures are visualized without distinction, it is difficult to specify which operation procedure is the main flow without knowledge of target business or knowledge of how to view and analyze a flowchart.

Solution to Problem

In order to solve the above problem and achieve the object, according to the present invention, there is provided a visualization display device including: an acquisition unit configured to acquire an operation log; an alignment unit configured to generate an operation sequence in units of operations arrayed in a time series order for each case on the basis of the operation log and to align the operation sequence in units of operations; and a visualization unit configured to specify operation types that appear with a threshold or more at the same position in the operation sequence in units of operations of each case as operation types of a main flow on the basis of the aligned operation sequence in units of operations and to arrange and visualize nodes corresponding to the operation types of the main flow in a time series order on one axis.

In addition, according to the present invention, there is provided a visualization display method which is executed by a visualization display device, the method including: an acquisition step of acquiring an operation log; an alignment step of generating an operation sequence in units of operations arrayed in a time series order for each case on the basis of the operation log and aligning the operation sequence in units of operations; and a visualization step of specifying operation types that appear with a threshold or more at the same position in the operation sequence in units of operations of each case as operation types of a main flow on the basis of the aligned operation sequence in units of operations and arranging and visualizing nodes corresponding to the operation types of the main flow in a time series order on one axis.

In addition, according to the present invention, there is provided a visualization display program causing a computer to execute: an acquisition step of acquiring an operation log; an alignment step of generating an operation sequence in units of operations arrayed in a time series order for each case on the basis of the operation log and aligning the operation sequence in units of operations; and a visualization step of specifying operation types that appear with a threshold or more at the same position in the operation sequence in units of operations of each case as operation types of a main flow on the basis of the aligned operation sequence in units of operations and arranging and visualizing nodes corresponding to the operation types of the main flow in a time series order on one axis.

Advantageous Effects of Invention

In the present invention, it is possible to easily specify a main flow in business analysis.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a configuration example of a visualization display device according to an embodiment.

FIG. 2 is a diagram illustrating an example of an operation log stored in a storage unit according to the embodiment.

FIG. 3 is a diagram illustrating an example of alignment processing according to the embodiment.

FIG. 4 is a diagram illustrating an example of alignment processing according to the embodiment.

FIG. 5 is a diagram illustrating an example of alignment processing according to the embodiment.

FIG. 6 is a diagram illustrating an example of alignment processing according to the embodiment.

FIG. 7 is a diagram illustrating an example of alignment processing according to the embodiment.

FIG. 8 is a diagram illustrating an example of alignment processing according to the embodiment.

FIG. 9 is a diagram illustrating an example of alignment processing according to the embodiment.

FIG. 10 is a diagram illustrating an example of alignment processing according to the embodiment.

FIG. 11 is a diagram illustrating an example of visualization processing related to the embodiment.

FIG. 12 is a diagram illustrating an example of visualization processing related to the embodiment.

FIG. 13 is a diagram illustrating an example of visualization processing related to the embodiment.

FIG. 14 is a flowchart illustrating an example of the overall flow of visualization display processing according to the embodiment.

FIG. 15 is a flowchart illustrating an example of a flow of operation log acquisition processing according to the embodiment.

FIG. 16 is a flowchart illustrating an example of a flow of alignment processing according to the embodiment.

FIG. 17 is a flowchart illustrating an example of a flow of visualization processing related to the embodiment.

FIG. 18 is a diagram illustrating an example of an operation log stored in a storage unit according to a modification example of the embodiment.

FIG. 19 is a diagram illustrating an example of alignment processing according to the modification example of the embodiment.

FIG. 20 is a diagram illustrating an example of alignment processing according to the modification example of the embodiment.

FIG. 21 is a diagram illustrating an example of alignment processing according to the modification example of the embodiment.

FIG. 22 is a diagram illustrating an example of alignment processing according to the modification example of the embodiment.

FIG. 23 is a diagram illustrating an example of alignment processing according to the modification example of the embodiment.

FIG. 24 is a diagram illustrating an example of alignment processing according to the modification example of the embodiment.

FIG. 25 is a diagram illustrating an example of alignment processing according to the modification example of the embodiment.

FIG. 26 is a diagram illustrating an example of alignment processing according to the modification example of the embodiment.

FIG. 27 is a diagram illustrating an example of alignment processing according to the modification example of the embodiment.

FIG. 28 is a diagram illustrating an example of alignment processing according to the modification example of the embodiment.

FIG. 29 is a diagram illustrating an example of alignment processing according to the modification example of the embodiment.

FIG. 30 is a diagram illustrating an example of alignment processing according to the modification example of the embodiment.

FIG. 31 is a diagram illustrating an example of alignment processing according to the modification example of the embodiment.

FIG. 32 is a diagram illustrating an example of alignment processing according to the modification example of the embodiment.

FIG. 33 is a diagram illustrating an example of visualization processing related to the modification example of the embodiment.

FIG. 34 is a diagram illustrating an example of visualization processing related to the modification example of the embodiment.

FIG. 35 is a flowchart illustrating an example of a flow of alignment processing according to the modification example of the embodiment.

FIG. 36 is a flowchart illustrating an example of a flow of visualization processing related to the modification example of the embodiment.

FIG. 37 is a diagram illustrating a computer that executes a program.

FIG. 38 is a diagram illustrating conventional visualization display processing.

FIG. 39 is a diagram illustrating conventional visualization display processing.

DESCRIPTION OF EMBODIMENTS

Hereinafter, an embodiment of a visualization display device, a visualization display method, and a visualization display program according to the present invention will be described in detail with reference to the accompanying drawings. Meanwhile, the present invention is not limited by an embodiment to be described below.

Embodiment

Hereinafter, processing of a visualization display system according to an embodiment (appropriately, the present embodiment), the configuration of a visualization display device 10, the details of the processing, and a flow of the processing will be described in order, and finally the effects of the present embodiment will be described.

[Processing of Visualization Display System]

Processing of a visualization display system (appropriately, the present system) according to the present embodiment will be described below. Processing of the present system will be described below in comparison with the related art.

(1. Conventional visualization display processing) As described above, it is effective for a business analyst to find a main flow which is expected to have a high operation frequency and a high improvement effect among operations for each case as a location where the application of RPA is effective. Therefore, it is called a visualization method effective for finding a main flow to collect operations performed for business execution as a log and to visualize the operation procedure in the form of a flowchart with operations of the same type as one node and the operations connected by edges as shown in FIG. 38. However, there are a wide variety of operation procedures, and thus in a case where all the operations procedures are visualized without distinction, it is difficult to specify which operation procedure is a main flow without knowledge of a target business and knowledge of how to view and analyze flowcharts (problem a).

Further, in a case where a flowchart is configured with operations of the same type as shown in FIG. 38 superimposed and expressed as one node and operations of the same type occur multiple times in a case, it is difficult to strictly ascertain the order of operations using the method shown in FIG. 38 (problem b). For example, in FIG. 38, the operation “0” is followed by the operation “1.” There are an operation to return from “1” to the same “1” and an operation to transition to “0.” The flow of the operations can be considered to be “0, 1, 0,” “0, 1, 1, 0,” and “0, 1, 1, 1, (1 is repeated), 0,” and the order of the operations cannot be accurately ascertained. If the main flow and the other operation procedures that appear variously can be separately found from the operation log, a business analyst can quickly examine a method of applying RPA to the automatically specified main flow, a method of leveling and simplifying locations where operations are diversified, or the like without specifying the main flow by himself/herself.

In addition, as a method of extracting a main flow from the operation log, there is a method of finding a main flow from the operation log by preparing operation sequences (operation sequences lined up in the order in which operations are executed) for each case from the operation log acquired from a terminal in a business which is performed repeatedly, specifying the type of operation sequence and the number of occurrences, and setting an operation sequence equal to or more than a threshold as a main flow (flow of operations occurring frequently) on the basis of a threshold set by a user and those that are not as exceptional flows (flows of exceptional operations).

In the above method, it is assumed that, for example, there are 100 operation sequences which are divided into seven types. When the user sets the threshold to 2, it is a method of determining two types having the highest number of occurrences among the seven types of operation sequences as the main flow and the remaining five types as exceptional flows. However, since there are various types of operations in the logs acquired in units of operations, it is unlikely that there is a completely coincident operation sequence even if an operation sequence is created for each case from the operation log and a main flow is extracted using the above method. Therefore, in the above method, it is difficult to classify the operation sequences in the first place, and the main flow cannot be extracted (problem c).

In addition, there is also a method of observing the operation sequence of a plurality of cases in the order of appearance of operations, and extracting the operations that appear in cases of a threshold or more in order and in consideration of the immediately preceding operation type (see FIG. 39). FIG. 39 shows the operation sequence for each case, and the numerals described for case ID represent the operation type (operation ID). Meanwhile, in FIG. 39, in a case where there is a common operation ID in the operation sequence of another case, the numeral which is the ID is underlined.

For example, in FIG. 39, there are five cases of operation sequences, and the threshold is set to 3. Among a plurality of cases, the first operation that appears most frequently is “0,” and the second operation that appears most frequently next to “0” is “1.” However, in this method, the main flow cannot be extracted in consideration of a case in which the operation extracted as the main flow is missing (problem d). That is, since “21” and “8” are missing in “A002” and “A003,” respectively, for the case ID shown in FIG. 39, the main flow cannot be extracted from the subsequent operation sequence.

(2. Visualization Display Processing According to the Present Embodiment)

In visualization display processing according to the present embodiment, the operation sequence (operations in the order of operation time) for each case related to the same work is used as an input, and the operation logs are aligned so that operations of the same type are arranged in the same sequence insofar as possible on the basis of a method of multiple sequence alignment. Finally, for all the cases, operations that appear with a threshold or more are extracted as the main flow, and operations less than the threshold are extracted as operations other than the main flow, so that these operations are visualized distinctly. That is, in the visualization display processing according to the present embodiment, in business which is performed repeatedly for each case, operation procedures performed for each case are roughly the same as each other, but in a case where a wide variety of operations occur in the middle, the same operation occurs multiple times in one case, or operations are missing in comparison with other cases, the main flow can be extracted from a set of operations for each of the plurality of cases in the same business on the basis of the method of multiple sequence alignment, and the main flow and the other operation procedures can be ascertained distinctly.

Therefore, in the visualization display processing according to the present embodiment, the main flow can be extracted in consideration of all the cases even if there are a variety of types and orders of operations that have occurred in the cases. In addition, in the visualization display processing according to the present embodiment, even if operations of the same type are included multiple times in the case, a process in which the order thereof is expressed can be drawn. Further, in the visualization display processing according to the present embodiment, attention can be paid only to the main flow by performing highlighting and visualization or by setting a threshold and visualizing only data that appears with a threshold or more. From the above, the visualization display processing according to the present embodiment contributes to solving the above problems a to d.

[Configuration of Visualization Display Device 10]

The configuration of the visualization display device 10 according to the embodiment will be described in detail with reference to FIG. 1. FIG. 1 is a block diagram illustrating a configuration example of a visualization display device according to the embodiment. The visualization display device 10 includes an input unit 11, an output unit 12, a communication unit 13, a storage unit 14, and a control unit 15.

(1. Input Unit 11)

The input unit 11 takes charge of input of various types of information to the visualization display device 10. For example, the input unit 11 is realized by a mouse, a keyboard, or the like, and receives input of setting information or the like to the visualization display device 10.

(2. Output Unit 12)

The output unit 12 takes charge of output of various types of information from the visualization display device 10. For example, the output unit 12 is realized by a display or the like, and outputs setting information or the like stored in the visualization display device 10. In addition, the output unit 12 displays the visualization result output by a visualization unit 15c of the control unit 15 to be described later.

(3. Communication Unit 13)

The communication unit 13 takes charge of data communication with another device. For example, the communication unit 13 performs data communication with each communication device. In addition, the communication unit 13 can perform data communication with an operator's terminal (not shown).

(4. Storage Unit 14)

The storage unit 14 stores various types of information referred to when the control unit 15 operates and various types of information acquired when the control unit 15 operates. The storage unit 14 has an operation log storage unit 14a. Here, the storage unit 14 can be realized, for example, by a semiconductor memory element such as a random access memory (RAM) or a flash memory, a storage device such as a hard disk or an optical disc, or the like. Meanwhile, in the example of FIG. 1, the storage unit 14 is installed inside the visualization display device 10, but it may be installed outside the visualization display device 10, and a plurality of storage units may be installed.

(4-1. Operation Log Storage Unit 14a)

An example of an operation log stored in the operation log storage unit 14a will be described with reference to FIG. 2. FIG. 2 is a diagram illustrating an example of an operation log stored in a storage unit according to the embodiment. The operation log storage unit 14a stores an operation log. For example, the operation log storage unit 14a stores “operation time,” “case ID,” “operation location (window information),” “operation location (GUI component, coordinates, etc.),” and “input value” as an operation log.

(5. Control Unit 15)

The control unit 15 takes charge of control of the visualization display device 10 as a whole. The control unit 15 includes an acquisition unit 15a, an alignment unit 15b, and the visualization unit 15c. Here, the control unit 15 is, for example, an electronic circuit such as a central processing unit (CPU) or a micro processing unit (MPU), or an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

(5-1. Acquisition Unit 15a)

The acquisition unit 15a acquires an operation log. For example, the acquisition unit 15a acquires an operation log including information on the operation location. Here, the information on the operation location is window information related to a user's operation, a GUI component name, coordinates, etc., but there is no particular limitation thereto. In addition, the acquisition unit 15a acquires an operation time, a case ID, and an input value. On the other hand, the acquisition unit 15a stores the acquired operation log in the operation log storage unit 14a of the storage unit 14. Meanwhile, the acquisition unit 15a may output the acquired operation log to the alignment unit 15b.

(5-2. Alignment Unit 15b)

The alignment unit 15b generates an operation sequence in units of operations arrayed in a time series order for each case on the basis of the operation log, and aligns the operation sequence in units of operations. For example, the alignment unit 15b specifies operations of the same type by extracting the information on the operation location from the operation log, aligns the operation sequence in units of operations through pairwise alignment, generates a distance matrix for each operation sequence in units of operations, generates a distance vector using the distance matrix, and further aligns the operation sequence through multiple sequence alignment in the order of the operation sequence in units of operations having the smaller distance vector.

That is, as alignment processing, the alignment unit 15b (1) creates a distance matrix using the difference caused by pairwise alignment of case pairs, and (2) generates a distance vector for each case from the distance matrix, to thereby perform multiple sequence alignment in order from the case where the distance between vectors is short. Here, the pairwise alignment involves aligning two sequences, and the multiple sequence alignment involves aligning three or more sequences, but the alignment processing to be executed is not particularly limited.

In addition, the alignment unit 15b outputs the alignment result to the visualization unit 15c. Meanwhile, the alignment unit 15b may store the alignment result in the storage unit 14.

(5-3. Visualization Unit 15c)

The visualization unit 15c specifies operation types that appear with a threshold or more at the same position in the operation sequence in units of operations for each case, as the operation type of the main flow, on the basis of the aligned operation sequence in units of operations, and arranges and visualizes nodes corresponding to the operation type of the main flow in a time series order on one axis. In addition, the visualization unit 15c arranges and visualizes nodes corresponding to operation types other than the operation type of the main flow in a time series order at positions different from on one axis where the nodes corresponding to the operation type of the main flow are arranged. Further, the visualization unit 15c sets a reference value (threshold′ of operation) of an operation to be visualized, arranges nodes corresponding to the operation type of the main flow in a time series order on one axis, among the nodes corresponding to the operation types that appear with the reference value or higher of the operation, and arranges and visualizes the nodes corresponding to the operation types other than the operation type of the main flow in a time series order at positions different from on one axis.

In addition, the visualization unit 15c outputs the visualization result to the output unit 12. Meanwhile, the visualization unit 15c may store the visualization result in the storage unit 14.

[Details of Processing]

The details of processing according to the present embodiment will be described with reference to FIGS. 3 to 13, formulas, and the like. Alignment processing and visualization processing will be described below in detail.

(1. Alignment Processing)

Alignment processing will be described with reference to FIGS. 3 to 10, formulas, and the like. FIG. 3 to FIG. 10 are diagrams illustrating an example of alignment processing according to the embodiment. Hereinafter, ID creation processing, operation sequence generation processing, operation sequence classification processing, sequence alignment processing, and position ID assignment processing will be described in this order.

(1-1. ID Creation Processing)

The ID creation processing will be described with reference to FIG. 3. The alignment unit 15b of the visualization display device 10 creates an operation ID from operation log data. At this time, the alignment unit 15b assigns the operation ID on the basis of (1) data matching GUI components, (2) data matching information on a combination of an active window and the GUI components, and (3) information such as designation performed by a user (for example, designation that an operation a and an operation b are set to have the same operation ID). In FIG. 3, the alignment unit 15b assigns, for example, the operation ID “0” to the combination of operation content “window A”+“operation a” on the basis of the above information of (2).

(1-2. Operation Sequence Generation Processing)

The operation sequence generation processing will be described with reference to FIG. 4. The alignment unit 15b generates an operation sequence based on the operation ID in the order of operation time for each case ID of the operation log acquired from the operation log storage unit 14a. In FIG. 4, the alignment unit 15b generates, for example, “0, 1, 1, 0, 2, 3, 3, . . . ” as an operation sequence based on the operation ID of the case ID “case 1,” “0, 1, 0, 2, 8, 3, 4, . . . ” as an operation sequence based on the operation ID of the case ID “case 2,” and the like. In addition, in FIG. 4, the alignment unit 15b generates an operation sequence based on the operation IDs in the order of operation time for ten cases with case IDs “case 1” to “case 10.”

(1-3. Operation Sequence Classification Processing)

The operation sequence classification processing will be described with reference to FIG. 5. The alignment unit 15b compares the types and orders of operations in the operation sequences and classifies them into types of operation sequences. At this time, the alignment unit 15b determines whether the operation sequence completely matches as a method of determining an operation type. In FIG. 5, for example, since “0, 1, 0, 2, 8, 3, 4, 4, 5, 3, 6, 7, 3, 2, 9, 2, 10, 16, 11, 12, 17, 2” completely matches as the operation sequences based on the operation IDs of the case IDs “case 2” and “case 6,” the alignment unit 15b classifies “case 2” and “case 6” into the operation sequence ID “operation sequence 2.” In addition, in FIG. 5, the alignment unit 15b similarly classifies only “case 1” into “operation sequence 1,” “case 3” and “case 7” into “operation sequence 3,” “case 4,” “case 8,” and “case 10” into “operation sequence 4,” and “case 5” and “case 9” into “operation sequence 5.”

(1-4. Sequence Alignment Processing)

The sequence alignment processing will be described with reference to FIGS. 6 to 9. Hereinafter, the outline of the sequence alignment processing will be described, and then the details of the sequence alignment processing will be described.

(1-4-1. Outline)

The alignment unit 15b uses an operation sequence for each operation sequence type and performs sequence alignment of operation sequences using a progressive alignment method which is one of multiple sequence alignment methods. Specifically, first, the alignment unit 15b applies a sequence alignment method (pairwise alignment) to two operation sequences, extracts a difference score between these operation sequences, and stores it in a distance matrix (procedure 1). Second, the alignment unit 15b calculates the distance matrix of the operation sequences as shown in FIG. 6 by applying the above procedure 1 to all the operations sequences, extracts the distance vector for each operation sequence from the distance matrix, obtains a distance between the operation sequences from the distance vector, and creates a tree diagram assembled from the operation sequences with a short distance therebetween (procedure 2). Third, the alignment unit 15b further performs sequence alignment (multiple sequence alignment) of the operation sequences with a short distance therebetween (procedure 3).

(1-4-2. Details)

(1-4-2-1. Procedure 1)

The details of procedure 1 of the sequence alignment processing will be described with reference to FIG. 7. After setting a specific score calculation recurrence formula, the alignment unit 15b defines a problem of finding a path for maximizing the score among the paths from the upper left cell to the lower right cell, and solves it using a dynamic programming method. However, the alignment unit 15b may use algorithms other than the dynamic programming method.

For example, the alignment unit 15b performs procedure 1 using a score calculation recurrence formula shown in the following Formula (1).

$\begin{array}{cc}\left[\mathrm{Formula}1\right]& \\ F\left(i,j\right)=\max \{\begin{array}{c}F\left(i-1,j-1\right)+s\left(x_i,y_j\right)\dots \lceil \searrow \rfloor \\ F\left(i-1,j\right)-d\dots \lceil \to \rfloor \\ F\left(i,j-1\right)-d\dots \lceil \downarrow \rfloor \end{array}& \left(1\right)\end{array}$

In Formula (1), (i, j) represents the coordinates of (column, row) in the table of FIG. 7. In addition, F(i, j) is a score on the path at coordinates (i, j). In addition, in Formula (1), the score penalty is set to d=0, and s(x_i, y_j) is set to +1 when the operation IDs match at a certain position and set to −1 when they do not match.

In FIG. 7, the alignment unit 15b calculates a difference score (distance matrix) from “operation sequence 2” and “operation sequence 4.” For example, in a case where the score on the path of F(1, 1) is calculated, the relation of s(1, 1)=1 is established because the operation ID of “operation sequence 2” (i=1) is “0,” the operation ID of “operation sequence 4” (j=1) is “0,” and the operation IDs match. Therefore, as shown in the following Formula (2), F(1, 1)=1 is calculated by obtaining the maximum value as a solution.

$\begin{array}{cc}\left[\mathrm{Formula}2\right]& \\ F\left(1,1\right)=\max \{\begin{array}{cc}F\left(0,0\right)+s\left(1,1\right)=1& \\ F\left(0,1\right)-d=0& =1\\ F\left(1,0\right)-d=0& \end{array}& \left(2\right)\end{array}$

Similarly, in a case where the score on the path of F(2, 2) is calculated, the relation of s(2, 2)=1 is established because the operation ID of “operation sequence 2” (i=2) is “1,” the operation ID of “operation sequence 4” (j=2) is “1,” and the operations match. Therefore, as shown in the following Formula (3), F(2, 2)=2 is calculated by obtaining the maximum value as a solution.

$\begin{array}{cc}\left[\mathrm{Formula}3\right]& \\ F\left(2,2\right)=\max \{\begin{array}{cc}F\left(1,1\right)+s\left(2,2\right)=1+1=2& \\ F\left(1,2\right)-d=1& =2\\ F\left(2,1\right)-d=1& \end{array}& \left(3\right)\end{array}$

The alignment unit 15b executes the sequence alignment of “operation sequence 2” and “operation sequence 4” by calculating the scores on the above paths for all the cells and performing a process of finding a path for maximizing the score among the paths from the upper left cell to the lower right cell in FIG. 7. At this time, in a case where there is “→” or “↓” in the operation sequence, the alignment unit 15b lines up operation ID sequences with a blank “-” in that portion. That is, “operation sequence 2” is aligned as “0, 1, 0, 2, 8, 3, 4, 4, 5, 3, 6, 7, 3, 2, 9, 2, 10, 16, 11, 12, 17, 2, -,” and “operation sequence 4” is aligned as “0, 1, 0, 2, 8, 3, 4, 4, 5, 3, 6, 7, 3, 2, 9, 2, 10, -, 11, 12, -, -, 16.”

The alignment unit 15b compares the aligned operation ID sequences, calculates the number of operation IDs that do not match at the compared positions as a difference score, and stores it in the distance matrix. That is, when the aligned operation ID sequences of “operation sequence 2” and “operation sequence 4” are compared with each other, the alignment unit 15b calculates the difference score as “4” and stores it in the distance matrix because four operation IDs are different in the corresponding sequences.

(1-4-2-2. Procedure 2)

The details of procedure 2 of the sequence alignment processing will be described with reference to FIG. 8. First, the alignment unit 15b calculates the distance matrix of the operation sequences as shown in FIG. 6 by applying the above procedure 1 to all the operations sequence, and calculates the distance vector for each operation sequence using the calculated distance matrix (see FIG. 8(1)).

In FIG. 8(1), the alignment unit 15b calculates, for example, the distance vector of “operation sequence 1” as [0, 24, 22, 24, 23], and calculates the distance vector of “operation sequence 2” as [24, 0, 4, 4, 9].

Next, the alignment unit 15b creates a tree diagram on the basis of a hierarchical clustering method using the distance vector for each operation sequence (see FIG. 8(2)). At this time, the alignment unit 15b performs calculation of the distance between operation sequences using a vector distance calculation method such as cosine similarity, but there is no particular limitation thereto.

In FIG. 8(2), the alignment unit 15b uses the distance vectors of “operation sequence 1” to “operation sequence 5” to visualize the distance between operation sequences (cases) using a tree diagram, and specifies “operation sequence 4,” “operation sequence 2,” “operation sequence 3,” “operation sequence 5,” and “operation sequence 1” as having a shorter distance in this order. Meanwhile, the purpose of the present processing is not to create a tree diagram, but to specify the cases in the order of shortest distance.

(1-4-2-3. Procedure 3)

The details of procedure 3 of the sequence alignment processing will be described with reference to FIG. 9. The alignment unit 15b aligns the operation sequences obtained through the above procedure 2 in the order of shortest distance (smallest value). At this time, the alignment unit 15b aligns the operation sequences in the same way as the sequence alignment method in the above procedure 1. That is, in the same way as the above procedure 1, after setting a specific score calculation recurrence formula, the alignment unit 15b defines a problem of finding a path for maximizing the score among the paths from the upper left cell to the lower right cell, and solves it using a dynamic programming method, and in a case where there is “→” or “↓” in the operation sequence, lines up the operation sequence with a blank “-” in that portion.

In FIG. 9, first, the alignment unit 15b aligns “operation sequence 2” and “operation sequence 4” (see FIG. 9(1)). Second, the alignment unit 15b aligns “operation sequence 3” with “operation sequence 2” and “operation sequence 4” after alignment (see FIG. 9(2)). Third, the alignment unit 15b aligns “operation sequence 5” with “operation sequence 2,” “operation sequence 3,” and “operation sequence 4” after alignment (see FIG. 9(3)). Fourth, the alignment unit 15b aligns “operation sequence 1” with “operation sequence 2,” “operation sequence 3,” “operation sequence 4,” and “operation sequence 5” after alignment (see FIG. 9(4)).

(1-5. Position ID Assignment Processing)

The position ID assignment processing will be described with reference to FIG. 10. First, the alignment unit 15b reflects the result after the alignment of the operation sequence IDs in the case ID. In FIG. 10, the alignment unit 15b reflects, for example, the operation sequence of “case 1” as the result after the alignment of “operation sequence 1,” and reflects the operation sequences of “case 2” and “case 6” as the result after the alignment of “operation sequence 2.” Next, the alignment unit 15b assigns a position ID to each sequence of the result after alignment. In FIG. 10, the alignment unit 15b assigns “1” to “46” as a position ID common to “case 1” to “case 10.”

(2. Visualization Processing)

The visualization processing will be described with reference to FIGS. 11 to 13. FIGS. 11 to 13 are diagrams illustrating an example of visualization processing related to according to the embodiment. Hereinafter, visualization processing of operations of the main flow, visualization processing of operations other than the main flow, and visualization processing using a threshold′ will be described in this order.

(2-1. Visualization Processing of Operations of Main Flow)

Visualization processing of operations of the main flow will be described with reference to FIG. 11. The visualization unit 15c highlights and visualizes operations of the main flow on one axis. At this time, the visualization unit 15c first extracts position IDs and operation IDs that appear with a threshold or more in the entire case. For example, in a case where the threshold is 80%, the visualization unit 15c extracts position IDs and operation IDs that appear in eight or more cases because there are ten cases in the example of FIG. 10. Next, the visualization unit 15c lines up and visualizes the extracted operation IDs from left to right (or from top to bottom) in the order of position IDs. For example, the visualization unit 15c performs visualization with the vertical direction defined as the y axis, the horizontal direction defined as the x axis, the y axis remaining fixed, and the x axis shifted in accordance with the position ID. In FIG. 11, the visualization unit 15c visualizes the operation ID “0” on the 1 portion of the x axis, for example, in a case where the position ID is “1.” In addition, the visualization unit 15c may change the color of the node to a conspicuous color or increase the size of the node as a method of highlighting the operation of the main flow.

(2-2. Visualization Processing of Operations Other than Main Flow)

Visualization processing of operations other than the main flow will be described with reference to FIG. 12. The visualization unit 15c visualizes operations other than the main flow. In this case, the visualization unit 15c extracts position IDs and operation IDs that appear less than a threshold in the entire case. For example, in a case where the threshold is 80%, the visualization unit 15c extracts position IDs and operation IDs that appear in less than eight cases because there are ten cases in the example of FIG. 10. Next, the visualization unit 15c lines up and visualizes the extracted operation IDs from left to right (or from top to bottom) in the order of position IDs. In addition, the visualization unit 15c visualizes the operation at this time by shifting it from the main flow. For example, in FIG. 12, in a case where the main flow is displayed from left to right, the visualization unit 15c visualizes the y axis of operations other than the main flow by shifting it unlikely the main flow.

(2-3. Visualization Processing Using Threshold′) The visualization processing using a threshold′ will be described with reference to FIG. 13. The visualization unit 15c sets a threshold′ as another visualization example, and visualizes only operation IDs associated with position IDs of the threshold′ or more. In addition, in the same way as the above-described visualization processing, a threshold for visualization is prepared so that the operation of the main flow and the operations of other flows can be distinguished from each other, and position IDs of the threshold or more are displayed as the main flow, and position IDs less than the threshold are displayed as operations of other flows, shifting them from the operation of the main flow. In addition, at this time, the operation that is the main flow may be highlighted.

In FIG. 13(1), the visualization unit 15c executes the visualization processing by setting threshold=80% and threshold′=80%. Therefore, the visualization unit 15c visualizes only the operations of the main flow. In FIG. 13(2), the visualization unit 15c executes the visualization processing by setting threshold=80% and threshold′=50%. Therefore, the visualization unit 15c visualizes operations that appear in 50% or more of operations of the main flow and operations other than the main flow. In FIG. 13(3), the visualization unit 15c executes the visualization processing by setting threshold=80% and threshold′=0%. Therefore, the visualization unit 15c visualizes all the operations.

[Flow of Processing]

The flow of processing according to the present embodiment will be described in detail with reference to FIGS. 14 to 17. Hereinafter, the overall flow of visualization display processing, the flow of operation log acquisition processing, the flow of alignment processing, and the flow of visualization processing will be described in this order.

(1. Overall Flow of Visualization Display Processing)

The overall flow of visualization display processing will be described with reference to FIG. 14. FIG. 14 is a flowchart illustrating the overall flow of visualization display processing according to the embodiment. Hereinafter, the overall flow of processing will be described, and then the outline of each processing will be described.

(1-1. Overall Flow of Processing)

First, the acquisition unit 15a of the visualization display device 10 executes the operation log acquisition processing (step S101). Next, the alignment unit 15b of the visualization display device 10 executes the alignment processing (step S102). The visualization unit 15c of the visualization display device 10 then executes the visualization processing (step S103) and ends the processing. Meanwhile, steps S101 to S103 described above can also be executed in a different order. In addition, some of steps S101 to S103 described above may be omitted.

(1-2. Flow of Each Processing)

First, the operation log acquisition processing performed by the acquisition unit 15a will be described. In the operation log acquisition processing, the acquisition unit 15a collects and accumulates an operation log (information composed of “operation time, case ID, and operation location”) from a terminal device. Meanwhile, the details of a flow of operation log acquisition processing will be described later in (2. Flow of operation log acquisition processing).

Second, the alignment processing performed by the alignment unit 15b will be described. In the alignment processing, the alignment unit 15b executes the following processes. First, the alignment unit 15b uses information on the operation log to generate an operation ID for identifying the operation. Next, the alignment unit 15b prepares an operation sequence in which operation IDs are lined up in the order of operation time for each case ID. Subsequently, the alignment unit 15b classifies the operation sequence into operation sequence types and assigns operation sequence IDs. The alignment unit 15b then extracts one operation sequence for each operation sequence type, and performs the alignment of the operation sequences using the method of multiple sequence alignment. Finally, the alignment unit 15b acquires the position ID and operation ID of the operation after alignment, and reflects them in the operation sequence for each case ID. Meanwhile, the details of a flow of alignment processing will be described later in (3. Flow of alignment processing).

Third, the visualization processing performed by the visualization unit 15c will be described. In the visualization processing, the visualization unit 15c executes the following processes. First, the visualization unit 15c uses the alignment result to extract position IDs that appear with a threshold or more from the entire case and operation IDs associated with the position IDs, and highlights and visualizes them by lining up the operations on one axis (main flow). Next, the visualization unit 15c uses the alignment result to extract position IDs that appear less than a threshold from the entire case and operation IDs associated with the position IDs. The visualization unit 15c then displays adjacent position IDs above and below (in a case where the main flow is visualized from left to right) or to the left and right of (in a case where the main flow is visualized from top to bottom) the visualized main flow, and connects edges on the basis of the order of position IDs for each case. Meanwhile, the details of a flow of visualization processing will be described later in (4. Flow of visualization processing).

(2. Flow of Operation Log Acquisition Processing)

A flow of operation log acquisition processing will be described with reference to FIG. 15. FIG. 15 is a flowchart illustrating a flow of operation log acquisition processing according to the embodiment. Meanwhile, the following steps S201 to S203 can also be executed in a different order. In addition, some of the following steps S201 to S203 may be omitted.

First, in a case where a user stops the process or drops a personal computer (PC) terminal (step S201: Yes), the acquisition unit 15a ends the process. On the other hand, in a case where the user has neither stop the process nor dropped the PC terminal (step S201: No), the acquisition unit 15a acquires an operation log (step S202). Next, the acquisition unit 15a accumulates the operation log by storing the acquired operation log in the operation log storage unit 14a (step S203), and proceeds to the process of step S201.

(3. Flow of Alignment Processing)

A flow of alignment processing will be described with reference to FIG. 16. FIG. 16 is a flowchart illustrating a flow of alignment processing according to the embodiment. Meanwhile, the following steps S301 to S305 can also be executed in a different order. In addition, some of the following steps S301 to S305 may be omitted.

First, the alignment unit 15b assigns an operation ID using information in the operation log of the operation log storage unit 14a (step S301). At this time, the alignment unit 15b may assign an operation ID using information in the operation log acquired by the acquisition unit 15a. Next, the alignment unit 15b generates an operation sequence for each case from the operation log (step S302). Subsequently, the alignment unit 15b classifies the operation sequence for each case into operation sequence types (step S303). At this time, the alignment unit 15b determines the classification into the operation sequence types by whether the operation sequence completely matches. Meanwhile, the amount of calculation of the process of step S304 can be reduced by executing the process of step S303.

The alignment unit 15b then performs multiple sequence alignment using the operation sequence for each operation sequence type (step S304). Here, in a case where progressive alignment which is one of the multiple sequence alignment methods is used, the alignment unit 15b creates a distance matrix of the operation sequence from the calculation of the sequence alignment of the operation sequence, creates a tree diagram from the distance matrix of the operation sequence, and aligns the operation sequence in the order of shortest distance on the tree diagram. Meanwhile, the alignment unit 15b may use a sequential improvement method, simulated annealing, or the like as methods other than the progressive alignment.

Finally, the alignment unit 15b acquires the position ID and operation ID of the operation after alignment, reflects them in the operation sequence for each case (step S305), and ends the process.

(4. Flow of Visualization Processing)

A flow of visualization processing will be described with reference to FIG. 17. FIG. 17 is a flowchart illustrating a flow of visualization processing related to according to the embodiment. Meanwhile, the following steps S401 to S404 can also be executed in a different order. In addition, some of the following steps S401 to S404 may be omitted.

First, in a case where a threshold′ is set (step S401: Yes), the visualization unit 15c sets the threshold′, extracts only position IDs and operation IDs that appear with the threshold′ or more for all the cases (step S404), and proceeds to the process of step S402. On the other hand, in a case where the threshold′ is not set (step S401: No), the visualization unit 15c sets the threshold, and highlights and visualizes position IDs and operation IDs (operations of the main flow) that appear with the threshold or more for all the cases by lining up the operation IDs on one axis in the order of the position IDs (step S402). At this time, the visualization unit 15c may change the color of the node to a conspicuous color or increase the size of the node as a method of highlighting the main flow.

The visualization unit 15c then visualizes position IDs and operation IDs (operations other than the main flow) that do not satisfy the threshold for all the cases on the visualization result visualized in the process of step S402, with these IDs shifted from one axis (step S403), and ends the process.

Effects of Embodiment

First, in the visualization display processing according to the present embodiment described above, an operation log is acquired, an operation sequence in units of operations arrayed in a time series order for each case is generated on the basis of the operation log, the operation sequence in units of operations is aligned, operation types that appear with a threshold or more at the same position in the operation sequence in units of operations of each case are specified as the operation types of the main flow on the basis of the aligned operation sequence in units of operations, and nodes corresponding to the operation types of the main flow are arranged and visualized in a time series order on one axis. Therefore, in the present processing, it is possible to easily specify a main flow in business analysis.

Second, in the visualization display processing according to the present embodiment described above, operations of the same type are specified by extracting information on the operation location from the operation log, the operation sequence in units of operations is aligned through pairwise alignment, the distance matrix for each operation sequence in units of operations is generated, the distance vector is generated using the distance matrix, and the operation sequence is further aligned through multiple sequence alignment in the order of the operation sequence in units of operations having the smaller distance vector. Therefore, in the present processing, the main flow can be easily and effectively specified in business analysis.

Third, in the visualization display processing according to the present embodiment described above, the reference value of an operation to be visualized is further set, nodes corresponding to the operation type of the main flow are arranged in a time series order on one axis, among the nodes corresponding to the operation types that appear with the reference value or higher of the operation, and the nodes corresponding to the operation types other than the operation type of the main flow are arranged and visualized in a time series order at positions different from on one axis. In the present processing, the main flow and flows other than the main flow desired to be visualized can be easily and effectively specified in business analysis.

Modification Example of Embodiment

Hereinafter, processing of a visualization display system according to a modification example of the embodiment (appropriately, the present modification example), the configuration of the visualization display device 10, the details of processing, and the flow of processing will be described in order, and finally the effects of the present modification example will be described. Meanwhile, the same configurations and processes as in the above-described embodiment will not be described.

[Processing of Visualization Display System]

Processing of the visualization display system according to the modification example of the embodiment will be described below. Processing of the above system will be described below in comparison with the related art.

(Conventional Visualization Display Processing)

The conventional visualization display processing also has the following problems in addition to the problems described in the processing of the visualization display system according to the embodiment. First, in a case where improvement of business centered on PC operations is performed in business analysis, business analysis is performed from the viewpoint of which work (specifically, which operation is performed on which screen) should be improved, but when the process is visualized at the granularity of operations, information is too detailed and business analysis is difficult, whereby business analysis cannot be performed efficiently (problem e). In addition, when processing is performed using a set of operations for each case, there is a problem in that as the number of cases increase, the processing time due to sequence alignment for the set of operations included in the case increases enormously. (problem f).

(Visualization Display Processing According to Present Modification Example)

In visualization display processing according to the present modification example, the operation sequence (operations in the order of operation time) for each case related to the same work is used as an input, and alignment is performed at the granularity of windows on the basis of the method of multiple sequence alignment, and then operation locations that appear in each window are aligned. Finally, for all the cases, operations that appear with a threshold or more are extracted as the main flow, and operations less than the threshold are extracted as operations other than the main flow, so that these operations are visualized distinctly. That is, in the visualization display processing according to the present modification example, in addition to the processing of the present embodiment, a set of operations for each case is first processed at the granularity of screens, the granularity of operations for each screen is next processed to reduce the processing time, the main flow is visualized at the granularity of screens and the granularity of operations, and thus it is possible to efficiently specify a screen to be improved by introducing RPA or the like and an operation in the screen.

Therefore, in the visualization display processing according to the present modification example, the process speed can be improved in addition to the processing of the present embodiment. In addition, in the visualization display processing according to the present modification example, it is possible to ascertain the main flow at the granularity of windows (screens) and the granularity of operations, making it easier to select screens and operations to be improved. From the above, the visualization display processing according to the present modification example contributes to solving the above problems e and f.

[Configuration of Visualization Display Device 10]

The configuration of the visualization display device 10 according to the present modification example will be described in detail. The configuration of the visualization display device 10 according to the present modification example is the same as that of the visualization display device 10 according to the present embodiment, and the processing of the input unit 11, the output unit 12, and the communication unit 13 of the visualization display device 10 according to the present modification example is the same as that of the visualization display device 10 according to the present embodiment, so the description thereof will be omitted. Hereinafter, only the storage unit 14 and the control unit 15 will be described, and description common to the storage unit 14 and the control unit 15 according to the present embodiment will be omitted.

(1. Storage Unit 14)

The storage unit 14 stores various types of information referred to when the control unit 15 operates and various types of information acquired when the control unit 15 operates. The storage unit 14 has the operation log storage unit 14a.

(1-1. Operation Log Storage Unit 14a)

An example of an operation log stored in the operation log storage unit 14a will be described with reference to FIG. 18. FIG. 18 is a diagram illustrating an example of an operation log stored in a storage unit according to a modification example of the embodiment. The operation log storage unit 14a stores an operation log. For example, the operation log storage unit 14a stores “operation time,” “case ID,” “operation location (window information),” “operation location (GUI component, coordinates, etc.),” and “input value” as an operation log.

(2. Control Unit 15)

The control unit 15 takes charge of control of the visualization display device 10 as a whole. The control unit 15 includes an acquisition unit 15a, an alignment unit 15b, and the visualization unit 15c. Meanwhile, the processing of the acquisition unit 15a is common to the processing of the present embodiment, and thus the description thereof will be omitted.

(2-1. Alignment Unit 15b)

The alignment unit 15b generates an operation sequence in units of windows arrayed in a time series order for each case, and aligns the operation sequence in units of windows.

(2-2. Visualization Unit 15c)

The visualization unit 15c specifies window operations that appear with a threshold or more at the same position in the operation sequence in units of windows of each case as windows of the main flow on the basis of the aligned operation sequence in units of windows, arranges the nodes corresponding to the windows of the main flow in a time series order on one axis, arranges the nodes corresponding to windows other than the windows of the main flow in a time series order at positions different from on the one axis, and superimposes and visualizes the nodes corresponding to the operations of each window. In addition, the visualization unit 15c further sets a reference value of windows to be visualized, arranges the nodes corresponding to the windows of the main flow in a time series order on one axis, among the nodes corresponding to windows that appear with the reference value or higher of the window, and arranges and visualizes the nodes corresponding to the windows other than the main flow in a time series order at positions different from on the one axis.

[Details of Processing]

The details of processing according to the present modification example will be described with reference to FIGS. 19 to 34, formulas, and the like. Hereinafter, alignment processing and visualization processing will be described in detail.

(1. Alignment Processing)

The alignment processing will be described with reference to FIGS. 19 to 32, formulas, and the like. FIGS. 19 to 32 are diagrams illustrating an example of alignment processing according to the modification example of the embodiment. Hereinafter, ID creation processing, operation sequence generation processing, operation sequence classification processing, sequence alignment processing 1, position ID assignment processing, ID association processing, and sequence alignment processing 2 will be described in this order.

(1-1. ID Creation Processing)

The ID creation processing will be described with reference to FIGS. 19 and 20. The alignment unit 15b of the visualization display device 10 creates a window ID and an operation ID from the operation log data. In FIG. 19, the alignment unit 15b assigns, for example, window IDs “A” to “C” to operation contents “window A” to “window C,” respectively (see FIG. 19(1)). Meanwhile, the assignment of the operation ID is the same as the processing according to the present embodiment, the description thereof will be omitted (see FIG. 19(2)).

In addition, the alignment unit 15b extracts the window ID and operation ID for each case from the operation log. In FIG. 20, the alignment unit 15b extracts, for example, “A” as the window ID of the case ID “case 1,” and “0” and “1” as the operation ID.

(1-2. Operation Sequence Generation Processing)

The operation sequence generation processing will be described with reference to FIG. 21. The alignment unit 15b generates an operation sequence based on window IDs in the order of operation time for each case ID of the operation log acquired from the operation log storage unit 14a. In FIG. 21, the alignment unit 15b generates, for example, an operation sequence of “A, C, B, D, E, A” as an operation sequence based on the window ID of the case ID “case 1,” and an operation sequence of “E, A, C, B, D, A” as an operation sequence based on the window ID of the case ID “case 3.” In addition, in FIG. 21, the alignment unit 15b generates an operation sequence based on window IDs in the order of operation time for eight cases with cases ID “case 1” to “case 8.”

(1-3. Operation Sequence Classification Processing)

The operation sequence classification processing will be described with reference to FIG. 22. The alignment unit 15b compares the types and orders of operations in the operation sequences and classifies them into types of operation sequences. At this time, the alignment unit 15b determines whether the operation sequence completely matches as a method of determining an operation type. In FIG. 22, the alignment unit 15b classifies, for example, “case 1” and “case 2” into the operation sequence ID “operation sequence 1” because “A, C, B, D, E, A” completely matches as the operation sequence based on the window ID of the case ID “case 1” and “case 2.” In addition, in FIG. 22, similarly, the alignment unit 15b classifies “case 3” and “case 7” into “operation sequence 2,” “case 4,” “case 5,” and “case 8” into “operation sequence 3,” and only “case 6” into “operation sequence 4.”

(1-4. Sequence Alignment Processing 1)

The sequence alignment processing 1 will be described with reference to FIGS. 23 to 27. Hereinafter, the outline of the sequence alignment processing 1 will be described, and then the details of the sequence alignment processing 1 will be described. Meanwhile, the present processing is similar to the sequence alignment processing according to the embodiment, but the present processing differs from the sequence alignment processing in that the processing target is not the operation sequence of operations but the operation sequence of windows. In addition, a method of obtaining s(x, y) in applying sequence alignment may use the matching degree of the operation type included in the window ID or the difference in the sequence alignment of the operation IDs.

(1-4-1. Outline)

The alignment unit 15b uses an operation sequence for each operation sequence type and performs sequence alignment of operation sequences using a progressive alignment method which is one of multiple sequence alignment methods. Specifically, first, the alignment unit 15b applies a sequence alignment method (pairwise alignment) to two operation sequences, extracts a difference score between these operation sequences, and stores it in a distance matrix (procedure 1). Second, the alignment unit 15b calculates the distance matrix of the operation sequences as shown in FIG. 23 by applying the above procedure 1 to all the operations sequences, extracts the distance vector for each operation sequence from the distance matrix, obtains a distance between the operation sequences from the distance vector, and creates a tree diagram assembled from the operation sequences with a short distance therebetween (procedure 2). Third, the alignment unit 15b further performs sequence alignment (multiple sequence alignment) of the operation sequences with a short distance therebetween (procedure 3).

(1-4-2. Details)

(1-4-2-1. Procedure 1-1)

The details of procedure 1-1 of sequence alignment processing 1 will be described with reference to FIG. 24. After setting a specific score calculation recurrence formula, the alignment unit 15b defines a problem of finding a path for maximizing the score among the paths from the upper left cell to the lower right cell, and solves it using a dynamic programming method. However, the alignment unit 15b may use algorithms other than the dynamic programming method.

For example, the alignment unit 15b performs procedure 1-1 using the score calculation recurrence formula shown in Formula (1) described in the sequence alignment processing according to the embodiment.

In Formula (1), (i, j) represents the coordinates of (column, row) in the table at the lower part of FIG. 24. In addition, F(i, j) is a score on the path at coordinates (i, j). In addition, in Formula (1), the score penalty is set to d=0. Further, s(x_i, y_j) is set to (the number of matching operation types+the number of all operation types) in a case where the window ID matches at a certain position, and set to −1 in a case where the window ID does not match. That is, in FIG. 24, the matching degree of the operation type included in the window ID is used as a method of obtaining s(x, y).

In FIG. 24, the alignment unit 15b calculates a difference score (distance matrix) from “operation sequence 1” and “operation sequence 2.” For example, in a case where the score on the path of F(3, 2) is calculated, the window ID of “operation sequence 2” (i=3) is “C,” the window ID of “operation sequence 1” (j=2) is “C,” and the window IDs match, so that the calculation is performed as s(x_i, y_j)=(the number of matching operation types+the number of all operation types). That is, in s(3, 2), the number of all operation types for the window ID “C” of “operation sequence 1” and “operation sequence 2” is “4, 5, 6, 25, 31,” which is five types, two of which are common. Thus, the relation of s(3, 2)=⅖=0.4 is established. From the above, as shown in the following Formula (4), F(3, 2)=1.4 is calculated by obtaining the maximum value as a solution.

$\begin{array}{cc}\left[\mathrm{Formula}4\right]& \\ F\left(3,2\right)=\max \{\begin{array}{cc}F\left(2,1\right)+s\left(3,2\right)=1+\left(\frac{2}{5}\right)=1.4& \\ F\left(2,2\right)-d=1& =1.4\\ F\left(3,1\right)-d=1& \end{array}& \left(4\right)\end{array}$

The alignment unit 15b executes the sequence alignment of “operation sequence 1” and “operation sequence 2” by calculating the score on the above path for all the cells and performing a process of finding a path for maximizing the score among the paths from the upper left cell to the lower right cell in FIG. 24. At this time, in a case where there is “-” or “I” in the operation sequence, the alignment unit 15b lines up the operation sequence with a blank “-” in that portion. That is, “operation sequence 1” aligns with “-, A, C, B, D, E, A,” and “operation sequence 2” aligns with “E, A, C, B, D, E, A.”

The alignment unit 15b then compares the aligned operation ID sequence, calculates the number of window IDs that do not match at the compared positions as a difference score, and stores it in the distance matrix. That is, when the window ID sequences of the aligned “operation sequence 1” and “operation sequence 2” are compared with each other, one window ID differs in the corresponding sequence, and thus the difference score is calculated as “1” and stored in the distance matrix.

(1-4-2-2. Procedure 1-2)

The details of procedure 1-2 of sequence alignment processing 1 will be described with reference to FIG. 25. Here, procedure 1-2 differs from the above procedure 1-1 in a method of obtaining s(x, y), and thus the description of the common processing will be omitted.

For example, the alignment unit 15b performs procedure 1-2 using the score calculation recurrence formula shown in Formula (1) described in the sequence alignment processing according to the embodiment.

In Formula (1), (i, j) represents the coordinates of (column, row) in the table at the lower part of FIG. 25. In addition, F(i, j) is a score on the path at coordinates (i, j). In addition, in Formula (1), the score penalty is set to d=0. Further, in s(x_i, y_j), sequence alignment may be performed when the window ID matches at a certain position, and +1 may be set when the difference is 0. When the difference is 1 or more, the number of operations after sequence alignment may be set to a denominator and the numerator may be set to (the number of operations after sequence alignment −difference). In addition, −1 is set when the window ID does not match. That is, in FIG. 25, the difference in sequence alignment of operation IDs is used as a method of obtaining s(x, y).

In FIG. 25, the alignment unit 15b calculates a difference score (distance matrix) from “operation sequence 1” and “operation sequence 2.” For example, in a case where the score on the path of F(3, 2) is calculated, the window ID of “operation sequence 2” (i=3) is “C,” the window ID of “operation sequence 1” (j=2) is “C,” and the window IDs match, so that sequence alignment is performed on the operation IDs of “C” of “operation sequence 1” and “operation sequence 2.” As a result of sequence alignment, the operation ID of “C” of “operation sequence 1” is “4, 5, 6, -, -,” the operation ID of “C” of “operation sequence 2” is “4, 5, -, 31, 25,” the difference is 3 (number of “-”), and the number of operations after sequence alignment is 5. Thus, the relation of s(3, 2)=(5-3)/5=⅖ is established. From the above, as shown in the following Formula (5), F(3, 2)=1.1 is calculated by obtaining the maximum value as a solution.

$\begin{array}{cc}\left[\mathrm{Formula}5\right]& \\ F\left(3,2\right)=\max \{\begin{array}{cc}F\left(2,1\right)+s\left(3,2\right)=0.7+\left(\frac{2}{5}\right)=1.1& \\ F\left(2,2\right)-d=0.7& =1.1\\ F\left(3,1\right)-d=0.7& \end{array}& \left(5\right)\end{array}$

The alignment unit 15b executes sequence alignment of “operation sequence 1” and “operation sequence 2” in the same way as the above procedure 1-1. As a result, “operation sequence 1” is aligned as “-, A, C, B, D, E, A” and “operation sequence 2” is aligned as “E, A, C, B, D, E, A,” the difference score is calculated as “1.”

(1-4-2-3. Procedure 2)

The details of procedure 2 of the sequence alignment processing 1 will be described with reference to FIG. 26. First, the alignment unit 15b calculates the distance matrix of the operation sequences as shown in FIG. 23 by applying the above procedure 1-1 or procedure 1-2 to all the operations sequence, and calculates the distance vector for each operation sequence using the calculated distance matrix (see FIG. 26(1)).

In FIG. 26(1), the alignment unit 15b calculates, for example, the distance vector of “operation sequence 1” as [0, 2, 1, 1], and calculates the distance vector of “operation sequence 2” as [2, 0, 1, 3].

Next, the alignment unit 15b creates a tree diagram on the basis of a hierarchical clustering method using the distance vector for each operation sequence (see FIG. 26(2)). At this time, the alignment unit 15b performs calculation of the distance between operation sequences using a vector distance calculation method such as cosine similarity, but there is no particular limitation thereto.

In FIG. 26(2), the alignment unit 15b uses the distance vectors of “operation sequence 1” to “operation sequence 4” to visualize the distance of operation sequences using a tree diagram, and specifies “operation sequence 3,” “operation sequence 1,” “operation sequence 2,” and “operation sequence 4” as having a shorter distance in this order. Meanwhile, the purpose of the present processing is not to create a tree diagram, but to specify the operation sequences in the order of shortest distance.

(1-4-2-4. Procedure 3)

The details of procedure 3 of the sequence alignment processing 1 will be described with reference to FIG. 27. The alignment unit 15b aligns the operation sequences obtained through the above procedure 2 in the order of shortest distance (smallest value). At this time, the alignment unit 15b aligns the operation sequences in the same ways as the sequence alignment method in the above procedure 1-1 or procedure 1-2.

In FIG. 27, first, the alignment unit 15b aligns “operation sequence 1” and “operation sequence 3” (see FIG. 27(1)). Second, the alignment unit 15b aligns “operation sequence 2,” and “operation sequence 1” and “operation sequence 3” after alignment (see FIG. 27(2)). Third, the alignment unit 15b aligns “operation sequence 4,” and “operation sequence 1,” “operation sequence 2,” and “operation sequence 3” after alignment (see FIG. 27(3)).

(1-5. Position ID Assignment Processing)

The position ID assignment processing will be described with reference to FIGS. 28 and 29. First, the alignment unit 15b reflects the result after the alignment of the operation sequence IDs in the case ID. In the lower part of FIG. 28, the alignment unit 15b reflects, for example, the operation sequences of “case 1” and “case 2” as the result after the alignment of “operation sequence 1,” and reflects the operation sequences of “case 3” and “case 7” as the result after the alignment of “operation sequence 2.” Next, the alignment unit 15b assigns a position ID (window) to each sequence of the result after alignment. In the lower part of FIG. 28, the alignment unit 15b assigns “1” to “7” as a position ID (window) common to “case 1” to “case 8.”

In addition, the alignment unit 15b assigns a position ID (window) to the original operation log. In FIG. 29, in the alignment unit 15b, for example, “case 1” continues with window IDs “A, B,” and a location where the window ID first appears is a position ID (window) “2” as compared with the position ID (window) in FIG. 28.

(1-6. ID Association Processing)

The ID association processing will be described with reference to FIG. 30. The alignment unit 15b prepares an operation sequence lined up in the order of operation time of the operation ID for each case ID, for each position ID (window). The alignment unit 15b associates, for example, the position ID (window) “4” and the operation ID for window ID “B,” and creates an operation sequence in units of operations for each case ID as shown in FIG. 30.

(1-7. Sequence Alignment Processing 2)

The sequence alignment processing 2 will be described with reference to FIGS. 31 and 32. The operation sequence for each case ID is aligned on the basis of the sequence alignment method for each position ID (window) using the same method as the processing executed in the operation sequence classification processing, the sequence alignment processing 1, the position ID assignment processing, and the ID association processing described above. The alignment unit 15b aligns, for example, the operation ID for each position ID (window) “4” and window ID “B,” and creates an operation sequence in units of operations for each case ID as shown in FIG. 31. Similarly, the alignment unit 15b aligns all the operation contents as shown in FIG. 32. Meanwhile, since the above processing can be performed independently for each position ID (window), the processing time can be reduced by performing parallel processing.

(2. Visualization Processing)

The visualization processing will be described with reference to FIGS. 33 and 34. FIGS. 33 and 34 are diagrams illustrating an example of visualization processing related to the embodiment. Hereinafter, the window visualization processing and the operation visualization processing will be described in this order.

(2-1. Window Visualization Processing)

The window visualization processing will be described with reference to FIG. 33. The visualization unit 15c uses the alignment result (the result of alignment at the granularity of windows) to extract the position IDs (windows) that appear with a threshold or more for all the cases and the window IDs associated with the position IDs (windows), and highlights and visualizes them by lining up the window IDs on one axis in the order of the position IDs (windows). At this time, the visualization unit 15c visualizes other window IDs that do not satisfy the threshold by shifting them up and down (in the case of visualization from left to right) or left and right (in the case of visualization from top to bottom).

For example, in a case where the position IDs (windows) that appear at a threshold of 80% for all the cases are visualized, the visualization unit 15c extracts position IDs (windows) and window IDs that appear in seven or more cases as windows of the main flows because there are eight cases in the example of FIG. 33. That is, the visualization unit 15c extracts “A” to “D” that appear in seven or more cases as windows of the main flow. Next, the visualization unit 15c lines up and visualizes the extracted operation IDs from left to right (or from top to bottom) in the order of position IDs. In addition, the visualization unit 15c may change the color of the node to a conspicuous color or increase the size of the node as a method of highlighting the operation of the main flow.

On the other hand, the visualization unit 15c extracts “E” that does not appear in seven or more cases as a window other than the main flow. In addition, the visualization unit 15c visualizes the window at this time by shifting it from the main flow.

(2-2. Operation Visualization Processing)

The operation visualization processing will be described with reference to FIG. 34. The visualization unit 15c extracts the position IDs (operations) that appear with a threshold or more for all the cases included in the extracted position IDs (windows) and the operation IDs associated with the position IDs (operations), and displays the operation IDs in each position ID (window) as the main flow on one axis in the order of the position IDs (operations). Other operations that do not satisfy the threshold are visualized by shifting them up and down (in the case of visualization from left to right) or left and right (in the case of visualization from top to bottom) within each position ID (window).

For example, in a case where the position IDs (windows) and position IDs (operations) that appear at a threshold of 80% for all the cases are visualized, as shown in FIG. 34, the visualization unit 15c displays the main flow (windows, operations) on one axis, and displays the operations other than the main flow so as to be superimposed on the corresponding window.

[Flow of Processing]

A flow of processing according to the present modification example will be described in detail with reference to FIGS. 35 and 36. Hereinafter, the overall flow of visualization display processing and the flow of operation log acquisition processing which are common to the processing according to the present embodiment will not be described, and the flow of alignment processing and the flow of visualization processing will be described.

(1. Flow of Alignment Processing)

A flow of alignment processing will be described with reference to FIG. 35. FIG. 35 is a flowchart illustrating a flow of alignment processing according to the modification example of the embodiment. Meanwhile, the following steps S501 to S509 can also be executed in a different order. In addition, some of the following steps S501 to S509 will be omitted.

First, the alignment unit 15b assigns a window ID using information in the operation log of the operation log storage unit 14a (step S501). Second, the alignment unit 15b uses the window ID to generate an operation sequence for each case (step S502). Third, the operation sequence for each case is classified into operation sequence types, and an operation sequence ID is assigned (step S503). At this time, the alignment unit 15b determines the classification into the operation sequence types by whether the operation sequence completely matches.

Fourth, the alignment unit 15b then performs multiple sequence alignment using the operation sequence for each operation sequence type (step S504). Here, in a case where progressive alignment which is one of the multiple sequence alignment methods is used, the alignment unit 15b creates a distance matrix of the operation sequence from the calculation of the sequence alignment of the operation sequence, creates a tree diagram from the distance matrix of the operation sequence, and aligns the operation sequence in the order of shortest distance on the tree diagram. Meanwhile, the alignment unit 15b may use a sequential improvement method, simulated annealing, or the like as methods other than the progressive alignment.

Fifth, the alignment unit 15b acquires the position ID (window) and window ID of the operation after alignment, and reflects them in the operation sequence for each case (step S505). Sixth, the alignment unit 15b uses the operation ID to generate an operation sequence for each case, for each position ID (window) (step S506). Seventh, the alignment unit 15b classifies the operation sequence for each case into operation sequence types (step S507). Eighth, multiple sequence alignment is performed using the operation sequence for each operation sequence type (step S508). Ninth, the alignment unit 15b acquires the position ID (window) and window ID of the operation after alignment, reflects them in the operation sequence for each case (step S509), and ends the process.

(2. Flow of Visualization Processing)

A flow of visualization processing will be described with reference to FIG. 36. FIG. 36 is a flowchart illustrating a flow of visualization processing related to the modification example of the embodiment. Meanwhile, the following steps S601 to S606 can also be executed in a different order. In addition, some of the following steps S601 to S606 will be omitted.

First, in a case where the threshold′ is set (step S601: Yes), the visualization unit 15c sets the threshold′, extracts only position IDs (windows), window IDs, position IDs (operations), and operation IDs that appear with the threshold′ or more for all the cases (step S606), and proceeds to the process of step S602.

On the other hand, in a case where a threshold′ is not set (step S601: No), the visualization unit 15c sets the threshold, and highlights and visualizes position IDs (windows) and window IDs that appear with the threshold or more for all the cases by lining up the window IDs on one axis in the order of the position IDs (windows) (step S602). At this time, the visualization unit 15c may change the color of the node to a conspicuous color or increase the size of the node as a method of highlighting the main flow. Subsequently, the visualization unit 15c visualizes the position IDs (windows) and window IDs that do not satisfy the threshold for all the cases on the visualization result visualized in the process of step S602, with these IDs shifted from one axis (step S603).

In addition, the visualization unit 15c sets a threshold, and highlights and visualizes the position IDs (operations) and operation IDs that appear with the threshold or more for all the cases for each position ID (window) by lining up the window IDs on one axis in the order of the position IDs (operations) within the position ID (window) (step S604). The position IDs (operations) and operation IDs that do not satisfy the threshold for all the cases for each position ID (window) are visualized within the position ID (window) on the visualization result visualized in the process of step S604, with these IDs shifted from one axis (step S605), and the process ends.

Effects of Modification Example of Embodiment

First, the visualization display processing according to the present modification example described above includes further generating an operation sequence in units of windows arrayed in a time series order for each case and aligns the operation sequence in units of windows, specifying window operations that appear with a threshold or more at the same position in the operation sequence in units of windows of each case as windows of the main flow on the basis of the aligned operation sequence in units of windows, arranging nodes corresponding to the windows of the main flow in a time series order on one axis, arranging nodes corresponding to windows other than the windows of the main flow in a time series order at positions different from on the one axis, and superimposing and visualizing nodes corresponding to operations of each window. Therefore, in the present processing, the main flow and flows other than the main flow can be easily specified in business analysis without spending processing time, and screens and operation to be improved can be easily selected.

Second, the visualization display processing according to the present modification example described above includes further setting a reference value of windows to be visualized, arranging the nodes corresponding to the windows of the main flow in a time series order on one axis, among the nodes corresponding to windows that appear with the reference value or higher of the window, and arranging and visualizing the nodes corresponding to the windows other than the main flow in a time series order at positions different from on the one axis. Therefore, in the present processing, the main flow and flows other than the main flow desired to be visualized can be easily specified in business analysis without spending processing time, and screens and operation to be improved can be easily selected.

[System Configuration Etc.]

Each component of each device shown according to the above embodiment is functionally conceptual, and does not necessarily have to be physically configured as shown in the drawing. That is, the specific form of distribution and integration of each device not limited to that shown in the drawing, and all or part of them can be functionally or physically distributed and integrated in arbitrary units according to various loads and usage conditions. Further, All or any part of processing functions which are performed by each device can be realized by a CPU and a program analyzed and executed by the CPU, or realized as hardware using wired logic.

In addition, among the processes described in the above embodiment, all or part of the processes described as being performed automatically can be manually performed, or all or part of the processes described as being performed manually can also be performed automatically using known methods. In addition, information including processing procedures, control procedures, specific names, and various types of data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified.

[Program]

In addition, It is also possible to create a program in which the processing executed by the visualization display device 10 described in the above embodiment is described in a computer-executable language. In this case, the same effects as in the above embodiment can be obtained by a computer executing a program. Further, such a program may be recorded in a computer-readable recording medium, and the program recorded in this recording medium may be read and executed by a computer to realize the same processing as in the above embodiment.

FIG. 37 is a diagram illustrating a computer that executes a program. As illustrated in FIG. 37, a computer 1000 includes, for example, a memory 1010, a CPU 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070, which are connected to each other through a bus 1080.

The memory 1010 includes a read only memory (ROM) 1011 and a RAM 1012 as illustrated in FIG. 37. The ROM 1011 stores a boot program such as, for example, a basic input output system (BIOS). The hard disk drive interface 1030 is connected to a hard disk drive 1090 as illustrated in FIG. 37. The disk drive interface 1040 is connected to a disk drive 1100 as illustrated in FIG. 37. A removable storage medium such as, for example, a magnetic disc or an optical disc is inserted into the disk drive 1100. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120 as illustrated in FIG. 37. The video adapter 1060 is connected to, for example, a display 1130 as illustrated in FIG. 37.

Here, as illustrated in FIG. 37, the hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, the above program is stored in, for example, the hard disk drive 1090 as a program module describing instructions to be executed by the computer 1000.

In addition, various types of data described in the above embodiment are stored as program data in, for example, the memory 1010 or the hard disk drive 1090. The CPU 1020 reads out the program module 1093 or the program data 1094 stored in the memory 1010 or the hard disk drive 1090 to the RAM 1012 as necessary, and executes various processing procedures.

Meanwhile, the program module 1093 or the program data 1094 related to the program is not limited to being stored in the hard disk drive 1090, and may be stored in, for example, a removable storage medium and read out by the CPU 1020 through a disk drive or the like. Alternatively, the program module 1093 or the program data 1094 related to the program may be stored in another computer connected through a network (such as a local area network (LAN) or a wide area network (WAN)) and read out by the CPU 1020 through the network interface 1070.

The above-described embodiments and modifications thereof are included in the claims and their equivalents as well as in the technique disclosed by the present application.

REFERENCE SIGNS LIST

• • 10 Visualization display device
  • 11 Input unit
  • 12 Output unit
  • 13 Communication unit
  • 14 Storage unit
  • 14 a Operation log storage unit
  • 15 Control unit
  • 15 a Acquisition unit
  • 15 b Alignment unit
  • 15 c Visualization unit

Claims

1. A visualization display device comprising: visualization display circuitry configured to: acquire an operation log;generate an operation sequence in units of operations arrayed in a time series order for each case on the basis of the operation log and align the operation sequence in units of operations; andspecify operation types that appear with a threshold or more at the same position in the operation sequence in units of operations of each case as operation types of a main flow on the basis of the aligned operation sequence in units of operations and arrange and visualize nodes corresponding to the operation types of the main flow in a time series order on one axis.

2. The visualization display device according to claim 1, wherein the visualization display circuitry configured to specify operations of the same type by extracting information on operation locations from the operation log, align the operation sequence in units of operations through pairwise alignment, generate a distance matrix for each operation sequence in units of operations, generate a distance vector using the distance matrix, and further align the operation sequence through multiple sequence alignment in the order of the operation sequence in units of operations having the smaller distance vector.

3. The visualization display device according to claim 1, wherein the visualization display circuitry configured to further set a reference value of an operation to be visualized, arrange the nodes corresponding to the operation types of the main flow in a time series order on the one axis, among nodes corresponding to operation types that appear with the reference value or higher of the operation, and arrange and visualize nodes corresponding to operation types other than the operation types of the main flow in a time series order at positions different from on the one axis.

4. The visualization display device according to claim 1, wherein the visualization display circuitry configured to: further generate an operation sequence in units of windows arrayed in a time series order for each case and aligns the operation sequence in units of windows, andspecify windows that appear with a threshold or more at the same position in the operation sequence in units of windows of each case as windows of the main flow on the basis of the aligned operation sequence in units of windows, arrange nodes corresponding to the windows of the main flow in a time series order on one axis, arrange nodes corresponding to windows other than the windows of the main flow in a time series order at positions different from on the one axis, and superimpose and visualize nodes corresponding to operations of each window.

5. The visualization display device according to claim 4, wherein the visualization display circuitry configured to further set a reference value of a window to be visualized, arrange nodes corresponding to the windows of the main flow in a time series order on one axis, among nodes corresponding to windows that appear with the reference value or higher of the window, and arrange and visualize nodes corresponding to windows other than the windows of the main flow in a time series order at positions different from on the one axis.

6. A visualization display method which is executed by a visualization display device, the method comprising: acquiring an operation log;generating an operation sequence in units of operations arrayed in a time series order for each case on the basis of the operation log and aligning the operation sequence in units of operations; andspecifying operation types that appear with a threshold or more at the same position in the operation sequence in units of operations of each case as operation types of a main flow on the basis of the aligned operation sequence in units of operations and arranging and visualizing nodes corresponding to the operation types of the main flow in a time series order on one axis.

7. A non-transitory computer-readable recording medium storing therein a visualization display program causing a computer to execute a process comprising: acquiring an operation log;generating an operation sequence in units of operations arrayed in a time series order for each case on the basis of the operation log and aligning the operation sequence in units of operations; andspecifying operation types that appear with a threshold or more at the same position in the operation sequence in units of operations of each case as operation types of a main flow on the basis of the aligned operation sequence in units of operations and arranging and visualizing nodes corresponding to the operation types of the main flow in a time series order on one axis.