The embodiment discussed herein is related to a recording medium in which an analysis program is recorded, an analytic method, and an analyzing apparatus.
Conventionally, as a method of evaluating a test of software, a method called mutation analysis is known, in which one element of a program is changed to embed a bug and how well the test detects the embedded bug is investigated. In the mutation analysis, each source code subjected to mutation is compiled and executed to generate all sorts of mutants that are changed elements, so that it takes a long time to perform a process. Therefore, a technology has been proposed, in which an element is replaced with a metafunction that is a function representing a specific mutant when a parameter is given, instead of performing mutations by replacing an element with a plurality of mutants, in order to reduce compiling cost.
Furthermore, a technology has been proposed in the literature such as K. N. King and J. Offutt, “A Fortran Language System for Mutation-Based Software Testing”, Software-Practice and Experience, 1991., in which executions are performed while splitting states (hereinafter, also referred to as states) in which elements are replaced with respective mutants, in order to reduce calculation cost.
Moreover, a technology has been proposed in the literature such as Rene Just, et al., Efficient mutation analysis by propagating and partitioning infected execution states. ISSTA2014, in which mutants having the same execution result as an original execution result are eliminated and mutants having the same execution result with one another are bundled in order to reduce calculation cost.
Patent Literature 1: Japanese Laid-open Patent Publication No. 2009-181549
Patent Literature 2: Japanese Laid-open Patent Publication No. 8-044590
However, when the mutants having the same execution result are bundled, execution paths of the mutants may be changed from an original execution path. Therefore, for example, whether the state is equivalent when a loop is executed multiple times is not found out until the mutants are executed. That is, in some cases, it may be difficult to bundle mutants having the same execution result, and it may become difficult to reduce the number of states of a higher order mutation.
According to an aspect of an embodiment, an analysis program recorded in a recording medium causes a computer to hold, when executing an intermediate code that is a compiled code in which an element in a source code is replaced with a metafunction that changes the element to a mutant, a set of mutation descriptors indicating a change in the mutant with respect to a mutation operation corresponding to the metafunction. The analysis program further causes the computer to evaluate a command of each of the mutation descriptors, select at least one mutation descriptor having a same command evaluation result from the set of the mutation descriptors, and calculate a direct product of the selected mutation descriptor and one of the mutation operation and a first state that is the set of the mutation descriptors before evaluation of the commands, thereby generating a second state. The analysis program further causes the computer to bundle, when evaluating a command of each mutation descriptor of the generated second state, if a plurality of the second states are present, the second states into a single group, evaluate commands of the second states in the group in parallel, and merge third states having the same evaluation result among third states that are based on the command evaluation results in the group.
The object and advantages of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the claims.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention, as claimed.
Preferred embodiments of a recording medium in which an analysis program of the disclosed technology is recorded, an analytic method, and an analyzing apparatus will be described in detail below with reference to the accompanying drawings. The disclosed technology is not limited by the embodiments below. The embodiments below may be combined appropriately as long as no contradiction is derived.
Mutation analysis will be described below.
Similarly, a source code 10b is a source code obtained by performing, for example, a mutation indicated by “mutant1” on the source code 10. For example, “mutant2” changes “0” in the second line to “1”. That is, in the source code 10b, a portion indicated by a mutated portion 12 is changed as compared to the source code 10. Similarly, a source code 10c is a source code obtained by performing, for example, a mutation indicated by “mutant3” on the source code 10. For example, “mutant3” changes “-” in the third line to “--”. That is, in the source code 10c, a portion indicated by a mutated portion 13 is changed as compared to the source code 10.
A test result 14 is a result obtained by performing a certain test on the mutated source codes 10a to 10c. For example, in “Test1”, “pass” is given when “2” is input to “abs” and “2” is output, and “fail” is given when other than “2” is output. Furthermore, for example, in “Test2”, “pass” is given when “−2” is input to “abs” and “2” is output, and “fail” is given when other than “2” is output. In this case, results of “mutant1” to “mutant3” are obtained as indicated in the test result 14.
As for “mutant1”, “fail” is given in each of “Test1” and “Test2”, so that it is possible to detect the embedded bug; therefore, “killed” is given. As for “mutant2”, “pass” is given in each of “Test1” and “Test2”, so that it is impossible to detect the embedded bug; therefore, “unkilled” is given. As for “mutant3”, “pass” is given in “Test1” and “fail” is given in “Test2”, so that it is possible to detect the embedded bug; therefore, “killed” is given. As a result, a mutation score indicating the ratio of mutants that are “killed” is 2/3=0.667.
Next, implementation cost of the mutation analysis will be described.
In contrast, for example, in Mutant Schema Generation (MSG), an element of the source code is replaced with a metafunction that is a function representing a specific mutant when a parameter is given, in order to reduce compiling cost. Furthermore, Split-Stream Execution has been proposed, in which executions are performed while splitting states in which elements are replaced with respective mutants, in order to reduce calculation cost.
In the state 27a, the operator “+” in the command 26 is used as it is, so that a calculation result of the command 26 is “%1=4”. A mutation descriptor 28a of the state 27a is “{}”. In the state 27b, the operator in the command 26 is mutated to “−”, so that a calculation result of the command 26 is “%1=0”. A mutation descriptor 28b of the state 27b is “{(1, −)}”. Meanwhile, the mutation descriptor 28b is represented by a pair of a mutation identifier (ID) of “1” indicating a mutated portion and a changed content “−”, and the same applies to the mutation descriptors described below.
In the state 27c, the operator in the command 26 is mutated to “*”, so that a calculation result of the command 26 is “%1=4”. A mutation descriptor 28c of the state 27c is “{(1, *)}”. In the state 27d, the operator in the command 26 is mutated to “/”, so that a calculation result of the command 26 is “%1=1”. A mutation descriptor 28d of the state 27d is “{(1, /)}”. In the state 27e, the operator in the command 26 is mutated to “%”, so that a calculation result of the command 26 is “%1=0”. A mutation descriptor 28e of the state 27e is “{(1, %)}”.
A method of eliminating mutants having the same execution result as an original execution result and bundling mutants having the same execution result with one another will be described below with reference to
Subsequently, in the propagation 30, a mutant that does not propagate is eliminated. In the example in
Subsequently, in the partitioning 31, mutants having the same result are bundled. In the example in
The method in
An expression 33 is an expression in which the mutated portions are replaced with the eval method calls. A table 34 indicates elements included in each of an ID “3e” corresponding to expr3 and an ID “5e” corresponding to expr5. A table 35 is a table indicating what mutation is possible, that is, a table indicating a monitoring target. For example, it is indicated that an ID “1m” changes “+” to “−” with respect to the ID “3e”. Furthermore, for example, it is indicated that an ID “3m” changes “>” to “>=” with respect to the ID “5e”.
Subsequently, in the example in
The third step is to embed the mutants in the original program. The fourth step is to execute a test on each of the mutants. That is, in
Furthermore, if the method of executions while splitting a state is applied as it is to the method of reducing the calculation cost, wasteful state splitting is increased. To split a state, a copy process, which is a heavy process, is included; therefore, it is preferable to reduce state splitting as much as possible.
To cope with the issue as described above, in the embodiment, commands in a source code are first evaluated, and states are bundled based on command evaluation results, so that it is possible to reduce the number of states of a higher order mutation.
Referring back to
The communication unit 110 is implemented by, for example, a network interface card (NIC) or the like. The communication unit 110 is a communication interface that is connected to other information processing apparatuses through a network (not illustrated) by wire or wireless, and controls communication of information with the other information processing apparatus. For example, the communication unit 110 receives, from the other information processing apparatuses, an analysis target source code, a test item, and a mutation operator list. The communication unit 110 outputs the received analysis target source code, the received test item, and the received mutation operator list to the control unit 130.
The input unit 111 is implemented by, for example, a media access device or the like for an external storage medium, such as an optical disk, a universal serial bus (USB) memory , or an SD memory card. The input unit 111 reads the analysis target source code, the test item, and the mutation operator list that are stored in the external storage medium, and outputs the read analysis target source code, the read test item, and the read mutation operator list to the control unit 130. Meanwhile, it is sufficient that the analysis target source code, the test item, and the mutation operator list are input to the control unit 130 from any of the communication unit 110 and the input unit 111, and in the following description, a case will be described in which they are input from the input unit 111 to the control unit 130.
The display unit 112 is a display device for displaying various kinds of information. The display unit 112 is implemented by, for example, a liquid crystal display or the like as the display device. The display unit 112 displays various screens, such as a result screen input from the control unit 130.
The operating unit 113 is an input device that accepts various operations from a user of the analyzing apparatus 100. The operating unit 113 is implemented by, for example, a keyboard, a mouse, or the like as an input device. The operating unit 113 outputs an operation input by the user as operation information to the control unit 130. The operating unit 113 may be implemented by a touch panel or the like as the input device. The display device of the display unit 112 and the input device of the operating unit 113 may be integrated with each other.
The storage unit 120 is implemented by, for example, a semiconductor memory device, such as a random access memory (RAM) or a flash memory, or a storage device, such as a hard disk or an optical disk. The storage unit 120 includes a command storage unit 121, an evaluation result storage unit 122, an execution state set storage unit 123, and a test result storage unit 124. Furthermore, the storage unit 120 stores therein information used for a process performed by the control unit 130.
The command storage unit 121 stores therein each command of an intermediate code in which a source code is compiled, in association with a command ID. The intermediate code may be, for example, a bit code.
The “command ID” is a number (ID) uniquely assigned to each command, and an identifier for identifying the command. The “command” is information indicating each command of an intermediate code.
Referring back to
Referring back to
The “state ID” is an identifier for identifying an execution state, that is, a state. The “state ID” may be taken over by a state obtained by evaluation of a command. The “command ID” an identifier for identifying a command. The “command result” is information indicating a command evaluation result, that is, an execution result. The “mutation descriptor” is information indicating a mutation descriptor of a command for a corresponding state ID. The “synchronous execution group” is information indicating a group for evaluating commands of a plurality of states in parallel. In the “synchronous execution group”, for example, circle marks are provided to the state IDs being evaluated in parallel, so that it is possible to identify the state IDs being evaluated in parallel.
Referring back to
The “test” is information indicating test inputs with respect to a command. The “mutant” is information indicating a mutant with respect to the command. The “mutant” is described using a mutation descriptor, such as “{(2, !=)}”. The “test pass-fail” is information indicating pass or fail of a test on a mutation of a test target source code, and is represented by “Pass” or “Fail”. “Pass” indicates that a bug embedded in a test item is not detected when an element in the source code is replaced with a mutant. “Fail” indicates that a bug embedded in a test item is detected when an element in the source code is replaced with a mutant. In the example in the first line in
Referring back to
The replacing unit 131, when a source code is input from the input unit 111, replaces an element in the source code with a metafunction that changes the element to a mutant. The metafunction has the mutation ID as a parameter. The replacing unit 131 outputs a source code, in which the element in the source code is replaced with the metafunction, that is, the replaced source code in which the metafunction is embedded, to the compiler 132.
Embedding of the metafunction will be described below with reference to
Referring back to
The intermediate code will be described below with reference to
Referring back to
The test item and the mutation operator list will be described below with reference to
The state managing unit 133, upon selecting the test case, generates an initial execution state. The state managing unit 133, upon generating the initial execution state, performs a state selecting process. As the state selecting process, the state managing unit 133 first refers to the execution state set storage unit 123, and determines whether a state that is included in a synchronous execution group (hereinafter, may be simply referred to as a group) is present in an execution state set. If the state included in the group is not present, the state managing unit 133 selects a state by an arbitrary method, for example, selects a state by giving priority to depth, and ends the state selecting process.
If the state included in the group is present, the state managing unit 133 determines whether the group includes states in which a command being evaluated is not executed. If the group includes the states in which the command being evaluated is not executed, the state managing unit 133 selects one arbitrary state from the states in which the command being evaluated is not executed in the group, and ends the state selecting process. If the group does not include the states in which the command being evaluated is not executed, the state managing unit 133 takes a next command as being evaluated, selects a state in which the next command is not executed, and ends the state selecting process. When the state selecting process ends, the state managing unit 133 outputs a state ID of the selected state to the command evaluating unit 134.
The state managing unit 133, when a group dissolution instruction is input from the command evaluating unit 134, deletes the field of the synchronous execution group in the execution state set storage unit 123 and dissolves the group. The state managing unit 133, when the group is dissolved, outputs dissolution completion information to the command evaluating unit 134.
The state managing unit 133, when a state split instruction is input from the command evaluating unit 134, performs a state splitting process. As the state splitting process, the state managing unit 133 first determines whether a metafunction has been executed in a state corresponding to the state split instruction. In the case of the executed metafunction, the state managing unit 133 stores a set M, in which mutation operations corresponding to the metafunction are bundled, that is, mutation descriptors used for the mutation operations among mutation descriptors of an original state, in the execution state set storage unit 123 in association with the state ID. The state managing unit 133, upon storing the set M in association with the state ID in the execution state set storage unit 123, outputs an evaluation instruction to the command evaluating unit 134.
In the case of not being the executed metafunction, the state managing unit 133 stores the set M of the mutation operations corresponding to the metafunction, that is, the mutation descriptors used for the mutation operations, in the execution state set storage unit 123 in association with the state ID. The state managing unit 133, upon storing the set M in association with the state ID in the execution state set storage unit 123, outputs an evaluation instruction to the command evaluating unit 134.
The state managing unit 133, when a group determination instruction is input from the command evaluating unit 134, refers to the execution state set storage unit 123 and determines whether a state before the split belongs to a group. If the state before the split belongs to the group, the state managing unit 133 includes a state after the split in the group before the split. That is, the state managing unit 133 stores circle marks in the field of the synchronous execution group for corresponding states in the execution state set storage unit 123. The state managing unit 133, upon classifying the split state into the group, ends the state splitting process.
If the state before the split does not belong to a group, the state managing unit 133 generates a new group and includes the state after the split in the generated group. That is, the state managing unit 133 stores circle marks in the field of the synchronous execution group for the state after the split in the execution state set storage unit 123. The state managing unit 133, upon classifying the split state into the group, ends the state splitting process.
The state managing unit 133, when the state splitting process ends or when command execution information is input from the command evaluating unit 134, determines whether the state is included in a group and whether the same command has been executed in all of states in the group. If the state is included in the group and the same command has been executed in all of the states in the group, the state managing unit 133 performs a state merging process. If the state is included in the group and the same command has not been executed in all of the states in the group, the state managing unit 133 performs a state completion checking process.
As the state merging process, the state managing unit 133 first determines whether the states in the group have the same evaluation result. If the states in the group have the same evaluation result, the state managing unit 133 ends the state merging process. If the states in the group do not have the same evaluation result, the state managing unit 133 performs merging of a state s in the group. The state managing unit 133 determines whether a different state having the same evaluation result is present in the group. If a different state having the same evaluation result is present in the group, the state managing unit 133 adds a mutation descriptor of the state s to the state having the same evaluation result, and deletes the state s. If a different state having the same evaluation result is not present in the group, the state managing unit 133 performs merging of a next state. The state managing unit 133, upon completing the merging of the state s in the group, ends the state merging process and subsequently performs the state completion checking process.
As the state completion checking process, the state managing unit 133 first determines whether a program has reached the end. If the program has reached the end, the state managing unit 133 refers to the execution state set storage unit 123, stores a test result in the test result storage unit 124, deletes the states in the execution state set storage unit 123, and ends the state completion checking process. That is, the state managing unit 133 clears the execution state set storage unit 123 in which information on the states related to the selected test case has been stored.
If the program has not reached the end, the state managing unit 133 determines whether a termination condition specified by the user is exceeded. If the termination condition specified by the user is exceeded, the state managing unit 133 refers to the execution state set storage unit 123, stores a test result in the test result storage unit 124, deletes the states in the execution state set storage unit 123, and ends the state completion checking process. If the termination condition specified by the user is not exceeded, the state managing unit 133 determines whether an abnormality, such as an unauthorized access to a memory, has occurred.
If an abnormality has occurred, the state managing unit 133 refers to the execution state set storage unit 123, stores a test result in the test result storage unit 124, deletes the states in the execution state set storage unit 123, and ends the state completion checking process. If an abnormality has not occurred, the state managing unit 133 ends the state completion checking process.
The state managing unit 133, upon ending the state completion checking process, determines whether all of the states are completed. If all of the states are not completed, the state managing unit 133 performs the state selecting process. If all of the states are completed, the state managing unit 133 determines whether all of test cases have been executed. If all of the test cases have not been executed, the state managing unit 133 selects a next case and repeats the process by using the selected test case. If all of the test cases have been executed, the state managing unit 133 outputs an output instruction to the output unit 135.
As described above, the state managing unit 133 is an example of a managing unit that, when executing an intermediate code that is a compiled code in which an element in a source code is replaced with a metafunction that changes the element to a mutant, manages a set of mutation descriptors indicating a change in the mutant with respect to a mutation operation corresponding to the metafunction. Furthermore, the state managing unit 133 is an example of a selecting unit that selects one or more mutation descriptors having the same command evaluation result from the set of the mutation descriptors. Moreover, the state managing unit 133 is an example of a merging unit that merges third states having the same evaluation result among third states that are based on the command evaluation results in the group. Furthermore, the state managing unit 133 is an example of a merging unit that merges third states in the group while commands without adverse effects are continued. Moreover, the state managing unit 133 is an example of a merging unit that does not perform merging when a state returns to the first state or the second state through merging.
The command evaluating unit 134, when the state ID is input from the state managing unit 133, refers to the command storage unit 121 and the execution state set storage unit 123 and reads a command. The command evaluating unit 134, upon reading the command, determines whether a state indicated by the state ID is included in a group and the command is other than a command without an adverse effect. Meanwhile, the command without an adverse effect is a command that does not change a content of a memory, such as a specified metafunction (function except for increment/decrement), a binary operation (ADD, OR, or the like), a comparison operation (ICMP or FCMP), a conversion operation (Trunc, ZEXT, or the like), a read statement (load), or a getelementptr instruction. In contrast, a command that may cause an adverse effect may be, for example, a branch instruction, a function call, a return command (return), substitution for a variable, or the like. Furthermore, the command that may cause an adverse effect is, for example, a command that changes a value of a global variable.
If the state is included in a group and the command is other than the command without an adverse effect, the command evaluating unit 134 outputs a group dissolution instruction to the state managing unit 133. If the state is included in the group and the command is not other than the command without an adverse effect or if dissolution completion information is input from the state managing unit 133, the command evaluating unit 134 determines whether the command is a call for the metafunction. If the command is not a call for the metafunction, the command evaluating unit 134 executes the command and outputs command execution information to the state managing unit 133.
If the command is a call for the metafunction, the command evaluating unit 134 determines whether the metafunction is included in the mutation operator list. If the metafunction is not included in the mutation operator list, the command evaluating unit 134 executes the command and outputs command execution information to the state managing unit 133. If the metafunction is included in the mutation operator list, the command evaluating unit 134 outputs a state split instruction to the state managing unit 133.
The command evaluating unit 134, when the evaluation instruction is input from the state managing unit 133, refers to the command storage unit 121 and the execution state set storage unit 123 and performs an eager evaluation on mutation operations m in the set M. The command evaluating unit 134 stores command evaluation results of the mutation operations m in the evaluation result storage unit 122. The command evaluating unit 134 determines whether a state that already has the same evaluation result is present. If a state that already has the same evaluation result is present, the command evaluating unit 134 copies an original state, stores the evaluation result in the state, and selects the state. That is, the command evaluating unit 134 updates a record corresponding to the state ID of the state in the execution state set storage unit 123.
If a state that already has the same evaluation result is not present, the command evaluating unit 134 generates a state that has the same evaluation result and selects the state. Furthermore, the command evaluating unit 134 stores the generated state in the execution state set storage unit 123. That is, the command evaluating unit 134 splits the state.
The command evaluating unit 134 calculates direct products of the mutation operations m and the mutation descriptor of the original state, and adds the direct products to a mutation descriptor of the selected state, thereby updating the execution state set storage unit 123. The command evaluating unit 134 performs an eager evaluation, determination of the need of state splitting, and a calculation of the direct product with respect to each of the mutation operations m in the set M. The command evaluating unit 134, upon completing the eager evaluation, the determination of the need of state splitting, and the calculation of the direct product with respect to each of the mutation operations m in the set M, outputs a group determination instruction to the state managing unit 133.
As described above, the command evaluating unit 134 is an example of a first evaluating unit that evaluates a command of each of mutation descriptors. Furthermore, the command evaluating unit 134 is an example of a generating unit that calculates a direct product of a selected mutation descriptor and a mutation operation or a first state that is a set of the mutation descriptors before evaluation of the commands, thereby generating a second state. A part of the process performed by the generating unit may be performed by the state managing unit 133. Moreover, the command evaluating unit 134 is an example of a second evaluating unit that, when evaluating a command of each mutation descriptor of the generated second state, if a plurality of the second states are present, bundles the second states into a single group and evaluates commands of the respective second states in the group in parallel. Furthermore, the command evaluating unit 134 is an example of the second evaluating unit that dissolves the group and evaluates the commands of the second states when a command other than a command without an adverse effect appears.
The output unit 135, when the output instruction is input from the state managing unit 133, refers to the test result storage unit 124 and extracts mutants with “Pass” in the item of the test pass-fail. Furthermore, the output unit 135 calculates, as a mutation score, a value by dividing the number of mutants with “Fail” in the item of the test pass-fail in the test result storage unit 124 by the total number of mutants. The output unit 135 generates a result report based on the calculated mutation score and the mutants with “Pass” in the item of the test pass-fail. The output unit 135 generates a result screen including the result report, and outputs and displays the generated result screen to the display unit 112.
Next, operation performed by the analyzing apparatus 100 of the embodiment will be described.
The replacing unit 131, when a source code is input from the input unit 111, replaces an element in the source code with a metafunction that changes the element to a mutant, and embeds the metafunction in the source code (Step S1). The replacing unit 131 outputs the replaced source code to the compiler 132. When the replaced source code is input from the replacing unit 131, the compiler 132 compiles the replaced source code into an intermediate code (Step S2). The compiler 132 outputs the compiled intermediate code to the state managing unit 133.
The state managing unit 133 receives an input of a test item and a mutation operator list from the input unit 111. Furthermore, the state managing unit 133 receives an input of the intermediate code from the compiler 132. The state managing unit 133, when the intermediate code is input, selects a test case on the basis of the test item and the mutation operator list (Step S3). The state managing unit 133, upon selecting the test case, generates an initial execution state (Step S4). The state managing unit 133, upon generating the initial execution state, performs the state selecting process (Step S5).
The state selecting process will be described below with reference to
As the state selecting process, the state managing unit 133 first refers to the execution state set storage unit 123, and determines whether a state that is included in the group is present in an execution state set (Step S51). If the state included in the group is not present (NO at Step S51), the state managing unit 133 selects a state by an arbitrary method (Step S52), and returns to the original process.
If the state included in the group is present (YES at Step S51), the state managing unit 133 determines whether the group includes states in which a command being evaluated is not executed (Step S53). If the group includes the states in which the command being evaluated is not executed (YES at Step S53), the state managing unit 133 selects one arbitrary state from the states in which the command being evaluated is not executed in the group (Step S54), and returns to the original process.
If the group does not include the states in which the command being evaluated is not executed (NO at Step S53) the state managing unit 133 takes a next command as being evaluated, selects a state in which the next command is not executed (Step S55), and returns to the original process. When the process return to the original process, the state managing unit 133 outputs a state ID of the selected state to the command evaluating unit 134. Therefore, the analyzing apparatus 100 can select a state.
Referring back to
The state managing unit 133, when the group dissolution instruction is input from the command evaluating unit 134, deletes the field of the synchronous execution group in the execution state set storage unit 123 and dissolves the group (Step S8). The state managing unit 133, when the group is dissolved, outputs dissolution completion information to the command evaluating unit 134.
If the state is included in the group and the command is not other than the command without an adverse effect (NO at Step S7) or if dissolution completion information is input from the state managing unit 133, determines whether the command is a call for the metafunction (Step S9). If the command is not a call for the metafunction (NO at Step S9), the command evaluating unit 134 executes the command (Step S11), and outputs command execution information to the state managing unit 133.
If the command is a call for the metafunction (YES at Step S9), the command evaluating unit 134 determines whether the metafunction is included in the mutation operator list (Step S10). If the metafunction is not included in the mutation operator list (NO at Step S10), the command evaluating unit 134 executes the command (Step S11), and outputs command execution information to the state managing unit 133. If the metafunction is included in the mutation operator list (YES at Step S10), the command evaluating unit 134 outputs a state split instruction to the state managing unit 133.
The state managing unit 133, when the state split instruction is input from the command evaluating unit 134, performs the state splitting process (Step S12). The state splitting process will be described below with reference to
In the case of not being the executed metafunction (NO at Step S121), the state managing unit 133 stores a set M of mutation operations corresponding to the metafunction in the execution state set storage unit 123 in association with the state ID (Step S123). The state managing unit 133, upon storing the set M in association with the state ID in the execution state set storage unit 123, outputs an evaluation instruction to the command evaluating unit 134.
The command evaluating unit 134, when the evaluation instruction is input from the state managing unit 133, repeats the process from Step S124a to Step S124b in order to perform an eager evaluation, determination of the need of state splitting, and a calculation of a direct product for each of the mutation operations m in the set M. The command evaluating unit 134 refers to the command storage unit 121 and the execution state set storage unit 123 and performs an eager evaluation on commands of the mutation operations m (Step S125). The command evaluating unit 134 stores command evaluation results of the mutation operations m in the evaluation result storage unit 122.
The command evaluating unit 134 determines whether a state that already has the same evaluation result is present (Step S126). If a state that already has the same evaluation result is present (YES at Step S126), the command evaluating unit 134 copies an original state, stores the evaluation result in the state, and selects the state (Step S127). If a state that already has the same evaluation result is not present (NO at Step S126), the command evaluating unit 134 generates a state that has the same evaluation result and selects the state (Step S128).
The command evaluating unit 134 calculates direct products of the mutation operations m and the mutation descriptor of the original state, and adds the direct products to a mutation descriptor of the selected state, thereby updating the execution state set storage unit 123 (Step S129). The command evaluating unit 134, upon completing the eager evaluation, the determination of the need of state splitting, and the calculation of the direct product with respect to each of the mutation operations m in the set M, outputs a group determination instruction to the state managing unit 133.
The state managing unit 133, when the group determination instruction is input from the command evaluating unit 134, refers to the execution state set storage unit 123 and determines whether a state before the split belongs to a group (Step S130). If the state before the split belongs to the group (YES at Step S130), the state managing unit 133 includes a state after the split in the group before the split (Step S131). If the state before the split does not belong to a group (NO at Step S130), the state managing unit 133 generates a new group and includes the state after the split in the generated group (Step S132). The state managing unit 133, upon classifying the split state into the group, returns to the original process. Therefore, the analyzing apparatus 100 can split the state.
Referring back to
The state merging process will be described below with reference to
The state managing unit 133 determines whether a different state having the same evaluation result is present in the group (Step S143). If a different state having the same evaluation result is present in the group (YES at Step S143), the state managing unit 133 adds a mutation descriptor of the state s to the state having the same evaluation result, and deletes the state s (Step S144). If a different state having the same evaluation result is not present in the group (NO at Step S143), the state managing unit 133 performs merging of a next state. The state managing unit 133, upon completing the merging of the state s in the group, returns to the original process. Therefore, the analyzing apparatus 100 can merge the state.
Referring back to
If the program has not reached the end (NO at Step S151), the state managing unit 133 determines whether a termination condition specified by the user is exceeded (Step S152). If the termination condition specified by the user is exceeded (YES at Step S152), the state managing unit 133 proceeds to Step S154. If the termination condition specified by the user is not exceeded (NO at Step S152), the state managing unit 133 determines whether an abnormality, such as an unauthorized access to a memory, has occurred (Step S153). If an abnormality has occurred (YES at Step S153), the state managing unit 133 proceeds to Step S154. If an abnormality has not occurred (NO at Step S153), the state managing unit 133 returns to the original process. Therefore, the analyzing apparatus 100 can check completion of the state.
Referring back to
The output unit 135, when the output instruction is input from the state managing unit 133, refers to the test result storage unit 124 and calculates a mutation score. Furthermore, the output unit 135 generates a result report based on the calculated mutation score and mutants with “Pass” in the item of the test pass-fail. The output unit 135 generates a result screen including the result report, and outputs and displays the generated result screen to the display unit 112 (Step S18). Therefore, the analyzing apparatus 100 can reduce the number of states of a higher order mutation. That is, the analyzing apparatus 100 can reduce calculation cost of the mutation analysis.
Next, a specific example of the analysis process will be described with reference to
In
Subsequently, in
In
Subsequently, in
In
The register value “%2” in the command 52a is a value corresponding to a mutant of the operator “>” as illustrated in a table 56b. In this case, each of “%1>c”, “%1!=c”, and “%1<c” has the register value “%2” of “0”, that is, the same result. Therefore, in
Subsequently, in
In
Subsequently, in
In
As illustrated in
As illustrated in
As illustrated in
The state 68a in
In
In
In
In
As illustrated in
In
The state 76 in
Subsequently, the state 77a is evaluated with respect to the command 52a. The register value “%2” in the command 52a is a value corresponding to a mutant of the operator “>” as illustrated in a table 56g. In this case, in the command 52a, a mutation descriptor “{}” and mutation descriptors with the mutation ID of “2” among mutation descriptors in the state 77a are evaluated. Here, each of “%1>c”, “%1!=c”, and “%1>=c” has the register value “%2” of “1”, that is, the same result. Therefore, in
The state 77b is evaluated with respect to the command 52a. The register value “%2” in the command 52a is a value corresponding to a mutant of the operator “>” as illustrated in a table 56h. In this case, in the command 52a, a mutation descriptor “{}” and mutation descriptors with the mutation ID of “2” among mutation descriptors in the state 77b are evaluated. Here, “%1<=c” has the register value “%2” of “1, and there is no other mutant having the same result. Therefore, in
The state 77c is evaluated with respect to the command 52a. The register value “%2” in the command 52a is a value corresponding to a mutant of the operator “>” as illustrated in a table 56i. In this case, in the command 52, a mutation descriptor “0” and a mutation descriptor with the mutation ID of “2” among mutation descriptors in the state 77c are evaluated. Here, “%1>=c” has the register value “%2” of “0”, and there is no other mutant having the same result. Therefore, in
In
The state 79a in
Subsequently, the state 80a is evaluated with respect to the command 52a. The register value “%2” in the command 52a is a value corresponding to a mutant of the operator “>” as illustrated in a table 56j. In this case, in the command 52a, a mutation descriptor “{}” and mutation descriptors with the mutation ID of “2” among mutation descriptors in the state 80a are evaluated. Here, each of “%1>c”, “%1!=c”, and “%1>=c” has the register value “%2” of “1”, that is, the same result. Therefore, in
The state 80b is evaluated with respect to the command 52a. The register value “%2” in the command 52a is a value corresponding to a mutant of the operator “>” as illustrated in a table 56k. In this case, in the command 52a, a mutation descriptor “{}” and mutation descriptors with the mutation ID of “2” among mutation descriptors in the state 80b are evaluated. Here, “%1<=c” has the register value “%2” of “1, and there is no other mutant having the same result. Therefore, in
In
The state 81a in
Subsequently, the state 82a is evaluated with respect to the command 52a. The register value “%2” in the command 52a is a value corresponding to a mutant of the operator “>” as illustrated in a table 561. In this case, in the command 52a, a mutation descriptor “{}” and mutation descriptors with the mutation ID of “2” among mutation descriptors in the state 82a are evaluated. Here, “%1>c” and “%1!=c” has the register value “%2” of “0”, that is, the same result. Therefore, in
In the group 82, there are no states having the same calculation result of the register value “%2”, so that merging is not performed. Furthermore, the group 82 is dissolved. Moreover, the state 83a exits the while statement, returns “4”, and is terminated. Furthermore, the state 83b is subjected to the while loop again, returns “5”, and is terminated.
The state 68b in
In
The state 84c is evaluated with respect to the command 52a. The register value “%2” in the command 52a is a value corresponding to a mutant of the operator “>” as illustrated in a table 56n. In this case, in the command 52a, mutation descriptors with the mutation ID of “2” among mutation descriptors in the state 84c are evaluated. Here, each of “%1>c”, “%1!=c”, and “%1>=c” has the register value “%2” of “1”, that is, the same result. Therefore, in
In
In
As described above, the analyzing apparatus 100, when executing an intermediate code that is a compiled code in which an element in a source code is replaced with a metafunction that changes the element to a mutant, holds a set of mutation descriptors indicating a change in the mutant with respect to a mutation operation corresponding to the metafunction. The analyzing apparatus 100 evaluates a command of each of the mutation descriptors. The analyzing apparatus 100 selects one or more mutation descriptors having the same command evaluation result in the set of the mutation descriptors. The analyzing apparatus 100 calculates a direct product of the selected mutation descriptor and the mutation operation or a first state that is the set of the mutation descriptors before evaluation of the commands, thereby generating a second state. The analyzing apparatus 100, when evaluating a command of each mutation descriptor of the generated second state, if a plurality of the second states are present, bundles the second states into a single group and evaluates commands of the second states in the group in parallel. The analyzing apparatus 100 merges third states having the same evaluation result among third states that are based on the command evaluation results in the group. Consequently, it is possible to reduce the number of states of a higher order mutation.
Furthermore, the analyzing apparatus 100 merges the third states in the group while commands without adverse effects are continued. Consequently, it is possible to reduce the states, that is, the number of the states.
Moreover, if a command other than a command without an adverse effect appears, the analyzing apparatus 100 dissolves the group and evaluates the commands of the second states. Consequently, it is possible to prevent wasteful state splitting.
Furthermore, if a state returns to the first state or the second state through merging, the analyzing apparatus 100 does not perform merging. Consequently, it is possible to prevent repetition of splitting and merging.
Incidentally, while an LLVM bit code is used as an example of the intermediate code in the above-described embodiment, the code is not limited to this example. For example, it may be possible to use a Java (registered trademark) bytecode or the like that is executable on a JAVA VM.
The components illustrated in the drawings need not necessarily be physically configured in the manner illustrated in the drawings. In other words, specific forms of distribution and integration of the apparatuses are not limited to those illustrated in the drawings, and all or part of the apparatuses may be functionally or physically distributed or integrated in arbitrary units depending on various loads or use conditions. For example, the compiler 132, the state managing unit 133, and the command evaluating unit 134 may be integrated with one another. Furthermore, the processes illustrated in the drawings need not necessarily be performed in the same order as described above, and the processes may be performed simultaneously or in different order as long as the processing contents do not contradict one another.
Moreover, all or part of various processing functions implemented by the apparatuses may be implemented on a CPU (or a micro computer such as an MPU or a micro controller unit (MCU)). Furthermore, all or part of various processing functions implemented by the apparatuses may, of course, be implemented on a program analyzed and executed by a CPU (or a micro computer such as an MPU or an MCU) or a hardware using wired logic.
Incidentally, the various processes described in the above embodiment are implemented by causing a computer to execute a program prepared in advance. Therefore, an example of a computer that executes a program with the same functions as those of the above-described embodiment will be described below.
As illustrated in
The hard disk device 208 stores therein an analysis program having the same functions as the processing units such as the replacing unit 131, the compiler 132, the state managing unit 133, the command evaluating unit 134, and the output unit 135 illustrated in
The CPU 201 reads each program stored in the hard disk device 208, loads the program on the RAM 207, and executes the program to perform various processes. Furthermore, the program can cause the computer 200 to function as the replacing unit 131, the compiler 132, the state managing unit 133, the command evaluating unit 134, and the output unit 135 illustrated in
Incidentally, the above-described analysis program need not necessarily be stored in the hard disk device 208. For example, the computer 200 may read and execute a program stored in a storage medium that the computer 200 can read. Examples of the storage medium that the computer 200 can read include a portable recording medium, such as a CD-ROM, a DVD disk, or a USB memory, a semiconductor memory, such as a flash memory, and a hard disk drive. Furthermore, it may be possible to store the analysis program in a device connected to the public line, the Internet, a LAN, or the like, and cause the computer 200 to read the analysis program from the device and execute the analysis program.
It is possible to reduce the number of states of a higher order mutation.
All examples and conditional language recited herein are intended for pedagogical purposes of aiding the reader in understanding the invention and the concepts contributed by the inventor to further the art, and are not to be construed as limitations to such specifically recited examples and conditions, nor does the organization of such examples in the specification relate to a showing of the superiority and inferiority of the invention. Although the embodiment of the present invention has been described in detail, it should be understood that the various changes, substitutions, and alterations could be made hereto without departing from the spirit and scope of the invention.