This application is based upon and claims the benefit of priority of the prior Japanese Patent Application No. 2008-163075, filed on Jun. 23, 2008, the entire contents of which are incorporated herein by reference.
This technique relates to a technique to effectively carry out design verification of hardware and/or a test for software.
Conventionally, when the design verification of the hardware or the test of the software is carried out, a method is adopted in which boundary conditions (e.g. conditions of branches and/or loops and the like) are extracted from an internal structure of a verification target (i.e. hardware or software), and input patterns to operate the verification target at values close to the boundary conditions are generated. However, when the conditions are complex, it is difficult to create the input patterns from the boundary conditions automatically or manually. Therefore, there is a problem that a long time is required to generate effective patterns.
In addition, there is a technique that the test is carried out by using patterns created in advance, and for the test, the total number of patterns is reduced by selecting or creating patterns that improves coverage (e.g. line coverage). However, this technique does not consider the boundary conditions of the verification target.
In order to efficiently carry out the design verification of the hardware or the test of the software, it is necessary to operate the verification target at points close to the boundary conditions. However, it is very difficult for the conventional techniques to generate the input patterns to operate the verification target at the points close to the boundary conditions, because the input patterns are generated based on the internal structure of the verification target.
Namely, it is impossible for the conventional technique to automatically extract test patterns to operate the verification target at the points close to the boundary conditions without analyzing the internal structure of the verification target.
According to an aspect of this technique, this test pattern extraction method includes obtaining an identifier of a processing executed for a test pattern by a verification target, and storing the identifier of the processing into a test result data storage device in association with the test pattern; calculating a distance between the test patterns whose identifiers of the processing are different each other and which are stored in the test result data storage device, identifying, for each group of the identifiers of the processing, a group of the test patterns whose distance satisfies a predetermined condition, and storing data of the identified group of the test patterns into a pattern data storage device.
The object and advantages of the embodiment will be realized and attained by means of the elements and combinations particularly pointed out in the claims.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the embodiment, as claimed.
The verification target 51 is, for example, software to be verified, hardware described, for example, by Hardware Description Language (HDL) or the like. The test execution unit 3 is, for example, a debugger for the verification target 51, and by a coverage obtaining function that the debugger normally has, an identifier of the processing (i.e. the function), which represents which processing (i.e. function) is carried out, is obtained.
For example, a case is considered where a function F(x, y) having two arguments as depicted in
In such a case, a correspondence relation between values of arguments and processing types (i.e. function types) is schematically depicted in
In this embodiment, within the aforementioned value ranges of x and y, test patterns (x, y) are generated, for example, randomly, and are inputted to a function F(x, y) to actually operate the function F(x, y), and it is identified by the test execution unit 3 including the coverage obtaining function of the debugger and the like, which processing of the processing A to D (i.e. the functions A to D) was carried out. As depicted in
Incidentally, for example, (x, y) may be prepared and stored in advance in the test pattern storage 1 in the examples of
Next, processing contents of the test pattern extraction apparatus depicted in
Then, the text execution unit 3 carries out a test processing (step S3). The test processing will be explained by using
Returning to the explanation of the processing of
Then, the pattern selector 9 carries out a pattern selection processing (step S7). The pattern selection processing will be explained by using
As schematically depicted in
Then, the pattern selector 9 identifies one unprocessed processing identifier s0 (i.e. function identifier s0) among the processing identifiers (i.e. the function identifiers) stored in the test result storage 7 (step S31). In addition, the pattern selector 9 identifies one unprocessed processing identifier s1 (i.e. function identifier s1) different from s0 among the processing identifiers (i.e. the function identifiers) stored in the test result storage 7 (step S33). Then, the pattern selector 9 initially set the infinite to a variable mid_dist storing the distance (step S35). Furthermore, the pattern selector 9 identifies an unprocessed test pattern p0 relating to the processing identifier s0, among the test patterns stored in the test result storage 7 (step S37). In addition, the pattern selector 9 identifies an unprocessed test pattern p0 relating to the processing identifier s1, among the test patterns stored in the test result storage 7 (step S39).
Then, the pattern selector 9 calculates the distance Dist (p0, p1) between the test patterns p0 and p1 (step S41). The distance between X and Y is calculated according to the following equation, for example.
Incidentally, it is assumed that X and Y respectively include n elements.
Then, the pattern selector 9 judges whether or not Dist (p0, p1) is less than min dist (step S43). In the first processing, because min dist is infinite, Dist (p0, p1)<min_dist is always satisfied. After the first processing, the judgment result varies in every cases.
When Dist (p0, p1) is equal to or greater than min_dist, the process shifts to the process of
Shifting to the explanation of the process in
When all test patterns relating to the processing identifier s1 have been processed, the pattern selector 9 judges whether or not min_dist is shorter than the first neighboring value δ1 (step S51). When min_dist is equal to or longer than the first neighboring value δ1, the process shifts to step S55. Namely, the candidate test pattern pair pp including the test patterns p0 and p1, whose distance is min_dist, is not adopted. In the example of
On the other hand, when min dist is less than the first neighboring value δ1, the pattern selector 9 additionally registers the candidate test pattern pair pp (i.e. pertinent test patterns p0 and p1) whose distance is min_dist, and the processing identifiers s0 and s1 for the candidate test pattern pair pp to a solution set (step S53). Namely, data as depicted in
In the example of
After that, the pattern selector 9 judges whether or not all test patterns relating to the processing identifier s0 have been processed (step S55). When an unprocessed test pattern exists among the test patterns relating to the processing identifier s0, the process returns to the step S37 of
On the other hand, when all test patterns relating to the processing identifier s0 have been processed, the pattern selector 9 judges whether or not all processing identifiers different from the processing identifier s0 have been processed (step S57). When an unprocessed processing identifier different from the processing identifier s0 exists, the process returns to the step S33 of
In addition, when all processing identifiers different from the processing identifier s0 have been processed, the pattern selector 9 judges whether or not all of the processing identifiers to be set to the processing identifier s0 have been processed (step S59). When an unprocessed processing identifier to be set to the processing identifier s0 exists, the process returns to the step S31 of
On the other hand, when all of the processing identifiers to be set to the processing identifier s0 have been processed, the process returns to the original process.
By carrying out such a processing, the test patterns relating to the different processing (i.e. the different functions) and whose distance is the shortest and shorter than the first neighboring value δ1 can be identified.
Incidentally, in the aforementioned processing flow, the test patterns identified for a pair of the processing A and B (i.e. the functions A and B) are identical with the test patterns identified for a pair of the processing B and A (i.e. the functions B and A), and the combination of the processing A and B and the combination of the processing B and A are separately processed. However, only the combination of the processing A and B may be processed.
Returning to the explanation of the process in
When the pattern generation is not carried out, the output unit 17 outputs data of the processing result (e.g.
On the other hand, the pattern generator 13 carries out a first pattern generation processing (step S13). The first pattern generation processing will be explained by using
First, the pattern generator 13 sets a second neighboring value δ2 (step S61). For example, a fixed value, which was set in advance, may be used, and after prompting the user to input a value, the value inputted by the user may be used.
Next, the pattern generator 13 identifies one unprocessed test pattern pair pp stored in the solution set data storage 11 (step S63). In addition, the pattern generator 13 identifies the test patterns p0 and p1 included in the identified test pattern pair pp (step S65).
Then, the pattern generator 13 carries out a second pattern generation processing for the test patterns p0 and p1 and the second neighboring value δ2 (step S67). This second pattern generation processing will be explained by using
First, an outline of the second pattern generation processing will be explained by using
When the distance between the test pattern a and the midpoint c1 is longer than the second neighboring value δ2, a mid point c2 (=(c2x, c2y)=((ax+c1x)/2, (ay+c1y)/2)) is calculated. Then, it is identified which processing (i.e. which function) the verification target 51 carries out for this midpoint c2. Here, it is assumed that the processing A (i.e. the function A) is carried out for the midpoint c2. Then, the test pattern a is replaced with the midpoint c2. Here, the distance between the midpoints c2 and c1 is calculated and it is judged whether or not the distance is equal to or shorter than the second neighboring value δ2.
When the distance between the midpoints c2 and c1 is longer than the second neighboring value δ2, a midpoint c3 (=(c3x, c3y)=((c2x+c1x)/2, (c2y+c1y)/2)) is calculated. Then, it is identified which processing (i.e. which function) the verification target 51 carries out for this midpoint c3. Here, it is assumed that the processing A (i.e. the function A) is carried out for the midpoint c3. Then, the midpoint c2 is replaced with the midpoint c3. Here, the distance between the midpoints c3 and c1 is calculated, and it is judged whether or not the distance is equal to or shorter than the second neighboring value δ2. In this example, it is judged here that the distance between the midpoints c3 and c1 is equal to or shorter than the second neighboring value δ2, and the second pattern generation processing is completed.
According to the aforementioned outline, the pattern generator 13 calculates the distance Dist (p0, p1) between the test patterns p0 and p1 (step S71). For example, the distance is calculated according to the equation (1). Then, the pattern generator 13 judges whether or not the distance Dist (p0, p1) exceeds the second neighboring value δ2 (step S73). When the distance Dist (p0, p1) exceeds the second neighboring value δ2, it is determined that a test pattern that is much closer to the boundary can be generated, and when the distance Dist (p0, p1) is equal to or shorter than the second neighboring value δ2, it is determined that it is not necessary to generate a test pattern much closer to the boundary.
When the distance Dist (p0, p1) exceeds the second neighboring value δ2, the pattern generator 13 calculates a midpoint p2 between the test patterns p0 and p1 (step S75) The midpoint between X and Y is calculated as follows:
As described above, X and Y respectively include n elements, and an average is calculated for each element.
Then, the pattern generator 13 causes the test execution unit 3 to execute a processing (i.e. a function) for the test pattern p2 and causes the test execution unit 3 to identify an identifier of the executed processing (i.e. the executed function) (step S77). The identifier of the processing (i.e. the function) is obtained from the test execution unit 3. Then, the pattern generator 13 judges whether or not the processing identifier (i.e. the function identifier) of the test pattern p0 is identical with the processing identifier of the test pattern p2 (step S79).
When the processing identifier of the test pattern p0 is identical with the processing identifier of the test pattern p2, the pattern generator 13 replaces the test pattern p0 with the test pattern p2 (step S81). Then, the processing returns to the step S71.
On the other hand, when the processing identifier of the test pattern p0 is different from the processing identifier of the test pattern p2, the pattern generator 13 judges whether or not the processing identifier of the test pattern p1 is identical with the processing identifier of the test pattern p2 (step S83). When the processing identifier of the test pattern p1 is identical with the processing identifier of the test pattern p2, the pattern generator 13 replaces the test pattern p1 with the test pattern p2 (step S85). Then, the processing returns to the step S71.
On the other hand, when the processing identifier of the test pattern p1 is different from the processing identifier of the test pattern p2, it is determined that a state as depicted in
In such a case, the pattern generator 13 carries out the second pattern generation processing for the test patterns p0 and p1 as depicted in (1) of
Thus, test patterns close to the boundaries of the region in which the processing A (i.e. the function A) is carried out and the region in which the processing E (i.e. the function E) is carried out are generated, and test patterns close to the boundaries of the region in which the processing E (i.e. the function E) is carried out and the region in which the processing B (i.e. the function B) is carried out are generated.
When it is judged at the step S73 that the distance Dist (p0, p1) is shorter than the second neighboring value δ2, the pattern generator 13 stores the test patterns p0 and p1 at that time into the generated pattern data storage 15 (step S90). Then, the process returns to the original process.
The generated pattern data storage 15 stores data as depicted in
By carrying out the aforementioned process, the test pattern which is much closer to the boundary can be generated.
Returning to the explanation of the process in
Returning to the explanation of the process of
By carrying out such a processing, it becomes possible to automatically extract the test patterns for operating the verification target 51 at points close to the boundary conditions without analyzing the internal structure of the verification target 51.
Incidentally, in the aforementioned embodiment, an example is indicated that elements (i.e. variables) included in the test pattern is equally handled. However, because the value range of the control variable is narrower than that of the data variable, there is a tendency that a large number of value combinations of the control variable are tested. On the other hand, as for the data variable, only typical value combinations are often tested. In addition, when there is a processing branch by the data variable in the verification target 51, there are a lot of cases where the workload to generate the pattern for the boundary conditions of the data variable is larger than the work load to generate the patterns for the boundary conditions by the control variable, and there is high possibility that the boundary conditions depending on the data variable are missed.
In addition, compared with the control variable, there is a tendency that the value range of the data variable is wide, and the difference between the variable values of test patterns is large, and it becomes easy to select the boundary values depending on the data variable in the equation for calculating the distance. Then, by reducing the weight of the difference of the values of the control variable rather than that of the difference of the values of the data variable, it is made easy to select the test pattern pair according to the difference of the values of the data variable.
For example, in the example as depicted in
On the other hand, a weight coefficient wx=10000 is set for the control variable x whose value range is 0≦x≦5, and a weight coefficient wy=5 is set for the data variable y whose value range is 0≦y≦10000. Then, the distance WeightedDist (W, X, Y) is defined as follows:
Incidentally, the greater value the weight coefficient is, the longer the distance is. Therefore, the smaller weight is assigned to the variable relating to such weight coefficient.
In the example, WeightedDist(W, A, B)=22<WeightedDist (W, A, C)=100 is satisfied. Therefore, test patters A and B are selected.
Therefore, when the data variable or control variable and its weight coefficient are designated, for example, from the user, the pattern selector 9 and pattern generator 13 calculates the distance according to the aforementioned equation (2). In addition, the variable whose value range is narrow may be identified as the control variable, and the value range of the data variable may be designated for the weight coefficient of the control variable. Similarly, the variable whose value range is wide may be identified as the data variable, and the value range of the control variable may be designated for the weight coefficient of the data variable.
Although this embodiment is explained above, the embodiment is not limited to this embodiment. For example, the functional block diagram of
Furthermore, when the processing result is not changed, the processing order may be replaced or may be executed in parallel.
This embodiment described above can be summarized as follows:
This test pattern extraction method includes obtaining an identifier of a processing executed for a test pattern by a verification target, and storing the identifier of the processing into a test result data storage device in association with the test pattern; calculating a distance between the test patterns whose identifiers of the processing are different each other and which are stored in the test result data storage device, identifying, for each pair of the identifiers of the processing, a pair of the test patterns whose distance satisfies a predetermined condition, and storing data of the identified pair of the test patterns into a pattern data storage device.
Thus, it becomes possible to automatically extract the pair of the test patterns close to the boundary conditions without analyzing the internal structure of the verification target.
In addition, the test pattern extraction method may further include generating a pair of test patterns whose distance is much shorter from the pairs of the test patterns, which are stored in the pattern data storage device, and storing the generated pair of test patterns into the generated pattern data storage device. Thus, it becomes possible to generate the pair of test patterns which are much closer to the boundary. For example, the test patterns may be brought to the boundary until a pair of test patterns whose distance is less than a second threshold can be obtained.
Incidentally, the aforementioned predetermined condition may be a condition that the distance between the test patterns is minimum for the pair of identifiers of the processing and is shorter than a predetermined threshold. The predetermined condition may be only a condition that the distance is minimum.
Furthermore, the identifier of the processing is registered in the aforementioned pattern data storage in association with the test pattern. Then, the aforementioned generating may include: calculating a candidate test pattern that is a midpoint between the first and second test patterns, and obtaining the identifier of the processing executed by the verification target for the candidate test pattern; judging whether or not the identifier of the processing executed for the candidate test pattern is identical to the identifier of the processing executed for the first or second test pattern; when the identifier of the processing executed for the candidate test pattern is identical with the identifier of the processing executed for the first test pattern, replacing the first test pattern with the candidate test pattern, and calculating a second distance between the first and second test patterns; when the identifier of the processing executed for the candidate test pattern is identical with the identifier of the processing executed for the second test pattern, replacing the second test pattern with the candidate test pattern, and calculating the second distance between the first and second test patterns; when the second distance is shorter than a second predetermined threshold, storing the first and second test patterns into the generated pattern data storage.
This enables the test pattern much closer to the boundary to be automatically generated.
In addition, the aforementioned generating may further include, when the identifier of the processing executed for the candidate test pattern is different from the identifiers of the processing executed for the first and second test patterns, carrying out the generating for the first test pattern and the candidate test pattern, and carrying out the generating for the second test pattern and the candidate test pattern.
Incidentally, the aforementioned distance may be a distance weighted according to the variable included in the test pattern. By calculating the distance weighted, for example, on the data variable, not simple Euclid distance, it becomes possible to select or generate appropriate test patterns.
Incidentally, it is possible to create a program causing a computer to execute the aforementioned method, and such a program is stored in a computer readable storage medium or storage device such as a flexible disk, CD-ROM, DVD-ROM, magneto-optic disk, a semiconductor memory, and hard disk. In addition, the intermediate processing result is temporarily stored in a storage device such as a main memory or the like.
In addition, the aforementioned test pattern extraction apparatus is a computer device as shown in
All examples and conditional language recited herein are intended for pedagogical purposes to aid the reader in understanding the invention and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions, nor does the organization of such examples in the specification relate to a showing of the superiority and inferiority of the invention. Although the embodiments of the present inventions have been described in detail, it should be understood that the various changes, substitutions, and alterations could be made hereto without departing from the spirit and scope of the invention.
Number | Date | Country | Kind |
---|---|---|---|
2008-163075 | Jun 2008 | JP | national |