Patent Application
20060136783
1. Field of the Invention
This invention relates to the field of data processing systems. More particularly, this invention relates to reducing the size of test programs whilst ensuring that those test programs still produce a predetermined event.
2. Description of the Prior Art
It is known to utilise test programs to perform test operations upon data processing systems for various purposes. Such test programs may, for example, be used to validate the design or manufacture of a data processing apparatus. Another example would be to investigate critical timing path limitations, power consumption maxima or minima, particular combinations of events which stress other components within the overall system, etc. Test programs to investigate these types of event can be manually written by engineers who understand the data processing system under test and seek to produce the predetermined event that they wish to investigate. Alternatively, some test programs may be automatically generated, such as by the use of genetic algorithm techniques or random test instruction generation.
It is generally desirable for a variety of reasons that a test program should be short whilst still producing the predetermined event to be investigated. An example of a situation in which short test programs are particularly desirable is built-in self test in which on-chip memory storage is provided for the test programs and accordingly keeping the test program size small represents a significant saving in circuit overhead for the storage of those test programs on-chip.
Viewed from one aspect the present invention provides a method of analyzing a test sequence of program instructions for an apparatus for processing data, said test sequence producing a predetermined event for said apparatus for processing data, said method comprising:
(i) forming a plurality of copies of said test sequence to serve as parent test sequences;
(ii) removing at least one program instruction from each of said parent test sequences to produce a plurality of candidate test sequences;
(iii) detecting if executing respective candidate test sequences continues to produce said predetermined event for said apparatus for processing data; and
(iv) if at least one of said candidate test sequences continues to produce said predetermined event for said apparatus for data processing, then using said at least one of said candidate test sequences to form parent test sequences; and
(v) repeating steps (ii), (iii) and (iv) until a termination condition is detected.
This technique provides a way in which a test program may be automatically reduced in size whilst still producing a predetermined event under investigation. One way of considering this technique is that it represents a genetic algorithm in which continuing to produce a predetermined event is required for survival between generations and there is a weighted selection in favour of shorter test programs. Such a genetic algorithm can be run for many iterations until the test program reaches a quasi-minimum length whilst still producing the required predetermined event.
It will be appreciated that the predetermined event to be tested could take a wide variety of different forms. The present technique is particularly useful when seeking to develop shorter tests which still produce test failure, stimulation of a critical path, power consumption reaching a predetermined level, occurrence of a predetermined temporal sequence of events and/or occurrence of a predetermined concurrent combination of events. Whilst the present technique is particularly useful in producing test programs to investigate such events, the technique is not limited to these events.
The termination of the iteration can be controlled in a variety of different ways. Particularly useful termination conditions are that no candidate test sequences produced the predetermined event or that one or more of the candidate test sequences which continues to produce the predetermined event falls below a predetermined length established as a desired target length.
If an iteration of the technique produces no candidate programs which still produce the predetermined event, then the system may revert to the parent test programs and reiterate these with a view to a different random removal of program instructions having a chance of producing a test sequence that will continue to produce the predetermined event.
In the context of a limiting condition in which none of the candidate test sequences are producing the predetermined event, then a useful termination condition is that such behaviour is exhibited for a continuous predetermined number of iterations whereupon the iterative process is terminated.
Whilst it will be appreciated that the instructions removed on each iteration can be selected in a variety of different ways, in preferred embodiments it is effective to randomly select the instruction to be removed from the different candidate programs. Such random selection is simple to provide and unlikely to suffer from the risk of an in-built bias in how the evolution is steered which could result from an attempt to use a more sophisticated way in which the instruction to be removed is selected.
Viewed from another aspect the present invention provides apparatus for processing data operable to analyze a test sequence of program instructions for an apparatus for processing data, said test sequence producing a predetermined event for said apparatus for processing data, said apparatus comprising logic operable to perform the steps of:
(i) forming a plurality of copies of said test sequence to serve as parent test sequences;
(ii) removing at least one program instruction from each of said parent test sequences to produce a plurality of candidate test sequences;
(iii) detecting if executing respective candidate test sequences continues to produce said predetermined event for said apparatus for processing data; and
(iv) if at least one of said candidate test sequences continues to produce said predetermined event for said apparatus for data processing, then using said at least one of said candidate test sequences to form parent test sequences; and
(v) repeating steps (ii), (iii) and (iv) until a termination condition is detected.
Viewed from a further aspect the present invention provides a computer program product bearing a computer program for controlling a computer to perform a method of analyzing a test sequence of program instructions for an apparatus for processing data, said test sequence producing a predetermined event for said apparatus for processing data, said method comprising:
(i) forming a plurality of copies of said test sequence to serve as parent test sequences;
(ii) removing at least one program instruction from each of said parent test sequences to produce a plurality of candidate test sequences;
(iii) detecting if executing respective candidate test sequences continues to produce said predetermined event for said apparatus for processing data; and
(iv) if at least one of said candidate test sequences continues to produce said predetermined event for said apparatus for data processing, then using said at least one of said candidate test sequences to form parent test sequences; and
(v) repeating steps (ii), (iii) and (iv) until a termination condition is detected.
The above, and other objects, features and advantages of this invention will be apparent from the following detailed description of illustrative embodiments which is to be read in connection with the accompanying drawings.
Embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings in which:
Returning to
When the candidate test sequences 6 are evaluated, two of these continue to produce the predetermined event and one does not. This behaviour is indicated by the ticks and crosses in
The two candidate test sequences which survive are retained whilst the other candidate test sequence is discarded. Candidate test sequences are discarded if they either do not continue to produce the predetermined event or unpredictably produce the predetermined event.
At this stage two candidate test sequences which are both shorter than the original test sequence 2 have been produced and both of these candidate test sequences still produce the predetermined event. The system then checks for termination conditions. These termination conditions may be that no suitable candidate test sequences still showing the predetermined event have been found, that the candidate test sequences have reached a desired level of shortness, or other termination conditions. If the termination conditions are not met, then the passing candidate test sequences are then used as parent test sequences for and the iterative process repeated. The passing candidate test sequences can be considered to be a population of candidate test sequences from among which parent test sequences for a next generation are selected in accordance with genetic algorithm techniques. At a simple level all the candidate test sequences could be reused as parents, but in other embodiments there may be a tournament between the candidate test sequences that have passed to establish a ranking in their fitness to serve as parents for the next generation and a selection then made from amongst these ranked parents.
If no passing candidate test sequences were produced, then the parent test sequences from the previous generation may be reinstated and reused with different randomly selected program instructions being likely to be removed therefrom upon the next iterative generation. If such reinstatement occurs a predetermined number of times without any passing candidate test sequences being produced, then this may be detected as another form of termination condition.
Step 18 determines whether or not there are any candidate programs that produce the predetermined event after the mutation. If there are no such candidate programs that still produce the predetermined event, then step 20 serves to reinstate the previous parent programs as candidate programs that produce the event for another mutation cycle. Alternatively, step 20 is bypassed if at least one candidate event passing in the sense of still producing the predetermined event is detected at step 18. Step 22 detects if the termination conditions for the iterative process have yet been met. These termination conditions may be that no passing candidate test sequences have been identified, that the candidate test sequences have fallen below a predetermined size or that reinstatement of a previous generation of test sequences has occurred a predetermined number of times. If any of these termination conditions is met, then the iterative process terminates. Alternatively, processing proceeds to step 24 where the surviving candidate programs are used to form new parent programs and processing is returned to step 12 for another mutation and evaluation iteration.
Although illustrative embodiments of the invention have been described in detail herein with reference to the accompanying drawings, it is to be understood that the invention is not limited to those precise embodiments, and that various changes and modifications can be effected therein by one skilled in the art without departing from the scope and spirit of the invention as defined by the appended claims.
