Patent Application
20220067238
This application claims priority to EP Application No. 20193626.7, having a filing date of Au. 31, 2020, the entire contents of which are hereby incorporated by reference.
The following relates to a computer-implemented method and to a computer program product for testing a technical system having a plurality of technical components. Embodiments of the present invention further relate to a computerized device for testing a technical system having a plurality of technical components and to an arrangement comprising a technical system and such a computerized device for testing the technical system.
Testing a technical system may particularly include testing regarding functional behavior of the technical system as well as regarding safety of the technical system.
For example, a technical system to be tested may be a safety-critical system or safety-relevant system. The importance of safety-critical systems in many application domains of embedded systems, such as aerospace, railway, health care, automotive and industrial automation is continuously growing. Thus, along with the growing system complexity, also the need for safety assessment as well as its effort is increasing drastically to guarantee the high-quality demands in these application domains.
A goal of the safety assessment process is to identify all failures that cause hazardous situations and to demonstrate that their probabilities are sufficiently low. In the application domains of safety-critical systems the safety assurance process is defined by the means of safety standards, such as IEC61508 or others.
Traditionally, the analysis of a system in terms of safety consists of bottom-up safety analysis approaches, such as Failure Mode and Effect Analysis (FMEA), and top-down approaches, such as Fault Tree Analysis (FTA) to identify failure modes, their causes, and effects with impact on the system safety. With Component Fault Trees (CFTs) there is a model- and component-based methodology for FTA, which supports a modular and compositional safety analysis strategy.
In particular, for safety-critical systems, there are regulatory requirements to show that the safety-related functions or safety-relevant functionalities perform the intended functionality correctly and that all relevant hazards are dealt by the technical system adequately.
The relevant hazards are captured in safety analyses or safety models, for example. The technical system may then be designed and built to cope with these hazards. However, the safety analyses are usually not connected directly to the system design and test environments. This disconnect between the different engineering disciplines leads to a loss of information, to additional effort required to specify tests or test cases and the check if all measures realized to prevent hazards have been tested.
As technical systems get more complex, a model-based approach is currently being adopted in many parts of the system development. Conventionally, the models for different purposes, like functional behavior testing and safety testing, are created separately by experts. For instance, there is one model for the system architecture of the technical system, one model for testing purposes and one model for safety analysis. Testing and system models have been integrated to some extent to support a partial generation of tests for this technical system from the test models.
However, safety models conventionally consisting of manual analyses performed by safety together with system experts are not integrated with system or test models.
For tests relating to safety analyses, a manual approach has therefore been pursued so far. The safety analyses are looked through by system designers, safety experts and testers to define required tests. This manual approach takes a lot of effort, is limited in scope, as the tests have to be defined manually.
As systems get more and more complex, model-based approaches are used in the system development, but also for testing and safety it is difficult to conventionally evaluate if the defined tests cover the relevant safety functionality adequately.
An aspect relates to enhance the testing of a technical system.
According to a first aspect, a computer-implemented method for testing a technical system having a plurality of technical components is suggested. The computer-implemented method comprises:
By analyzing the testing, i.e. the executed tests, based on the test model linked with the safety model, suitable coverage criteria for the safety-relevant functionality of the technical system are provided. By the provided coverage criteria, a quantitative measure is available to determine how well the safety-relevant functionality is covered by the test cases or tests of the test model, in particular by the already executed test cases.
The provided coverage criteria are a suitable test coverage metric that can be evaluated based on the test model and the safety model. In this regard, the linking of the test model and the safety model enables a tracing between the test cases of the test model and the safety relevant functionality of the safety model.
In particular, the test model is an abstraction of the technical system that can be used to drive test design and analysis, in particular having a focus on the functional behavior of the technical system from a given initial state under certain actions and/or events to an expected final state.
In particular, the safety model is adapted to capture causes and reasons that lead to the undesired event of violating the safety-relevant functionality that needs to be prevented, managed or kept quantitatively at a certain level, e.g., the probability of occurrence needs to meet a threshold defined in standards, by regulations or implicitly defined by societal acceptance.
For example, if the safety model consists of a Component Fault Tree (CFT), the safety-relevant functionality that is analyzed is the top event of the CFT. The top event of the CFT is triggered by events occurring together, so-called cut-sets. A cut-set may be a combination of basic events of the CFT and may be adapted to cause said top event. Linking the safety model and the test model captures the connections between the top event to the part of the test model that captures the corresponding functionality that influences the occurrence of the top event. In particular, the elements of the cut-set that trigger the top event are linked to the elements of the test model, e.g., test data, test activities and/or system elements like stimulated interfaces.
In the following, several embodiments for the computer-implemented method for testing a technical system having a plurality of technical components are described.
According to an embodiment, in step a), the safety model is provided as a tree of logic, in particular as a Component Fault Tree, such that it includes a top event associated to a violation of the safety-relevant functionality.
For example, if the technical system is a railway and the safety-relevant functionality is decelerating the railway to a certain velocity within a certain time period, then a violation of said safety-relevant functionality may be if the railway cannot be decelerated to said certain velocity within the certain time period.
According to a further embodiment, in step a), the safety model is provided such that it includes, for each of the technical components having an input port and/or an output port,
According to a further embodiment, in step a), the safety model is provided such that it includes, for each of the technical components having an input port and/or an output port,
According to a further embodiment, the safety model includes a number of cut-sets, each of the cut-sets combining a number of basic events and adapted to cause the top event. In particular, the basic events constituting a cut-set are leaves in a Component Fault Tree (CFT) constituting the safety model, for example.
According to a further embodiment, in step e), the coverage criteria are provided such that they include probabilistic criteria for each respective test case of the test cases used in step d), said probabilistic criteria indicating a probability of occurrence during operation of the technical system, wherein the probability of occurrence is derived by a probability of the cut-set corresponding to the respective test case via the linking of the test model and the safety model.
In particular, the probabilistic criteria indicate an importance of each test case of the test model. For example, depending on the goals of the testing process using said test model, there may be a focus on very likely cases (high probability), e.g., to ensure that basic functionality is correctly implemented, but also in extremely rare cases (very low probability) to show that the system is able to deal even with rare cases. For example, different categories for relevant cases may be defined. Moreover, to each of said categories, a suitable approach and coverage target may be defined.
According to a further embodiment, in step e), the coverage criteria are provided such that they include qualitative criteria for each respective test case of the test cases used in step d), the qualitative criteria being derived from the number of basic events of the cut-set corresponding to the respective test case via the linking of the test model and the safety model.
According to a further embodiment, as part of the qualitative criteria, a plurality N of different classes for the test cases used in step d) are provided, each of said N different classes being defined by a different number M of basic events configured to cause the top event in combination, with Mϵ[1, . . . , N].
In short words, the qualitative criteria may be based on the number of elements of a cut-set. Thus, the importance of each test or test case of the test model may be based on the number of basic events that need to be fulfilled in order to trigger the top event. Here, different classes may be defined, e.g., cases that are triggered with only one event (test all), cases that are triggered by two events (test all combinations, or a certain percentage of these), cases that are triggered by M events, and cases that are triggered by more than M events.
According to a further embodiment, in step e), the coverage criteria are provided such that they include criteria of occurrence and/or of probability of occurrence for minimal cut-sets corresponding to the test cases used in step d).
In particular, a cut-set in a logic tree or fault tree is a set of basic events whose (simultaneous) occurrence ensures that the top event occurs. A cut-set is particularly said to be a minimal cut-set if, when any basic event is removed from the cut-set, the remaining events collectively are no longer a cut-set. In particular, for minimal cut-sets, further tests can be derived that ensure that the safety-relevant functionality is not triggered if any one of the required cut-set elements does not evaluate to true.
In particular, the tests or test cases defined in the test model and the link between the test model and the safety model are used to determine test coverage for each safety-relevant functionality with regard to the possibilities given above, i.e. probabilistic criteria or qualitative criteria or both, and if deemed relevant for each defined class within these categories. Based on this information, it becomes apparent which part of the safety-relevant functionality is tested adequately, and where more effort should be spent, or alternatively if testing effort for a certain safety-relevant functionality can be shifted to more efficient testing levels, e.g. from system to integration or software, as the feasible number of system level test cases is far more limited than on lower testing levels.
According to a further embodiment, in step c), the top event of the safety model is linked with those elements of the test model capturing a functionality that is configured to influence an occurrence of the top event.
According to a further embodiment, in step c), the safety model and the test model are linked such that
a certain input failure mode of the safety model is linked with a certain input interface of the test model,
a certain output failure mode of the safety model is linked with a certain output interface of the test model,
a certain basic event of the safety model is linked with an internal component state of the test model, and/or
the top event of the safety model is linked with a certain test case of the test cases of the test model.
According to a further embodiment, the method comprises:
According to a second aspect, a computer program product (non-transitory computer readable storage medium having instructions, which when executed by a processor, perform actions) is suggested, wherein the computer program product comprises a program code for executing the method of the first aspect or of an embodiment of the first aspect when the program code is run on at least one computer.
A computer program product, such as a computer program means or a computer program, may be embodied as a memory card, USB stick, CD-ROM, DVD or as a file which may be downloaded from a server in a network. For example, such a file may be provided by transferring the file comprising the computer program product from a wireless communication network.
According to a third aspect, a computerized device for testing a technical system having a plurality of technical components is suggested. The computerized device comprises:
In particular, the computerized device may be a computer or workstation. Moreover, the computerized device may be or may include a computer-aided or computer-related system or a computer system.
The respective unit, e.g. first providing unit, the second providing unit, the linking unit and the analyzing unit, may be implemented in hardware and/or in software. If said unit is implemented in hardware, it may be embodied as a device, e.g. as a computer or as a processor or as a part of a system, e.g. a computer system. If said unit is implemented in software, it may be embodied as a computer program product, as a function, as a routine, as a program code or as an executable object.
The embodiments and features according to the first aspect are also embodiments of the third aspect.
According to a fourth aspect, an arrangement comprising a technical system having a plurality of technical components and a computerized device for testing the technical system according to the third aspect is suggested.
Further possible implementations or alternative solutions of embodiments of the invention also encompass combinations—that are not explicitly mentioned herein—of features described above or below with regard to the embodiments. The person skilled in the art may also add individual or isolated aspects and features to the most basic form of embodiments of the invention.
Some of the embodiments will be described in detail, with reference to the following figures, wherein like designations denote like members, wherein:
In the Figures, like reference numerals designate like or functionally equivalent elements, unless otherwise indicated.
In this regard, the left part of
The technical system TS may be a safety-critical system, for example used in an application domain of embedded systems, such as aerospace, railway, health care, automotive or industrial automation. The technical system TS includes a plurality of technical components TC, for example including actors, sensors and/or receivers.
As indicated above, the method of
In particular, the safety model 10 is provided such that it includes a top event TE associated to a violation of the safety-relevant functionality. For example, if the technical system TS is a railway and the safety-relevant functionality is to decelerate the railway to a certain velocity within a certain time period, then a violation of said safety-relevant functionality may be if the railway cannot be decelerated to said certain velocity within the certain time period. This violation is mapped to the top event TE of the safety model 10 of
The classic fault tree of the safety model 10 of
Furthermore, the classic fault tree of the safety model 10 of
Moreover, the classic fault tree of the safety model 10 of
In particular, the safety model 10 includes a number of cut-sets, each of the cut-sets combining a number of basic events 13 and adapted to cause the top event TE.
In step S2, a system model 20 is provided, said system model 20 modeling a functional behavior of the technical system TS. AS shown in the middle part of
The system model 20 may include a number of input ports 21, a number of output ports 22 and a number of internal component failures 23. In the middle part of
In step S3, a test model 30 is provided, said test model 30 describing or including test cases C for testing the technical system TS. As shown in the right part of
In step S4, elements of the safety model 10 are linked with elements of the test model 30, in particular using elements of the system model 20, for enabling a tracing between the test cases C of the test model 30 and the safety-relevant functionality of the safety model 10.
In particular, the top even TE of the safety model 10 is linked with those elements of the test model 30 capturing a functionality that is configured to influence an occurrence of the top event TE.
In step S5, the technical system TS is tested using at least one of the test cases C generated based on the test model 30 linked with the safety model 10.
In step S6, the testing, i.e. the executed tests, is analyzed to provide coverage criteria for the safety-relevant functionality.
In particular, in said step S6, the coverage criteria are provided such that they include probabilistic criteria for each respective test case of the test cases C used in step S5. Said probabilistic criteria may indicate a probability of occurrence during operation of the technical system TS, wherein the probability of occurrence may be derived by a probability of the cut-set corresponding the respective test case C via the linking of the test model 30 and the safety model 10.
Moreover, in step S6, the coverage criteria may be provided such that they include qualitative criteria for each respective test case C used in step S5. The qualitative criteria may be derived from the number of basic events 13 of the cut-set corresponding to the respective test case C via the linking of the test model 30 and the safety model 10. The coverage criteria may be provided based on all combinatorial possible cases. Alternatively, or additionally, the coverage criteria may be provided based on the total probability of occurrence of the top event, and the test coverage may be the total probability of all cut-sets considered.
As part of the qualitative criteria, a plurality N of different classes for the test case C used in step S5 may be provided, wherein each of said N different classes may be defined by a different number M of basic events 13 configured to cause the top event TE in combination, with Mϵ[1, . . . , N].
Moreover, in step S6, the coverage criteria may be provided such they include criteria of occurrence and/or of probability of occurrence for minimal cut-sets corresponding to the test cases C used in step S5.
Details and examples for this linking of the safety model 10 and the test model 30 via the system model 20 are shown in
Moreover,
Furthermore,
Moreover,
In
The first providing unit 101 is configured to provide a safety model 10 modeling a safety-relevant functionality of the technical system TS.
The second providing unit 102 is configured to provide a test model 30 describing test cases C for testing the technical system TS.
The linking unit 103 is configured to link elements of the safety model 10 with elements of the test model 30 for enabling a tracing between the test cases C of the test model 30 and the safety-relevant functionality of the safety model 10.
The testing unit 104 is configured to test the technical system TS using at least one of the test cases C generated based on the test model 30 linked with the safety model 10.
The analyzing unit 105 is configured to analyze the testing for providing coverage criteria for the safety-relevant functionality.
Furthermore,
Although the present invention has been disclosed in the form of preferred embodiments and variations thereon, it will be understood that numerous additional modifications and variations could be made thereto without departing from the scope of the invention.
For the sake of clarity, it is to be understood that the use of “a” or “an” throughout this application does not exclude a plurality, and “comprising” does not exclude other steps or elements.
| Number | Date | Country | Kind |
|---|---|---|---|
| 20193626.7 | Aug 2020 | EP | regional |
