Patent Application
20130076899
An incorrect representation of image data on a display unit, i.e. on a screen, monitor or other display, can have serious consequences, depending on the purpose of the display unit. This is particularly true in cases in which display units are applied in circumstances that are relevant or even critical to safety. Examples of this might include applications in fields such as railway, aeronautical, automobile, military, medical or power engineering, such as for instance in nuclear power stations.
Generally speaking, hardware can fail as a result, for instance, of ageing, wear or of external influences. Presentday secure systems generally operate with computer support, in which the status of the system concerned is represented on a display unit. Because the operating activity of the personnel often depends on the information represented on the display unit, it is necessary that faults in the display, i.e. an incorrectly represented system status, can be detected promptly and reliably.
One way of achieving this is the provision of redundant information in the image data to be displayed. This can, as an example, be done in the case of a display unit employed in a railway signal box by representing a railway track on the one hand through the display of a green section of track and on the other hand through an adjacent image of a signal, also represented in green. Only if both items of information are present in the image shown on the display unit can the user assume that the representation of the image on display unit is correct.
The procedure described however has the disadvantage that at least the safety-critical information, in the form of corresponding image data to be displayed, must be generated redundantly. In addition, it is not entirely possible to exclude the possibility that a faulty representation may, depending on the situation at the time, not be detected by the operator, or only after some delay. It must also be borne in mind that display units in the form, for instance, of the liquid crystal displays (LCDs) that are usual nowadays have a relatively complex structure, and can comprise, for example, their own memory and their own computing unit. As a result of this there is, for instance, a risk that under some circumstances the display may show an old image from the memory, and this may be consistent in itself without in fact representing the actual status of the monitored system.
The present invention is based on the task of disclosing a method that permits particularly reliable and also prompt detection of an incorrect representation of image data on a display unit.
This task is fulfilled according to the invention by a method for detecting an incorrect representation of image data on a display unit, wherein test data is acquired through electronic acquisition of at least one part of the image represented on the display unit and through an electronic evaluation of at least one part of the acquired test data an incorrect representation of the image data on the display unit is detected.
In the context of the description of the present invention, the term display unit refers to those components that perform the actual reproduction of the image data, i.e. the two-dimensional component on which a person can read or observe the image represented using, or on the basis of, the image data. The display unit can receive the image data that is to be displayed from, for instance, a graphics card, a graphics controller or from some other control unit. The components from which the display unit receives the image data that is to be displayed may here be combined with the display unit to form a common component, or may be a separate component, only connected to the display unit through communication equipment.
The image data is the information that represents the input signal to the display unit or the basis for the representation of the image. Preferably the image data here is a quantity of individual, preferably digital, data values, each of which provides image values for individual pixels, i.e. points on the image that is to be displayed.
According to the method according to the invention, test data is initially acquired through electronic acquisition of at least one part of the image represented on the display unit. This means that the image that is displayed on the display unit on the basis of the image data is automatically detected or read back. On the one hand the entire image represented on the display unit can be detected hereby. On the other hand it is also feasible for merely a part, or multiple parts of the image represented on the display unit to be detected or read back. This is particularly expedient when an examination of the representation of the image data on the display units is only required for selected parts or regions of the image represented on the display unit.
An incorrect representation of the image data on the display unit is then detected through electronic evaluation of at least one part of the acquired test data. This means that the electronically acquired test data is subjected to an electronic, i.e. in particular to an automated evaluation, and that an incorrect representation of the image data on the display unit is detected on the basis of the evaluation carried out.
The method according to the invention offers the particular advantage that it permits the representation of the image data on the display unit to be checked automatically, independently of operating or supervisory personnel.
Advantageously the entire path from the transmission of the image data to be displayed on the display unit through to the actual representation on the display unit can hereby be checked, so that faults occurring at any location along this path can be reliably detected, and that, due to the electronic acquisition and evaluation, this detection is also prompt.
Preferably the method according to the invention can be further developed in such a way that the test data is acquired by means of an electronic camera aimed at the display unit. This has the advantage that the acquisition of the test data by means of an electronic camera aimed at the display unit is a particularly simple method that can be comparatively economically implemented for acquiring the test data. Preferably here the camera can be fastened to the display unit itself.
According to a further particularly preferred embodiment of the method according to the invention, the test data is acquired by means of photo-sensors arranged on the display unit. This means that the photo-sensors are arranged on the display unit itself, as a result of which direct, undistorted acquisition of the test data is enabled. A precondition for this is that the photo-sensors are arranged on the display unit in such a way that the image that is represented on the display unit continues to be recognizable.
Preferably the method according to the invention can here be further developed in such a way that the test data is acquired by means of a translucent photo-sensitive membrane attached to the display unit. This is advantageous, since by means of an appropriate sensor membrane, it is possible to acquire the test data for any display units, without the necessity of modifying the display unit itself for this purpose. The photo-sensors in the photo-sensitive membrane could here advantageously be independent of the size and number of the pixels, i.e. the resolution, of the monitor.
Because of the fact that a suitable translucent photo-sensitive membrane can be applied directly to the display unit, it is advantageously not necessary to provide focusing.
According to a further particularly preferred embodiment, the method according to the invention is developed in such a way that the test data is acquired by means of photo-sensors integrated into the display unit, implemented so as to detect the light from at least one pixel of the display unit that is physically adjacent to the photo-sensor concerned. This means that, according to this embodiment, the photo-sensors are arranged in intermediate spaces between the pixels of the display unit. Preferably here the photo-sensors are constructed in such a way that they are shaded towards the outside, i.e. to the front of the display unit, and to a large extent only acquire the light from the at least one pixel physically adjacent to the photo-sensor concerned. As a result, the method is made insensitive to stray light. The image can be acquired or recorded by acquiring the test data using a resolution that matches that of the display unit or with a lower resolution. Using a display unit with integrated photo-sensors for acquiring the test data, i.e. to read back the represented image, offers the advantage that the display unit can maintain a comparatively flat structure, and moreover in particular that any impairment of the quality of the representation of the image data on the display unit by the recording equipment, i.e. the photo-sensors, is avoided. Optical focusing here is advantageously again not required in this case.
It should be noted that it is also possible to employ more than one of the possibilities for acquiring the test data mentioned above at the same time. It is thus, for instance, feasible for the test data to be acquired on the one hand by means of an electronic camera aimed at the display unit and on the other hand, for acquisition of the test data by means of photo-sensors integrated into the display unit to also be carried out at the same time. Depending on the particular application and its associated features, an appropriate redundant acquisition of the test data can in particular increase the quality or the speed of the electronic evaluation.
According to a further particularly preferred embodiment, the method according to the invention is configured in such a way that the electronic evaluation of the at least one part of the acquired test data comprises a comparison between the at least one part of the acquired test data with at least one part of the image data. With the help of suitable hardware and/or software, the acquired test data, or at least a part of that data, can be compared with the corresponding image data. Preferably it should be ensured that the hardware components used for representing the image data and for acquiring the test data are sufficiently independent from one another that the data used for comparison with the image data can only be provided by the electronically acquired test data. Inasmuch as the comparison, which may be carried out in a variety of ways and which can, for instance, comprise processing of the image data and/or of the test data, shows that they are in agreement, the representation of the image data on the display unit is correct; on the other hand, in the case of a difference, it can be assumed that the representation of the image data on the display unit is incorrect. The significance or interpretation of the inconsistency can vary, depending on the type and purpose of the representation of the image data on the display unit.
Preferably the method according to the invention can, in addition or as an alternative, also be implemented in such a way that the electronic evaluation of the at least one part of the acquired test data comprises a comparison between the at least one part of the acquired test data with the reference data associated with the image data concerned. This is advantageous since in this way the comparison of the at least one part of the acquired test data with the corresponding image data is simplified. To the extent that certain image data occurs frequently, it is thus in this way possible to determine and acquire reference data for this image data, which permits a particularly simple, fast comparison of the at least one part of the acquired test data with the reference data. For this purpose, the reference data can differ from the associated image data both in terms of the data format and in terms of its content. The reference data can thus, for instance, comprise “target test data” determined from the corresponding image data.
According to another particularly preferred implementation of the method according to the invention, the electronic evaluation of the at least one part of the acquired test data comprises an interpretation of the content of the test data. This means that in the course of the electronic evaluation of the test data, as an alternative or in addition to a comparison between target and actual values, an interpretation of the content of the image read back, i.e. the test data, is also possible. An appropriate interpretation of the contents of the test data using, for instance, automatic text recognition such as, for instance, OCR (Optical Character Recognition), and/or automatic image recognition, can be performed here. The focus here is thus less on the representation of the image data itself, and more on the information conveyed by means of the representation. This has the advantage that, even without knowledge of the contents of each specific pixel on the target display, i.e. the image data, a suitable image interpretation unit can be used to check the image represented on the display unit in a manner that is appropriate for safety purposes. Advantageously this is also possible in the case of complex, changing representations.
According to another particularly preferred embodiment, the method according to the invention is configured in such a way that the acquisition of the test data and the electronic evaluation of the at least one part of the acquired test data are carried out continuously or at regular intervals. This is advantageous because an incorrect representation of the image data on the display unit can be detected particularly promptly.
According to another preferred embodiment of the method according to the invention, image data to be displayed is transmitted in the form of a test pattern to the display unit. This offers the advantage that the use of test patterns simplifies the electronic evaluation of the at least one part of the acquired test data, since in this case the expected test data is known. In addition, through the use of a test pattern, those functions of the display unit that quite possibly are not being used at the time concerned but which nevertheless may be necessary in situations that will occur in the future, can be tested. This might, for instance, concern the representation of particular colors, or the driving of specific areas of the display unit. Preferably, suitable test patterns can be regularly transmitted and displayed for very short times whereby the duration of the display can preferably be shorter than the perceptive capability of the human eye, so that an observer of the display unit is not, or is only insignificantly, disturbed by the transmission of the test pattern. Through appropriate synchronization of the transmission or representation of the test pattern with the acquisition or the reading back, it is possible for the time for which the test pattern is represented to be reduced to durations of, for instance, less than 0.1 second.
In terms of the equipment, the present invention is based on the task of disclosing equipment that permits particularly reliable and at the same time prompt detection of an incorrect representation of image data on a display unit.
This task is fulfilled according to the invention by equipment for detecting an incorrect representation of image data on a display unit, the equipment comprising the display unit for representing an image, a first means for acquiring test data through the electronic acquisition of at least one part of the image represented on the display unit, and a second means for detecting an incorrect representation of the image data on the display unit employing an electronic evaluation of at least one part of the acquired test data.
The advantages of the equipment according to the invention correspond to a large extent to those of the method according to the invention, so that in this respect reference is made to the corresponding explanations above. The same applies to the preferred further developments of the equipment according to the invention described below, so that in this respect again reference is made to the associated explanations given in connection with the corresponding preferred further development of the method according to the invention.
Preferably the equipment according to the invention is designed in such a way that the first means comprises an electronic camera aimed at the display unit.
According to another particularly preferred further development of the equipment according to the invention, the first means comprises photo-sensors arranged on the display unit.
Preferably the equipment according to the invention can also be configured in such a way that the first means comprises a translucent photo-sensitive membrane attached to the display unit.
According to another particularly preferred implementation, the equipment according to the invention is configured in such a way that the first means comprises photo-sensors integrated into the display unit that are implemented in order to detect the light from at least one pixel of the display unit that is physically adjacent to the photo-sensor concerned.
According to a particularly preferred further development, the equipment according to the invention is designed to execute the method according to the invention, or to execute the method according to one of the preferred further developments of the method according to the invention described above.
The invention is described with the help of exemplary embodiments in more detail below. Here
For reasons of clarity, the same reference numbers have been used in the figures for components that are identical or substantially the same in function.
In order to be able to detect an incorrect representation of the image data B on the display unit 10, the equipment also comprises a first means of acquiring test data P in the form of a camera 30. As is indicated in
In the context of the exemplary embodiment described it is to be assumed that the test data P related to the whole of the image represented on the display unit 10 is acquired by the camera 30. The acquired test data P is transmitted from the camera 30 to the control unit 20, where it is subjected to electronic evaluation. In the context of the exemplary embodiment described it can be assumed here that, in the light of the specific application of the display unit 10 in the present situation, the representation of the image is only relevant or critical to safety in the parts or regions 40, 50. For this reason the electronic evaluation of the acquired test data P is only carried out on the test data P acquired for the parts 40, 50 of the image. Alternatively, or in addition to this, it would in principle of course also be feasible for the test data in the first place only to be acquired for a part of the image. This could, for instance, be done by having the camera 30 only aimed at a part of the display unit 10 or of the image displayed on it.
The equipment shown in
The testing can preferably be carried out continuously or cyclically, i.e. regularly repeated. Here the monitoring of the representation of the image data in the context of the method is advantageously executed on the contents of the target image; in other words it is not restricted to checking whether the display unit 10 is in principle capable of representing image data.
In addition, or in some cases also as an alternative to this, it is possible to perform a comprehensive test of the functions for representing the image data on the display unit 10 by inserting test patterns for what can be only a very short duration. This has the advantage that sources of error, in particular including those within the display unit 10 itself, can be detected or made visible promptly.
The exemplary embodiment of the equipment according to the invention represented in
In addition to the possibility of distortions, it is possible with the exemplary embodiment of
In the exemplary embodiment of
The photo-sensitive membrane or layer 60 covers a part of the image represented on the display unit 10, so that as a rule the content of the image is darkened. The photocells of the photo-sensitive membrane 60 face towards the display unit 10, and are therefore significantly less sensitive to light scattered from the environment than to the light of the display unit 10, i.e. to light transmitted by the pixels.
In the exemplary embodiment of
In order to avoid an influence on the photo-sensors by pixels of the display unit located further away, screens can advantageously be provided. This is indicated in
The reduced resolution used to acquire the test data, i.e. the smaller number of photo-sensors as compared with the number of pixels, advantageously ensures that the image represented on the display unit 10 continues to be recognizable for the operating or monitoring personnel who are using the display unit 10. Advantageously, the two-dimensional application of the translucent membrane 60 avoids distortion in the course of the image acquisition, as a result of which no focusing is needed in order to acquire the test data.
Preferably the arrangement can be calibrated for the particular screen position and image geometry on the basis of software. It is possible here, for example, for significant representations to be detected during a learning phase, and the positions of the representations concerned to be stored.
In the context of the exemplary embodiment of
The photo-sensors 61a to 64a are constructed in such a way that they are shaded to the outside, and only receive the light from the pixels that surround them. As far as the rest of their structure is concerned, the display unit 10 according to the exemplary embodiment of
In the context of the exemplary embodiment of
In general, the display unit according to the exemplary embodiment of
Regardless of which kind of acquisition of test data described in connection with the exemplary embodiments of
Fundamentally it is expedient for the acquisition of the test data and the electronic evaluation of the at least one part of the test data to be carried out continuously or at regular intervals. The specification of the testing interval and of the areas of the display unit or of the represented image that are to be checked is carried out in relation to the particular application and to the fundamental risk considerations applicable to the case. The check interval will here generally be based on the danger that arises as a result of a possible incorrect representation. It is, for instance, feasible for the aim to be to avoid an incorrect representation of image data remaining for a period of more than 1 second. In that case a standard check interval of 1 second could be used as a basis or specified for the acquisition of the test data and the subsequent electronic evaluation.
As has already been explained, it is also possible for the region of the image section that is to be checked to be specified or determined depending on the particular application, so that the image data represented on the display unit can comprise a mixture of information that is to be checked with information that does not have to be checked. In this way, an unwarranted reaction of the system in a case in which a fault is detected in regions of the display or of the image data represented that are uncritical for safety, is avoided.
Through the acquisition or reading back of the test data it is advantageously possible to interpret the meaning of the represented content. On the one hand this can comprise text recognition, similar to an OCR system, or a general image recognition process. This has the advantage that even without knowing the specific pixel-by-pixel image content of the target display, i.e. of the image data to be represented, a meaningful examination, from the point of view of safety, of the information represented is possible. An example of this would be a tachometer that displays a speed on a display unit in graphic form. In this case, reading back the displayed speed by applying image evaluation to the acquired test data would be appropriate, whereby only the question of whether the speed to be displayed is shown to the observer on the display unit in a recognizable form is checked.
Through the use of reference data or target image patterns assigned to the image data concerned, the electronic evaluation of the test data is possible with relatively little computing. In this case, the expensive correlation comparison of the meaning of the image is advantageously not required, since the comparison image appropriate for the particular method of acquisition of the test data is already available.
Through the use of test patterns, the display unit can be checked for its capability for image representation particularly comprehensively, quickly and reliably. In this case test patterns or test image samples can be shown on the display unit for long enough to permit the corresponding test data to be acquired. This kind of use of test patterns or test images is expedient, since an image currently on display does not necessarily exploit all the possibilities of the display. In other words, for instance, with the system in a normal condition, it may be the case that a represented image does not include the color red. If therefore, the capacity to display the color red remains unused for a long time and if, because of a fault, the corresponding display capabilities are not working reliably, a regular check is expedient so that, if necessary, the display of a red warning is possible, and that it is not only when dangerous circumstances have arisen that the fact that a fault is preventing its display is detected.
In the context of the exemplary embodiment of
In the context of a method for detecting an incorrect representation of image data on a display unit, it is now possible to acquire test data only for the parts of the image 90 to 118, or if the test data is acquired for the whole of the represented image, only to carry out electronic evaluation of the acquired test data for the parts of the image 90 to 118 concerned. In this connection it should be stressed again that the parts or regions of the image that are to be checked, i.e. the test areas 90 to 109 for tracks, or 110 to 118 for signals, are only sketched in
The image data, as illustrated in
In accordance with the explanations of this given above, it is also possible to carry out a check of the representational capability of the display unit by means of briefly output test patterns, which can also be referred to as complete test images, at regular intervals.
It can also be seen from
In addition to application in the field of railway signaling or of railway safety engineering, i.e. for instance in connection with control panel displays in signal boxes or displays in train drivers' cabins, the method described above is generally capable of application in any other safety fields where the reliable, prompt and autonomous detection of an incorrect representation of image data on a display unit, independently of human intervention, is significant.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10 2010 023 891.0 | Jun 2010 | DE | national |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/EP11/59260 | 6/6/2011 | WO | 00 | 12/6/2012 |
