Patent Application
20200042808
The present invention relates to image acquisition and image processing systems in motor vehicles.
In particular, the present invention relates to an apparatus and a method for detecting a traffic light phase for each camera for a motor vehicle.
The publication DE 102015003847 A1 describes a method for acquiring light signals from at least one light signaling device of a modulated light signal installation and for signaling the light signals to a driver of a vehicle. The following steps are provided for this purpose:
1. a visual acquisition of a region of the surroundings in front of the vehicle by means of at least one vehicle-mounted image acquisition device;
2. an evaluation of the image data acquired by the at least one vehicle-mounted image acquisition device by an image evaluation device, which evaluates the acquired image data for the presence of a red-light signal of at least one light signaling device of a modulated light signal installation;
3. displaying the red-light signal of at least one light signaling device identified in the evaluated image data by means of a vehicle-mounted display device.
Regarding step 2, it is explained in the publication: “Such image evaluation devices and methods for evaluating images are likewise known from the prior art so there is no need to go into more detail about these here”.
However, the process of establishing the currently switched traffic light phase, for instance red, amber, green, of a previously detected traffic light, by means of computerized visual image recognition methods in the field of road traffic hazard and situation recognition, is made difficult by the following factors:
i) glare or overexposure of the captured image occurring;
ii) reflections on the traffic light glass in the captured image which do not originate from the traffic light, for instance caused by extraneous light;
iii) poor contrast during the capturing of the image, for instance during imaging of the traffic light with a light background (sky) directly at the stop line in front of an overhead traffic light;
iv) steep angle of vision, for instance during imaging of the traffic light likewise directly at the stop line in front of an overhead traffic light; or
v) partial masking of the region of the red, green or amber light by the lampshade of the traffic light, for instance during lateral capturing, for instance of a traffic light located on the right side of the road when stopping directly at the stop line.
It is an object of the present invention to provide an improved visual image recognition for traffic light phases of traffic light systems.
This object is achieved by the subject matter of the independent claims. Further developments and embodiments can be inferred from the dependent claims, the description and the figures of the drawings.
A first aspect of the present invention relates to an apparatus for detecting a traffic light phase for a motor vehicle.
The apparatus comprises an image sensor device, which is designed to capture an image of a traffic light and to provide same in the form of image data.
The apparatus further comprises a segmentation device, which is designed to define at least one partial region of the captured image and to assign same to at least one signal lamp of the plurality of signal lamps of the traffic light.
The apparatus further comprises a scaling device, which is designed to determine a maximum color saturation and/or a maximum brightness in the at least one partial region. In particular, this means the maximum color saturation or respectively brightness for a pixel or respectively a small connected pixel region in the partial region.
The apparatus further comprises a computer device, which is designed to establish a signal status of the traffic light on the basis of the determined maximum color saturation and/or the determined maximum brightness.
In other words, the present invention advantageously makes it possible to guarantee a recognition of traffic lights and their respective traffic light phases.
The present invention also advantageously makes it possible to ensure that an overhead traffic light is recognized from a vehicle directly at the stop line—i.e. with an unfavorable, steep viewing angle of the traffic light—with a wide-angle lens.
Advantageous configurations of the present invention can be inferred from the subordinate claims.
According to a preferred embodiment, the segmentation device is designed to define a partial region of the captured image for each signal lamp, meaning each traffic light color, of a detected traffic light and to assign same to the (respective) signal lamp. The scaling device is preferably designed to determine the maximum and mean color saturation and/or maximum and mean brightness for each of the partial regions. The computer device is preferably designed to establish the signal status of the traffic light from a comparison of maximum to mean color saturation (or respectively brightness) for all partial regions of a traffic light.
In other words, a center having maximum brightness and/or color saturation is sought in each signal lamp (separated into red, amber and green) and is compared with the brightness in the remaining region of the respective signal lamp. The traffic light phase is calculated from the ratio of the maximum value and the average in the region of the respective signal lamp.
Using “just the brightness” of the respective signal lamps for recognition would not be sufficient to be able to establish the traffic light phase in most cases where signal lamps “only” reflect brightly, e.g. due to extraneous light.
In an advantageous embodiment of the present invention, it is provided that the computer device is designed to establish the signal status of the traffic light on the basis of
i) contrast values of the captured image;
ii) prevailing light conditions during the capturing of the image; or
iii) performance parameters of the camera used
with the aid of
i) the determined maximum color saturation alone;
ii) the determined maximum brightness alone; or
iii) both the determined maximum color saturation and the determined maximum brightness.
In an advantageous embodiment of the present invention, it is provided that the scaling device is designed to determine the maximum color saturation and the maximum brightness in the at least one partial region on the basis of an evaluation of the HSV color space of the image. The evaluation of the HSV color space can also be restricted to the partial region.
In an advantageous embodiment of the present invention, it is provided that the segmentation device is designed to assign the partial region of the signal lamp of the traffic light on the basis of a visual image recognition of signal light glasses of the traffic light.
In an advantageous embodiment of the present invention, it is provided that the image sensor device comprises a wide-angle lens.
The term “wide-angle lens” as used by the present invention comprises a lens having an angle of view which is larger than that of normal lenses, for example an angle of view which is larger than 40° or larger than 50° or larger than 60°.
In an advantageous embodiment of the present invention, it is provided that the image sensor device has an upwardly pointing viewing direction and is designed to capture an image of an overhead traffic light. This advantageously makes it possible to compensate for steep viewing angles and the resulting reduced contrast quality of the captured image.
The described configurations and further developments can be combined in any way with one another.
Further possible configurations, further developments and implementations of the present invention also comprise combinations of features of the present invention, which are described above or below with respect to the embodiments, including those which are not explicitly indicated.
The appended drawings are intended to provide a further understanding of the embodiments of the present invention.
The appended drawings illustrate embodiments and, in connection with the description, serve to explain concepts of the present invention.
Other embodiments and many of the indicated advantages are set out with respect to the figures of the drawings. The represented elements of the figures of the drawings are not necessarily shown to scale with respect to one another.
In the figures of the drawings, the same reference numerals denote elements, parts or components or method steps which are the same or which have the same function, unless otherwise indicated.
The motor vehicle or respectively vehicle is, for example, a motor vehicle or a hybrid vehicle, for example a hybrid vehicle having a coasting function, for example a motorcycle, a bus or a truck or a bicycle.
A signal generator of an optical signaling system in road traffic is designated as the traffic light or traffic light signals by the present invention.
These visual signal generators in the form of light signals are various road signs which each have different meanings depending on their shape and color and only operate against the direction of travel of the traffic which is to be regulated.
Driver assistance systems require an evaluation of the respective statuses of the traffic lights or respectively traffic light systems, i.e. assessment of the traffic light phase, generally by means of visual image recognition and processing.
The term “HSV color space” as used by the present invention designates a color space on the basis of a color model, in which the color is defined on the basis of a color value, “hue”, the color saturation, “saturation”, and the brightness value, “value”.
In an embodiment of the present invention, an HSL color space having relative lightness, “lightness”, or an HSB color space having absolute brightness, “brightness”, or an HSI color space having light intensity, “intensity”, can also be used.
The apparatus 1 comprises an image sensor device 10, a segmentation device 20, a scaling device 30, and a computer device 40.
The image sensor device 10 is designed to capture an image of a traffic light and to provide same in the form of image data.
The segmentation device 20 is designed to define a partial region of the captured image and to assign same to at least one signal lamp of the plurality of signal lamps of the traffic light.
The scaling device 30 is designed to determine a maximum color saturation and/or a maximum brightness in the partial region.
The computer device 40 is designed to establish a signal status of the traffic light on the basis of the determined maximum color saturation and/or the determined maximum brightness.
The method for detecting a traffic light phase for each camera for a motor vehicle comprises the following method steps:
A first method step involves capturing S1 an image of a traffic light by means of an image sensor device 10 and providing the image in the form of image data. The image data is, for example, binary coded data in an electronic form for reproducing image information.
A second method step involves defining S2, also segmenting, at least one partial region of the captured image by means of a segmentation device 20 and assigning the at least one partial region to at least one signal lamp of the traffic light by means of the segmentation device 20.
A third method step involves determining S3 a maximum color saturation and/or a maximum brightness in the at least one partial region by means of a scaling device 30.
A fourth method step involves establishing S4 a signal status of the traffic light on the basis of the determined maximum color saturation and/or the determined maximum brightness by means of a computer device 40.
A segmentation was performed for the seven traffic lights represented, i.e. partial regions SM1-1, . . . , SM7-3 of the captured image were defined, for example, by the segmentation device 20 and assigned to the signal lamps, i.e. red (e.g. SM1-1, SM2-1, . . . , SM7-1), amber (SM1-2, . . . , SM7-2), green (SM1-3, . . . , SM7-3), of the traffic light.
The traffic lights represented in
The circles show the partial regions SM1-1, . . . , SM7-3, within which the scaling device 30 determines a maximum and an average color saturation and a maximum and an average brightness in each case, the maximum brightness being more or less constant even as the viewing angle increases, unlike the total brightness of the partial region SM1-1, . . . , SM7-3, from left to right, i.e. from 1 to 7. In the case of the first five traffic lights represented (SM1- . . . , . . . , SM5- . . . ), the lowest partial region (SM1-3, . . . , SM5-3) shows, in each case, the highest mean color saturation and the highest mean brightness (compared with the two upper partial regions) . On the other hand, the sixth traffic light (SM6- . . . ) shows the highest mean color saturation and the highest mean brightness in the upper partial region (SM6-1). A traffic light recognition in accordance with the prior art would therefore reach the conclusion that the traffic light is showing a red signal.
On the other hand, the maximum color saturation and brightness is also at its highest in the lowest partial region (SM6-3) in the case of the sixth traffic light. In the case of the seventh traffic light (SM7- . . . ) the mean color saturation and brightness is, on the other hand, at its highest in the middle partial region (SM7-2). Here, a traffic light recognition in accordance with the prior art would reach the conclusion that the traffic light is showing an amber signal. On the other hand, the maximum color saturation and brightness is also at its highest here, in the case of the sixth traffic light, in the lowest partial region (SM7-3). It can therefore always be correctly recognized that all seven traffic lights are showing a green signal with the apparatus described.
In a particularly preferred embodiment, it is possible to determine even more clearly which traffic light signal a segmented traffic light is showing by forming ratios of the maximum to average brightness or respectively color saturation: it is the one with the highest ratio value.
As is clear from
In particular, modern signal lamps have beam distributors or respectively free-form reflectors, including stepped reflectors, for LED light sources. In the case of specific, unfavorable high viewing angles, beam distributors only make a small part of the light area appear illuminated and, consequently, the segmented partial region of the signal lamp only partially appears to be illuminated from the high (for example >30° viewing angles, i.e. it is not imaged with full color saturation and full brightness over the entire surface of the signal lamp.
As can likewise be seen from
This advantageously shows that an image data evaluation on the basis of the maximum color saturation and/or the maximum brightness is less prone to failure with respect to images at different viewing angles and is therefore more reliable overall.
Although the present invention has been described above on the basis of preferred exemplary embodiments, it is not restricted to these, but can be modified in many ways. In particular, the invention can be amended or modified in multiple ways, without deviating from the core of the invention.
In addition, it is pointed out that “comprising” and “having” do not exclude any other elements or steps and “a” or “one” does not exclude a plurality.
It is additionally pointed out that features or steps which have been described with reference to one of the above exemplary embodiments can also be used in combination with other features or steps of other exemplary embodiments described above. Reference numerals in the claims are not to be viewed as restrictions.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10 2017 203 236.7 | Feb 2017 | DE | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/DE2018/200018 | 2/27/2018 | WO | 00 |
