Patent Application
20240135720
The invention relates to a method as well as a device for the detection and determination of the height of objects in a vehicle environment.
Sensor systems, which can also detect small obstacles reliably at large distances, are required for highly automated driving. At present, LIDAR systems and stereo camera systems are being developed for this task since they offer the possibility of mapping the environment in 3D. Thus, the size and distance of the obstacles and the free space can be detected simultaneously. Stereo camera systems offer a significantly higher lateral resolution than LIDAR systems, whilst LIDAR systems allow very accurate distance determination. However, it is not possible to detect small objects over a long distance with the current solutions.
Accordingly, it is an object of the present disclosure to provide a method as well as a system by means of which small objects can be reliably detected and the height of the objects can be reliably determined.
This object is achieved by the subject-matter of the independent claims 1 and 6. Advantageous configurations and embodiments are the subject-matter of the subclaims.
Initial considerations were that it is essential to accurately estimate the height of an object in order to be able to judge a danger due to small obstacles.
Previous stereo methods are predominantly based on the SGM (Semi Global Matching) method. Recently, machine learning methods based on deep neural networks, which are trained using ground truth data from LIDAR measurements, have also been increasingly used.
In order to be able to detect small objects of a height of 10 cm at a distance of 100 m, an angular resolution of at least 0.03 degrees is required. High-resolution LIDAR systems are still a factor of 3 to 6 above said value and are therefore not sufficiently suited to this task. High-resolution camera systems having 8 Mpix image sensors and a lens having a 30° aperture angle offer an angular resolution of 130 pixels/degree and map a 10 cm high object at a distance of 100 m on 8 pixels.
A stereo camera system having 2 high-resolution cameras could theoretically detect such a small object at a distance of 100 m. However, the signal noise in the depth map which is obtained from the stereo images is, as a general rule, too poor to irrefutably detect the object and, simultaneously, to also determine the height of the object.
According to the present disclosure, there is proposed a method for the detection and determination of the height of objects by means of an environment detection system, including a first environment detection sensor and a second environment detection sensor of a vehicle, wherein at least one of the environment detection sensors is a camera, the method including the following steps:
The camera may be a high-resolution telephoto camera. The camera has, for example, a resolution of 8 Mpix and an aperture angle of 30° as well as an angular resolution of 130 pixels/degree. The second environment detection sensor is, for example, a stereo camera. In this case, the environment detection system would be designed as a stereo camera and the mono camera may be a part of the stereo system in this configuration. It would also be conceivable to use a single mono camera and a separate stereo camera. In this configuration, the environment representation of the second environment detection sensor would be a depth image. It would also be conceivable to use a radar or lidar sensor as the second environment detection sensor. In the case of a radar sensor, the environment representation would be an object list or a radar signature list in which detected objects or radar signatures and the distances thereof from the vehicle are entered. The data can then be fused with the mono image. If a lidar sensor serves as the second environment detection sensor, the environment representation would be a point cloud. The singular is used for the object in the claims, to make them easier to read. However, it goes without saying that multiple objects can also be detected. It should further be noted that the two steps for object detection can take place simultaneously or after one another in any order. For example, the object detection can be carried out first in the environment representation of the second environment detection sensor and only then can the object be detected in the mono image. A reverse order or a simultaneous process would also be conceivable, as already indicated.
The detection of the object in the mono image indicates the size of the object in pixels. The distance from the object may be measured by means of the second environment detection sensor, in particular if the latter is a radar or a lidar sensor.
In a particular configuration, the height of the object is determined based on a height of the object in pixels in the mono image, the determined distance of the object as well as a known angular resolution of the environment detection sensor. The angular resolution of the respective sensors follows from the specific sensor data.
In a further configuration, the object detection is carried out in the mono image by means of a semantic segmentation based on a trained convolutional neural network CNN. This is advantageous since objects can be efficiently detected by means of the semantic segmentation, and not only can the detection of objects be improved by means of a correspondingly trained CNN, but also the shape of the object can be determined. The information can be helpful in order to specify whether it would be possible to drive over an object, if necessary, without damaging the vehicle.
In a configuration, after carrying out the object detections, the object is confirmed by means of a comparison of the mono image and the second environment representation. In this configuration, it is compared whether the positions of the detections correspond. If a correspondence exists in the comparison, the object is unequivocally confirmed.
In a particular embodiment, the object detection of one of the environment detection sensors further specifies a region of interest for the other environment detection sensor. That means that, for example, the object detection of the second environment detection sensor, for example by means of a radar sensor, specifies a region of interest (ROI) or a search area for the camera. Consequently, it is ensured that only the determined region in the mono image is examined for an object. This leads to a more efficient object detection. It would also be conceivable that the object detection in the mono image allows an increase in sensitivity in the radar search field. As a result, it can be ensured that very weak radar detections, for example, can be recognized as objects. Here as well, this leads to a more efficient and, above all, safer object detection.
According to the present disclosure, an environment detection system is further provided for a vehicle, comprising a first environment detection sensor and a second environment detection sensor having a determined angular resolution and a computing unit, wherein at least the first environment detection sensor is configured as a camera, wherein a mono image is captured using the camera and a further environment representation is captured using the second environment detection sensor, wherein the computing unit is configured to detect an object in the mono image as well as in the environment representation of the second environment detection sensor, wherein the computing unit is further configured to carry out distance determination as well as height determination of the object.
In a configuration, the first environment detection sensor is a telephoto camera and the second environment detection sensor is a stereo camera, a radar sensor or a lidar sensor.
If the second environment detection sensor is configured as a stereo camera, the telephoto camera is further a part of the stereo camera. The second environment detection sensor would consequently be the stereo camera and the first, as before, would be a mono camera. In this case, the stereo camera serves as a height-determining sensor by height determination in a depth image.
Further advantageous configurations and embodiments are the subject-matter of the drawings, wherein:
| Number | Date | Country | Kind |
|---|---|---|---|
| 10 2021 201 713.4 | Feb 2021 | DE | national |
The present application is a National Stage Application under 35 U.S.C. § 371 of International Patent Application No. PCT/DE2022/200009 filed on Jan. 27, 2022, and claims priority from German Patent Application No. 10 2021 201 713.4 filed on Feb. 24, 2021, in the German Patent and Trademark Office, the disclosures of which are herein incorporated by reference in their entireties.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/DE2022/200009 | 1/27/2022 | WO |
