METHOD FOR EVALUATING SPATIALLY RESOLVED ACTUAL SENSOR DATA

Abstract

A method for evaluating spatially resolved actual sensor data recorded with at least one sensor. The actual sensor data are read first. A location and an orientation of the actual sensor data are ascertained. A spatially resolved map with spatially resolved expectations of sensor data is read. The spatially resolved expectations of the sensor data are compared to the actual sensor data. A property can be ascertained from the comparison. An estimation can take place as to what influence an error source has on the comparison of the spatially resolved expectations of the sensor data to the actual sensor data. It is determined whether or not the property is fulfilled, based on the comparison of the spatially resolved expectations of the sensor data to the actual sensor data and based on the estimation of the influence of the error source. The property and a property probability are output.

Description

CROSS REFERENCE

The present application claims the benefit under 35 U.S.C. § 119 of German Patent Application No. DE 10 2023 204 610.5 filed on May 17, 2023, which is expressly incorporated herein by reference in its entirety.

FIELD

The present invention relates to a method for evaluating spatially resolved actual sensor data, to a method for controlling a movable object, and to a computing unit with which one of the methods or both methods can be carried out.

BACKGROUND INFORMATION

Driver assistance systems and systems for the at least partially automated driving of vehicles or robots are described in the related art. In these systems, the environment of the vehicle or robot is sensed with one or more sensors, and a plan of the future behavior of the vehicle or robot is ascertained therefrom. Neural networks are frequently used to ascertain such a plan, or an environmental representation as a pre-product for the plan. German Patent Application No. DE 10 2018 008 685 A1 describes a method for training a neural network for determining a path prediction for a vehicle.

It can be provided that spatially resolved actual sensor data are evaluated to check whether they are in line with an expectation. As a result, free areas, i.e., areas into which the vehicle or the robot is to be moved, can be ascertained more easily. However, such methods require a high resolution of the involved sensors so that the spatially resolved actual sensor data are sufficiently accurate to ascertain the free areas. In these methods, it can be provided to ascertain whether free areas can possibly include a hidden object.

SUMMARY

An object of the present invention is to provide an improved method for evaluating spatially resolved actual sensor data in which a lower sensor resolution can be used. A further object of the present invention is to provide a method for controlling a movable object that accesses the evaluated sensor data. A further object of the present invention is to provide a computing unit with which one of the methods or both methods can be carried out. A further object of the present invention is to specify a computer program for carrying out the method. These objects may be achieved by features of the present invention disclosed herein. Advantageous example embodiments and developments of the present invention are disclosed herein.

The present invention relates to a method for evaluating spatially resolved actual sensor data recorded with at least one sensor. According to an example embodiment of the present invention, in this case, the actual sensor data are read first. Furthermore, a location and an orientation of the actual sensor data are ascertained. They can each relate to a two-dimensional or a three-dimensional environment of the sensor. In particular, location and environment can include two- or three-dimensional vectors. In addition, a spatially resolved map with spatially resolved expectations of sensor data is read. The spatially resolved expectations of the sensor data can, for example, relate to a road space for a vehicle or to a movement area of a robot. Now, the spatially resolved expectations of the sensor data are compared to the actual sensor data and a property is ascertained from the comparison of the spatially resolved expectations of the sensor data and the actual sensor data. Furthermore, an estimation can take place as to what influence an error source has on the comparison of the spatially resolved expectations of the sensor data to the actual sensor data. On the basis of the comparison of the spatially resolved expectations of the sensor data to the actual sensor data, it is subsequently determined whether or not a property is fulfilled in a drivable area. Furthermore, an estimation of the influence of the error source takes place. Lastly, the property and a property probability are output.

According to an example embodiment of the present invention, the spatially resolved expectations of the sensor data can, in particular, include expectations of raw sensor data and/or expectations of processed sensor data. The actual sensor data can include raw actual sensor data and/or expectations of processed actual sensor data. The spatially resolved expectations of the sensor data can, for example, include which areas are drivable for the vehicle or the robot, i.e., free of other objects. However, it can also be provided that the spatially resolved expectations of the sensor data include where objects such as, for example, but not exclusively, walls, posts, traffic signs, signs, machines, holes, which are to be avoided by the vehicle or the robot, are arranged on a roadway. The property can, in particular, include whether the area is really drivable and/or whether the objects are really arranged at these locations.

The property probability can, in particular, include with what certainty the property is present. In particular, a percent probability with which the property applies can be specified with the property probability. This can, for example, include that a hidden object of a specified size is ruled out to one hundred percent in a particular area, and that there is only a residual probability for a smaller object.

In particular, according to an example embodiment of the present invention, it can be provided that it is determined with what probability no object is present in an area. Alternatively, it may be provided that it is indicated to what object size an object is ruled out in an area. It can, for example, be provided that it is ascertained and output that no object greater than a few centimeters is arranged in an area. This information can then optionally be taken into account in a movement plan.

According to an example embodiment of the present invention, it can be provided that more than one error source is evaluated in order to ascertain the influence of a plurality of error sources on the comparison of the spatially resolved expectations of the sensor data to the actual sensor data. Then, an even more accurate estimation of the influence of the error sources and an even more accurate determination of the property probability can take place.

According to an example embodiment of the present invention, it can be provided that the comparison of the spatially resolved expectations of the sensor data and the actual sensor data takes place on the basis of a spatially resolved three-dimensional geometry and/or a texturing and/or a reflectance amplitude and/or a multispectral response and/or a magnetic resonance. In this case, the sensor can comprise a camera. Alternatively, or additionally, the sensor can comprise a radar sensor and/or a LIDAR sensor.

In one example embodiment of the method of the present invention for evaluating spatially resolved actual sensor data, the error source includes a measurement inaccuracy of the sensor. The measurement inaccuracy of the sensor can be propagated through a signal path so that a statement can be made as to which areas are free and thus drivable and which areas are occupied and thus not drivable, and a measurement inaccuracy of the sensor is taken into account in the process.

In one example embodiment of the method of the present invention for evaluating spatially resolved actual sensor data, the error source includes a location and orientation inaccuracy of the actual sensor data. The location and orientation inaccuracy can be propagated through a signal path so that a statement can be made as to which areas are free and thus drivable and which areas are occupied and thus not drivable, and inaccuracies of the location determination and of the orientation determination are taken into account in the process.

In one example embodiment of the method of the present invention for evaluating spatially resolved actual sensor data, the error source includes a map data inaccuracy. This map data inaccuracy can also be propagated through a signal path so that a statement can be made as to which areas are free and thus drivable and which areas are occupied and thus not drivable, and the map data inaccuracy can be taken into account in the process.

In one example embodiment of the method of the present invention for evaluating spatially resolved actual sensor data, the determination of whether the property is fulfilled takes place on the basis of a residual of a possibly existing object, wherein the residual, including an error estimate, is compared to a threshold value. In particular, the property can be fulfilled if the residual, including an error estimate, is above the threshold value. This makes a simple mathematical implementation of the method according to the present invention possible.

In one example embodiment of the method of the present invention for evaluating spatially resolved actual sensor data, error limits are taken into account when estimating the influence of the error source. In particular, maximum values and minimum values for estimating the error can be used in each case. This likewise makes a simple mathematical implementation possible.

In one example embodiment of the method of the present invention for evaluating spatially resolved actual sensor data, the error limits are furthermore checked as to whether an adjustment of the error limits is required. For example, outliers can be detected or checked for a number of deviations.

In one example embodiment of the method of the present invention for evaluating spatially resolved actual sensor data, an uncertainty volume is considered for the actual sensor data, wherein a property linked to the actual sensor data is assumed for the entire uncertainty volume. The uncertainty volume can in this case be a representation of measured values of the actual sensor data, with which a simple estimation can be made.

The present invention also relates to a method for controlling a movable object. This method for controlling a movable object can, in particular, be based on the method according to the present invention for evaluating spatially resolved actual sensor data. In particular, the method for evaluating spatially resolved actual sensor data can be used to determine a property and a property probability, and the method for controlling the movable object can subsequently be carried out. In this case, a movement of the movable object is first planned on the basis of the properties, movement data are subsequently ascertained on the basis of the planned movement, and the movable object is then moved on the basis of the movement data.

The present invention furthermore relates to a computing unit for carrying out one of the methods according to the present invention. In this case, it can be provided that the computing unit carries out the method for evaluating spatially resolved actual sensor data and/or the method for controlling a movable object. In particular, it can be provided that the computing unit carries out both methods. The computing unit can, for example, be part of a control unit of the vehicle or part of a control unit of the robot.

The present invention furthermore relates to a computer program containing machine-readable instructions which, when executed on one or more computers, cause the computer(s) to perform the method according to the present invention. Such a computer program can also be stored on the computing unit.

The present invention furthermore relates to a machine-readable data carrier and/or download product with the computer program according to the present invention.

Embodiment examples of the present invention are explained with reference to the figures.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a flowchart of a method for evaluating spatially resolved actual sensor data, according to an example embodiment of the present invention.

FIG. 2 shows a vehicle with a sensor, according to an example embodiment of the present invention.

FIG. 3 shows a robot with a sensor, according to an example embodiment of the present invention.

FIG. 4 shows a flowchart of a further method for evaluating spatially resolved actual sensor data, according to an example embodiment of the present invention.

FIG. 5 shows an uncertainty volume.

FIG. 6 shows a flowchart of a method for controlling a movable object, according to an example embodiment of the present invention.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

FIG. 1 shows a flowchart 100 of a method for evaluating spatially resolved actual sensor data recorded with at least one sensor. In a reading step 101, the actual sensor data are read. In an ascertainment step 102, a location and an orientation of the actual sensor data are ascertained. In a map reading step 103, a spatially resolved map with spatially resolved expectations of sensor data is read. The map reading step 103 can also be carried out before the reading step 101, or between the reading step 101 and the ascertainment step 102, and optionally also in parallel with the reading step 101 or ascertainment step 102. In a comparison step 104, the spatially resolved expectations of the sensor data are compared to the actual sensor data and a property is ascertained from the comparison of the spatially resolved expectations of the sensor data and the actual sensor data. In an estimation step 105, an influence of an error source on the comparison of the spatially resolved expectations of the sensor data to the actual sensor data is estimated. In a determination step 106, it is determined whether the property is fulfilled in a drivable area, on the basis of the comparison made in the comparison step 104 of the spatially resolved expectations of the sensor data to the actual sensor data and on the basis of the estimation made in the estimation step 106 of the influence of the error source. In an output step 107, the property and a property probability are output.

The property probability can, in particular, include with what certainty the property is present. In particular, a percent probability with which the property applies can be specified with the property probability. This can, for example, include that a hidden object of a specified size is ruled out to one hundred percent in a particular area, and that there is only a residual probability for a smaller object. The property probability can be ascertained in particular in the determination step 106.

FIG. 2 shows a vehicle 1 with a sensor 10 on a roadway 2. The sensor 10 can, for example, be a LIDAR sensor 11. By means of the LIDAR sensor 11, the roadway 2 can be scanned and it can be determined whether the roadway 2 is free so that the vehicle 1 can optionally be moved on the roadway 2 or in particular on free areas of the roadway 2. By means of the sensor 10, an object 3 on or above the roadway 2 can be detected, for example. In particular, it can be provided here to check, on the basis of a map read in the map reading step 103, whether an object 3 is always present at the location where the object 3 was ascertained, for example a building or a tree, or whether the object 3 is temporarily arranged at this location, for example in the form of a further vehicle parked there. Drivable areas can, in particular, be the parts of the roadway 2 into which the vehicle 1 can be moved.

The vehicle 1 furthermore comprises a computing unit 9 connected to the sensor 10. The computing unit 9 can serve to carry out the method according to the present invention of FIG. 1. The computing unit 9 can furthermore be configured to carry out a method for controlling the vehicle 1, which is explained below. In particular, it can be provided that the computing unit 9 carries out both methods. The computing unit 9 can, for example, be part of a control unit of the vehicle 1.

FIG. 3 shows a robot 4 with a sensor 10. The sensor 10 can, for example, be a LIDAR sensor 11. By means of the LIDAR sensor 11, a movement area 5 can be scanned and it can be determined whether the movement area 5 is free so that the robot 4, or in particular, for example, a tool 6 of the robot 4, can optionally be moved into the movement area 5. Here, too, an object 3 can be determined by means of the sensor 10 and a movement of the robot 4 can optionally be planned such that the tool 6 does not move toward the object 3. Drivable areas can, in particular, be the parts of the movement area 5 into which the robot 4 can be moved.

Furthermore, a computing unit 9 is connected to the sensor 10. The computing unit 9 can serve to carry out the method according to the present invention of FIG. 1. The computing unit 9 can furthermore be configured to carry out a method for controlling the robot 4, which is explained below. In particular, it can be provided that the computing unit 9 carries out both methods. The computing unit 9 can, for example, be part of a control unit of the robot 4.

It can, in particular, be problematic in the embodiment examples of FIGS. 2 and 3 if a hidden object 7 is present and is overlooked by the sensor 10 since, in this case, a movement in this area possibly results in a collision. The method explained in connection with FIG. 1 can, in particular, use the property probability to specify a probability with which a particular area is free of a hidden object of a specified size. In particular, it can thus be provided that it is determined with which probability no object is present in a drivable area. Alternatively, it may be provided that it is indicated to what object size an object is ruled out in a drivable area. It can, for example, be provided that it is ascertained and output that no object greater than a few centimeters is arranged in a drivable area. This information can then optionally be taken into account in a movement plan.

It can be provided that the comparison of the spatially resolved expectations of the sensor data and the actual sensor data takes place on the basis of a spatially resolved three-dimensional geometry and/or a texturing and/or a reflectance amplitude and/or a multispectral response and/or a magnetic resonance.

The sensor 10 can comprise a LIDAR sensor 11, as shown in FIGS. 2 and 3. Alternatively, or additionally, the sensor 10 can comprise a radar sensor and/or a camera.

FIG. 4 shows a flowchart 100 of a further method for evaluating spatially resolved actual sensor data, in which the method steps already described in FIG. 1 are denoted by the same reference signs. Additionally, a calibration step 108 is performed, in which the sensor 10 can be calibrated. After the calibration step 108, the ascertainment step 102 is carried out. Furthermore, a transformation step 109 can be carried out, in which the actual sensor data are transformed on the basis of the location ascertained in the ascertainment step 102 and on the basis of the orientation, ascertained in the ascertainment step 102, of the sensor.

In one embodiment example, the error source includes a measurement inaccuracy of the sensor. This measurement inaccuracy can be ascertained in the reading step 101 or on the basis of the actual sensor data read in the reading step 101. This can, for example, take place by specifying a measurement data uncertainty for each value of the actual sensor data.

In one embodiment example, the error source includes a location and orientation inaccuracy of the actual sensor data. The location and orientation inaccuracy of the actual sensor data can be ascertained in the ascertainment step 102 or on the basis of the location processed in the ascertainment step 102 and on the basis of the orientation processed in the ascertainment step 102. This can, for example, take place by specifying a location uncertainty for the location and also an orientation uncertainty for the orientation.

In one embodiment example, the error source includes a map data inaccuracy. This map data inaccuracy can be ascertained in the map reading step 103 or on the basis of the map read in the map reading step 103. The map data inaccuracy can, for example, comprise an uncertainty of a position of an object included in the map.

In one embodiment example, the determination as to whether the property is fulfilled takes place on the basis of a residual of a possibly existing object. The residual can then be compared to a threshold value. This can, for example, take place such that a residual of the object is ascertained in the comparison step 104, optionally using the described error sources, and the presence of the object is subsequently estimated in the estimation step 105 if the residual, optionally including an error estimate based on the error sources, is above a threshold value.

The error estimation can, in particular, include that intervals are specified for each error source and the error limits of the intervals are used to estimate a total error. In such an embodiment example, error limits are thus taken into account when estimating the influence of the error source.

In one embodiment example, the error limits are furthermore checked as to whether an adjustment of the error limits is required. This can, for example, be used for outlier detection or for a check for a number of deviations. The outlier detection can include that the actual sensor data are assessed on the basis of assumptions regarding a smoothness and/or a temporal consistency and/or a spatial consistency.

In one embodiment example, it is furthermore checked whether a plurality of the measured values or of the actual sensor data matches the map data. An upper limit for the influences of the error sources can thereby be estimated.

FIG. 5 shows an uncertainty volume 120, which links a physical object point 121 to a measured object point 122 from the actual sensor data. The uncertainty volume 120 results from the error sources and the consideration of how these error sources can influence the actual sensor data. The uncertainty volume 120 can then be projected onto a surface so that a projection 123 of the uncertainty volume 120 results. Furthermore, a rectangle estimation 124 of the projection 123 can take place. The projection 123 or the rectangle estimation 124 of the projection 123 allows estimating which grid cells 125 are affected by the measured object point 122, i.e., for which grid cells the object must be assumed.

In one embodiment example, an uncertainty volume 120 is considered for the actual sensor data, wherein a property linked to the actual sensor data is assumed for the entire uncertainty volume 120. The linked property can be the presence of the object described with the measured object point 122.

FIG. 6 shows a flowchart 100 of a method for controlling a movable object, for example the vehicle 1 or the robot 4. This method for controlling a movable object can, in particular, be based on the method for evaluating spatially resolved actual sensor data as explained in connection with FIG. 1 to 5. In particular, properties and property probabilities ascertained with the method for evaluating spatially resolved actual sensor data can be used in the method for controlling a movable object. First, a movement of the movable object is planned on the basis of the properties in a planning step 131. In this case, it can, in particular, be provided that property probabilities for the drivable areas are also considered. For example, it can be taken into account for which areas a hidden object 7 with a minimum size can be ruled out with a probability above a specified threshold value, which can, for example, also be 99 or 100 percent. Subsequently, movement data are ascertained on the basis of the planned movement in a movement data ascertainment step 132 and the movable object is moved on the basis of the movement data in a movement step 133. It is thus possible to be able to control movements of the vehicle 1 or of the robot 4. The computing unit 9 shown in FIGS. 2 and 3 can also be configured for carrying out this method.

The present invention furthermore relates to a computing unit 9 for carrying out one of the methods according to the present invention. It can be provided that the computing unit 9 carries out the method for evaluating spatially resolved actual sensor data and/or the method for controlling a movable object. In particular, it can be provided that the computing unit 9 carries out both methods. The computing unit can, for example, be part of a control unit of the vehicle 1 or part of a control unit of the robot 4.

The present invention furthermore relates to a computer program containing machine-readable instructions which, when executed on one or more computers, cause the computer(s) to perform the method according to the present invention. Such a computer program can also be stored on the computing unit 9.

The present invention furthermore relates to a machine-readable data carrier and/or download product with the computer program according to the present invention.

Although the present invention has been described in detail by means of the preferred embodiment examples, the present invention is not limited to the disclosed examples and other variations may be derived therefrom by a person skilled in the art without departing from the scope of protection of the present invention.

Claims

1. A method for evaluating spatially resolved actual sensor data recorded with at least one sensor, comprising the following steps: reading actual sensor data;ascertaining a location and an orientation of the actual sensor data;reading a spatially resolved map with spatially resolved expectations of sensor data;comparing the spatially resolved expectations of the sensor data to the actual sensor data, and ascertaining a property from the comparison of the spatially resolved expectations of the sensor data to the actual sensor data;estimating an influence of an error source on the comparison of the spatially resolved expectations of the sensor data to the actual sensor data;determining whether the property is fulfilled or not fulfilled, based on the comparison of the spatially resolved expectations of the sensor data to the actual sensor data and based on the estimation of the influence of the error source; andoutputting the property and a property probability.

2. The method according to claim 1, wherein the error source includes a measurement inaccuracy of the sensor.

3. The method according to claim 1, wherein the error source includes a location and orientation inaccuracy of the actual sensor data.

4. The method according to claim 1, wherein the error source includes a map data inaccuracy.

5. The method according to claim 1, wherein the determination of whether the property is fulfilled takes place based on a residual of a possibly existing object wherein the residual, including an error estimate, is compared to a threshold value.

6. The method according to claim 1, wherein error limits are taken into account when estimating the influence of the error source.

7. The method according to claim 6, wherein the error limits are checked as to whether an adjustment of the error limits is required.

8. The method according to claim 1, wherein an uncertainty volume is considered for the actual sensor data, wherein a property linked to the actual sensor data is assumed for the entire uncertainty volume.

9. A method for controlling a movable object, comprising the following steps: planning a movement of the movable object based on properties;ascertaining movement data based on the planned movement; andmoving the movable object based on the movement data.

10. A computing unit configured to evaluate spatially resolved actual sensor data recorded with at least one sensor, the computing unit configured to: read actual sensor data;ascertain a location and an orientation of the actual sensor data;read a spatially resolved map with spatially resolved expectations of sensor data;compare the spatially resolved expectations of the sensor data to the actual sensor data, and ascertain a property from the comparison of the spatially resolved expectations of the sensor data to the actual sensor data;estimate an influence of an error source on the comparison of the spatially resolved expectations of the sensor data to the actual sensor data;determine whether the property is fulfilled or not fulfilled, based on the comparison of the spatially resolved expectations of the sensor data to the actual sensor data and based on the estimation of the influence of the error source; andoutput the property and a property probability.

11. A non-transitory machine-readable data carrier on which is stored a computer program for evaluating spatially resolved actual sensor data recorded with at least one sensor, the computer program, when executed by one or more computers, causing the one or more computers to perform the following steps: reading actual sensor data;ascertaining a location and an orientation of the actual sensor data;reading a spatially resolved map with spatially resolved expectations of sensor data;comparing the spatially resolved expectations of the sensor data to the actual sensor data, and ascertaining a property from the comparison of the spatially resolved expectations of the sensor data to the actual sensor data;estimating an influence of an error source on the comparison of the spatially resolved expectations of the sensor data to the actual sensor data;determining whether the property is fulfilled or not fulfilled, based on the comparison of the spatially resolved expectations of the sensor data to the actual sensor data and based on the estimation of the influence of the error source; andoutputting the property and a property probability.