Some radar sensor systems use evidential grid mapping algorithms to process the radar sensor data in the context of environmental understanding, where a grid map is used as a representation or approximation of the real environment. Each cell in a grid map represents a location in the environment, typically using a fixed frame, and is assigned a state, such as an “occupied” state that indicates a probability that an object is present at that location or a “free” state that indicates a probability that the location is unoccupied. The occupied state can be further refined to a “dynamic” state that indicates the probability that the object at the location is moving (that is, “dynamic”) and a “static-dynamic” state used to represent an uncertainty whether the object at the location is stationary, that is, “static,” or dynamic. When a radar sensor is mounted on a moving platform, it can be difficult to determine whether a measurement is indicative of a static object or a dynamic object. The static-dynamic state is introduced to represent this uncertainty.
The evidence from a radar sensor at a particular instant in time about a state is called the “instantaneous mass” of the state. Evidential grid mapping algorithms iteratively process the instantaneous mass for a particular time with instantaneous masses at prior times to formulate a more accurate grid map. Some evidential grid mapping algorithms struggle to distinguish artifacts due to noise in the radar sensor data, for example, from artifacts due to dynamic objects in the radar sensor data, which reduces the quality of the resulting representation of the environment. In addition, some evidential grid mapping algorithms do not account for sensor characteristics in determining the probability that a cell is in a free, dynamic, or static-dynamic state. For example, the antenna gain pattern of a radar sensor can vary over azimuth angle, which influences the reliability and/or accuracy of measured radar data points in determining the probability that a cell is in free, dynamic, or static-dynamic state.
An apparatus comprises an inverse radar sensor model processor and a grid mapping processor. The inverse radar sensor model processor receives radar sensor data for a time k from a radar sensor and generates object data for the time k based on the radar sensor data. The inverse radar sensor model processor calculates and outputs instantaneous masses at the time k for each cell in a field of view (FOV) of the radar sensor based on the object data for the time k and a sensor characteristic. The grid mapping processor receives the calculated instantaneous masses for each cell in the FOV at the time k and accumulated masses for each cell in the FOV for a time period 0:k−1 prior to the time k. An accumulated mass for a cell in the FOV for the time period 0:k−1 represents a combination of instantaneous masses for the cell at each time increment in the time period 0:k−1. The grid mapping processor generates, for each cell in the FOV, updated accumulated masses for a time period 0:k based on the calculated instantaneous masses for the time k and the accumulated masses for the time period 0:k−1.
In some embodiments, the radar sensor data for the time k comprises point cloud data, and each data point in the point cloud data comprises a range with respect to the radar sensor, an azimuth angle with respect to the radar sensor, and a radial velocity with respect to the radar sensor. The apparatus further comprises a motion sensor, a motion calculator, and a motion compensator in some implementations. The motion sensor outputs motion data for the apparatus at the time k, and the motion calculator receives the motion data and determines calculated motion data based on the motion data. The motion compensator receives the calculated motion data and the radar sensor data for the time k and adjusts the radial velocity of each data point in the point cloud data based on the calculated motion data.
The inverse radar sensor model processor comprises an object level data calculator and an instantaneous mass calculator in some embodiments. The object level data calculator receives the radar sensor data for the time k and the adjusted radial velocity data, and generates the object data for the time k. The instantaneous mass calculator receives the object data for the time k and calculates the instantaneous masses at the time k for each cell in the FOV based on the object data for the time k, the adjusted radial velocity data, and the sensor characteristic. The instantaneous mass calculator then outputs the calculated instantaneous masses at the time k.
In some implementations, the sensor characteristic is at least one of an antenna gain as a function of angle with respect to the radar sensor, an antenna gain as a function of range with respect to the radar sensor, a signal-to-noise ratio as a function of angle with respect to the radar sensor, and a signal-to-noise ratio as a function of range with respect to the radar sensor.
In some embodiments, the object data for the time k includes locations at which radar reflections are centered at the time k, and the instantaneous mass calculator calculates the instantaneous masses at the time k for each cell (i,j) in the FOV by determining a probability of occupancy Pocc(i,j;k) at the time k based on a range of the cell (i,j) from the locations at which the radar reflections are centered. The instantaneous mass calculator then determines a position of the cell (i,j) with respect to a position of the radar sensor and updates the Pocc(i,j;k) based on an ambiguity of the radar sensor data associated with the position of the cell (i,j). The instantaneous mass calculator determines whether the Pocc(i,j;k) satisfies an occupancy criterion Åocc. In response to the Pocc(i,j;k) not satisfying the ∈occ, the instantaneous mass calculator performs a first set of operations, and in response to the Pocc(i,j;k) satisfying the ∈occ, the instantaneous mass calculator performs a second set of operations.
While performing the first set of operation in some implementations, the instantaneous mass calculator determines whether the position of the cell (i,j) is between the position of the radar sensor and a position of a cell (a,b) with a corresponding Pocc(a, b; k) that satisfies the ∈occ. In response to the position of the cell (i,j) not being between the radar sensor and the position of the cell (a,b), the instantaneous mass calculator calculates an instantaneous mass for a free state with low confidence and sets an instantaneous mass for a static-dynamic state and an instantaneous mass for a dynamic state to zero. In response to the position of the cell (i,j) being between the radar sensor and the position of the cell (a,b), the instantaneous mass calculator calculates the instantaneous mass for the free state with high confidence and sets the instantaneous mass for the static-dynamic state and the instantaneous mass for the dynamic state to zero.
The instantaneous mass m(i,j;k)(F) for the free state of a cell (i,j) at the time k with low confidence is represented as:
m(i,j;k)(F)=pƒ,1*wƒ,1(θi,j)*wƒ,1(ri,j)
where θi,j represents an angle of the position of the cell (i,j) with respect to the position of the radar sensor, ri,j represents a range of the position of the cell (i,j) with respect to the position of the radar sensor, pƒ,1 represents a constant, wƒ,1(θi,j) represents a first weight based on the sensor characteristic associated with the angle θi,j, and wƒ,1(ri,j) represents a second weight based on the sensor characteristic associated with the range ri,j.
The instantaneous mass m(i,j;k)(F) for the free state of a cell (i,j) at the time k with high confidence is represented as:
m(i,j;k)(F)=pƒ,2*wƒ,2(θi,j)*wƒ,2(ri,j)
where θi,j represents an angle of the cell position with respect to the position of the radar sensor, ri,j represents a range of the cell position with respect to the position of the radar sensor, pƒ,2 represents a constant, wƒ,2(θi,j) represents a first weight based on the sensor characteristic associated with the angle θi,j, and wƒ,2(ri,j) represents a second weight based on the sensor characteristic associated with the range ri,j.
While performing the second set of operation in some implementations, the instantaneous mass calculator determines whether an adjusted radial velocity associated with the cell (i,j) satisfies a velocity criterion ∈ν. In response to the adjusted radial velocity not satisfying the ∈ν, the instantaneous mass calculator calculates an instantaneous mass for a static-dynamic state and sets an instantaneous mass for a free state and an instantaneous mass for a dynamic state to zero. In response to the adjusted radial velocity satisfying the ∈ν, the instantaneous mass calculator calculates the instantaneous mass for the dynamic state and sets the instantaneous mass for the free state and the instantaneous mass for the static-dynamic state to zero.
The instantaneous mass m(i,j;k)(SD) for the static-dynamic state of a cell (i,j) at the time k is represented as:
m(i,j;k)(SD)=Pocc(i,j;k)*ρocc(radar)
where ρocc(radar) represents a weight based on the sensor characteristic. The instantaneous mass m(i,j;k)(D) for the dynamic state of a cell (i,j) at the time k is represented as:
m(i,j;k)(D)=Pocc(i,j;k)*ρocc(radar)
where ρocc(radar) represents a weight based on the sensor characteristic.
For a detailed description of various examples, reference will now be made to the accompanying drawings in which:
The grid mapping system described herein includes a radar sensor, an inverse radar sensor model processor, and a grid mapping processor. The radar sensor outputs sensor data, such as point cloud data with range, azimuth angle, radial velocity, and an associated reflectivity or an associated signal-to-noise ratio for each point, to the inverse radar sensor model processor, which generates object data representative of locations in two-dimensional space at which the radar reflections are centered based on the sensor data. The inverse radar sensor model processor then calculates instantaneous masses for each cell in a field of view of the radar sensor based on the object data and a sensor characteristic. The instantaneous masses for each cell can be determined based on a probability of occupancy for the cell, a radial velocity for the cell, and the sensor characteristic associated with the position of the cell. The grid mapping processor then iteratively processes the calculated instantaneous mass for each cell at a particular time with instantaneous masses from prior times to generate an accumulated mass and a more accurate grid map.
The motion calculator 130 receives the motion data 125 at an input 127 and determines calculated motion data 135, such as the position, the velocity, and the attitude of the system 100. An output 132 of the motion calculator 130 is coupled to an input 182 of the motion compensator 180, and the motion compensator 180 compensates a radial velocity for each radar measurement in the sensor data 115 to account for the estimated motion of the radar sensor 110. For example, the system 100 is mounted in a vehicle as part of a driver assistance system, and the motion compensator 180 compensates the radial velocities based on the calculated motion data 135 to account for at least some of the vehicle motion.
An output 186 of the motion compensator 180 is coupled to an input 137 of the inverse radar sensor model processor 140 and provides the compensated motion data 188 to the inverse radar sensor model processor 140. The inverse radar sensor model processor 140 calculates an instantaneous mass for each cell in the grid mapping based on the sensor data 115 and the compensated motion data 188 and outputs the instantaneous masses 145 through an output 142. An input 147 of the grid mapping processor 150 is coupled to the output 142 of the inverse radar sensor model processor 140, and receives the instantaneous masses 145. The grid mapping processor 150 processes the instantaneous masses 145 with prior instantaneous masses to update an accumulated grid mapping and outputs the accumulated masses 155 through an output 152. The accumulated masses 155 can be used to generate an approximate representation of the environment 160.
In turn, the approximate representation of the environment 160 can be used by other system to provide alerts, update parameters, and perform other actions. Returning to the example in which the system 100 is mounted in a vehicle as part of a driver assistance system, the approximate representation of the environment 160 can be used to identify obstacles such as parked and moving vehicles, pedestrians, bicyclists, and the like; present driver alerts such as a proximity alert; adjust a vehicle speed during a parking assistance operation; and the like. The feedback loop 165 receives the accumulated masses 155 at an input 162 and determines feedback data 170 for the grid mapping processor 150. An output 167 of feedback loop 165 is coupled to an input 172 of the grid mapping processor 150 to provide the feedback data 170.
The output 215 of the object level data calculator 220 is coupled to an input 230 of the instantaneous mass calculator 240. The instantaneous mass calculator 240 computes the instantaneous mass for each cell being free (F), occupied by a dynamic object (D), or occupied by an object that might be moving or stationary (SD). For each cell (i,j), the instantaneous mass calculator 240 uses the probability of occupancy Pocc(i,j) based on the object level data, the position of the cell with respect to the radar sensor and other cells with a positive probability of occupancy, and the radial velocity for the cell. The accuracy of the sensor data 115 varies based on the position (i,j) of the cell with respect to the radar sensor 110, and the position (i,j) of the cell with respect to other occupied cells can identify occlusions by objects in between the particular location and the radar sensor 110. An example process for calculating the instantaneous mass is described further herein with respect to
Table 1 summarizes the scenarios 300 shown in
ν(i,j;k) represents the compensated radial velocity associated with a cell (i,j) at a time k. The first entry in table 1 corresponds to the scenario 300A shown in
The second entry in table 1 corresponds to the scenario 300B shown in
The third entry in table 1 corresponds to scenario 300C shown in
The fourth entry in table 1 corresponds to scenario 300D shown in
The process 400 begins at step 405, where the instantaneous mass calculator 240 determines, for each cell (i,j) in the FOV 320 of the radar sensor 110, a probability of occupancy Pocc(i,j) based on the distance of the particular cell (i,j) from the radar reflections 330A and 330B indicated in the sensor data 115. For example, the probability of occupancy Pocc(i,j) at a time k can be represented with a two-dimensional Gaussian probability density in polar coordinates as follows:
Pi,j;kocc=1{{tilde over (P)}
where
The expression 1{{tilde over (P)}
and σr,l
At step 410, the instantaneous mass calculator 240 determines, for each cell (i,j) in the FOV 320, a position of the cell with respect to the sensor 110 and updates the corresponding Pocc(i,j) based on the ambiguity of the sensor data 115. The SNR of the radar sensor 110 can vary with respect to the angle and distance from the radar sensor 110, and thus the ambiguity of the sensor data 115 varies with respect to the angle and distance as well.
Returning to
The instantaneous mass calculator 240 proceeds to step 425 and determines whether the cell position (i,j) is in the path 345A between the sensor 110 and an occupied cell in the region 340A or in the path 345B between the sensor 110 and an occupied cell in the region 340B. For example, the instantaneous mass calculator 240 can compare the azimuth angle θi,j of the cell with respect to the radar sensor 110 to the azimuth angle θ330A of the radar reflection 330A and the azimuth angle θ330B of the radar reflection 330B with respect to the radar sensor 110. The instantaneous mass calculator 240 can determine a range of angles [θ330A−ϵθA, θ330A+ϵθA] around the radar reflection 330A and a range of angles [θ330B−ϵθB, θ330B+ϵθB] around the radar reflection 330B, where ϵθA is chosen to represent the azimuth angle for a cell at the edge of the radar reflection 330A with a Pocc that satisfies ∈occ and ϵθB is chosen to represent the azimuth angle for a cell at the edge of the radar reflection 330B with a Pocc that satisfies ∈occ. The instantaneous mass calculator 240 can then determine whether the azimuth angle θi,j of the cell is within the range of angles [θ330A−ϵθA, θ330A++ϵθA] or [θ330B−ϵθB, θ330B+ϵθB].
If the cell is not between an occupied cell and the sensor 110 at step 430, as illustrated in scenario 300A in
For example, the instantaneous mass m(i,j;k)(F) with a low confidence pƒ,1 can be represented as:
m(i,j;k)(F)=pƒ,1*wƒ,1(θi,j)*wƒ,1(ri,j) w
here pƒ,1 represents a constant between zero and one, wƒ,1(θi,j) represents a first weight based on the gain of the sensor 110 at the particular angle θi,j of the cell position with respect to the sensor 110, and wƒ,1(ri,j) represents a second weight based on the gain of the radar sensor 110 at the particular range ri,j of the cell position with respect to the radar sensor 110. The weights wƒ,1(θi,j) and wƒ,1(ri,j) can be looked up from a table based on the known behavior of the gain of the radar sensor 110.
In some implementations, the first weight wƒ,1(θi,j) is a function based on the radar sensor gain 550 over azimuth angle θi,j as shown in
where θmax represents the angles of the edges of the FOV 320 for sensor 110. In some implementations, the second weight wƒ,1(ri,j) is a function in which the weight wƒ,1 decreases monotonically with range (ri,j). For example, the second weight wƒ,1(ri,j) can be based on the radar sensor gain over range, and represented as:
where r1 represents a range of an object, rmax represents a maximum range for the sensor 110, and b represents a constant chosen based on the characteristics of sensor 110.
Returning to step 430, if the cell is between an occupied cell and the radar sensor 110, as illustrated in scenario 300B in
For example, the instantaneous mass m(i,j;k)(F) with high confidence pƒ,2 can be represented as:
m(i,j;k)(F)=pƒ,2*wƒ,2(θi,j)*wƒ,2(ri,j)
where pƒ,2 represents a constant greater than pƒ,1 and less than one, wƒ,2(θi,j) represents a third weight based on the gain of the sensor 110 at the particular angle θi,j of the cell position with respect to the sensor 110, and wƒ,2(ri,j) represents a fourth weight based on the gain of the radar sensor 110 at the particular range ri,j of the cell position with respect to the radar sensor 110. The weights wƒ,2(θi,j) and wƒ,2(ri,j) can be looked up from a table based on the known behavior of the gain of the radar sensor 110. Because there is a higher confidence in step 340 than at step 335, the constant pƒ,2 is greater than the constant pƒ,1 used in calculating the instantaneous mass m(i,j;k)(F) with low confidence. In addition, the weights wƒ,2(θi,j) and wƒ,2(ri,j) are normalized to a larger number between zero and one than the weights wƒ,1(ri,j) and wƒ,1(ri,j).
Returning to step 420, if the Pocc(i,j) satisfies the occupancy criterion ∈occ, it is likely the cell is occupied.
If the compensated radial velocity does not satisfy ∈ν at step 450, for example, the compensated radial velocity is less than or equal to ∈ν as illustrated in scenario 300C shown in
m(i,j;k)(SD)=Pocc(i,j;k)*ρocc(radar)
where ρocc(radar) represents a fifth weight based on the overall SNR of the radar sensor 110. The instantaneous mass m(i,j;k)(D) for a dynamic cell and the instantaneous mass m(i,j;k)(F) for a free cell at the cell position (i,j) and the time k are set to zero.
Returning to step 450, if the compensated radial velocity satisfies the velocity criterion ∈ν, for example, the compensated radial velocity is greater than ∈ν as illustrated in scenario 300D in
m(i,j;k)(D)=Pocc(i,j;k)*ρocc(radar)
The instantaneous mass m(i,j;k)(SD) for a static-dynamic cell and the instantaneous mass m(i,j;k)(F) for a free cell at the cell position (i,j) and the time k are set to zero.
From each of steps 435, 440, 455, and 460, the instantaneous mass calculator 240 proceeds to step 470 and determines whether there are cells remaining in the FOV 320. If there are cells remaining at step 475, the instantaneous mass calculator 240 returns to step 420 and iterates through the remaining steps of process 400 until an instantaneous mass has been calculated for each cell in the FOV 320. Once there are no cells remaining at step 465, the process 400 ends and the instantaneous mass calculator 240 outputs the instantaneous masses 145 to the grid mapping processor 150.
The grid mapping processor 150 generates the accumulated masses 155 for the cells at the present time k. That is, the grid mapping processor calculates the updated accumulated mass for the free state m(i,j;0:k)(F), the updated accumulated mass for the static-dynamic state m(i,j;0:k)(SD), and the updated accumulated mass for the dynamic state m(i,j;0:k)(D). The updated accumulated masses are calculated based on the instantaneous mass and the prior accumulated mass of an uncertain state (SDF) which represents the ambiguity of sensor data between the free state, dynamic state, and static-dynamic state.
The instantaneous mass of the uncertain state SDF m(i,j;k)(SDF) can be represented as:
m(i,j;k)(SDF)=1−m(i,j;k)(F)−m(i,j;k)(D)−m(i,j;k)(SD)
The prior accumulated mass of the uncertain state SDF m(i,j;0:k−1)(SDF) can be represented as:
m(i,j;0:k−1)(SDF)=1−m(i,j;0:k−1)(F)−m(i,j;0:k−1)(D)−m(i,j;0:k−1)(SD)
The updated accumulated mass for the free state m(i,j;0:k)(F) can be represented as:
m(i,j;0:k)(F)=[m(i,j;k)(F)][m(i,j;0:k−1)(F)]+[m(i,j;k)(F)][m(i,j;0:k−1)(SDF)]+[m(i,j;k)(SDF)][m(i,j;0:k−1)(F)]
The updated accumulated mass for the static-dynamic state m(i,j;0:k)(SD) can be represented as:
m(i,j;0:k)(SD)=[m(i,j;k)(SD)][m(i,j;0:k−1)(SD)]+[m(i,j;k)(SD)][m(i,j;0:k−1)(SDF)]+[m(i,j;k)(SDF)][m(i,j;0:k−1)(SD)]
The updated accumulated mass for the dynamic state m(i,j;0:k)(D) can be represented as:
m
(i,j;0:k)(D)=[m(i,j;k)(D)][m(i,j;0:k−1)(D)]+[m(i,j;k)(SD)][m(i,j;0:k−1)(D)]+[m(i,j;k)(D)][m(i,j;0:k−1)(SD)]+[m(i,j;k)(SDF)][m(i,j;0:k−1)(D)]+[m(i,j;k)(D)][m(i,j;0:k−1)(SDF)]
The instantaneous and prior accumulated masses of the static-dynamic state (SD) contribute to the updated accumulated mass of the dynamic state (D) because the static-dynamic state (SD) is an ambiguity whether a cell is stationary or dynamic. Accounting for this ambiguity enables the grid mapping processor 150 to change state from static-dynamic to dynamic more quickly, which makes the resulting representation of the environment 160 more accurate in the presence of moving objects.
While the system 100 described herein with reference to
In this description, the term “couple” may cover connections, communications, or signal paths that enable a functional relationship consistent with this description. For example, if device A generates a signal to control device B to perform an action: (a) in a first example, device A is coupled to device B by direct connection; or (b) in a second example, device A is coupled to device B through intervening component C if intervening component C does not alter the functional relationship between device A and device B, such that device B is controlled by device A via the control signal generated by device A.
Modifications are possible in the described embodiments, and other embodiments are possible, within the scope of the claims.
Number | Name | Date | Kind |
---|---|---|---|
5123057 | Verly | Jun 1992 | A |
6125339 | Reiser | Sep 2000 | A |
7421419 | Maren | Sep 2008 | B2 |
10657393 | Foroughi | May 2020 | B2 |
20020019697 | Cong | Feb 2002 | A1 |
20030191610 | Chen | Oct 2003 | A1 |
20210131823 | Giorgio | May 2021 | A1 |
Number | Date | Country |
---|---|---|
102018100907 | Jul 2019 | DE |
102019109332 | Oct 2020 | DE |
WO2020094170 | May 2020 | WO |
Entry |
---|
C. Seeger et al., “Locally Adaptive Discounting in Multi Sensor Occupancy Grid Fusion”; Proceedings of the 2016 IEEE Intelligent Vehicle Symposium (IV); Gothenburg. SWEEDEN; Jun. 19-22, 2016. (Year: 2016). |
Godaliyadda, G. et al., “Multi-Sensor Fusion in Dynamic Environment Using Evidential Grid Mapping.” Electronic Imaging, Jan. 2020, 6 pages. |
Number | Date | Country | |
---|---|---|---|
20230089552 A1 | Mar 2023 | US |