The subject disclosure relates to multipath ghost mitigation in a vehicle radar system.
A vehicle (e.g., automobile, motorcycle, truck, construction equipment, farm equipment, automated factory equipment) may include sensors to obtain information about the vehicle and its environment. Exemplary sensors include radio detection and ranging (radar) systems, light detection and ranging (lidar) systems, and cameras. Certain scenarios may lead to inaccurate information from a sensor. In the case of a radar system, transmitted energy is reflected by an object in the field of view of the radar system. The resulting reflected energy is received by the radar system and analyzed to determine information about the object such as its relative location to the radar system. When transmitted energy is deflected by one object to a second object, the reflected energy received from the second object is referred to as a multipath reflection and the second object is detected as a ghost object (i.e., a wrong location is perceived). Accordingly, it is desirable to provide multipath ghost mitigation in a vehicle radar system.
In one exemplary embodiment, a method for a radar system of a vehicle includes detecting two or more objects using the radar system of the vehicle. Tracks of the two or more objects in a track database are initiated. The tracks store data, respectively, for the two or more objects and are updated based on additional detections of the two or more objects. The tracks of the two or more objects initially are unclassified tracks in the track database. Two tracks corresponding to two of the two or more objects from the track database are selected as a candidate pair. Criteria are applied to the candidate pair to determine whether one track of the two tracks of the candidate pair is a track of a ghost object resulting from multipath reflection and another track of the two tracks of the candidate pair is a track of a true object corresponding with the ghost object. The ghost object represents detection of the true object in an incorrect location.
The candidate pair is classified in the track database as tracks of a true object and ghost object pair based on determining that the one track of the two tracks of the candidate pair is the track of the ghost object and the other track of the two tracks of the candidate pair is the track of the true object corresponding with the ghost object. Information from the track database is reported and, based on the classifying, only the data for the track of the true object is reported. The information is used to control an operation of the vehicle.
In addition to one or more of the features described herein, the selecting the two tracks of the candidate pair is based on a difference between a first range obtained from the one track of the two tracks of the candidate pair and a second range obtained from the other track of the two tracks of the candidate pair being within a threshold range value.
In addition to one or more of the features described herein, the method also includes iteratively selecting different candidate pairs from the track database, applying the criteria, and classifying the different candidate pairs.
In addition to one or more of the features described herein, the method also includes classifying one or both of the two tracks of the candidate pair as the track of the true object or leaving a prior classification of the two tracks of the candidate pair based on the criteria not determining that the one track of the two tracks of the candidate pair is the track of the ghost object and the other track of the two tracks of the candidate pair is the track of the true object corresponding with the ghost object.
In addition to one or more of the features described herein, the applying the criteria to the candidate pair includes applying a first criterion that includes estimating a shape of a reflective surface giving rise to the multipath reflection that would result in the one track of the two tracks of the candidate pair being the track of the ghost object.
In addition to one or more of the features described herein, the applying the first criterion includes determining that the one track of the two tracks of the candidate pair is the track of the ghost object based on a linearity of the reflective surface.
In addition to one or more of the features described herein, the applying the criteria to the candidate pair includes applying a second criterion that includes determining if dynamics indicated by the one track of the two tracks of the candidate pair matches dynamics indicated by the other track of the two tracks of the candidate pair such that velocity and heading indicated by the two tracks of the candidate pair are correlated within a threshold amount over multiple updates of the two tracks.
In addition to one or more of the features described herein, the applying the second criterion includes determining a first velocity indicated by the other track of the two tracks of the candidate pair using the one track of the two tracks of the candidate pair, determining a second velocity indicated by the other track of the two tracks of the candidate pair using the updates of the other track of the two tracks of the candidate pair, and classifying the one track of the two tracks of the candidate pair as the track of the ghost object and the other track of the two tracks of the candidate pair as the track of the true object corresponding with the ghost object based on the first velocity and the second velocity being within a specified threshold of each other.
In addition to one or more of the features described herein, the applying the criteria to the candidate pair includes determining an incident angle of transmit energy from the radar system on a reflective surface that gives rise to the multipath reflection that would result in the one track of the two tracks of the candidate pair being the track of the ghost object and determining whether the incident angle is below a threshold value, the threshold value being range-dependent.
In addition to one or more of the features described herein, the method also includes estimating a velocity of the true object based on the ghost object based on the determining that the one track of the two tracks of the candidate pair is the track of the ghost object and the other track of the two tracks of the candidate pair is the track of the true object corresponding with the ghost object.
In another exemplary embodiment, a system in a vehicle includes a radar system to detect two or more objects. The system also includes a processor to initiate tracks of the two or more objects in a track database, the tracks storing data, respectively, for the two or more objects and being updated based on additional detections of the two or more objects, and the tracks of the two or more objects initially being unclassified tracks in the track database. The processor additionally selects two tracks corresponding to two of the two or more objects from the track database as a candidate pair, and applies criteria to the candidate pair to determine whether one track of the two tracks of the candidate pair is a track of a ghost object resulting from multipath reflection and another track of the two tracks of the candidate pair is a track of a true object corresponding with the ghost object. The ghost object represents detection of the true object in an incorrect location. The processor additionally classifies the candidate pair as tracks of a true object and ghost object pair based on determining that the one track of the two tracks of the candidate pair is the track of the ghost object and the other track of the two tracks of the candidate pair is the track of the true object corresponding with the ghost object. The processor reports information from the track database. The report provides the data only for the track of the true object, not the ghost object. The information is used to control an operation of the vehicle.
In addition to one or more of the features described herein, the processor selects the two tracks of the candidate pair based on a difference between a first range obtained from the one track of the two tracks of the candidate pair and a second range obtained from to the other track of the two tracks of the candidate pair being within a threshold range value.
In addition to one or more of the features described herein, the processor iteratively selects different candidate pairs from the track database, applies the criteria, and classifies the different candidate pairs.
In addition to one or more of the features described herein, the processor classifies one or both of the two tracks of the candidate pair as the track of the true object or leave prior classification of the two tracks of the candidate pair based on the criteria not determining that the one track of the two tracks of the candidate pair is the track of the ghost object and the other track of the two tracks of the candidate pair is the track of the true object corresponding with the ghost object.
In addition to one or more of the features described herein, the processor applies the criteria by applying a first criterion that includes estimating a shape of a reflective surface giving rise to the multipath reflection that would result in the one track of the two tracks of the candidate pair being the track of the ghost object.
In addition to one or more of the features described herein, the processor applies the first criterion to determine that the one track of the two tracks of the candidate pair is the track of the ghost object based on a linearity of the reflective surface.
In addition to one or more of the features described herein, the processor applies the criteria by applying a second criterion that includes determining if dynamics of the one track of the two tracks of the candidate pair matches dynamics of the other track of the two tracks of the candidate pair such that velocity and heading indicated by the two tracks of the candidate pair are correlated within a threshold amount over multiple updates of the two tracks.
In addition to one or more of the features described herein, the processor applies the second criterion to determine a first velocity indicated by the other track of the two tracks of the candidate pair using the one track of the two tracks of the candidate pair, determines a second velocity of the other track of the two tracks of the candidate pair using the updates of the other track of the two tracks of the candidate pair, and classifies the one track of the two tracks of the candidate pair as the track of the ghost object and the other track of the two tracks of the candidate pair as the track of the true object corresponding with the ghost object based on the first velocity and the second velocity being within a specified threshold of each other.
In addition to one or more of the features described herein, the processor applies the criteria to the candidate pair to determine an incident angle of transmit energy from the radar system on a reflective surface that gives rise to the multipath reflection that would result in the one track of the two tracks of the candidate pair being the track of the ghost object and to determine whether the incident angle is below a threshold value, the threshold value being range-dependent.
In addition to one or more of the features described herein, the processor estimates a velocity of the true object based on the ghost object based on the classifying indicating that the one track of the two tracks of the candidate pair is the track of the ghost object and the other track of the two tracks of the candidate pair is the track of the true object corresponding with the ghost object.
The above features and advantages, and other features and advantages of the disclosure are readily apparent from the following detailed description when taken in connection with the accompanying drawings.
Other features, advantages and details appear, by way of example only, in the following detailed description, the detailed description referring to the drawings in which:
The following description is merely exemplary in nature and is not intended to limit the present disclosure, its application or uses. It should be understood that throughout the drawings, corresponding reference numerals indicate like or corresponding parts and features.
As previously noted, deflection (rather than reflection) of transmitted energy can result in a multipath reflection. That is, for example, the energy may traverse multiple paths to travel from the radar system to a first object that deflects the transmitted energy, to a second object that reflects the deflected energy back to the radar system via the first object. By contrast, a direct reflection results from transmitted energy being reflected back to the radar system by the first object that is encountered. In the case of multipath reflection, the position of the second object will appear to be in a different place than the actual position of the second object (i.e., as a ghost object). Essentially, the ghost object will appear as a mirror image of the second object with the first object acting as the mirror surface.
In vehicle applications, roads, guardrails, buildings, and other surfaces may increase the likelihood of multipath reflections. At the same time, ghost objects may create challenges in vehicle radar systems because information from a radar system may be used to control autonomous or semi-autonomous operation of the vehicle. The detection of a ghost object may result in unnecessary automatic braking or other unwarranted operation of the vehicle. In prior radar systems, an approach to dealing with multipath reflections involved identifying ghost objects. However, this approach considered each detected object individually. Embodiments of the systems and methods detailed herein relate to multipath ghost mitigation in a vehicle radar system based on correlating ghost objects with real objects and tracking both. That is, ghost objects are identified based on considering a pair of detected objects at a time. Correlation metrics are used to identify ghost objects and the true objects to which they correspond. Ghost objects are then tracked along with their true counterparts. Tracking ghost objects avoids re-identifying the same ghost object multiple times. According to embodiments herein, because a ghost object and true object pair are correlated, the velocity of a true object may be more accurately estimated by using the Doppler of its corresponding ghost object.
In accordance with an exemplary embodiment,
Processing of the reflections (direct reflections 205 and multipath reflections 215) received by a radar system 110 (
Tracks of ghost objects 220 are not reported for use in controlling vehicle systems in the way that tracks of true objects 210 are. However, as previously noted, the tracking of ghost objects 220 may result in efficiencies and advantages. The classification of the tracks of ghost objects 220 is detailed with reference to
At block 320, the method 300 includes determining candidate pairs among the unclassified tracks in the track database 305. This process involves sorting the unclassified tracks in the track database 305 by range. As previously noted, for a true object 210 and its corresponding ghost object 220, the range to the ghost object 220 will be longer than the range to the true object 210. In addition, the ranges to the ghost object 220 and to the true object 210 will not be too different (i.e., difference will be less than a threshold value). Thus, based on the range (or set of ranges in the case of a track of a moving object) stored for a given unclassified track in the track database 305, a track of a candidate ghost object 220 or a track of a candidate correlated true object 210 may be identified to form the candidate pair. The track of the candidate ghost object 220 will have a stored range (or ranges) that is longer than the stored range (or ranges) for the given unclassified track. The track of the candidate correlated true object 210 will have a stored range (or ranges) that is shorter than the stored range (or ranges) for the given unclassified track. In addition, the track of the candidate correlated true object 210 may already be classified as a track of a true object 210 (in a previous iteration). In that case, the processes at blocks 330 and 340 will determine if the true object 210 gives rise to the ghost object 220 corresponding with the given unclassified track. Based on the process at block 320 that involves the ranges at which unclassified tracks were detected, every candidate pair that is currently in the track database 305 may be identified. When additional objects are detected and processed to be added to the track database 305 as new tracks (at block 310), the method 300 may be repeated to address new unclassified tracks in the track database 305.
At block 330, selecting a candidate pair refers to choosing one of the candidate pairs identified at block 320. The selected candidate pair includes two tracks, at least one of which is an unclassified track. As previously noted, a candidate pair may include a track that is already classified as the track of a true object 210. In this case, the classification of the candidate pair according to the processes determines if the true object 210 whose track is part of the candidate pair gives rise to the ghost object 220 whose track is the unclassified track in the candidate pair. At block 340, applying criteria involves three processes. The first of the three processes at block 340 involves determining the shape of the reflective surface 225, at block 350. This is because a reflective surface 225 that gives rise to multipath reflections 215 is typically linear. The second criteria applied at block 340 includes determining whether dynamics (e.g., location, velocity) of the tracks of the candidate pair correlate with each other, at block 360. This is because a true object 210 and ghost object 220 will typically have matching dynamics. The third criteria applied at block 340 includes determining the incident angle of the transmitted energy 201 on the reflective surface 225 that would have given rise to the candidate ghost object 220, at block 370. This is because multipath reflections 215 typically occur based on low incident angles. Each of these criteria used at block 340 is further discussed.
At block 380, classifying according to the criteria refers to different potential outcomes. Based on the criteria applied at block 340, the candidate pair, selected at block 330, may be classified as tracks of a ghost object 220 and the true object 210 that gives rise to that ghost object 220 (i.e., a true object 210 and ghost object 220 pair). However, if, based on the criteria applied at block 340, the candidate pair is deemed not to be tracks of a true object 210 and ghost object 220 pair, then the tracks of the candidate pair may either retain their prior status or one or both tracks may be classified as a track of a true object 210, as detailed. A track that is part of the candidate pair but is either not part of any other candidate pair or is not the track of a candidate ghost object 220 in another candidate pair (determined at block 320) may be classified as the track of a true object 210. If the candidate pair included a track of a candidate ghost object 220 and a track of a candidate correlated true object 210, with the track of the candidate correlated true object 210 already being classified as the track of a true object 210, then, even if the criteria of a true object 210 and ghost object 220 pair is not met (at block 380), then the true object 210 classification is retained.
According to an exemplary embodiment, all three of the criteria that are checked at block 340 must indicate that the candidate pair is a true object 210 and ghost object 220 pair in order for the candidate pair to be so declared. This is because, on balance, it may be deemed better to not miss-classify a true object 210 as a ghost object 220. According to alternate embodiments, minimizing false alarms and braking or other actions may be deemed more important. In that case, a candidate pair may be declared as a true object 210 and a ghost object 220 based on only one or two of the three criteria at block 340. As indicated in
At block 510, the process includes obtaining the detected position of the candidate true object 210 (xt, yt) and the detected position of the candidate ghost object 220 (xg, yg) from the track database 305. At block 520, calculating the reflection line is based on the line equation:
y=tan(α)x+b [EQ. 1]
In EQ. 1, the line slope, tan(α), is based on the reflection surface angle α, which is the angle between the reflective surface 225 and the direction of travel X of the vehicle 100, as shown in
In EQ. 2, m is based on the detected position of the candidate true object 210 (xt, yt) and the detected position of the candidate ghost object 220 (xg, yg) and is given by:
The intercept, b, in the line equation shown in EQ. 1 is given by:
Calculating the ghost line 420, at block 530, includes using the line equation:
y=tan(β)x [EQ. 7]
The slope, tan(β), may be derived from the position of the candidate ghost object 220:
Once the reflection line is calculated (at block 520) based on EQ. 1 and the ghost line 420 is calculated (at block 530) based on EQ. 7, the processes include obtaining the reflection point 410, at block 540. An estimate of the reflection point 410 is at position (xr, yr), and xr and yr are obtained as:
The processes at blocks 510 through 540 are repeated, as indicated in ,
). At block 550, performing criteria testing refers to three different criteria that may be tested.
At block 550, one criteria includes determining whether the mean square distance (MSD) is within a threshold value. The MSD is given by:
E((−xr)2+(
−yr)2) [EQ. 11]
In EQ. 11, E indicates a mean operation. Another criteria involves determining whether the percentage of estimations that exceed a threshold are within a specified value (i.e., a specified percentile of error). Yet another criteria examines the error between the vector of reflection surface angles α (results of EQ. 2 for multiple track updates) and the surface angle estimate {circumflex over (r)} calculated by the linear fit. The error is given by:
E(α−{circumflex over (r)}) [EQ. 12]
Based on the criteria, a determination is made (at block 350,
The process at block 360 (
υg=Rυt [EQ. 13]
In EQ. 13, R is the rotation matrix and is given by:
The angle γ is the angle difference between the vector of true object velocities υt and the reflective surface 225 and is given by:
In EQ. 15, Sr is a normalized vector corresponding with the reflective surface 225 and is given by:
Sr=(cos α, sin α) [EQ. 16]
The true object Doppler Dt and the ghost object Doppler Dg are projections of the true object velocity υt and the ghost object velocity υg, respectively, towards the vehicle 100. They are given by:
The Doppler values are measured (i.e., obtained from the direct reflection 205 and multipath reflection 215 as previously noted). The velocity of the candidate true object 210 (υt) is estimated as:
υt=(V cos ϕ,V sin ϕ) [EQ. 19]
In EQ. 19, V is the velocity of the candidate true object 210 and is given by:
The angle μ is the angle between the heading of the true object 210 and the vehicle 100, as shown in
The angle η is the angle between the direct reflection 205 and the multipath reflection 215, as shown in
η=2θ−ρ [EQ. 22]
The incident angle θ is the angle between the reflection surface 225 and the ghost line 420, as shown in
In EQ. 19, ϕ is the heading of the candidate true object 210 and is given by:
By obtaining the velocity V of the candidate true object 210, according to EQ. 20, and the heading ϕ of the candidate true object 210, according to EQ. 23, for every track update of the candidate true object 210, a velocity vector υt may be obtained. This estimated velocity vector υt may be compared with the velocity vector that is obtained from processing the direct reflections 205 and stored in the track database 305. Because the estimated velocity vector υt is derived using the candidate ghost object 220 while the velocity vector
is obtained from processing the direct reflections 205, a close match between the two suggests that the candidate ghost object 220 is a real ghost object 220.
Specifically, two criteria may be checked. One criteria is the MSE between the estimated velocity vector υt and the velocity vector , which is given by:
E((−υt
−υt
In EQ. 24, υtx and υtυ are the x and y components of the estimated velocity vector υt. A determination may be made about whether the MSE is below a threshold value. A second criteria that may be checked is whether the percentage of estimation differences between the estimated velocity vector υt and the velocity vector (i.e., the percentile error) is within a specified threshold.
The process at block 360 (
|l()−l(
)| [EQ. 25]
Generally, length l is defined as:
l(x)=Σi=oN−2∥xi+1−xi∥ [EQ. 26]
In addition, the candidate pair should have the same velocity magnitude. Thus, an MSE may be defined as:
E((||−|
|)2) [EQ. 27]
Finally, as another check of object dynamics at block 360 ( (determined and stored in the track database 305) and for the candidate ghost object velocity vector
(determined and stored in the track database 305). Accordingly, an MSE may be defined as:
Thus, a check may be done of whether the MSE is within a threshold value.
The process at block 370 (
While the above disclosure has been described with reference to exemplary embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from its scope. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the disclosure without departing from the essential scope thereof. Therefore, it is intended that the present disclosure not be limited to the particular embodiments disclosed, but will include all embodiments falling within the scope thereof.
Number | Name | Date | Kind |
---|---|---|---|
20180120842 | Smith | May 2018 | A1 |
Entry |
---|
G. Gennarelli and F. Soldovieri, “Multipath Ghosts in Radar Imaging: Physical Insight and Mitigation Strategies,” in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 8, No. 3, pp. 1078-1086, Mar. 2015, doi: 10.1109/JSTARS.2014.2363233. (Year: 2015). |
O. Longman, S. Villeval and I. Bilik, “Multipath Ghost Targets Mitigation in Automotive Environments,” 2021 IEEE Radar Conference (RadarConf21), 2021, pp. 1-5, doi: 10.1109/RadarConf2147009.2021.9455253. (Year: 2021). |
Number | Date | Country | |
---|---|---|---|
20210104027 A1 | Apr 2021 | US |