The present application is a National Stage of International Patent Application No. PCT/IB2019/055145, filed on Jun. 19, 2019, which claims priority to and all the benefits of Italian Patent Application No. 102018000006594, filed on Jun. 22, 2018, both of which are hereby expressly incorporated herein by reference in their entirety.
The description relates to map building techniques and vehicle environment mapping.
One or more embodiments may be related to processing at least one fused grid map to determine objects occupancy, for instance to facilitate detecting the position of obstacles and how to avoid contact with such obstacles.
Multi-sensor data fusion may facilitate combining information from different sources in order to form a unified picture, for instance by providing fused occupancy grid maps representing an environment based on data from multiple sources.
Different mapping techniques may provide different types of maps, for instance:
These two approaches are by no means mutually exclusive, but they in fact supplement each other.
Occupancy grid maps are well suited for path planning and obstacle avoidance tasks. Conversely, feature-based maps may be well suited for localization purposes, where relative pose of the objects may be required for accurate estimation of vehicle self-position and orientation.
Occupancy grid maps, briefly grid maps, represent the (vehicle surrounding) environment as a two-dimensional grid map with fixed dimensions and spatial resolution. The accuracy of a grid map may be defined in terms of the resolution of the discretized binary mesh. Moreover, they may be oriented in the direction of the longitudinal and transverse axes of the vehicle, and may have edge segments disposed at right angles to one another.
A grid map may be provided as a representation of the environment of a vehicle as divided into grid cells, each cell storing a feature describing the environment. For instance, each cell may store a probability/likelihood value, e.g. the probability that a cell is occupied or not. In fact, the map building can be formulated as an optimization problem that maximizes the probability that particular grid cell is in fact occupied (or free) given the prior knowledge of the environment, vehicle position and sensor measurements.
A grid map may facilitate providing to the drive assistance system information about the presence of obstacles and their space occupancy in the surrounding mapped environment. Consequently, the driver assistance system may operate vehicle functions on the basis of that information. For instance, the drive assistance system may react to traffic conditions.
The grid map may represent the environment in a variety of coordinate systems, for instance in Cartesian or polar coordinates, as known to those skilled in the art.
An environment may be scanned via a plurality of sensors/detectors, for instance Global Positioning System (GPS) antenna and/or Geographic Information System (GIS) antenna, LIDAR, camera, etc.
Multiple grid maps may be generated as a function of individual sensors data, providing correlated information about the environment.
Fusing the multiple occupancy grid maps into a unified coherent map combining (e.g. by data fusion) the information of all sensors with high precision and accuracy is an object of the present description, especially with respect to, e.g., drive assistance systems and methods for vehicles or automated vehicles.
Occupancy grid map may facilitate mapping the environment surrounding a vehicle looking for possible obstacles, in order to determine drivable spaces in such a way that an autonomous system capable of estimating its own position may be able to safely navigate through the environment mapped.
As opposed to single sensor mapping, grid maps built as a result of multi-sensor fusion rely heavily on sensor measurement weighting. Traditional approaches to mapping, i.e. the Bayes Occupancy Filter (BOF) or the Dempster-Shafer theory (DST), apply equal weights unless specified by the user. Empirically adjusted sensor weights are tailored for certain scenarios, specifically in situations where the sensors readings are not contradictory.
An object of one or more embodiments is to overcome the limitations inherent in the solutions achievable from the prior art.
According to one or more embodiments, that object is achieved thanks to a method for vehicle (V) environment mapping, including the steps of receiving a set of input values (Y1, Y2, Yi) from a plurality of sensors (S1, S2, Si), applying temporal fusion processing to the set of input values (Y1, Y2, Yi), resulting in a respective set of occupancy grid maps (G1, G2, Gi), applying data fusion processing to the set of occupancy grid maps (G1, G2, Gi), resulting in a set of fused occupancy grid maps (FF), and processing the set of fused occupancy grid map (FF). The processing operation includes performing an arbitration of conflict in the set of fused occupancy grid maps (FF) and outputting a set of arbitrated fused occupancy grid maps (MM), in particular an arbitrated fused occupancy grid map (M) of drivable spaces. One or more embodiments may refer to a corresponding system, vehicle and computer program product.
The claims form an integral part of the technical teaching provided herein in relation to the various embodiments.
According to the solution described herein, the method comprises providing a fused grid map representation of a vehicle environment, e.g. in the form of an occupancy grid map, as a function of data from multiple sources.
In one or more embodiments, adaptive sensor weights may be applied to multi-sensor fused maps, e.g. iteratively.
Other objects, features and advantages of the present invention will be readily appreciated as the same becomes better understood after reading the subsequent description taken in connection with the accompanying drawings.
The embodiments will now be described purely by way of a non-limiting example with reference to the annexed drawings, in which:
In the ensuing description, one or more specific details are illustrated, aimed at providing an in-depth understanding of examples of embodiments of this description. The embodiments may be obtained without one or more of the specific details, or with other methods, components, materials, etc. In other cases, known structures, materials, or operations are not illustrated or described in detail so that certain aspects of embodiments will not be obscured.
Reference to “an embodiment” or “one embodiment” in the framework of the present description is intended to indicate that a particular configuration, structure, or characteristic described in relation to the embodiment is comprised in at least one embodiment. Hence, phrases such as “in an embodiment” or “in one embodiment” that may be present in one or more points of the present description do not necessarily refer to one and the same embodiment.
Moreover, particular conformations, structures, or characteristics may be combined in any adequate way in one or more embodiments.
The references used herein are provided merely for convenience and hence do not define the extent of protection or the scope of the embodiments.
In
For instance, said vehicle V may be an autonomous land vehicle such as a self-driving car. Other examples of vehicles may be car-like vehicles, self-moving robots, etc.
The plurality of sensors/detector devices S1, S2, Si includes sensors, known per se, such as LIDAR/radar sensor, camera, GPS antenna and sources using the GPS information such as (GPS/GIS) context map, etc., and it is coupled to the system 100. Specifically, in the example shown the sensors are a LIDAR, a camera and a GPS/GIS system producing a (GPS/GIS) context map on the basis of the detected position.
The set of sensors S1, S2, Si may in general comprise sensors having:
The system 100 in general makes available the arbitrated map M either to a user, e.g. as aid for assisted driving, or to a further system which uses the decided map M to control the vehicle V, such as a system for automated driving. In
The system 100 is configured to process the measurements data (“readings”) Y1, Y2, Yi from the set of sensors S1, S2, Si, providing a set FF of fused occupancy-grid map F, E (out of data fusion block 40) toward the output interface IA.
As mentioned, the arbitrated fused occupancy-grid map of drivable spaces M available at the interface IA may be used by a control system of the vehicle to operate the vehicle itself, e.g. navigating the environment represented in the fused occupancy grid map M provided as output by the system 100.
As exemplified in
In particular, the plurality of sensors S1, S2, Si is brought as input to a set of temporal fusion stages 30. The set of temporal fusion stages 30 receives the set of sensor readings Y1, Y2, Yi.
The set of temporal fusion stages 30 may comprise, as shown in
For instance, the set of temporal fusion stages 30 may be distributed across respective sensors, providing local decentralized processing of sensor readings, in a manner known per se (e.g. via software).
In the exemplary embodiment shown in
Thus, each temporal fusion stage 301, 302, 30i in the set of temporal fusion stages 30 provides a set of occupancy grid maps G1, G2, Gi expressed in vehicle frame, as a function of respective sensor readings Y1, Y2, Yi from the set of sensors S1, S2, Si.
Alternatively, the i-th sensor/source may also provide as reading directly a grid map Gi, such as it may be the case when acquisition of a GIS/GPS map from a remote sensing source is involved.
As mentioned, in variant embodiments, the operation of processing sensor readings Y1, Y2, Yi in the set of temporal fusion stages 30 may comprise:
Preferably, at least one i-th map Gi in the set of maps G1, G2, Gi provided by the set of temporal fusion stages 30 provides information about specific features of the environment, as a function of the type of measurement provided by the i-th sensor Si.
An occupancy grid map typically comprises a number N=n*m of individual cells gijk, where n is the number of lines and m is the number of columns, indexed by indices i and j, respectively.
Specifically, any individual cell gijk in the grid map G0 preferably has a fixed size and predefined geometry, e.g. square, as in a Cartesian coordinate system.
An occupancy grid map G0=G0,k may evolve in time, thus being a function of the time k.
Each cell gij of the grid G0 may be a variable, e.g. a random variable, which can assume a certain number of “state” values with a certain probability (at a time k).
Each cell gijk of the grid in the embodiment shown comprises a random variable numeric value and a “state” value (classification) associated, for instance:
Typically, if a probability value stored in a cell is:
The grid map is usually described in terms of a two-dimensional grid, e.g. Gk={gijk: i≤i≤n, 1≤j≤m}.
When visualizing the grid map on a display or in any graphic form, the information regarding the classification label of each grid cell may be encoded as a pattern/color filling the grid cell area, as shown e.g. in
The probability distribution p(gijk) associated to random variables in each grid cell gijk evolves in time as fresh sensor readings are acquired from sensors and it is updated accordingly taking into account the sensor measurements Yi and a vehicle pose V, at every time instant k.
Temporal fusion processing 30i refers to aggregating the information received in sensor readings, e.g. readings Yi,k taken from the i-th source Si at time k are aggregated with readings Yi,k−1 from the i-th source Si at a previous time k−1.
Thus, each temporal fusion stage 301, 302, 30i provides a set of occupancy grid maps G1, G2, Gi, expressed in vehicle frame, as a function of respective sensor readings Y1, Y2, Yi from the set of sensors S1, S2, Si.
The temporal fusion processing stage 30i may comprise operations of coordinate system transformations 301, 302, 30i, from a sensor frame to a vehicle frame, as:
As exemplified in
Similarly, the vehicle state Vk (e.g. its position) at a given time k may be known (e.g. stored in stage 312) and it may be described as an m-dimensional vector, e.g. Vk=(v1k, v2k, . . . , vmk). For instance, the pose of the vehicle Vk may be parametrized by centroid position and orientation angle of the vehicle V with respect to the ground plane.
The vehicle pose/state Vk and sensor readings Yk at a certain time k are combined, for instance to provide an input in the form of likelihood value, e.g. p(Yk|Vk), to be fed into the temporal fusion processing stage 30i.
A Bayesian Occupancy Filter (BOF) is adequate for use as described herein. The BOF is for instance a valid formulation to compute a posterior probability p(Gk|Yk,Vk) of the occupancy grid map Gk at the time k for use in calculating the updated probability values of grid cells of the occupancy grid map Gi, taking into account the fresh set of sensor readings.
To perform the temporal fusion processing 30i, an initial occupancy grid map M0 is provided, e.g. stored/generated in stage 324. The initial grid M0 for instance is initialized:
Also, a “previous” grid map Gk−1 is generated/stored in stage 307 of the temporal fusion stage 30i, with grid cells having probability values computed in a time k−1 preceding the time k of acquisition of the sensor readings.
It is noted that at the start-up of the system, the initial grid M0 and the “previous” grid map Mk−1 may have equal values, e.g. Mk−1=M0.
The temporal fusion stage 30i fuses a priori map knowledge M0 and/or the “previous” grid map Mk−1, e.g. in stage 322, with fresh data in the form of measurement likelihoods p(yk|xk).
The Bayes theorem states that the probability p(gijk|yk,xk) that a cell gijk of the grid map Mk at the time k is occupied (or free) is proportional:
The Bayesian gain is computed in a gain stage 520, shown in
It is noted that in the hypothesis of a priori unknown boundaries of the map, the Bayesian gain is computed solely on the bases of the measurements.
A forgetting factor 327 is also applied, in the exemplary embodiment shown in
The temporal fusion processing 30 may thus be formulated as a recursive solution to the problem of at maximizing the probability p(gkij) that a particular grid cell gkij is in fact occupied (or free), given the prior knowledge of the environment G0, vehicle position Vk and sensor readings Yk.
It is noted that grid map G, e.g. G=Gi, at time k (indicated as Gk) is the result of processing sensor readings at time k, e.g. Yi,k, and the prior map G at time k−1 (indicated as Gk−1). Also, the prior map Gk−1 contains not only information relative to time k−1 itself, but it contains information regarding the entire evolution of the map from the time origin to the time k−1 (e.g., from time 1 up to time k−1).
Moreover, when indicating a sensor occupancy-grid map Gi reference will be made to a single “time frame” of the evolution of the map G in time k, without limitation to the embodiments, according to first order Markov assumption. Said otherwise, a full map history, from time 1 to k−1, is represented/captured within a single map G at time k−1.
The respective “state” labels classifying the content of grid cells of respective sensor occupancy grid maps may vary as a function of the sensor readings parameters/working principles of each sensor.
In fact, respective sensor occupancy grid maps G1, G2, Gk may each provide information about specific features of the environment.
For instance, in the example shown:
Specifically, after respective sensor occupancy grid maps G1, G2, Gi are generated in the temporal fusion stage 30 via temporal fusion processing 30i of sensor readings Y1, Y2, Yi, for every time step k, the grid maps in the set of sensor occupancy grid maps G1, G2, Gi are “fused” together in the set FF of fused maps which includes at least one “fused” occupancy grid map F, E.
In order to do so, the grid fusion stage 40 receives the set of maps G1, G2, Gi and provide as an output at least one “fused” occupancy grid map F, E by performing fusion operations on data in (e.g., timed frames G1,k, G2,k, Gi,k) of the set of maps G1, G2, Gi.
Different approaches may be used for data fusion 40, which may include at least one of combining, associating, estimating, correlating operation, in data in (timed frames) of the set of maps G1, G2, Gi.
These two approaches are summarized in the two embodiments represented in
For instance, a data fusion based on Bayesian inference 50, as shown in
The fused grid map may thus facilitate accurately predicting and/or detecting the occupancy of an obstacle along the path of travel of the vehicle V, facilitating the online correction for the path of travel of the vehicle V by a drive assistance system/interface.
For instance, it may facilitate avoidance of a collision between the vehicle and the obstacle when the vehicle is operated by an autonomous driving system (not visible in the figures).
As anticipated above, for instance, grid fusion processing 40 may comprise at least one of:
In fact, at least one of Bayesian Occupancy Filter (BOF) (see, e.g.
As exemplified in
In a general form, the result of BOF (grid fusion processing) 50 can be expressed in terms of a weighted (linear) combination of sensor occupancy grid maps resulting from temporal fusion, for instance as:
Unless specified a priori by the user and/or by design, sensors are assumed to bring equal contribution to the BOF grid fusion process 50, i.e. maps are combined with equal weights, e.g. wk,i=wk.
In the classical Bayesian setting, which assumes a spatio-temporal relationship between grid cells gkij, the truthfulness (probability) of the map p (Fk|Gk) as a combination of multiple sensor grids Gk,i is evaluated via the Bayes theorem, i.e.
in which the estimate about the map occupancy is compared to each individual sensor grid Gk in the form of a likelihood p(Gk|Fk).
However, grid-based approaches do not have the ability to predict the behavior of the objects, as this will require the implementation of feature recognition and data association techniques, and thus solely rely on measurement weighting. Similarly, as in temporal fusion, without the model based prediction the expected map change from time k−1 to k is p(Fk|Fk−1)=0. Therefore, the Bayesian gain ƒ{p(not(Fk)|Gk)−p(Fk|Gk)} may not be able to determine the sensor weights wk. Empirically adjusted sensor weights may be tailored only for certain scenarios and thus may not be suited for general purpose mapping. It therefore becomes apparent, that sensor weights need to be adjusted recursively during the grid fusion process.
Therefore, for instance, an error δwk,i between a true weight wk,i and an actual value ŵk,i of the i-th sensor weight may be expressed as:
δwk,i=wk,i−ŵk,i
A feedback network providing iteratively adjustable weights, e.g. in dataset W provided by the arbitration stage 90 as discussed in the following, may facilitate automatic sensor weights wk adjustments during BOF fusion.
Alternatively, a set of combination rules known as the Dempster-Shafer theory of evidence (DST) may be employed, for instance in block 60 of
In fact, according to Bayes' theorem, the occupancy of the cell is evaluated based on single probability per cell, while in DST the decision about occupancy is made based on belief function of four “masses”. Furthermore, the belief transition in the Bayes' theorem is possible only between two states and is symmetrically restricted.
DST processing 60, as exemplified in
Similar to what was discussed in relation to
The “belief” (i.e. evidence that supports the state proposition, the degree to which the classification is believed to be true) about the environment occupancy is classified by either occupied (O) or free (FR), which results into a set of all possible states of a random variable X={O, FR}. The set of the possible subsets of X, i.e. the “frame of discernment” (briefly, FOD), is denoted as 2X={occupied (O), free(FR), unknown (Ω), conflict (Ø)}. The main objective of DST is to compute the belief Bel(X), a resulting probability that accounts for all the evidence that supports any proposition X in the frame of discernment 2X. Thus, the belief function Bel(X) represents the degree to which X is believed to be true, and may be expressed as:
where:
The content of cells in an evidential grid map E (indicated also as “likelihood”) is an indicator of the degree of reliability of detection of the presence of objects/obstacles in the environmental space surrounding the vehicle V.
The process of DST fusion 60 usually starts with a basic mass assignment (BMA) stage 610, comprising a process based on hard constrains that redistributes the four masses across the occupancy grid map cells p(gk,i). The result of BMA is subsequently provided into the Dempster-Shafer combination rule stage 690, as discussed in the following.
For instance, two sources of information, e.g. sensor occupancy grid maps G1 and G2, are combined into a fused belief function having a “joint mass” m through the XOR operator ⊕, whose value is usually calculated as:
The order in which the sensors are fused is irrelevant given the commutative property.
In the evidential approach, the belief can be transferred among four states, in which each of these transitions has a different meaning, dynamic and importance. While the DST models a higher level of confidence in cells than just a single probability value, sensor grid weighting wk remains unsolved at this stage.
As shown in the embodiment of
The discrepancy assessment stage 70 receives the set of maps G1, G2, Gi and provides a set of detected discrepancies D comprising information about the result of an operation of comparing the maps, e.g. one-to-one. The comparation operation can be performed by any different technique suitable to identify at least the equal and different, i.e. conflicting, cells between maps.
For instance, the set of detected discrepancies D may comprise:
For instance, the abovementioned information may be combined in a matrix form.
As illustrated in
As different sensors provide different information, possibly conflicting between each other, a vast majority of grid cells of the fused map F, E may have associated values corresponding to highly conflicting or uncertain information about the space occupancy, i.e. p(gkij≈0.5).
Consequently, the discrepancy process stage 70 may provide a set of detected discrepancies D containing a structured information about the presence of conflicts between groups of occupancy grid maps, in particular for specifying where the conflict is and to which group, e.g. pair, of compared grid maps the conflict is related.
Specifically, the set of detected discrepancies D comprises, for instance:
For example, such information may be stored/encoded in the set of detected discrepancies D in the form of an array having discrepancy components Δ12, Δ2i, Δi1, each storing the result of each comparison of groups of sensor grid cells, e.g. containing a “conflict” value for every cell in respective grid maps cells.
The “conflict” value may be calculated by subtracting probability values in cells of a map from the probability values in cells of a different map.
An effect of the presence of the discrepancy process stage 70 is to facilitate automatic calibration of merging and manipulating the temporally fused grid maps G1, G2, Gi generated from readings Y1, Y2, Yi provided by sensors S1, S2, Si. In fact, knowledge of “conflict” values Δ12, Δ21, Δi1 in the set of discrepancies D can be used to “tip” the probability of “uncertain” cells of the fused map F, E to either an “occupied” or a “free” state via conflict arbitration stage processing 90, providing a “harmonized” or arbitrated drivable fused map M with a reduced number of conflicting cells.
As exemplified in
In the case of the embodiment of
The arbitration stage 90, 90′ as exemplified in
The conflict arbitration stage 90 may use different approaches for resolving the possible conflicting, e.g. contradictory, information provided by the sensors on the occupancy state of the cells of the fused grid F and/or evidential grids E, as a function of the approach used in the data fusion processing stage 40, e.g. BOF fusion 50 and/or DST fusion 60.
Specifically, in a variant embodiment as exemplified in
In the embodiment considered, the conflict arbitration stage 90 includes a further a situation assessment stage 80, based on probabilistic models or alternatively based on encoded rules and logic formulas to resolve conflicting cell values in a fused map F by processing the set of detected discrepancies D. Therefore, the conflict arbitration stage 90 receives as input the fused map F from the BOF fusion processing stage 50 and the set of detected discrepancies D from the discrepancy detection stage 70.
In the embodiment as exemplified in
The conflict detection stage 83 in the occupancy grid processing stage 85 receives the fused map F from the BOF processing stage 50 and provides at output said map F and/or a set CA of conflicting cells and/or a trigger signal.
Specifically, the conflict detection stage 83 performs operations of flagging a set of conflicting cells CA within grid cells gkij in the input fused map F, which may comprise, for instance:
The conflict detection stage 83 may thus provide as output the set of flagged cells CA, signaling that a conflict has occurred. It is noted that within a fused grid F a plurality of flagged cells CA might be flagged at a time k. Thus, a same flag may be enabled per group of cells CA within the fused map F. Alternatively, multiple flag values may be associated to multiple grid cells at a same time k, e.g. to distinguish a higher “conflict level” from a lower one.
The trigger signal may be generated periodically, i.e. at given time instants, or as a function of probability values in the set of conflicting cells CA. The trigger signal and it may be used internally or provided to user processing stages 87, 84.
For example, the conflict detection stage 83 may receive multiple time frames of the fused map F from the BOF processing stage 50 in the data fusion processing 40 before applying flagging. The conflict detection stage 83 may wait for a certain time k, e.g. a certain number of fusion processing cycles, and/or wait for cells probability values to be stable before flagging said set of conflicting cells CA.
Alternatively, the presence itself of a non-null set of conflicting cells CA may be used as trigger signal.
In the embodiment considered, the conflict detection stage 83 provides said set of flagged cells CA and/or the fused map F and/or a trigger signal to the conflict assessment stage 84 in the situation assessment stage 80 and to the conflict resolution stage 87 in the occupancy grid map processing stage 85.
In the exemplary embodiment of
The discrepancy-based relational models DM provided by stage 82 represent a set of rules to apply to the conflicting grid cells in the set of conflicting cells CA and/or the fused grid map F, to perform at least one of:
The conflict assessment stage 84 is thus configured to use, e.g. combining, the set of detected discrepancies D and the discrepancy-based relational models DM, thus providing ways to solve the conflict which maintains “uncertain/unknown” the state of the grid cell.
More in detail, in order to obtain information to “tip” the occupancy of ambiguous cells towards either a first state, e.g. “occupied” p(gkij»0.5), or a second state, e.g. “free” p(gkij«0.5) state, the conflict assessment stage 84 may adopt a discrepancy resolution logic as discussed in the following.
A first exemplary resolution logic is developed via observation of the existence of discrepancy-based relations between different sensor measurements, similarly to the foregoing discussed with reference to
In portions b), c) and d) of
Therefore, as exemplified in
Obstacles detection is a concern, in particular, with respect to accurate determination of the presence of temporary jerseys which might occur on the road and/or lanes which determine road boundaries. These objects are likely sources of uncertainties as their presence/existence may generate/be in conflict within at least one pair of sensor reading.
The presence of said objects may be evaluated on the basis of detected discrepancies D by using the following logic/rules, for each cell:
The “conflict” value, thus, is a parameter of discrepancy evaluation models DM provided by the discrepancy modeling stage 82. The “conflict” may be parametrized in models DM provided by stage 82. For example, “conflict”=0.5 may imply that grid cell gijk is labeled as free using data from a first sensor and occupied according to a grid map from second sensor reading, as discussed in the foregoing.
The conflict assessment stage 83 provides information about conflicting cells and types of conflicts, e.g. in a dataset CR, as an output of the situation assessment stage 80. Consequently, as exemplified in
The conflict resolution stage 87 processes inputs, for instance evaluating a probability of conflict as a spatial and temporal confidence in the sensor measurements. Thus, adaptive weight values W may be computed for further processing. For instance, weights values may comprise adaptive weights W, which may be passed “back” to previous stages, e.g. via a feedback network, to the data fusion processing stage 40, in particular in stage 520 of the BOF fusion stage 50, to apply to sensor grid values in the data fusion processing.
The conflict resolution stage 87 processes said inputs, providing a set of values CC.
In one or more embodiments, the set of values CC is indicative of a correction δSF to apply to the fused map F in case of feed-forward corrections, or of a weight correction δW in case the correction are applied in feed-back variant.
This would thus “tip”, i.e. push or reassign, the probability of said conflicting cells to a defined state, e.g. either occupied or empty state. For instance, said resolved grid cells values may be computed as a function of the measure of the “conflict” values in the set of detected discrepancies D, e.g. by applying rules in the dataset CR provided by the situation assessment stage 80.
The output of the conflict resolution stage 87 is communicated to the conflict correction stage 89.
Consequently, the conflict correction stage 89 receives:
The conflict correction stage 89 provides as output the drivable fused map M, which may be provided to a driver assistance system/interface IA as discussed in the foregoing. Said fused grid map M, for instance, is obtained in the conflict correction stage 89 by merging and/or replacing values of conflicting grid cells in the fused map F with the corresponding values CC, e.g. resolved grid cells values.
Processing in a conflict correction stage 89 may be expressed as: M=F−δF, the difference between the fused map F and the correction δF from stage 87, or W={tilde over (W)}−δW, the difference between a previously computed weight values set {tilde over (W)} and the correction δW from stage 87, respectively in feed-forward or feed-back variant.
The fused grid map M thus facilitate accurately predicting and/or detecting the occupancy of an obstacle along the path of travel of the vehicle V, facilitating the online correction for the path of travel of the vehicle V by a drive assistance system/interface IA.
For instance, it may facilitate avoidance of a collision between the vehicle and the obstacle when the vehicle is operated by an autonomous driving system (not visible in the figures).
In an example as in
The calculations shown here above represent of course a simple example, although in other embodiments different calculations to determine the discrepancies may be performed.
Processing the set of detected discrepancies in the conflict assessment stage 84 in combination with models DM, provide the dataset CR value, e.g. CR=⅔.
Moreover, again as an example, the conflict resolution stage 87 may receive the initial values of weight used in the fusion stage, e.g. CA={tilde over (w)}0={tilde over (w)}1={tilde over (w)}2={tilde over (w)}i=⅓=0.33.
Consequently, the conflict resolution stage 87 may compute the updated weight values, for instance to provide to the BOF fusion stage 50 in the grid fusion processing stage 40. For example, weights may be computed as follow: e.g.:
CC=[w1,w2,w3]
where:
Either in a feed-forward or a feed-back configuration, thus, the sensor grid maps probabilities of occupancy may be fused and weighted accordingly a function of these weights in the fused grid map M.
In a further variant embodiment of the arbitration stage 90, the value CC may comprise adaptive weights W to feed back to the BOF fusion processing 50 in the data fusion processing stage 40, e.g. in particular in the stage 520 of the BOF fusion processing, as visible in
As already mentioned, in a variant embodiment as exemplified in
Specifically, in a variant embodiment as exemplified in
As exemplified in
In a variant embodiment as exemplified in
In a variant embodiment as exemplified in
In a variant embodiment as exemplified in
In fact, even if GPS information states there is no wall, in case a LIDAR sensor states otherwise, it would be best to highlight this by assigning a “conflict” value, distinguishing from the case in which neither GPS nor LIDAR give information about a certain area, to be thus marked as “unknown”. In the BOF fusion 50, in both cases of the previous example the grid cell of the fused map F would have been assigned a same probability value, e.g. of 0.5 and a same state, e.g. “unknown”. On the other hand, DST enables to differentiate more “evidence” states, e.g. four states free, occupied, conflict, unknown.
Nevertheless, it has been observed that DST fusion 60 may produce counter-intuitive results when combining information from (highly) conflicting sources, thus PCR processing 87′ may be used for the purpose of coping with an unpredictable “evidence” grid maps E, extending DST capability to automatically resolve potential conflicts.
For instance, one or more embodiments may comprise operations 87′1 of conflicting masses redistribution, e.g., proportionally to non-empty sets, according to all integrity constraints.
PCR stage 87′ is for instance configured to compute the conjunctive combination of all sources n for all subsets (free, occupied, conflict, unknown)∈A., generalizing the basic DST formula as:
The total conflicting mass can be expressed as follows:
where:
In one or more embodiments using evidential grids E, thus, each partial conflicting mass is redistributed 89′ to its subsets in proportion to the basic probability mass already assigned to these subsets in the conflict resolution stage 87′ of the arbitration stage 90′, as exemplified in
It is noted again that the variant embodiments of
In fact, the use of proportional conflict redistribution rules 87′, briefly PCR, on evidential grid maps E and of conflict resolution via rules CR provided by situation assessment 80 of discrepancies D are not mutually exclusive approaches.
The two approaches may be combined in a variant embodiment, comprising, for instance in the arbitration stage 90:
The latter arrangement may prove useful, e.g., if including both processing stages 50 and 60 in the processing data fusion stage 40. In fact, DST processing 60 and BF processing 50 may be performed, e.g. in parallel, in the data fusion stage 40 in order to improve consistency of the both methods and reduce each other weaknesses, e.g. to facilitate improved robustness.
The conflict arbitration stage 90 may thus facilitate coherent and reliable correction of conflicts in grid cells of at least one fused grid map F, E, thus outputting a drivable fused map M.
It will be appreciated that the representation of the processing stages discussed in the foregoing—in particular data fusion stage 40, BOF fusion 50, DST fusion 60, arbitration stage 90 and/or 90′ and discrepancy detection stage 70—as distinct entities is merely for the sake of clarity and simplicity. In one or more embodiments, one or more of those stages may be integrated in the form of a multi-functional stage and/or circuit, e.g. in a single processor or DSP.
The method 1000 for mapping the environment of a vehicle V, e.g. a land vehicle, comprises the steps of:
As described, the operation 1090 processing includes, in particular, at least one of:
As described, the operation 1090a feed-forward processing includes, in particular:
As described, the operation 1090b processing includes, in particular:
As already mentioned, the method can include either the using the blocks 50 and 90 for BOF or the blocks 60 and 90′ for DST, e.g. performing either the operations 1050, 1090a and 1060, 1090b. However, in a further variant embodiment the data fusion block 40 can included both blocks 50 and 60, and the system 100 can include both the arbitration blocks 90, 90′, this meaning that the method can include and perform all the operations shown in
To sum up, one or more embodiments:
It will be otherwise understood that the various individual implementing options exemplified throughout the figures accompanying this description are not necessarily intended to be adopted in the same combinations exemplified in the figures. One or more embodiments may thus adopt these (otherwise non-mandatory) options individually and/or in different combinations with respect to the combination exemplified in the accompanying figures.
Without prejudice to the underlying principles, the details and embodiments may vary, even significantly, with respect to what has been described by way of example only, without departing from the extent of protection. The extent of protection is defined by the annexed claims. Thus, the invention has been described in an illustrative manner. It is to be understood that the terminology which has been used is intended to be in the nature of words of description rather than of limitation. Many modifications and variations of the invention are possible in light of the above teachings. Therefore, within the scope of the appended claims, the invention may be practiced other than as specifically described.
Number | Date | Country | Kind |
---|---|---|---|
102018000006594 | Jun 2018 | IT | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/IB2019/055145 | 6/19/2019 | WO |
Publishing Document | Publishing Date | Country | Kind |
---|---|---|---|
WO2019/244060 | 12/26/2019 | WO | A |
Number | Name | Date | Kind |
---|---|---|---|
20150154471 | Ruan et al. | Jun 2015 | A1 |
20170197311 | Garcia et al. | Jul 2017 | A1 |
Number | Date | Country |
---|---|---|
102014200279 | Jul 2015 | DE |
2012033193 | Feb 2012 | JP |
2013077202 | Apr 2013 | JP |
2015106382 | Jun 2015 | JP |
2017521761 | Aug 2017 | JP |
Entry |
---|
C. Seeger, M. Manz, P. Matters and J. Hornegger, “Locally adaptive discounting in multi sensor occupancy grid fusion,” 2016 IEEE Intelligent Vehicles Symposium (IV), Gothenburg, Sweden, 2016, pp. 266-271, doi: 10.1109/IVS.2016.7535396. (Year: 2016). |
J. Dezert, J. Moras and B. Pannetier, “Environment perception using grid occupancy estimation with belief functions,” 2015 18th International Conference on Information Fusion (Fusion), Washington, DC, USA, 2015, pp. 1070-1077. (Year: 2015). |
Machine Translation DE-102014200279-A1 (Year: 2015). |
International Search Report and Written Opinion of the International Searching Authority for PCT International Application No. PCT/IB2019/055145 dated Nov. 15, 2019. |
Adarve, Juan David, et al., “Computing Occupancy Grids from Multiple Sensors using Linear Opinion Pools,” 2013 IEEE International Conference on Robotics and Automation (ICRA), Karlsruhe, Germany, May 6-10, 2013, pp. 4074-4079 (May 1, 2012). |
Dezert, Jean, et al., “Environment Perception Using Grid Occupancy Estimation with Belief Functions,” available at https://www.onera.fr/sites/default/files/297/C109.pdf (Jul. 1, 2015) (8 pages). |
Fernández, Carlos Gálvez Del Postigo, “Grid-Based Multi-Sensor Fusion for On-Road Obstacle Detection: Application to Autonomous Driving,” available at https://www.diva-portal.org/smash/get/diva2:852457/FULLTEXT01.pdf (Jan. 1, 2015). |
Moras, Julien, et al., “Credibilist Occupancy Grids for Vehicle Perception in Dynamic Environments,” 2013 IEEE International Conference on Robotics and Automation (ICRA), Karlsruhe, Germany, May 6-10, 2013, pp. 84-89 (May 1, 2011). |
Pathak, Kaustubh, et al., “3D Forward Sensor Modeling and Application to Occupancy Grid based Sensor Fusion,” 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2059-2064 (Oct. 1, 2007). |
Seeger, Christoph, et al., “Locally Adaptive Discounting in Multi Sensor Occupancy Grid Fusion,” 2016 IEEE Intelligent Vehicles Symposium (IV), Gothenburg, Sweden, Jun. 19-22, 2016, pp. 266-271, IEEE (Jun. 19, 2016). |
Office Action issued for Japanese Patent Application No. 2020-560369 dated Feb. 21, 2023. |
Second Office Action issued for Japanese Patent Application No. 2020-560369 dated Aug. 15, 2023. |
Number | Date | Country | |
---|---|---|---|
20210131823 A1 | May 2021 | US |