This application claims priority to foreign European patent application No. EP 19315027.3, filed on Apr. 29, 2019, the disclosure of which is incorporated by reference in its entirety.
The invention relates to a method of performing simultaneous localization of a mobile body and mapping of its environment using distance (or “range”) and motion sensors. It also relates to an apparatus for carrying out such a method.
The invention applies, in particular, to the field of the navigation of robots, drones, autonomous vehicles, etc. and more generally to that of perception.
In the following, “mapping the environment” will designate the task of determining the position, in a suitable reference frame, of physical bodies within a region of space surrounding in whole or in part the mobile body (also called a “carrier”). “Localization” refers to the task of determining at least the position, and preferably also the heading, of the mobile body within such a region of space. “Performing simultaneous localization and mapping” refers to a situation wherein signals provided by sensors are processed to determine the position/heading of the mobile body (carrier) and simultaneously detect and localize obstacles in the surrounding space.
The moving body may be a robot, drone, autonomous vehicle or the like, or even a device which cannot move autonomously but is carried by a person or a machine (e.g. an “electronic white cane” assisting visually impaired people).
By “physical body” (also called an “obstacle”) it is meant any physical object or substance that exhibits individuality and can be detected by an appropriate sensor. Thus, inanimate objects, be they natural or artificial, plants, animals, human beings, but also liquid or solid particles in suspension in the air, such as clouds, or indeed liquid or gaseous masses, are considered to be physical bodies.
A distance (or “range”) sensor is a sensor which provides information about at least the distance to a nearest obstacle in its field of view. Its physical operating principle is irrelevant: sensors as diverse as LIDARs, radars, sonars, etc. may be used to implement the inventive method and apparatus. Some distance sensors may provide additional information, e.g. the distance of all obstacles in a field of view and in some cases the azimuth/elevation of said obstacles.
A distance sensor may be considered to be of the “narrow-angle” or of the “wide-angle” type. A distance sensor is considered to be “wide-angle” when there is a non-negligible likelihood that several different obstacles are simultaneously present inside its field of view, and at a same distance from it (taking into account the finite resolution of the distance measurement). Conversely, a distance sensor is considered to be “narrow-angle” when such likelihood is negligible. Therefore, these definitions are application-dependent. While in most cases a sensor having a field of view of 15° or more can be considered to be a “wide-angle” sensor this is not always the case, and conversely a narrower field of view may nevertheless be considered “wide” enough to benefit from the invention.
A motion sensor is a sensor which provides information about the movements (translation and/or rotation) of the moving body with respect to its environment. Its physical operating principle is irrelevant: sensors as diverse as rotary encoders, inertial sensors, Doppler radars or sonars etc. may be used to implement the inventive method and apparatus.
Environment mapping can be performed according to several different techniques, most of which belong to one of two main families: geometric procedures, which are aimed at identifying the geometry of the objects of the surrounding space, and occupancy grid based procedures, which are aimed at determining whether a certain location is occupied by an obstacle. The occupancy grid technique is disclosed e.g. by [1]
Localization, also called “position estimation”, can also be performed according to different techniques. In particular, the so-called “Markov localization” [2] approach aims at estimating the probability distribution in space of the carrier's position, and its evolution as the carrier moves and/or new signals are acquired by different sorts of sensors. It is important to note that it is not possible to use motion (or “odometric”) sensor alone to perform position estimation, as the growth of the estimation error over time would be unbounded.
In the simplest cases, Markov localization assumes that the environment is known and static, i.e. there is a known “map” of the environment on which the carrier has to be localized. Researchers, however, have also tackled the more complex problem of simultaneous localization and mapping (SLAM) wherein the environment is unknown and/or dynamic, and a map of it has to be built simultaneously with the estimation of the position of the carrier.
Document [3] discloses a SLAM algorithm called FastSLAM wherein sensors (e.g. laser sensors) are used for detecting “landmarks” of the environment. The measurements issued by the sensors are used for updating a carrier position using particle filters, while the landmarks themselves are localized using extended Kalman filters. The method supposes that a measurement can be unambiguously associated to a given landmark, which is a rather stringent hypothesis. Moreover, it is computationally heavy (it requires a high number of Kalman filters) and the error of the pose estimation may diverge in some cases.
Document [4] discloses a SLAM algorithm wherein the environment is represented using an occupancy grid. The heading of the carrier is supposed to be known and its position is determined using the “scan matching” method. A drawback of this method is that the carrier heading is simply supposed to be known, which is not usually the case in real-word applications. Moreover, the method does not account for the impact of the uncertainty of the carrier position on the updating of the occupancy grid.
Document [5] discloses a SLAM method combining odometry and sonar measurements. The environment is represented using an occupancy grid updated by a Kalman filter, while a particle filter is used for tracking the pose of the carrier. A drawback of this method is that it does not account for the impact of the uncertainty of the positions of the obstacle on the updating of the carrier position.
Document [6] discloses a SLAM method using an occupancy grid for representing the environment and a pose grid for representing the carrier position and orientation. This approach has an exponential computational complexity, which makes it difficult to apply to embarked systems, having a limited computing power. Moreover, it relies on restrictive hypotheses (an insufficiently general sensor model, a specific motion model).
U.S. Pat No. 8,340,852 discloses the application of SLAM using an inertial measurement unit (IMU) and range sensors to autonomous vehicles. SLAM is performed using an optimization algorithm, which does not allow taking into account the measurement incertitude.
US 2004/013295 discloses a method of performing SLAM using a pair of cameras for acquiring images of the environment, detecting obstacles and identifying landmarks, and odometric sensors.
US 2016/0062361 discloses a navigation method fora mobile robot, including a first step wherein the robot moves along a path following input commands from an operator and detecting obstacle, and a second step wherein the robot moves autonomously upgrading a map of the environment and estimating its own position.
The invention aims at overcoming, in full or in part, at least some of the drawbacks of the prior art. More particularly it aims at providing a SLAM method having an acceptable computational complexity, general applicability (i.e. avoiding overly-restrictive hypothesis) and explicitly accounting for uncertainties on sensor measurements. Advantageously, such a method should be able to be carried out by an embarked processor with limited computing power.
According to the invention, an object of the invention allowing achieving these aims in full or in part is a method of performing simultaneous localization of a mobile body and mapping of its environment, the method comprising the following steps, carried out by a data processor:
Another object of the invention is an apparatus for performing simultaneous localization of a mobile body and mapping of its environment, comprising:
Yet another object of the invention is a mobile robot carrying such an apparatus.
Additional features and advantages of the present invention will become apparent from the subsequent description, taken in conjunction with the accompanying drawings, wherein:
A range (or distance) sensor may be represented by use of a probabilistic model, which accounts for its finite precision, resolution and for possible measurement errors.
The idea is that a measurement output by the sensor does not necessarily indicate the exact distance between the sensor and the nearest obstacle in its field of view. Instead, for a given distance d, the output z of the sensor is a random variable characterized by a probability distribution p(z|d), constituting the “direct model” of the sensor.
Hereinafter, Ω will denote a spatial reference frame in one, two or three dimensions; an occupancy grid OG is a partition of a continuous and bounded region of a space generated by Ω into a number N of parts, called “cells” and designated by an index i∈[0, N−1]. The cell of index i is indicated by ci. Without loss of generality, we shall consider a one-dimensional occupancy grid observed by a single distance sensor C (or a plurality of co-located sensors), the index i increasing with the distance from the sensor (c0 therefore being the cell closest to the sensor and cN−1 the one furthest away). This configuration is illustrated on
An obstacle T is a bounded continuous subset of Ω. A cell ci is said to be occupied by an obstacle T if T∩ci≠Ø, to be not occupied by T if A∩ci=Ø. Stated otherwise, if the obstacle covers the cell even partially, the latter is considered to be occupied. Other conventions are possible, but in any event a cell must be either free (vacant), or occupied.
For each of the cells of the grid, we consider the binary random experiment “state” which can have one of the two outcomes {occupied; vacant} consisting in knowing whether or not the cell contains an obstacle or not. The state of the cell ci is denoted si∈{ei, oi}. oi will denote the realisation si=oi (I;e. the cell is occupied), and ei will denote the realisation si=ei (i.e. the cell is empty).
An occupancy probability P(oi)—or, equivalently, a vacancy probability P(ei)=1−P(oi)—is associated to each cell ci of an occupancy grid (in the following, a capital “P” will denote a probability, and a lowercase “p” a probability density). Before any measurement, an a priori occupancy probability—often, but not necessarily equal to 0.5, is associated to each cell.
It is also considered that the position of the obstacles can only be known with the aid of uncertain distance sensors, characterized by a probabilistic model such as described above which may be written, in the single obstacle approximation, p(z|{right arrow over (x)}), {right arrow over (x)} being the position of the nearest obstacle (in several dimensions, it is a vector, expressed in cartesian, spherical, polar coordinates, etc. and not a simple scalar). The sensors may be telemetric lasers (also called LIDARs), sonars, infrared radars, time-of-flight cameras, etc. More generally, by taking into account the possibility of having k>1 obstacles in the field of view of the sensor, the inverse model may be written as p(z|{right arrow over (x)}1, {right arrow over (x)}2, . . . , {right arrow over (x)}k), where {right arrow over (x)}i is the position of the i-th obstacle.
A measurement z arising from a sensor makes it possible to determine the conditioned probability of occupancy P(oi|z) of a cell ci. For a given measurement z, the set of probabilities P(oi|z)∀i∈[0, N−1] constitutes what is called the “inverse model” of the sensor on the grid. Whilst the direct model of the sensor describes the response of the sensor as a function of the material world, the inverse model expresses the impact of the measurement on the occupancy grid, which is the model of the material world that is adopted, thereby justifying the term inverse model.
The inverse model, applied to range measurement, allows updating the occupancy probabilities of the occupancy grid.
Documents WO2017/050890 and EP 3 364 213 describe methods for updating the occupancy probabilities of an occupancy grid using a time series of range measurements from one or more “narrow-angle” distance sensors. European Patent Application 18305955 filed on Jul. 13, 2018 generalizes these methods to the case of “wide-angle” sensors. The teaching of these documents can be applied to the present invention
In accordance with the usage which prevails in the literature,
Moreover, for reasons which are explained in WO2017/050890 and which will become apparent in the following, it is preferable to quantize the probability values by allowing them to only take values belonging to a finite set. Suitable quantizing schemes will be described later, see equations (13)-(28).
Similarly, the pose of the carrier may be represented by a “pose grid” Π, comprising a plurality of cells, each associated to a pose probability. In a preferred embodiment of the invention, a “pose” is constituted by a position in two- or three-dimensions and a heading direction expressed by one or two angles. For some applications (e.g. a terrestrial robot such as an autonomous vehicle), the position will be defined in two dimensions and the heading direction will be expressed by an angle. The invention will be discussed with reference to this case, which is easier to represent on drawings, without loss of generality.
For instance, the position Xr of the carrier may take its values in L={xi, i=0 . . . M−1} and its heading αr may take its values in A={αi, i=0 . . . M′−1}. The pose xr=(Xr, αr) will then take its values in Π=L×A, which is the pose grid. In the following, the pose grid Π will be represented by a two-dimensional grid; however, it should be understood that it often represents a parameter space of higher dimensions (e.g. three dimensions: a two-dimensional position and a one-dimensional heading) and that it may be represented, in a computer memory, by a one-dimensional, two-dimensional or higher-dimensional array.
As it will be discussed below, the pose grid is updated using both range measurement, provided by range sensors, and odometry measurement, provided by motion sensors. To this aim, a motion sensor may be represented by a “motion model” P(xr*|xr, u) which expresses the probability that the carrier takes a pose xr* knowing that, at a previous time, it had a pose xr and that an odometry measurement u has been received.
As illustrated on
Before time t1 the occupancy grid G and the pose grid Π are in an initial state st0. The state is characterized by an occupancy probability value for each cell of the occupancy grid, and a pose value for each cell of the pose grid.
At time t1, a first odometry measurement u1 is received, and used for updating the pose grid Π (more precisely, the pose probability of at least some of its cells) while the occupancy grid G remains unchanged. This corresponds to state st1.
At time t2, a first distance measurement z1 is received. The measurement z1 and the pose grid Π are used to compute an updated occupancy grid (more precisely: updated occupancy probabilities of at least some of its cells), while the pose grid remains unchanged (state st2).
At time t3, a second distance measurement z2 is received. This time, this measurement is used—together with the occupancy grid G—to update the pose grid Π (more precisely, the pose probability of at least some of its cells). The occupancy grid remains unchanged (state st3).
At time t4, a second odometry measurement u2 is received, which is used for updating the pose grid Π (i.e. its pose probabilities) while the occupancy grid G remains unchanged (state st4).
It is important to note that the distance and odometry measurements may arrive asynchronously, and it is even possible that several measurements arrive simultaneously. However, two simultaneous odometry measurements must be processed sequentially, while two simultaneous distance measurements may be processed simultaneously, provided that one of them is used for updating the pose grid and the other one for updating the occupancy grid. Similarly, if an odometry measurement and a position measurement are received simultaneously, they may be processed simultaneously provided that the latter is used for updating the occupancy grid. It should be understood that two measurements are deemed to be “received simultaneously” when they are separated by a time interval smaller than a time resolution of the data processor, and that they are “processed simultaneously” is they are applied to a same state sti of the occupancy and pose grids.
In the example of
It is apparent from
In the following, possible embodiments of each one of these three steps will be described in detail.
The following notations will be used:
xr=(Xr, αr) represents a pose—i.e. a position and a heading. P(xr) is the probabilty that the carrier takes pose xr.
z is the result of a distance (or range) measurement, representing the distance between the carrier (more precisely, a point of the carrier) and an obstacle. In the following, it will be assumed that z can only take discrete values, which is required for digital processing.
u is the result of an odometry measurement, representing a movement (a translation, a rotation or a combination thereof) of the carrier. In the following, it will be assumed that u can only take discrete values, which is required for digital processing.
P(oi|z, xr) is the inverse model of a range sensor (ISM—inverse sensor model), associating an occupancy probability to each cell of the occupancy grid for a given range measurement result z and a pose of the carrier, xr, deemed to be known. The ISM is supposed to be tabulated for all possible values of z and xr.
P(xr*|xr, u) is the motion model, which expresses the probability that the carrier takes a pose xr* knowing that, at a previous time, it had a pose xr and that an odometry measurement u has been received. Again, this model is supposed to be tabulated for all possible values of u and xr.
As illustrated on
where the second equality accounts for the fact that the pose probability is independent from the range measurement.
Then, the current occupancy probabilities—represented by P(oi|zt−1)—is “fused” with the occupancy probabilities of the temporary grid Pxr(oi|z) to obtain updated occupancy probabilities (i.e. an updated occupancy grid) P(oi|z, zt−1) as follows:
See WO2017/050890 and EP 3 364 213.
Updating the pose grid upon reception of a range measurement means computing P(xr|z). For xr∈Π:
The denominator p(z) is a constant factor, therefore (3) can be written
P(xr|z)=η·p(xr, z) (4)
η being a constant.
For the sake of simplicity, it will be considered that the position grid L coincides with the occupancy grid G. Therefore, let xr=(cn, αm). For given n and m, one may define Gxr be the projection of the occupancy grid G onto the range sensor field of view when the sensor itself is located at Xr=cn and having a heading αr=αm. Then, summing over all the possible configurations Gkxr of Gxr:
and therefore (4) can be written
P(xr|z)=η·P(xr)·Σkp(z|xr, Gkxr)·P(Gkxr) (6)
All the terms of (6) are either known or easily computed:
P(Gkxr) is the probability of a given configuration of the occupancy grid, which in turn depends on the occupancy probabilities P(oi). Assuming that the projected grid Gxr has Nxr elements, the a priori probability of any configuration is
The computation of P(xr|z) is illustrated on
The number of terms involved in the sum of (6) depends on the kind of range sensor considered. For instance, while in the most general case this number of terms is an exponential function of the number of cells Nxr, in the case of a LIDAR sensor it is equal to Nxr (i.e. the complexity is linear); the same is true for wide-field sensor if the nearest-target approximation applies (see below, equations 9 and 10, and above-cited European Patent Application 18305955). Indeed, a LIDAR only detects the nearest target, therefore two different configurations having a same nearest occupied cell give identical contributions. Let Ljxr be the event “the first occupied cell in Gkxr is ĉj”. This means that all cells ĉi, i<j, are empty and that cells ĉi, i>j can have any state. Summing all possible events Ljxr, j={0, . . . Nxr−1}, (6) can be written as:
wherein:
For sensors having a large field of view, the occupancy grid may take the form illustrated on
If the nearest-target hypothesis can be applied, for two configurations Gk1xr and Gk2xr sharing the same nearest occupied arc, p(z|xr, Gk1xr) can be taken to be equal to p(z|xr, Gk2xr). Thus for Gxr={ĉ0, ĉ1, . . . , }={, . . . , }Qjxr is defined, for j=0, . . . , Nxr−1, as the event “Âj is the first occupied arc in Gxr” (as a consequence, all cells in arcs Âii<j are empty and cells in a Âii>j can be in any state. Summing all possible events Qjxr, equation (6) may be written:
where P(Qjxr) is the probability of having the arc Âj as the first occupied arc, i.e. the probability of having all arcs Âii<j empty, which is given by:
Different kind of motion sensors can be used in order to update the carrier pose, such as IMU (Inertial Measurement Units), GPS, wheel encoders . . . The pose distribution at time t=0, P0(xr) is deemed to be known, and updated when an odometry measurement u is received. The problem is then to compute Pt(xr|u) from Pt−1(xr).
For a fixed pose xr*, Pt(xr*|u) is computed by summing over all possible poses:
Taking into account the fact that P(xr|u)=Pt−1(xr), (11) becomes:
wherein:
A potential problem with the inventive method described above is that, in a nave implementation, computations would need to be performed using floating point arithmetic. The use of floating-point computation requires significant resources in terms of computing power, which are hardly compatible with the constraints specific to embedded systems.
According to some advantageous embodiments of the invention, however, occupancy and pose probability may be updated by making use of much simpler integer computation, while keeping an acceptable accuracy level.
It is worth noting that WO2017/050890 (already cited) discloses a method for updating occupancy grids only using integer computations. This method alone, however, is not sufficient to allow updating pose grids, too, without floating point computations.
Evaluating equations (1)-(12) requires five kind of elementary operations:
Two alternative embodiments will be described, allowing carrying out all these operations using integer computation only. According to both embodiments of the invention, the probability values are discretized, i.e. they can only take values belonging to a finite set. The first embodiment uses a first, non-uniform discretization scheme and the second embodiment uses a second, uniform discretization scheme. A third discretization scheme (already disclosed in WO2017/050890) is used for carrying out “integer fusion”—it is therefore necessary to map the first or second discretization scheme onto the third and vice-verse, which can be done very cheaply using a lookup table.
Let ϵ be a real value in ]0,1[. The discretization set Dlog of size Nlog is defined as:
D
log={1, (1−ϵ), (1−ϵ)2, . . . , 0.5, . . . , (1−ϵ)N
It should be noted that ϵ is chosen such that it exists an integer n (usually n=Nlog/2 or n=Nlog/4) satisfying (1−ϵ)n=0.5.
It can be easily verified that, for i,j∈{0 . . . , Nlog−1}, the product of Dlog[i] by Dlog[j] given by:
Therefore, the product of two values (probabilities, probability densities) discretized according to the discretization set Dlog can be computed using an integer operation on integer indexes.
The division of Dlog[i] by Dlog[j] is given by:
Interestingly, in the inventive method, division is only used to compute the normalization factor η<1, therefore the first condition never occurs. Again, only an integer operation on indexes needs to be performed.
The addition of Dlog[i] and Dlog[j], however, gives a result which—in general—does not belong to Dlog. Indeed, one should solve the following equation
(1−ϵ)k
which gives:
Clearly, k+i,j is not necessarily an integer. However, one may take the floor (or, alternatively, the ceiling, or the closest integer etc.) of (17), so that k+i,j is redefined as:
Addition is then represented by:
Computing the addition using (18) and (19) requires floating point operations. However, it is possible to compute (19) offline for all pairs (i,j) and store them in a lookup table. Note that, since addition is symmetric, the table only needs to store Nlog2/2 terms.
Additive inversion is performed in a similar way:
1⊖Dlog[j] is then defined as:
Like for addition, (21) is computed offline and stored in a table of size Nlog.
As mentioned above, fusion requires mapping Dlog to a different discretization set Gp, the result being then mapped back to Dlog. The theory of integer probability fusion was disclosed in WO2017/050890 and will be briefly recalled here.
Let p be a probability value that verifies
0.5<p<1 (22)
A sequence pn(n∈) is defined as follows:
where F(.,.) is the fusion operator
The definition of pn is then extended to negative integer values of n as follows:
The discretization schemes, depending on parameter p∈]0.5;1[ are then defined as follows:
G
p
−={(pn), n≤0} (26)
G
p
+={(pn), n≥0} (27)
It can be shown that the fusion of two elements pi and pj belonging to Gp=Gp+∪Gp− is defined by:
F(pi, pj)=pi+j (28)
Therefore, Gp is a discretization scheme which does not introduce errors when used for performing probability fusion.
As explained above, according to the present embodiment of the invention, when fusion has to be performed in order to update the occupancy grid upon reception of a range measurement, Dlog is mapped onto Gp, fusion is performed, and the result is mapped back onto Dlog, the mapping being performed using lookup tables stored in a memory device.
More precisely, let fG
The values of this function are then stored offline in a lookup table. Similarly, for pi∈Gp, let fD
Let N*∈*. The discretization set Dunif of size N*+1 is defined as:
The set Dunif represents a discretization of the probabilities in [0, 1].
For i,j∈{0, . . . N*} the addition of Dunif[i] and Dunif[j] is simply given by:
Therefore, the sum of two values (probabilities, probability densities) discretized according to the discretization set Dunif can be computed using an integer operation on integer indexes. The same applies to additive inversion. Indeed, for i∈{0, . . . N*}, 1−Dunif[i] is given by:
For j∈{0, . . . N*} the product of Dunif[i] and Dunif[j] cannot be exactly represented in Dunif. Indeed, one should solve the following equation for k×i,j:
which yields
It can be easily understood that k×i,j is not necessarily an integer. As in the first embodiment, one can take the floor of (34)—or, alternatively, the ceiling, or the closest integer etc.—and redefine k×i,j as:
Thus, multiplication in Dunif is represented by
The values of Dunif[i]⊗Dunif[j], as defined in (36), for all i,j pairs are then computed offline and stored in a lookup table.
Division is dealt with similarly. One should solve the following equation for kdi,j:
which yields:
As (38) does not necessarily gives an integer value, kdi,j is redefined by taking the floor of (38):
Division in Dunif is then represented by:
Note that, in the inventive method, the first condition (j=0, leading to an undefined result) can never occur. As for product, the values of Dunif[i]∅Dunif[j], as defined in (40), for all i,j pairs (except j=0) are then computed offline and stored in a lookup table.
Fusion is dealt with in a similar way than in the first embodiment. gG
Both the first and the second embodiments allow then to carry out the inventive method using only:
The invention has been described with reference to embodiments wherein a single mobile body, carrying odometry and distance sensors, is localized, and its environment is mapped. Other embodiments are possible. For instance, some or even all of the sensors may not be embarked on the mobile body—e.g. an external Doppler radar may be used to obtain relevant odometry measurements. Moreover, several sensor-carrying mobile bodies moving within a same environment may share—and contribute to update—a single occupancy grid, while having each a respective pose grid.
[1] A. Elfes, “Occupancy grids: a stochastic spatial representation for active robot perception”, Sixth Conference on Uncertainty in AI, 1990.
[2] D. Fox, W. Burgard, and S. Thrun, “Markov localization for mobile robots in dynamic environments”, Journal of Artificial Intelligence Research, 1999.
[3] M. Montemerlo, S. Thrun, D. Koller, and B. Wegbreit “FastSLAM: A factored solution to the simultaneous localization and mapping problem” in AAAI National Conference on Artificial Intelligence. AAAI, pp. 593-598, 2002.
[4] T. Duckett and U. Nehmzow “Mobile robot self-localisation using occupancy histograms and a mixture of gaussian location hypotheses” Robotics and Autonomous Systems, vol. 34, no. 2, pp. 117-129, 2001.
[5] J. Nordh and K. Berntorp “Extending the occupancy grid concept for low-cost sensor-based slam” in IFAC Proceedings Volumes, vol. 45, no. 22, Elsevier, pp. 151-156, 2012.
[6] K. P. Murphy, “Bayesian map learning in dynamic environments” in Advances in Neural Information Processing Systems, pp. 1015-1021, 2000.
Number | Date | Country | Kind |
---|---|---|---|
19315027.3 | Apr 2019 | EP | regional |