Patent Application
20220350329
This patent application claims the benefit and priority of Chinese Patent Application No. 202110448691.1, filed on Apr. 25, 2021, the disclosure of which is incorporated by reference herein in its entirety as part of the present application.
The present disclosure relates to the field of inspection robots, and in particular, to a neural network-based method for calibration and localization of an indoor inspection robot.
Safe production is the premise and important assurance for stable development of enterprises. Once a safety accident happens, it would cause casualties and property losses to seriously restrict the rapid and sound development of enterprises. Particularly in hazardous working environments such as nuclear power stations, steelworks, chemical plants, oil depots and substations, the safe production depends on the safe and stable operation of various devices and instruments, and thus the regular and continuous inspection is very necessary. At present, the hazardous working environments such as substations and oil depots are mainly inspected manually, specifically, devices and instruments in plants are inspected based on sensory organs of workers and by reading meters, making records and adopting some assorted detectors. Due to various external factors, the manual inspection is faced with the undesirable inspection quality and the low inspection efficiency. In order to improve the inspection efficiency, and ensure the safe and reliable operation of the devices and instruments, it has been a trend to replace a part of the manual inspection with an automatic inspection robot.
However, localization for the inspection robot in plants is an arduous task for various reasons. On the basis of the widespread use of vision, the odometer as a proprioceptive sensor of the robot inevitably accumulates trajectory error over time and distance. Concerning existing odometry localization methods for the indoor inspection robot, the odometer has a large computational error due to complicated indoor environmental factors as well as systematic and non-systematic factors.
In view of the above problems in the prior art, the present disclosure is intended to solve the technical problem that the odometer in existing odometry localization methods for the indoor inspection robot has a large error due to complicated indoor environmental factors.
To solve the above-mentioned technical problem, the present disclosure employs the following technical solutions. A neural network-based method for calibration and localization of an indoor inspection robot includes the following steps:
S100: presetting indoor positions for N label signal sources, the label signal sources being capable of transmitting radio frequency (RF) signals;
S200: providing a reader-writer for receiving the signals of the label signal sources on an indoor robot, and computing, during an indoor traveling process of the robot, an actual path of the robot according to numbers of signal labels received by the reader-writer at different moments, specifically:
S210: establishing a log-normal propagation loss model according to a signal strength of received labels:
P(d)=P(d0)−10α log(d/d0)−xσ (2-1)
where, P(d) represents a signal strength from labels received by a reader, P(d0) represents a signal strength from labels received by the reader at a reference point d0, αrepresents a scale factor between a path length and a path loss, X, represents a Gaussian random variable having an average of 0, d0 represents a distance from the reference point to each of the labels, and d represents a distance from a label to be computed to the reader;
S220: obtaining a distance d from each of the label signal sources to the reader-writer by transforming eq. (2-1):
d=10[P(d
where, P(d0) represents a signal strength at 1 m, and α represents a signal propagation constant;
S230: randomly selecting I moments within a time interval T, and assuming that a number of label signals received by the reader-writer at a tth moment is n, then computing an actual coordinate of the robot at the tth moment as:
where,
E
x1
=x
1
−x
n
,E
x2
=x
2
−x
n
, . . . ,E
xn−1
=x
n−1
−x
n;
E
y1
=y
1
−y
n
,E
y2
=y
2
−y
n
,. . . ,E
yn−1
=y
n−1
−y
n;
K=(x1−xn)(y1−yn)+(x2−xn)(y2−yn)+. . . +(xn−1−xn)(yn−1−yn);
K
1
=x
1
2
−x
4
2
+y
1
2
−y
4
2
+d
4
2
−d
1
2;
K
2
=x
2
2
−x
4
2
+y
2
2
−y
4
2
+d
4
2
−d
2
2;
K
n
=x
n−1
2
−x
n
2
+y
n−1
2
−y
n
2
+d
n
2
−d
n−1
2;
M=(x1−x4)2+(x2−x4)2+ . . . +(xn−1−xn)2;
N=(y1−y4)2+(y2−y4)2+ . . . +(yn−1−yn)2;
S240: establishing a model with coordinates of the robot in the I moments to obtain an actual path of the robot:
RSSI={RSSIx,RSSIy }={x1′,x2′, . . . ,xn′,y1′,y2′, . . . ,yn′} (2-4)
where, RSSI ={RSSIx,RSSIy} represents the actual path of the robot, and RSSIx and RSSIy represent an coordinate and a coordinate computed through RF identification (RFID)-based localization; and
S250: obtaining an actual path of the robot in the I moments with the method in S210-S240;
S300: computing positional information moved by the robot at the tth moment:
where, ut=(ΔDt,Δθt)T represents a pose increment, St=(xt,yt,θt) represents a state of the robot at the tth moment, (x,y) represents a coordinate of the robot at the tth moment, θ represents a direction angle at the tth moment, ΔDt is an arc length moved by the robot within time Δt, and Δθt is a change of a direction angle in a pose of the robot within the time Δt;
S400: processing the positional information of the robot at the tth moment in step S300 with a generalized linear model (GLM) to obtain a predicted path of the robot at the tth moment:
GLM={x
1
,x
2
, . . . ,x
n
,y
1
,y
2
, . . . ,y
n} (4-1)
where, X 1 . . . nis an xcoordinate of the predicted path estimated by the GLM, and y 1 . . . nis a y coordinate of the predicted path estimated by the GLM, thereby obtaining a predicted path corresponding to each moment;
S500: establishing an odometry error model with the neural network:
E=Σ
n
|O−RSSI|2 (5-1)
where, O represents an optimized predicted path;
S600: presetting a maximum number of iterations, taking the actual path in the I moments and a corresponding predicted path as an input of a neural network model to train the error model E, updating parameters of the model through back propagation (BP) during training, and stopping training when E≤e−5 to obtain a well-trained odometry error model; and
S700: repeating steps S300-S400 to obtain a predicted traveling path of a robot R′ to be predicted, and inputting the predicted traveling path to the well-trained odometry error model in step S600 to obtain an optimized predicted traveling path that is a predicted value for the traveling path of the robot R′.
Preferably, the computing an actual coordinate of the robot at the tth moment in step S230 may specifically include:
S231: assuming that the inspection robot R provided with the reader-writer has a coordinate of (x, y), and coordinates of the n label signal sources are (x1, y1) (x2, y2) (x3, y3) . . . (xnyn):
where, d1, d2, d3 , . . . , dn represent distances from the label signal sources to the inspection robot R , respectively;
S232: substituting eq. (2-4) into eq. (2-5) to obtain:
representing eq. (2-6) with a linear equation as:
AX=b (2-7)
where:
[43]
S233: processing eq. (2-7) with a standard least squares estimation method to obtain:
J=[−A{circumflex over (X)} ]T [b−A{circumflex over (X)} ]=[bT −{circumflex over (X)}T AT][b−A{circumflex over (X)}] (2-8)
where, A is a non-singular matrix; M>2 , namely an overdetermined system in which a number of equations is greater than a number of unknowns; a reciprocal of {circumflex over (X)} is obtained with eq. (2-8); and xT ATb=bTA{circumflex over (X)}leads to:
obtaining a predicted coordinate when the inspection robot read an nth label:
{circumflex over (X)}=(AT A)−1ATb (2-10)
S234: computing an actual coordinate when the inspection robot reads the nth label, the actual coordinate being specifically given by:
where, t represents the tth moment, and
S235: substituting eq. (2-7) into eq. (2-11) to obtain eq. (2-3), eq. (2-3) being the actual coordinate of the robot at the tth moment.
At a moment when the RF reader-writer on the robot acquire different numbers of label signals, different localization algorithms are generally used to acquire the actual path of the robot more accurately. Usually, a probabilistic localization method is used for two or less label signals, a trilateral localization method is used for three label signals, and a maximum likelihood localization method is used for four or more label signals. Different localization algorithms are used depending on different numbers of label signals, which can effectively reduce influences on errors from different amounts of input data.
Preferably, the computing positional information moved by the robot in step S300 may specifically include:
S310: computing a theoretical resolution of an odometer:
where, δ represents the resolution of the odometer, D represents a diameter (mm) of a wheel, η represents a reduction ratio of a drive motor, and P represents an accuracy of an encoder;
S320: assuming that a sampling interval is Δt, then computing a displacement increment ΔdL (ΔdR) of a left (right) wheel:
ΔdL=δ·NL (3-3)
where, NL (NR) represent a pulse increment output by a photoelectric encoder on the left (right) wheel;
S330: computing the pose increment of the robot with the displacement increment of the wheel:
where, β represents a distance between the left drive wheel and the right drive wheel;
S340: obtaining, in case of a difference |Δθt|>0between direction angles at an end pose and a start pose of the robot, the positional information of the robot.
Preferably, the establishing an odometry error model E in step S500 may specifically include:
S510: assuming that the neural network structurally includes an input layer having q neurons, a hidden layer having l neurons, and an output layer having m neurons, connections between the neurons of the input layer and the neurons of the hidden layer each have a weight value of wij, connections between the neurons of the hidden layer and the neurons of the output layer each have a weight value of wjk , the hidden layer has a bias term of a, and the output layer has a bias term of b ;
S520: taking the GLM as an input of the neural network to obtain an output result of the hidden layer:
where, pi represents a vector for coordinates (xn−1,yn−1and (xn ,yn ) of the input GLM, f represents an activation function, and i={1, 2, . . . ,q}, j={1,2, . . . ,1}; and
a result from the output layer of the neural network is:
S530: taking the result from the output layer of the neural network at the tth moment and the actual path at the present moment as an input of a loss function to establish the odometry error model E.
This part is mainly intended to construct the odometry error model through the neural network. The neural network mainly includes two stages: forward propagation of the signal from the input layer to the hidden layer and to the output layer, and back propagation of the error. The neural network estimates an error for the directly previous layer of the output layer with the output error, and then estimates an error for the more previous layer with the present error, thereby implementing error estimation on all layers in the neural network; and through the loss function E, parameters of the neural network are obtained sequentially.
The disclosure has the following advantages over the prior art:
1. The odometry calibration method for an indoor inspection robot based on a neural network maximizes the localization accuracy by minimizing the odometry error.
2. The neural network method can be applied to any odometry model and high-speed working environment; and the method is irrelevant to the platform and does not need to provide the error variable of the odometer before model establishment.
3. The present disclosure is applied to common indoor environments for mapping and other tasks, and can also work well in complicated environments.
The present disclosure will be further described below in detail.
Due to complicated environmental factors, localization of the inspection robot is a research bottleneck to be solved urgently. On the basis of the widespread use of vision, the self-contained odometer can predict the trajectory of the robot according to the kinematic model, and achieve the desirable localization within a short distance and short time. The robot can be accurately localized by optimizing the error of the odometer. The RFID-based indoor localization can serve as an auxiliary localization device for odometry optimization of the indoor inspection robot because of its unnecessity for manual intervention, strong anti-interference performance, and capability of working in various harsh environments.
Referring to
S100: Indoor positions for N label signal sources are preset, the label signal sources being capable of transmitting RF signals.
S200: A reader-writer for receiving the signals of the label signal sources is provided on an indoor robot, and during an indoor traveling process of the robot, an actual path of the robot is computed according to numbers of signal labels received by the reader-writer at different moments, specifically:
S210: A log-normal propagation loss model is established according to a signal strength of received labels. The signal strength of each label is a constant value, unless the reader keeps a different distance from the label. The attenuation in the strength is called the propagation loss of the signal. With theoretical and empirical propagation loss models, the propagation loss can be transformed into the distance:
P(d)=P(d0)−10α log (d/d0)−xσ (2-1)
where, P(d) represents a signal strength from labels received by a reader, P(d0) represents a signal strength from labels received by the reader at a reference point d0 , all labels having a same initial signal strength, αrepresents a scale factor between a path length and a path loss and depends on a structure and a material of an obstacle, x94 represents a Gaussian random variable having an average of 0, namely attenuation that the signal passes through the obstacle, d0 represents a distance from the reference point to each of the labels, and represents a distance from a label to be computed to the reader.
S220: A distance d from each of the label signal sources to the reader-writer is obtained by transforming Eq. (2-1):
P(d)=P(d0)−10α log(d/d0)−xσ (2-1)
where, P(d0) represents a signal strength from labels at at 1m, and α represents a signal propagation constant. As can be seen from Eq. (2-2), P(d0)and α determine the relationship between an RSS and a signal transmission distance, and both can be a constant once the service environment is determined.
S230: I moments are randomly selected within a time interval T, and assuming that a number of label signals received by the reader-writer at a tth moment is n, an actual coordinate of the robot at the Tthmoment is computed as:
where,
E
x1
=x
1
−x
n
,E
x2
=x
2
−x
n
,. . . ,E
xn−1
=x
n−1
−x
n;
E
y1
=y
1
−y
n
,E
y2
=y
2
−y
n
,. . . ,E
yn−1
=y
n−1
−y
n;
K=(x1−xn)(y1−yn)+(x2−xn)(y2−yn)+. . . +(xn−1−xn)(yn−1−yn);
K
1
=x
1
2
−x
4
2
+y
1
2
−y
4
2
+d
4
2
−d
1
2;
K
2
=x
2
2
−x
4
2
+y
2
2
−y
4
2
+d
4
2
−d
2
2;
K
n
=x
n−1
2
−x
n
2
+y
n−1
2
−y
n
2
+d
n
2
−d
n−1
2;
M=(x1−x4)2+(x2−x4)2+ . . . +(xn−1−xn)2;
N=(y1−y4)2+(y2−y4)2+ . . . +(yn−1−yn)2;
where, E, K, M and N are all process parameters for computation, without the practical significance.
During specific implementation, the actual coordinate of the robot at the tth moment is specifically computed as follows:
S231: It is assumed that the inspection robot R provided with the reader-writer has a coordinate of (x,y), and coordinates of the n label signal sources are (x1,y1), (x2, y2), (x3,y3) . . . (xn ,yn):
where, d1 d2 d3 represent distances from the label signal sources to the inspection robot R , respectively;
S232: Eq. (2-4) is substituted into Eq. (2-5) to obtain:
Eq. (2-6) is represented with a linear equation as:
AX=b (2-7)
where:
S233: Eq. (2-7) is processed with a standard least squares estimation method to obtain:
J=[−A{circumflex over (X)} ]T [b−A{circumflex over (X)} ]=[bT −{circumflex over (X)}T AT][b−A{circumflex over (X)}] (2-8)
where, A is a non-singular matrix; M>2 , namely an overdetermined system in which a number of equations is greater than a number of unknowns; a reciprocal of {circumflex over (X)} is obtained with Eq. (2-8); and xTATb=bTA{circumflex over (X)} leads to:
A predicted coordinate when the inspection robot reads an nth label is obtained:
{circumflex over (X)}=(AT A)−1 AT b (2-10)
S234: An actual coordinate when the inspection robot reads the nth label is computed, the actual coordinate being specifically given by:
where, t represents the tth moment, and
During actual localization, the localization information acquired by the sensor changes to some extent in different environments. In order to reduce the error arising from fluctuations of the wireless signal, signal strengths acquired in unit time are averaged as follows to improve the accuracy:
where, j represents a number of coordinates computed within the unit time, and Ū represents a coordinate in unit average time.
The averaging method is used during the traveling process of the robot. Therefore, when the robot moves from the point P(t−1) to the point P(t) within the time Δt, the coordinate computed by the localization system is approximately estimated as the point P(
S235: Eq. (2-7) is substituted into Eq. (2-11) to obtain Eq. (2-3), Eq. (2-3) being the actual coordinate of the robot at the tth moment.
S240: A model is established with coordinates of the robot in the I moments to obtain an actual path of the robot:
RSSI={RSSIx,RSSIy }={x1′,x2′, . . . ,xn′,y1′,y2′, . . . ,yn′} (2-4)
where, RSSI={RSSIx,RSSIy} represents the actual path of the robot, which is achieved by modeling the coordinates from the RFID-based localization, and the target set of the neural network; and RSSIx and RSSIyrepresent an x coordinate and a y coordinate computed through the RFID-based localization.
S250: An actual path of the robot in the I moments is obtained with the method in S210-S240.
S300: Positional information moved by the robot at the tth moment is computed:
where, ut=(ΔDt,Δθt)Trepresents a pose increment, St =(xt,yt,θt) represents a state of the robot at the tth moment, (x,y) represents a coordinate of the robot at the tth moment, θ represents a direction angle at the tth moment, is an arc length moved by the robot within time Δt, and Δθt is a change of a direction angle in a pose of the robot within the time Δt.
During specific implementation, the positional information moved by the robot is specifically computed as follows:
S310: A 360 PPR incremental photoelectric encoder capable of outputting a dual pulse spaced at 90° is provided, and thus a rotation direction of a wheel can be determined through a phase change of the dual pulse. The photoelectric encoder is provided on a shaft extension of a drive motor to directly measure the rotation of the motor. The motor drives the wheel through a 15-fold decelerator, which means that when the wheel rotates one revolution, the motor rotates 15 revolutions. While the wheel has a diameter of 250 mm, a theoretical resolution of an odometer is computed:
where, δ represents the resolution of the odometer, D represents the diameter (mm) of the wheel, η represents a reduction ratio of the drive motor, and P represents an accuracy of the encoder.
S320: Assuming that a sampling interval is Δt, a displacement increment ΔdL(ΔdR) of a left (right) wheel is computed:
ΔdL=δ·NL (3-3)
where, NL (NR) represent a pulse increment output by a photoelectric encoder on the left (right) drive wheel.
S330: The robot moves from the state St−1 =(xt−1, yt−1 ,θt−1) at the (t−1 )th moment to the state St =(xt,yt,θt) at the tth moment, and the pose increment of the robot is computed with the displacement increment of the wheel:
where, β represents a distance between the left drive wheel and the right drive wheel.
S340: In case of a difference |Δθt>0between direction angles at an end pose and a start pose of the robot, the positional information of the robot is obtained.
S400: The positional information of the robot at the tth moment in Step S300 is processed with a GLM to obtain a predicted path of the robot at the tth moment:
GLM={x
1
,x
2
, . . . ,x
n
,y
1
,y
2
, . . . ,y
n} (4-1)
where, x1 . . . n is an x coordinate of the predicted path estimated by the GLM, and y1 . . . nis a y coordinate of the predicted path estimated by the GLM, thereby obtaining a predicted path corresponding to each moment.
S500: An odometry error model is established with the neural network in the prior art:
E=Σ
n
|O−RSSI|2 (5-1)
where, O represents an optimized predicted path, namely an optimized output result after the predicted path is trained by the neural network.
Referring to
S510: It is assumed that the neural network structurally includes an input layer having q neurons, a hidden layer l having neurons, and an output layer having m neurons, connections between the neurons of the input layer and the neurons of the hidden layer each have a weight value of wij , connections between the neurons of the hidden layer and the neurons of the output layer each have a weight value of wjk, the hidden layer has a bias term of a, and the output layer has a bias term of b.
S520: The GLM is taken as an input of the neural network to obtain an output result of the hidden layer:
where, pi represents a vector for coordinates (xn−1,yn−1) and (xn,yn) of the input GLM, f represents an activation function, and i={1, 2, . . . ,q}, j={1,2, . . . ,1}. In the embodiment, the activation function is the sigmoid function and is specifically expressed as:
Considering that the number of nodes on the hidden layer of the neural network has a certain impact on the final result, the number of nodes on the hidden layer is optimized:
l=√{square root over (q+m)}+ζζ∈[1, 10] (5-4)
where, l represents a number of neurons on the hidden layer. In the embodiment, the best performance can be obtained with l=13 .
A result from the output layer of the neural network is:
S530: The result from the output layer of the neural network at the tth moment and the actual path at the present moment are taken as an input of a loss function to establish the odometry error model E.
S600: A maximum number of iterations is preset, the actual path in the I moments and a corresponding predicted path are taken as an input of a neural network model to train the error model E, parameters of the model are updated through BP during training, and training is <stopped when E≤e−5 to obtain a well-trained odometry error model. The odometry error model of the robot is nonlinear. The neural network is widely recognized as a nonlinear modeling tool. The two-layer feed-forward neural network is used in the odometry error model of the robot, and sensory and perceptual training is performed on the two layers with a BP algorithm. The working principle of the BP is to apply the descent rule to the feed-forward network and optimize network parameters.
S700: Steps S300-S400 are repeated to obtain a predicted traveling path of a robot R′ to be predicted, and the predicted traveling path is input to the well-trained odometry error model in Step S600 to obtain an optimized predicted traveling path that is a predicted value for the traveling path of the robot R′.
As can be seen from the above embodiments, compared with the predicted path processed by the GLM, the final position of the robot estimated with the neural network is almost the same as the actual position of the robot computed with the RFID-based localization, as shown in
Finally, it should be noted that the above embodiment is only intended to explain, rather than to limit, the technical solution of the present disclosure. Although the present disclosure is described in detail with reference to the preferred embodiment, those ordinarily skilled in the art should understand that modifications or equivalent substitutions made to the technical solutions of the present disclosure without departing from the spirit and scope of the technical solution of the present disclosure should be included within the scope of the claims of the present disclosure.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202110448691.1 | Apr 2021 | CN | national |
