Indoor navigation system and method

Abstract

This document discusses, among other things, a portable device that generates position and orientation data to facilitate movement within a building. The device can be, for example, handheld or affixed to a cane. Data for navigation is derived from reflections from interior surfaces and dead reckoning sensors.

Description

TECHNICAL FIELD

This document pertains generally to navigation, and more particularly, but not by way of limitation, to indoor navigation.

BACKGROUND

Human mobility entails execution of several tasks including path planning, navigation, and obstacle avoidance. Visually impaired people cannot accurately assess the surrounding environment. Seeing-eye dogs and white canes are widely used for the purpose of guidance and environment sensing. The former, however, has costly and often prohibitive training requirements, while the latter can only provide cues about ones immediate surroundings.

Traditional systems for navigating within a building are generally costly or ineffective. For example, the installation and operating costs associated with an installed base of radio frequency markers within a building are substantial barriers not readily overcome. In addition, poor reception of radio frequency navigation signals within a building, such as that used by satellite-based navigation systems (for example, global positioning systems), precludes widespread acceptance.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, which are not necessarily drawn to scale, like numerals describe substantially similar components throughout the several views. Like numerals having different letter suffixes represent different instances of substantially similar components. The drawings illustrate generally, by way of example, but not by way of limitation, various embodiments discussed in the present document.

FIGS. 1 and 2 include block diagrams of an indoor navigation system.

FIG. 3 includes an illustration of a cane mounted indoor navigation system.

FIG. 4 depicts a trace of attitude covariance of a cane with respect to time.

FIG. 5 depicts measurement residuals based on the trace of FIG. 4.

FIG. 6 depicts the yaw component of a cane's attitude estimate before and after low-pass filtering.

FIG. 7 depicts a trace of position covariance for a cane.

FIGS. 8A and 8B depicts x and y axis position measurement residuals for a cane.

FIG. 9 illustrates an estimated trajectory and a floor diagram.

FIG. 10 illustrates a method according to the present subject matter.

DETAILED DESCRIPTION

The following detailed description includes references to the accompanying drawings, which form a part of the detailed description. The drawings show, by way of illustration, specific embodiments in which the present subject matter may be practiced. These embodiments, which are also referred to herein as “examples,” are described in enough detail to enable those skilled in the art to practice the present subject matter. The embodiments may be combined, other embodiments may be utilized, or structural, logical and electrical changes may be made without departing from the scope of the present subject matter. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present subject matter is defined by the appended claims and their equivalents.

In this document, the terms “a” or “an” are used, as is common in patent documents, to include one or more than one. In this document, the term “or” is used to refer to a nonexclusive or, unless otherwise indicated. Furthermore, all publications, patents, and patent documents referred to in this document are incorporated by reference herein in their entirety, as though individually incorporated by reference. In the event of inconsistent usages between this document and those documents so incorporated by reference, the usage in the incorporated reference(s) should be considered supplementary to that of this document; for irreconcilable inconsistencies, the usage in this document controls.

The present subject matter provides information derived from environmental sensing (e.g., position, orientation, or local geometry) based on the use of appropriate sensors and sensor fusion algorithms.

One example of the present subject matter includes a portable system for indoor human localization having a pedometer and a white cane, on which a laser range finder, and a 3-axis gyroscope are disposed. An algorithm executed by a processor estimates the heading and position of a person navigating in an indoor space. The algorithm includes a two-layered, Extended Kalman Filter (EKF) for attitude and position estimation wherein a first layer maintains an attitude estimate of the white cane and a second layer estimates a position of the user.

One example of the present subject matter includes a sensor package configured for mounting on a white cane. An exemplary embodiment of this configuration is unobtrusive to the user, is operable without calibration to a particular body type, and preserves the ability of the user to physically touch the environment with the white cane.

In one example, a pair of sensors (a laser scanner and a 3-axis gyroscope) are mounted beneath the white cane and provide attitude information about the cane, and a lightweight, foot-mounted pedometer measures the user's walking speed. Information from these three sources can be fused in a single pose estimator. All the sensors, however, can move in three dimensions (3-D) and the coordinate transformation from the pedometer to the laser/gyro is unknown and time varying.

The present subject matter includes a two-stage (2.5-D) estimator. In the first stage, rotational velocity measurements from the 3-axis gyroscope are combined with relative attitude measurements inferred from the laser scan data to estimate the 3-D attitude of the cane. The second stage incorporates corner features extracted from the laser data, linear velocity measurements from the pedometer, and a filtered version of the cane's yaw to compute 2-D pose estimates of the user. The laser data is utilized in both filters and statistical correlations in the estimates are avoided by using the even-indexed data points in the attitude filter, and the odd-indexed data points in the position filter. By exploiting a priori information about the location of environmental features (corners), and considering that the primary structural planes (floor, ceiling, walls) of a building lie perpendicular to each other, an example of the method generates a reliable localization estimate of a human traveling indoors.

FIG. 1 illustrates block diagram of system 100A configured for indoor navigation. Data received from rotational velocity sensor 30A and distance sensor 40A is processed by processor 50A to generate navigation data that is rendered using output device 60. Rotational velocity sensor 30A can include a gyroscope, an inertial measurement unit, an accelerometer, an inclinometer, a magnetometer or other device that generates data corresponding to rotational velocity. Distance sensor 40A can include a range finding sensor such as a laser-based or ultrasonic-based sensor or other distance measuring equipment. In various examples, distance sensor 40A includes an acoustical sensor or an optical flow sensor or other camera-based sensor.

Processor 50A, in various examples, includes a handheld computer, such as a personal digital assistant (PDA), a laptop computer or other processor.

Output device 60 can include a headphone, a speaker or other audio transducer. In addition, output device 60 can include a haptic transducer that provides a human-perceivable signal corresponding to navigation data derived from processor 50A.

FIG. 2 illustrates a block diagram of system 100B having rotational velocity sensor 30A and distance sensor 40B coupled to processor 50B. In the figure, distance sensor 40B includes range sensor 42A and linear velocity sensor 44A. Range sensor 42A, in one example, includes a laser-based rangefinder. Linear velocity sensor 44A includes a pedometer, an accelerometer or an optical flow sensor.

Processor 50B executes a filter algorithm as described elsewhere in this document and in various examples, includes an extended Kalman filter. In the figure, processor 50B includes heading estimator 52 and position estimator 54. Heading estimator 52 includes a filter that generates an estimate of heading based on data from rotational velocity sensor 30A and data from range sensor 42A. Range sensor 42A provides data based on selected point locations and line segments.

Data corresponding to the heading estimate is provided to position estimator 54. Position estimator 54 receives inputs from linear velocity sensor 44A and, in the example shown, data from range sensor 42A. Data provided by range sensor 42A can include features and landmarks.

In addition, processor 50B is shown coupled to additional data input 70. In one example, additional data input 70 includes a digital file corresponding to a stored map. The stored map data can be received wirelessly by a transceiver or from a user-accessible storage device.

FIG. 3 illustrates exemplary system 300. In the example illustrated, sensor 30B and sensor 42B are mounted using housing 20. Housing 20 is shown affixed to white cane 10. Housing 20 also includes power module 80 which, in various examples includes a rechargeable or replaceable battery. Processor 50A is coupled to power module 80 as well as memory 55. Memory 55 can provide storage for instructions used by processor 50A in executing an algorithm as well as data storage corresponding to the navigation data generated by processor 50A. Wireless interface 75 is coupled to processor 50A and in various examples, includes a Bluetooth transceiver or other radio frequency telemetry link. In the example illustrated, output device 60 is coupled by wireless interface 75.

Linear velocity sensor 44B, in the example illustrated, includes a pedometer configured for attachment to the user. In various examples, the pedometer is attached to a leg, hip or foot. Sensors 30B, 42B and 44B are interfaced to processor 50A via USB, RS-232, and Bluetooth, respectively. The real-time software components are written in C++, and the software for simulation and data plotting is written in Matlab. The sensor bay, or housing 20, is mounted beneath white cane 10 which measures 1.27 m when extended and 0.33 m when retracted. In one example, cane 10 is a light-weight carbon fiber cane and the total weight, including sensors, is approximately the same as a standard white cane.

Laser scanner of sensor 42B, in one example, includes an URG-X002S which measures 5 cm by 5 cm by 7 cm. It has an angular scan range of 240 degrees, with an accuracy of ±1% of the measurement for distances 1 m to 4 m. Closer than 1 m, the measurement accuracy is ±10 mm. The laser scanner can measure distances ranging from 0.02 m to 4 m. The scanner weighs 160 g and consumes 2.5 W at 5V.

The 3-axis gyroscope of sensor 30B, in one example, includes an ISIS Inertial Measurement Unit (IMU), with an angular-rate range of ±90 deg/sec. Over an RS-232 connection, the ISIS IMU provides measurements at 100 Hz. The weight of the sensor is 363 g, and the power consumption is 6.72 W at 12V. The IMU measures 5.5 cm by 6.5 cm by 7 cm.

FIG. 3 also illustrates pedometer of sensor 44B which provides linear velocity measurements as the person walks. The size of this sensor is 8.5 cm by 3.5 cm by 3.5 cm. Communication packets are received via Bluetooth at a rate of 1 Hz.

Method

An exemplary method includes generating an attitude estimate of the white cane, determining the heading direction and determining a position of the person. The attitude of the white cane is estimated using a 3-axis gyroscope, and laser-scan measurements of structural planes in the building. The heading direction of the person is extracted from the yaw component of the white cane's attitude estimate with the use of a low-pass filter. This provides a heading measurement to the position filter. The position of the person is estimated using the heading estimates from the low-pass filter, the linear velocity measurements from the pedometer, and the relative coordinates of known corner features detected by the laser scanner.

A. Attitude Estimation of the White Cane

Estimating the attitude of an object in 3-D entails parametrization of the elements involved. One method for specifying an orientation in 3-D is by means of roll, pitch, and yaw (rpy) angles. While the rpy representation is intuitive and readily visualized, it suffers from singularities which can cause instability in an estimator. A more robust attitude representation is the quaternion of rotation

$\begin{array}{cc}q={\begin{array}{cc}[\hat{k}\sin \frac{\theta }{2}& \cos \frac{\theta }{2}]\end{array}}^T& \left(1\right)\end{array}$ where {circumflex over (k)} is the axis of rotation, and θ signifies the magnitude of rotation. The quaternion representation of an attitude is compact and singularity-free. For clarity, the quaternion notation employed in this paper results in “natural order” quaternion multiplication. As such, the symbol denotes quaternion multiplication fulfilling _L3^L1q=_L2^L1q_L3^L2q, which is the attitude rotation between successive frames.

Attitude estimation is accomplished through the use of an EKF which fuses measurements from proprioceptive and exteroceptive sensing devices. Rotational velocity measurements from a 3-axis gyroscope are integrated to propagate the attitude estimate, and lines extracted from the laser-scan data are used to update the computed estimate.

1) Attitude Propagation

Propagation of the attitude estimate is performed through the incorporation of proprioceptive information and knowledge about the state propagation model of the system. The gyroscope measures the rotational velocities about each of the local coordinate axes; these velocities are integrated at each time step to generate a new attitude estimate (cf. (4)). 4)). The state vector X_k includes the quaternion q and the gyroscope bias ^b. The error state {tilde over (X)}_k includes the attitude angle-error vector δθ and the gyroscope bias error state Δb=b−{circumflex over (b)}.

The state vector X_k is 7×1 and the error state {tilde over (X)}_k is 6×1. Some EKF formulations maintain equal sized state and error state vectors. The quaternion of rotation, however, is defined to have unit length which causes the corresponding covariance matrix to lose rank. To account for this, only the attitude angle-error vector δθ (cf. (2)) is used in the error state.

Some EKF applications employ an additive error model. It is convenient though, when using quaternions, to quantify the error as a multiplicative error term

$\begin{array}{cc}\delta q=q\otimes {\hat{q}}^{-1}\cong \left[\begin{array}{c}\frac{1}{2}\mathrm{\delta \theta }\\ 1\end{array}\right]& \left(2\right)\end{array}$

The error quaternion δq denotes a small rotational error between the true, q, and the estimated, {circumflex over (q)}, attitude of the cane.

The state model for the quaternion representation of attitude is governed by the quaternion time derivative (3), which is computed in terms of the instantaneous rotational velocity ω.

$\begin{array}{cc}\stackrel{.}{q}\left(t\right)=\frac{1}{2}\Omega \left(\omega \right)q\left(t\right)\Omega \left(\omega \right)=\left[\begin{array}{cc}-\lfloor \omega \times \rfloor & \omega \\ -\omega & 0\end{array}\right]& \left(3\right)\end{array}$ where └ωx┘ denotes the skew symmetric cross-product matrix

$\lfloor \omega \times \rfloor =\left[\begin{array}{ccc}0& -{\omega }_3& {\omega }_2\\ {\omega }_3& 0& -{\omega }_1\\ -{\omega }_2& {\omega }_1& 0\end{array}\right]$

Propagating the quaternion from time step k to k+1 entails quaternion integration

$\begin{array}{cc}{q}_{k+1}=\left[\begin{array}{c}\frac{\omega }{\omega }\sin \left(\frac{\omega }{2}\Delta t\right)\\ \cos \left(\frac{\omega }{2}\Delta t\right)\end{array}\right]\otimes q_k\omega =\sqrt{{\omega }^T\omega }& \left(4\right)\end{array}$

The gyroscope noise model dictates that angular velocity measurements are corrupted by sensor bias b, as well as measurement noise n_r. The gyroscope measurement is:ω_m=ω+b+n_r  (5)where the turn rate noise n_r is distributed as zero-mean white Gaussian with covariance σ_r²I_3×3. The sensor bias b is modeled as a random walk with{dot over (b)}=n_ω  (6)where n_ω is also distributed as zero-mean white Gaussian noise with covariance σ_ω²I_3×3.

The error state equation is computed by taking the derivative of the error state:

$\begin{array}{cc}\stackrel{\stackrel{.}{~}}{X}=F_c·\tilde{X}+G_c·n\begin{array}{c}\left[\begin{array}{c}\delta \stackrel{.}{\theta }\\ \Delta \stackrel{.}{b}\end{array}\right]=\left[\begin{array}{cc}-\lfloor \hat{\omega }\times \rfloor & -I_{3\times 3}\\ 0_{3\times 3}& 0_{3\times 3}\end{array}\right]·\left[\begin{array}{c}\mathrm{\delta \theta }\\ \Delta b\end{array}\right]+\\ \left[\begin{array}{cc}-I_{3\times 3}& 0_{3\times 3}\\ 0_{3\times 3}& I_{3\times 3}\end{array}\right]·\left[\begin{array}{c}{n}_r\\ n_{\omega }\end{array}\right]\end{array}& \left(7\right)\end{array}$ where the matrix F_c is the continuous-time system matrix, and G_c is the corresponding noise matrix. The matrices 0_m×n and I_m×n are the m×n zero, and identity matrices, respectively.

While knowledge of the continuous-time system and noise matrices is useful for attaining theoretical results, implementation on a digital computer requires the corresponding discrete-time matrices. For this reason, the above equations are discretized to obtain the discrete-time Kalman filter equations. The discrete-time state transition matrix is defined in terms of its continuous-time counterpart as:

$\begin{array}{cc}\Phi \left(t_{k+1},t_k\right)=e^{{\int }_{t_k}^{t_{k+1}}{F}_c\left(\tau \right)d\tau }& \left(8\right)\end{array}$

Attaining the discrete-time noise matrix Q_d, is accomplished by integrating the equivalent continuous-time stochastic model over one time step.

$\begin{array}{cc}{Q}_d={\int }_k^{k+1}\Phi \left(t_{k+1},\tau \right)G_c\left(\tau \right)Q_c{G}_c^T\left(\tau \right){\Phi }^T\left(t_{k+1},\tau \right)d\tau & \left(9\right)\\ \mathrm{where}{Q}_c=\left[\begin{array}{cc}{\sigma }_r^2{I}_{3\times 3}& 0_{3\times 3}\\ 0_{3\times 3}& {\sigma }_{\omega }^2{I}_{3\times 3}\end{array}\right]& \end{array}$

With the required matrices now defined, the propagated state is computed in the following manner; the bias estimate is:{circumflex over (b)}_k+1|k={circumflex over (b)}_k|k  (10)

The turn rate measurement is estimated by subtracting the estimated bias from the actual measurement{circumflex over (ω)}_k|k=ω_mk−{circumflex over (b)}_k|k  (11)

The quaternion estimate is propagated using the quaternion integrator (4), and the current estimate of the turn rate {circumflex over (ω)}_k|k.

After computing Φ (cf. (8)) and Q_d (cf. (9)), the covariance propagation is completed using the general form of the EKF covariance propagationP_k+1|k=ΦP_k|kΦ^T+Q_d  (12)

2) Attitude Update

Even-indexed laser scan data points from a laser range finder are employed to measure the relative orientation between the sensor frame of reference {S} and the global frame {G}. Specifically, the laser sensing plane intersects the planar surfaces inside a building (e.g., walls, floor, ceiling) along lines which can be reliably detected and extracted from the laser data. The direction of each of the extracted lines is processed as a measurement for updating the attitude estimates.

While inside a building, frame {G} can be assigned such that its principal axes {e1,e2,e3} are perpendicular to the prominent structural planes of the building (i.e., e1⊥Wallx, e2⊥Wally, and e3⊥Floor, Ceiling). Measurements to these planes are denoted by their corresponding unit-vectors (e.g., an x-measurement is a measured line which is perpendicular to the e1 of the global frame). Let e_iε{e1,e2,e3} be one of the three orthogonal unit vectors which span the frame {G}, and let ^Gl denote the (unit vector) direction of the line of intersection between the laser-scan plane and the measured plane with respect to {G}. Then from the geometric constraint, their inner product should be zero.e_i^TGl=0,i=1,2,3  (13)

The measurement equation is derived by rewriting the constraint (cf. (13)) using the transformation relation ^Gl=_S^GC(q)^Sl=C^T(q)^Sl, i.e.:z=e_i^TC^T(q)^Sl=0  (14){circumflex over (z)}=e_i^TC^T({circumflex over (q)})^Sl_m  (15)the rotation matrix C^T(q) denotes the rotation from frame {S} to frame {G} Note that since ^Sl is the unit vector direction of a line on the x-y plane of the laser sensor frame, it can be written as ^Sl=[sin φ−cos φ0]^T where φ is the complimentary angle to the line direction. In order to derive the measurement matrix, the true line direction is replaced by its equivalent representation ^Sl=^Sl_m+^S{tilde over (l)}. Here, ^Sl_m denotes the measured line direction and ^S{tilde over (l)}=└e3x┘^Sl_mφ is the error in this measurement; the true rotation matrix is approximated asC^T(q)=C^T(δq{circumflex over (q)})≅C^T({circumflex over (q)})+C^T({circumflex over (q)})└δθx┘

Using (14) and (15), the measurement error {tilde over (z)}=z−{circumflex over (z)} is:

$\begin{array}{cc}\begin{array}{c}\tilde{z}\cong -e_i^T{C}^T\left(\hat{q}\right)\lfloor l_m\times \rfloor \mathrm{\delta \theta }+e_i^T{C^T\left(\hat{q}\right)}^S\tilde{l}\\ =\left[-e_i^T{C}^T\left(\hat{q}\right)\lfloor l_m\times \rfloor 0_{1\times 3}\right]\left[\begin{array}{c}\delta \theta \\ \Delta b\end{array}\right]+e_i^T{C}^T\left(\hat{q}\right)\lfloor \mathrm{e3}\times \rfloor l_m\tilde{\varphi }\\ =h^T\tilde{X}+n\end{array}& \left(16\right)\end{array}$

The line direction error {tilde over (φ)} accounts for measurement noise as well as inaccuracies in line fitting. Various line fitting methods and the corresponding error models can be utilized.

Updating the attitude filter is done by first computing the residualr=z−e_i^TC^T({circumflex over (q)})^Sl_m=−e_i^TC^T({circumflex over (q)})^Sl_m  (17)

Second, the covariance of the residual (cf. (18)), and the Kalman gain (cf. (19)) are computed:S=h^TPh+R  (18)k=PhS⁻¹  (19)where R=E{n²}=(e_i^TC^T({circumflex over (q)})└e3x┘^Sl_m)²σ_φ² and σ_φ²=E{{tilde over (φ)}²}. The error state update is computed as

$\begin{array}{cc}\Delta \hat{x}(+)=\left[\begin{array}{c}\delta \hat{\theta }(+)\\ \Delta \hat{b}(+)\end{array}\right]=\left[\begin{array}{c}2·\delta \hat{q}(+)\\ \Delta \hat{b}(+)\end{array}\right]=\mathrm{kr}& \left(20\right)\end{array}$ The error quaternion estimate is

$\begin{array}{cc}\delta \hat{q}=\frac{1}{\sqrt{1+\delta {\hat{q}(+)}^T\delta \hat{q}(+)}}·\left[\begin{array}{c}\delta \hat{q}(+)\\ 1\end{array}\right]& \left(21\right)\end{array}$ The quaternion estimate, and the bias estimate are updated as{circumflex over (q)}_k+1|k+1=δ{circumflex over (q)}{circumflex over (q)}_k+1|k  (22){circumflex over (b)}_k+1|k+1={circumflex over (b)}_k+1|k+Δ{circumflex over (b)}(+)  (23)

The last step in the attitude filter update is to update the covariance [22]:P_k+1|k+1=(I−kh^T)P_k+1|k(I−kh^T)^T+kRk^T  (24)

3) Attitude Observability

Due to physical limitations of planar laser scanning, it is impossible to resolve all three degrees of rotational freedom from a stationary vantage point. The attitude of the cane, however, is stochastically observable when the cane is in motion. The observability requirements are satisfied when the laser sensor detects all three main directions in space, at different time steps, over a given time interval.

Since observability of the attitude entails observability of the bias, a simplified measurement model is used with only the quaternion as state variable, and measurement matrixh_i^T=└−e_i^TC^T({circumflex over (q)})└^Sl_mx┘┘  (25)

In order to establish stochastic observability of the attitude, it suffices to show that the observability Gramian

$\begin{array}{cc}M=\sum _{\mu =v-N+1}^v\sum _i{\Phi }_{t_{\mu },t_v}^T{h}_{i,t_{\mu }}{R}_{i,t_{\mu }}^{-1}{h}_{i,t_{\mu }}^T{\Phi }_{t_{\mu },t_v}& \left(26\right)\end{array}$ is of full rank for all v≧N [24].

Noting that the state transition matrix equals Φ_tμ,tv=C(_Sv^Sμq), the observability matrix can be expressed, after pre- and postmultiplication with the proper rotational matrices, as

$\begin{array}{cc}\begin{array}{c}M=\sum _{i,\mu }{R}_{i,t_{\mu }}^{-1}{C}^T\left({\hat{q}}_{t_{\mu }}\right)h_{i,t_{\mu }}{h}_{i,t_{\mu }}^{T}C\left({\hat{q}}_{t_{\mu }}\right)\\ =\sum _{i,\mu }{R}_{i,t_{\mu }}^{-1}\left(l_{i_m,t_{\mu }}\times e_i\right){\left(l_{i_m,t_{\mu }}\times e_i\right)}^T\end{array}& \left(27\right)\end{array}$ that is, as a weighted sum of outer products of vectors ^Gl_im. If, over a period of time, the sensor observes surfaces with normals e_i that span R³, and recalling that ^Gl_i^TGe_i=0(cf. (13)), then the vectors ^Gh_i also span the 3D-space, ensuring M to be of full rank.

FIG. 4 depicts graphic 240 showing the trace of the attitude covariance matrix with respect to time. As shown in graphic 240, while the cane is stationary (during bias initialization) its attitude is initially unobservable, however, it becomes observable while the cane is in motion.

During this experiment, the white cane was initially stationary for 10 sec for the purpose of gyroscope bias initialization. As evident from the experimental results, the trace of the attitude covariance becomes bounded (as expected since the system is observable) once the cane is in motion. In order to ensure consistency of the EKF estimator, the measurement residuals should lie within the 3σ bounds of the residual covariance. This is verified by the results shown in graphic 250 of FIG. 5. Line measurement residual 252 and 3σ bounds 254 are shown in this plot from the residual covariance. The units of the vertical-axis are omitted considering that this measurement residual is by nature unit-less (it is the difference of unit vector dot-products).

B. Heading Estimation

By estimating the attitude of the cane, the orientation of the person can be inferred. During regular operation, the yaw angle of the cane will be an asymmetric cyclostationary random process in which the amplitude, phase, frequency and degree of asymmetry may all change (FIG. 6). These characteristics result from swinging the white cane which helps the person identify a clear walking path. Extracting the heading direction of the person from the cane's yaw signal can be accomplished through signal processing. To facilitate real time operation, the following method can be used.

The heading of a person can be approximated as the mean value of the cane yaw over a period of swinging. One solution includes a combination of frequency estimation, and low pass filtering to remove the high frequency component of the yaw. This motivates the use of a Finite Impulse Response (FIR) filter. The attitude EKF provides yaw estimates at 100 Hz. Due to the nature of the motion of the cane, the high frequency component of the yaw does not fall below 0.5 Hz. In order to reduce the number of taps needed by the filter, the yaw signal is downsampled by a factor of 40. The signal is then filtered using a 7th order FIR filter with Kaiser window β=0.5, and a normalized cut-off frequency of 0.02.

FIG. 6 depicts graphic 260 showing the yaw component of the cane's attitude estimate (262) along with the filtered version (264). A byproduct of filtering the yaw signal is that a delay is introduced into the system. Although the filter has only seven coefficient-delay pairs, there is a delay of 1.2 sec because the downsampled yaw signal has a sampling frequency of 2.5 Hz. As a result, an equivalent time-lag exists in the position estimate. However, due to the relatively slow walking rate of a person carrying a white cane, this delay is acceptable for real-time operation.

C. Position Estimation of the Person

An EKF is employed for the purpose of map-based human localization. Estimating the position of a person within a building can be treated as a 2-D position estimation problem in which each floor of the building is a separate environment containing landmarks, in this case corner features, whose position is known. While a person is traversing a single floor, their motion will be constrained on the plane of that floor. This allows for the use of a generic odometry propagation framework.

1) Position Propagation

The non-holonomic formulation of the odometry state equations typically uses the linear and rotational velocity measurements during the propagation stage. These constraints are relaxed for the case of a person. Linear velocity measurements are provided by a foot-mounted wireless pedometer. Note that rotational velocity measurements need not be accessible as the person's heading direction is available from the FIR filter. This odometric model is referred to as direct heading odometry.x_k+1=x_k+Vδt cos ψ,y_k+1=y_k+Vδt sin ψ{circumflex over (x)}_k+1={circumflex over (x)}_k+Vδt cos ψ_m,ŷ_k+1=ŷ_k+Vδt sin ψ_m where x, y, and ψ are the position and orientation of the person, and V is the average velocity during the time interval δt. In the above equations “^” denotes estimates while the subscript m refers to measured quantities, i.e.,V_m=V+ω_v,ψ_m=ψ+ω_ψwhere the velocity, ω_v, and heading, ω_ψ, noises are zero-mean white Gaussian processes with variances σ_v² and σ_ψ² respectively. The error model based on these relations is:

$\begin{array}{cc}\left[\begin{array}{c}{\tilde{x}}_{k+1}\\ {\tilde{y}}_{k+1}\end{array}\right]=\left[\begin{array}{c}{\tilde{x}}_k\\ {\tilde{y}}_k\end{array}\right]+\left[\begin{array}{cc}-\delta t\cos {\psi }_m& V_m\delta t\sin {\psi }_m\\ -\delta t\sin {\psi }_m& -V_m\delta t\cos {\psi }_m\end{array}\right]\left[\begin{array}{c}{\omega }_v\\ {\omega }_{\psi }\end{array}\right]{\tilde{X}}_{k+1}={\tilde{X}}_k+\Gamma n& \left(28\right)\end{array}$

The covariance propagation is computed as

$\begin{array}{cc}{P}_{k+1k}=P_{kk}+\Gamma Q{\Gamma }^T,\mathrm{where}Q=\left[\begin{array}{cc}{\sigma }_v^2& 0\\ 0& {\sigma }_{\psi }^2\end{array}\right]& \left(29\right)\end{array}$

2) Position Update

The person's position estimate is updated by incorporating relative position measurements to known landmarks in the environment. Although the selection of features is arbitrary, using corners at hallway intersections is a good choice for an indoor environment because they occur with a relatively high frequency, and they can be extracted reliably from the laser-scan data. By extracting lines from the odd-indexed laser scan data points, corners are identified with the following characteristics: (i) two lines must be nearly perpendicular, (ii) the endpoints of the lines must be within 5 cm of each other, and (iii) the line orientations must match the possible wall orientations in the environment.

The relative position measurement is written as a 3-D vector from the sensor to the landmark which is aligned to the sensor frame and projected down to 2-D:

$\begin{array}{cc}z=\Pi C\left(q\right)\left(p_{L_i}-p_S\right),\mathrm{where}\Pi =\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\end{array}\right]& \left(30\right)\\ \hat{z}=\Pi C\left(\hat{q}\right)\left(p_{L_i}-\hat{p}_S\right)& \left(31\right)\\ \begin{array}{c}\tilde{z}\cong -\Pi C\left(\hat{q}\right)\tilde{p}_S+\Pi \lfloor C\left(\hat{q}\right)\Delta \hat{p}\times \rfloor \delta \theta +\Pi n_p\\ =H\tilde{X}+\mathrm{\Gamma \delta \theta }+\Pi n_p\end{array}& \left(32\right)\end{array}$

The vector Δ{circumflex over (p)} in (32) is the difference between the true position of the ith landmark ^Gp_Li and the measured location ^Gp_Lim. The measurement noise, denoted n_p, is due to noise in the laser scan and linearization errors, and has covariance R_p. The position of the sensor S in frame {G} is denoted as ^Gp_S.

The measurement update is performed by computing the measurement residualr_k+1=z_k+1−{circumflex over (z)}_k+1  (33)the covariance of the residualS_k+1=H_k+1P_k+1|kH_k+1^T+πR_pπ^T  (34)and the Kalman gainK_k+1=P_k+1|kH_k+1^TS_k+1⁻¹  (35)

With the matrices computed, the state estimate is updated using{circumflex over (X)}_k+1|k+1={circumflex over (X)}_k+1|k+K_k+1(z_k+1−{circumflex over (z)}_k+1)  (36)Lastly, the covariance is updated usingP_k+1|k+1=(I_2×2−K_k+1H_k+1)P_k+1|k(I_2×2−K_k+1H_k+1)^T+K_k+1RK_k+1^T  (37)

The system and method described herein can be used in an indoor environment such as the closed loop path shown in FIG. 9. The closed loop has a path length of 130 m. Twenty-one corners along this loop are known a priori from the building blueprints or from another source and are used as features for position updates. While walking around, the user swings the cane to and fro in a natural manner while searching for obstacles which might lie in the path. FIG. 9 shows the estimated trajectory super-imposed on the floor diagram.

FIG. 9 shows data for a 130 m test run, starting in the lower left corner, traveling counter-clockwise and ending back at the start location. The striped regions on the figure depict obstacles (such as furniture or garbage cans) and other objects not represented in the building blueprint.

Additionally, some of the doors along the hallway can be open, while others are closed. The data illustrated represents conditions of normal flow of pedestrian traffic through the hallways. The corners in the map are marked with a box, and measurements are used to update the position estimate as marked with a line to the corresponding corner.

Based on the data illustrated, the uncertainty in the position estimate is low (max σ=0.16 m), even though the number of position update measurements is small (only 9 corners were detected, approximately 110 total measurements). For the data shown, the attitude estimate of the cane is based on over 5,000 relative orientation measurements. When the orientation error is bounded, i.e., σ_ψ≦σ_ψ0, then the position covariance grows as:

$\begin{array}{cc}P\left(t\right)\approx \frac{{\sigma }_v^2+{\sigma }_{\psi 0}^2{V}^2\delta T^2}{2}t=\alpha t& \left(38\right)\end{array}$

For the data shown, δT=100 ms and α=9.8204×10⁻⁴. This means that for the case of direct heading odometry, the position uncertainty grows approximately linearly with time between consecutive position updates. Thus, when detecting a small number of corners, the position filter maintains a good estimate. This is corroborated by the time evolution of the trace of the position covariance; the value of the trace remains below 0.054 m2 which corresponds to approximately 0.16 m 1σ error in each direction. FIG. 7 illustrates that trace of the position covariance verifies that the positional uncertainty remains bounded, but grows linearly with time between consecutive position updates.

The filter maintains consistency as the x and y components of the measurement residuals fall within the 3σ bounds of the residual covariance, as shown in FIGS. 8A and 8B.

The present subject matter includes processing of data from a pair of cane-mounted sensors, and a foot-mounted pedometer involving unknown and time-varying relative coordinate transformation fused using a two-stage pose estimator. The first stage uses inertial measurements from a 3-axis gyroscope and relative orientation measurements from laser scan data to accurately estimate the attitude of the white cane. The second stage generates a pose estimate of the person holding the cane, by incorporating linear velocity measurements from the pedometer, a filtered version of the cane's yaw estimate, and corner features extracted from the laser scan data.

FIG. 10 illustrates method 370 according to one example. In method 370, rotational velocity data is received at 310 and distance data is received at 320. A processor executes an algorithm at 330 to filter the data and generate aheading estimate. At 340,velocity data is received and at 350, feature data is received. At 360, the processor generates a position estimate.

ALTERNATIVE EXAMPLES

The present subject matter can be used for path planning by suitable programming of the processor. Path planning can entail generating paths between indoor locations, as well as a haptic feedback system to give the person simple directional commands. A haptic feedback system can include piezoelectric or other vibration inducing transducers.

In one example, the present subject matter can be used to generate a map of an interior space or building where no map is otherwise available.

As used in this document, pose refers to orientation (attitude) and position of a mobile or portable platform with respect to an initial location. The platform can be attached to or carried by a user. In one example, orientation (attitude) is estimated by integrating the rotational velocities (measured by gyroscopes).

In various examples of the present subject matter, attitude corrections are provided intermittently from a) the lines detected by laser scanning measurement (intersection of laser scanning plane with the walls, ceiling and floor); and b) the inclinometers (or equivalently, accelerometers) that measure device tilt angle.

Position can be computed by integrating the distance traveled and/or velocity measurements from the pedometer with the orientation estimate.

Map-based localization affords the ability to intermittently provide corrections to the pose estimator. Such a system uses measurements from laser scanner (and/or camera sensor) and makes reference to any available map.

In one example, the present subject matter allows simultaneous localization and mapping wherein the system both constructs and uses a map of the area.

As used herein, path planning refers to determining and communicating a route to the user.

CONCLUSION

It is to be understood that the above description is intended to be illustrative, and not restrictive. For example, the above-described embodiments (and/or aspects thereof) may be used in combination with each other. Many other embodiments will be apparent to those of skill in the art upon reviewing the above description. The scope of the subject matter should, therefore, be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. In the appended claims, the terms “including” and “in which” are used as the plain-English equivalents of the respective terms “comprising” and “wherein.” Also, in the following claims, the terms “including” and “comprising” are open-ended, that is, a system, device, article, or process that includes elements in addition to those listed after such a term in a claim are still deemed to fall within the scope of that claim. Moreover, in the following claims, the terms “first,” “second,” and “third,” etc. are used merely as labels, and are not intended to impose numerical requirements on their objects.

The Abstract of the Disclosure is provided to comply with 37 C.F.R. § 1.72(b), requiring an abstract that will allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, various features may be grouped together to streamline the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter may lie in less than all features of a single disclosed embodiment. Thus the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separate embodiment.

Claims

1. A portable system comprising: a first sensor configured to provide rotational velocity data;a second sensor configured to provide distance data based on a reflected signal; anda processor coupled to the first sensor and the second sensor and configured to execute an algorithm using the rotation velocity data and the distance data to determine at least one of a position relative to a landmark and a heading relative to the landmark.

2. The system of claim 1 wherein the first sensor includes at least one of a gyroscope, an inertia-based sensor and an accelerometer.

3. The system of claim 1 wherein the second sensor includes at least one of a laser and an ultrasonic transceiver.

4. The system of claim 1 further including a distance measuring device coupled to the processor.

5. The system of claim 4 wherein the distance measuring device includes a pedometer.

6. The system of claim 1 further including an output device coupled to the processor and wherein the output device is configured to render a human perceivable output based on at least one of the position and the heading.

7. The system of claim 6 wherein the output device includes at least one of a haptic transducer, an audio transducer, a memory and a transceiver.

8. A portable system comprising: a scanner to provide distance data based on a distance to a surface;a gyroscope to provide rotational data based on rotational movement;a pedometer to provide travel data based on linear movement;a processor in communication with the scanner, the gyroscope, and the pedometer and having instructions for determining navigation data based on the distance data, the rotational data, and the travel data; andan output device in communication with the processor and configured to render the navigation data in a human perceivable manner.

9. The system of claim 8 wherein the scanner includes at least one of a laser scanner and an ultrasonic scanner.

10. The system of claim 8 wherein the gyroscope is sensitive to rotation about three axes.

11. The system of claim 8 wherein the pedometer includes an accelerometer.

12. The system of claim 8 wherein the processor includes a personal data assistant.

13. The system of claim 8 wherein at least one of the scanner and the gyroscope is coupled to the processor by a wired connection.

14. The system of claim 8 wherein the pedometer is coupled to the processor by a wireless connection.

15. The system of claim 8 wherein the output device includes at least one of a display, a memory device, an audio transducer, and a haptic device.

16. The system of claim 8 wherein the navigation data includes a map.

17. The system of claim 8 further including: a memory coupled to the processor; andan input port coupled to the memory and further wherein map data can be received at the input port and stored in the memory.

18. The system of claim 8 further including a housing having disposed thereon at least one of the scanner, the gyroscope, the processor, and the output device.

19. The system of claim 18 wherein the housing is coupled to a cane.

20. A method comprising: receiving reflected data corresponding to a surface;receiving rotational velocity data corresponding to a rotation;receiving linear motion data;receiving a landmark location; andexecuting an algorithm using the reflected data, the rotational velocity data, and the linear motion data to determine at least one of a position relative to the landmark location and a heading relative to the landmark location.

21. The method of claim 20 wherein receiving the landmark location includes receiving a map of an interior space.

22. The method of claim 20 further including determining a route within an interior space relative to the landmark location.

23. The method of claim 20 wherein receiving the linear motion data includes at least one of linear velocity data and linear acceleration data.

24. The method of claim 20 wherein executing the algorithm includes implementing a filter.

25. The method of claim 24 wherein implementing the filter includes implementing a Kalman filter.

26. The method of claim 24 wherein implementing the filter includes implementing an extended Kalman filter.

27. A system comprising: means for generating rotational velocity data;means for generating distance data based on a reflected signal; andmeans for determining, relative to a landmark, at least one of a position and a heading based on the rotational velocity data and the distance data.

28. A system comprising: means for providing distance data based on a distance to a surface;means for providing rotational data based on a rotation;means for providing travel data based on movement;means for determining navigation data based on the distance data, the rotational data, and the travel data; andoutput means for rendering the navigation data in a human perceivable manner.