Patent Grant
6975246
The present invention relates, in general, to collision avoidance and, more specifically, to collision avoidance using a range gated camera.
A vehicle often requires knowledge of its environment. This may be accomplished by integrating information derived from multiple onboard navigation sensors, such as a GPS (Global Positioning System) receiver, an IMU (Inertial Measurement Unit), an altimeter, etc. Using such sensors, the vehicle may obtain its own position, velocity and acceleration information. Information of obstacles in the path of the vehicle may also be obtained using onboard target tracking sensors to derive position and bearing information of targets proximate to the vehicle.
By optimally integrating navigation sensors and target tracking sensors, guidance laws may be used to generate a desired trajectory for the vehicle.
Generally, target tracking sensors produce range measurements of a target with respect to the vehicle by using an active sensor, such as a radio or acoustic radar or a laser range finder. The operation of an active sensor depends on measurement of the traveling time between the transmitted signal and the received reflected signal.
In various scenarios it is desirable to maneuver unmanned aerial vehicles (UAVs) into areas for surveillance and reconnaissance. One scenario may include a low flying UAV with hover capability, navigating through a wooded area under a tree canopy. Another scenario may include a UAV with hover capability, navigating in a cave and sharply maneuvering to avoid collision with a wall. Providing autonomous collision avoidance for such a UAV is desirable by using an onboard camera having intensified video.
One advantage of using an intensified camera is the ability to gate the intensifier. Gating provides a mechanism to gather range information that may be used to form spatial three-dimensional (3D) knowledge about a region the vehicle is maneuvering through. Potential collision obstacles may be located, relative to the sensor, within the 3D space-of-regard.
When attempting to traverse a space where obstacles prevent a straight-line path, the limiting factor becomes the accuracy with which objects may be located. A range-gated camera may be used with extremely narrow range gates to locate objects in the space. This approach fails, however, if the spatial separation between obstructions becomes less than the illuminator pulse width, times one half the speed of light. In a scenario of a UAV navigating through a wooded area or a cave, the separation between obstructions may be on the order of 2 feet, thus requiring an illuminator pulse of less than five nanoseconds (ns). Acquisition of a small, low cost illuminator with a pulse width this short, and with enough intensity to generate a usable image, presents a major hurdle in developing a low cost UAV with collision avoidance. A different approach is, therefore, needed.
The present invention addresses a collision avoidance system using a gated camera sensor, which does not require the illuminator to be high performance.
To meet this and other needs, and in view of its purposes, the present invention provides a collision avoidance system for a vehicle having a collision avoidance processor, and an active sensor for obtaining successive range measurements to objects along a path transversed by the vehicle. The active sensor includes a near range gate for obtaining a near range measurement to objects located at a range less than a first predetermined range, and a far range gate for obtaining a far range measurement to objects located at a range greater than a second predetermined range. The near and far range measurements are provided to the collision avoidance processor for maneuvering of the vehicle.
In another embodiment, the invention provides a method of collision avoidance for a vehicle including the steps of: (a) measuring a near range to objects located at a range less than a first predetermined range; (b) measuring a far range to objects located at a range greater than a second predetermined range; (c) steering the vehicle in response to the objects located at the range greater than the second predetermined range; and (d) modifying step (c) in response to the objects located at the range less than the first predetermined range.
In yet another embodiment, the invention provides a method of collision avoidance for a vehicle including the steps of: (a) imaging a scene located at a first predetermined range; (b) imaging a scene located at a second predetermined range, the second predetermined range being greater than the first predetermined range; (c) locating at least one low density region in the scene imaged at the second predetermined range; (d) selecting a low density region from the low density regions located in step (c), and using the selected low density region as a steering aim point for the vehicle; and (e) modifying at least one of direction and velocity of the vehicle, while being steered toward the steering aim point, to avoid objects located at the first predetermined range.
It is understood that the foregoing general description and the following detailed description are exemplary, but are not restrictive, of the invention.
The invention is best understood from the following detailed description when read in connection with the accompanying drawing. Included in the drawing are the following FIGS.:
Objects in a scene may be placed in 2D space using spherical coordinates, theta and phi. In order to determine position in the third dimension, rho (or range) is also required. Extracting accurate range information using a range gated camera is problematic due to limitations of the illuminator, with cost tending to increase with increases in resolution. The higher the desired range accuracy, the shorter the required illuminator pulse width.
The invention provides two special cases where range resolution is not a function of the illuminator pulse width. These special cases are at minimum range (0 to Rmin) and at maximum range (Rmax to R∞). These two ranges are referred to herein as R1 and R2, respectively, and described below.
When imaging at the minimum range, camera gating (or an intensifier in the case of I2C systems) may be used to limit R1.
The illuminator pulse may be arbitrarily large. The camera ignores light reflected off objects at ranges greater than R1. The range, R1, is defined by the present invention as follows:
R1=(Tn−T0)c/2
R2=R∞−RΔ
The camera is gated on (intensifier enabled) at 2R2/c seconds, after the end of the illuminator pulse, or Tf. The illuminator is shut off at time T2, which results in a wave pulse trailing edge. Since the illuminator pulse width, defined by (T2−T0) may be arbitrarily large, it will be appreciated that at time T2 objects at all ranges (out to R∞) return as reflected rays 12.
At ΔT=T−T2=40 ns (for example), the wave pulse trailing edge is just reaching object 10 at 40 ft, as shown in
R2=(Tf−T2)c/2
T2 is the time the illuminator is shut off.
R∞ is limited by both the amount of flux returning from objects 10 at R∞ and the on time duration of the camera gate. R2 is a function of the timing (Tf−T2). Again, the illuminator pulse width may be arbitrarily large. For both R1 and R2 the range and range resolution, in an embodiment of the invention, is determined by the timing of camera range gates 11 and 13, not the illuminator pulse width.
Having shown a manner in which limited range gated video (LRGV) information may be acquired by the invention, examples of how this incomplete, two-dimensional image data may be used to perform collision avoidance will be shown next. Referring now to
Referring next to
Since limited range gated video only provides Information about objects at near range and far range, the existence of objects between R1 and R2 are not known. Objects in the near range seem to suddenly appear as the vehicle moves forward. If complete information about the obstructions in the space of regard is known, navigation becomes easier, because the further away an obstruction is detected, the smaller the course correction required to avoid the obstruction.
Collision avoidance using information limited to objects at near range and far range, or limited range gated video (LRGV) is discussed next. A drawback to LRGV is that objects in mid-range are not visible and may obscure (shadow) objects in the far range.
In addition, objects in both natural and man made environments, tend to be defined by straight edges. A hidden line interpolation routine may be used by the navigator to identify shadowed regions. As shown in
Again, selecting a point on a visible object in the far field at least insures that there is nothing between the sensor and that point. If this point is also in a region with few or no interrupted edges (for example, point 65 of line 66 in
As an alternative, when the surroundings are complex, an imager having an ability to vary R1 permits the navigator to successively image the near field (near range) using step-wise increasing range. This capability allows the mid range uncertainty to be reduced to near zero. Providing for an adjustable R1, it will be appreciated, requires a programmable gate on the imager shutter (or I2C intensifier).
It is not possible to guarantee that a local path vector does not end on a mid range obstruction. In one embodiment, therefore, the invention provides a method for navigating around close obstructions, in order to overcome this limitation. In addition, position of objects encountered by a vehicle may be dynamic, or varying in time. A robust navigator may be provided by the invention, which is capable of navigating around a dynamic obstruction injected into the path at close range (discussed below). Furthermore, lack of mid range information does not add significant complexity to the navigator, if a robust solution is available for navigating around obstructions injected into the path at close range.
A method for short-range collision avoidance in a dynamic environment is discussed next. One method for collision avoidance in a dynamic environment is disclosed in an article by D. Fox, W. Burgard and S. Thrun, titled “The Dynamic Window Approach to Collision Avoidance”, published by IEEE Robotics and Automation Magazine, 4(1), at pages 23–33, dated March 1997, which is incorporated herein by reference in its entirety. As disclosed therein, a dynamic window (DW) is a computable model of a robot's environment in which trajectory curvatures are searched to minimize the distance to a local goal (a local target). Kinematic constraints are taken into account by directly searching a velocity space of the robot vehicle. The velocity space is defined by a set of tuples (ν,ω) of longitudinal and rotational velocities achievable by the robot. Acceptable velocity tuples are selected from a set of possible tuples which satisfy a condition that the robot vehicle be able to stop before it reaches an obstacle. To determine the next motion command to the vehicle, these acceptable velocity tuples are considered. From these, a velocity tuple is selected that maximizes an objective function. In general, this objective function is a linear combination of three goal-driven considerations given by:
G(ν, ω)=Max[K1*head(ν, ω)+K2*avoid(ν, ω)+K3*vel(ν, ω)]
The heading function in the above equation is the alignment of the vehicle's heading to that of the current local goal. The avoidance function in the equation is the miss distance to obstacles in the near image, and the velocity function in the equation is used to maintain the highest possible vehicle velocity while still avoiding obstacles. Constants K1, K2, and K3 are weighting factors for these three functions.
The heading and velocity functions are given by:
head(ν, ω)=1−|Θ|/π,
The avoidance function produces a set of acceptable velocity tuples. One method for reducing the search space is by defining the DW as the velocity space which the robot vehicle may reach within the next time interval, given its current velocity (ν,ω) and its acceleration capability ({dot over (ν)},{dot over (ω)}).
The LRGV of the invention provides a DW produced at R1, which may be very restricted. As a result, the number and values of acceptable velocity tuples for collision avoidance may be selected in advance. When selecting velocity tuples toward local goals in the R2 image, imminent collision avoidance need not be considered and the number of acceptable velocity tuples may be reduced to provide a ‘turn to a point’.
When an object comes into view, the collision avoidance processor (discussed later) may initiate a turn. While turning, the vehicle may under-go an angular displacement, ΔΘ, such that the video image stays centered on the vehicle's velocity vector. The sign of w may be chosen based on where the object appears in the image. An object in the left half of the image may initiate a turn to the right, and visa versa. The turn may be maintained at a constant radius until the field of view (FOV) is clear of objects. This is illustrated in
If the obstruction is centered in the image, than the direction of the turn may be chosen to minimize the heading difference to the local goal. A narrow object, such as tree 72 in
Referring now to
This new obstruction (seen at 71a cannot be closer than the first obstruction (tree 93), because it would have been in view prior to the start of the turn. Therefore, during a left turn, if a new obstruction has a clear space at the left edge, then it may be determined to be avoidable, and w and v may remain unchanged.
where R1=near field range, and r is the turning radius. r=R1−(½ vehicle width).
Up to now, the movement of the vehicle has been limited to motion along a single horizontal plane. In case of a flying vehicle moving through trees, changes in altitude are also required in order to avoid horizontal hanging branches. The approach discussed above, may be extended to motion along a vertical plane, if the vehicle maintains its FOV centered along the vertical component, as well as the horizontal component of the velocity vector. This may be achieved with a tilt capability added to the camera system. If this is not practical, then other methods of detecting obstacles above and below the vehicle is required.
The ground terrain in the LRGV images also influences vertical motion. The ground terrain may be treated as an obstacle and the collision avoidance processor may attempt to increase altitude until the ground is no longer visible. This has an additional benefit of providing a terrain following function where the hold altitude may be set by the camera vertical FOV. Since the lower the altitude, the lower the probability of encountering a branch, the vehicle may attempt to maintain the lowest practical altitude possible. This may be achieved, while maintaining a large optical FOV, by reducing the effective FOV in the collision avoidance processor. A portion of the bottom half of the image may be ignored. This has the effect of raising the bottom edge of the image used in deciding whether to climb or descend.
Referring now to
Strategic navigation system 1100 provides robot processes to achieve the mission goal. As such, strategic navigation system 1100 includes communications and interface for mission input 1101, mission planner 1102, inertial sensors/GPS 1103 and global path planning module 1104. Communications and interface for mission input 1101 provides an interface with the user for Inputting mission goals and reporting mission status to the user. Based on inputs provided by inertial sensors/GPS 1103, such as vehicle present position, velocity, acceleration, attitude (roll, pitch), heading, altitude, etc., mission planner 1102 provides steering information to achieve the vehicle's mission goal (steer to a global destination).
As shown, reactive local navigation system 1105 includes range gated imager 1106, image data preprocessor 1107, far field density mapper 1108, near field object detector 1109, local goal path planner 1110 and collision avoidance processor 1111. These are discussed below.
Range gated imager (camera) 1106 provides images of the scene at the far field (far range) and near field (near range). As discussed previously, images may be taken at ranges R1 and R2, thereby obtaining the LRGV.
Image preprocessor 1107 converts both, near range and far range data from imager 1106 from gray scale (or color) to binary, i.e. black and white. The image preprocessor then performs edge detection on the resulting image. The far field density mapper processes the far field image, identifying local goal candidates. It performs the ‘interrupted line’ detection function, and maps the destiny fields in the image. Although this may be the most processing intensive block, a moderate performance processor may be adequate for this function, since the far field image only needs to be processed at rates from 3 to 10 frames per second.
The near field object detector has a simpler task of detecting when obstructions enter the near field of view. This task may be executed at the full camera frame rate, so obstructions may be identified as soon as possible. The detector may be able to distinguish between vertical and horizontal edges.
Data from far field density mapper 1108 is provided, as shown, to local goal path planner 1110 to derive local goal steering information. Data from near field object detector 1109 and local goal path planner 1110 are provided to collision avoidance module 1111 and used to make course changes in heading, velocity and altitude. As will be explained, local goal path planner 1110 modifies global path planner 1104, so that the global goal may be achieved while reducing the probability of vehicle collision.
Local goal path planner 1110 includes two functions. The first function is to select local goals that reduce the probability of vehicle collision. The second function is to drive the vehicle toward the global goal (or global destination). The steps performed by the navigator are executed in two loops. A first loop is shown in
Referring first to
Eθ may be computed from the following:
PT=PTi such that Eθ=min(Eθi)
θT=f(PT)=θH±FOV
Eθi=dGtan(θG−θTi)
where
Having selected the optimal local target aim point in step 1206, the method loops back to step 1201 and obtains an image of a scene with the imager (camera) shutter opened at R2 (as shown in
Having an image at range R2, the method performs edge detection in step 1202 (as discussed, for example, with respect to
The target aim point, which has the minimum error, is then set as the current local goal. By this process, the local goal decision loop drives toward the global goal. In general, the heading to the global goal does not lie along one of the permissible aim points provided by the far field density mapper, i.e. Eθ≠0. As the vehicle approaches the global goal, if θG does not lie within the FOV, the local goal planner may not be able to close the distance to zero. If the local goal planner determines that the distance to the global goal is increasing, a heading alignment reset may be performed. This may be achieved by reducing the velocity ν to near zero, while the vehicle orients itself toward the heading of the global goal by setting θH=θG This is effective if the global goal is located in a region with few obstacles.
In heavily obstructed spaces, however, such a solution may quickly result in a local minimum in which the navigator may make no further progress toward the global goal. To resolve a local minimum, the method may add a step that forces the navigator to reposition the vehicle away from the area that produced the local minimum. This may include returning to the starting point.
Referring next to
Referring now to
As shown, the intensifier tube (which may be omitted) amplifies the is received signal and provides the amplified signal to camera 1402. The camera may be, for example, a CCD or a CMOS type of camera. Camera 1402 forms the image signal and sends the signal to data processor 1401. The data processor may process the received image signal. Since the intensifier tube is gated ON/OFF by intensifier power supply 1406, camera 1402 may form images of objects at ranges between zero and R1, as previously described, and may also form images of objects at ranges between R2 and infinity, as previously described.
Two controllers gate ON and OFF intensifier power supply 1406. As shown, near range gate controller 1405 turns ON the intensifier power supply at range zero and turns OFF the intensifier power supply at range R1. Far range gate controller 1407 turns ON the intensifier power supply at range R2 and turns OFF the intensifier power supply at range R2 plus a desired depth of field range, as previously described.
Pulse leading edge detector 1410 samples the leading edges of the transmitted pulses and starts clock 1404 at time t0. The clock provides an elapsed time to near range gate controller 1405 and far range gate controller 1407 for, respectively, gating ON/OFF the near range gate and the far range gate.
In summary, collision avoidance, using temporally acquired spatial data, via a range-gated camera, is feasible. Acquisition of limited range gated data using an arbitrarily long illuminator pulse, as provided by the invention, may be advantageously used for collision avoidance. Navigation through a wooded area by using only R1 and R2 image data is logically feasible, without need for a high cost, ultra short pulsed illuminator.
There are additional advantages to LRGV of the invention. Manipulating complete spatial information of the space of regard requires significant computational processing power. A large three-dimensional matrix of scene information must be continuously updated and processed as the vehicle moves through the space. This processing requires edge detection on multiple images (which must all be updated concurrently) and 3D matrix math functions to handle translation and rotation of objects in the matrix space. This is in addition to the processing required by the navigator, which must calculate multi-vector paths around obstructions. The processing power required to handle this may be limited by both power and cost constraints. In addition, image subtraction may be encumbered by registration errors due to vibration and temporal variations in successive images. With LRGV, however, there are only two 2D images to be processed, and only the near field image requires a high frame rate. In addition, only edge detection and tracking of aim point objects are required to be processed.
In the near field frames the number of edges may be small, minimizing the processing required at high frame rates. Far field frames potentially have a large number of edges but the far field need only be updated sporadically. This places more of the computational load on the navigator, instead of the image processor. The long-range navigator algorithms need not be complex; the long-range navigator just needs to advance the vehicle to an aim point, as previously described. Near field object avoidance, however, is a more formidable problem, but may be solved using a rule-based algorithm. Fuzzy logic may be suited for this application.
Although illustrated and described herein with reference to certain specific embodiments, the present invention is nevertheless not intended to be limited to the details shown. Rather, various modifications may be made in the details within the scope and range of equivalents of the claims and without departing from the spirit of the invention. For example, LRGV is suitable for other scenarios, such as autonomously traversing caves or navigating through a building. If hovering is permissible, then cartography is feasible using LRGV. For instance, accurate range to an object may be determined, using only R1, if successive images are taken while walking R1 out in steps until a return is established. The same approach may be used to find a door or a wall.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5448233 | Saban et al. | Sep 1995 | A |
| 5581250 | Khvilivitzky | Dec 1996 | A |
| 5631640 | Deis et al. | May 1997 | A |
| 6489922 | Lin et al. | Dec 2002 | B1 |
| 20020147534 | Delcheccolo et al. | Oct 2002 | A1 |
| 20030001772 | Woodington et al. | Jan 2003 | A1 |
