The subject disclosure relates to predictive and reactive field-of-view-based planning for autonomous driving.
Autonomous operation of vehicles relies on one or more types of sensors to detect and monitor both the vehicle and its environment. Exemplary vehicles include automobiles, trucks, motorcycles, construction equipment, farm equipment, automated factory equipment. Exemplary sensors include light detection and ranging (lidar) systems, radio detection and ranging (radar) systems, and cameras. Most sensors have a nominal field of view (FOV) associated with them, and the sensors detect objects or obtains images within their respective FOV. The nominal FOV of one or more sensors of an autonomous vehicle are considered for planning the future trajectory of the vehicle. For example, a static route plan is developed for travel from a given origin to a given destination. This route plan is then used during travel, along with detection data from the nominal FOV of the sensors, to generate a dynamic trajectory which dictates path points and velocities of the vehicle. But, the nominal FOV of a given sensor may be reduced because of an occlusion. Occlusions may be static (e.g., buildings, bushes) or dynamic (e.g., other vehicles in a current path). Accordingly, it is desirable to provide predictive and reactive field-of-view-based planning for autonomous driving.
In one exemplary embodiment, a method of controlling an autonomous vehicle to travel from an origin to a destination includes determining a route between the origin and the destination using a map. The method also includes determining an initial path along the route by optimizing a first cost function, the first cost function including a static cost component at a first set of locations along the route, and the static cost component at each location among the first set of locations along the route corresponding to a change in field of view of one or more sensors of the autonomous vehicle resulting from one or more static obstructions at the location that are indicated on the map. The method further includes controlling the autonomous vehicle to begin the travel on the route along the initial path.
In addition to one or more of the features described herein, the method also includes dynamically modifying the initial path in real time during the travel.
In addition to one or more of the features described herein, the modifying the initial path includes optimizing a second cost function in real time.
In addition to one or more of the features described herein, the optimizing the second cost function includes using a dynamic cost component at a second set of locations along the route, the dynamic cost component at each location among the second set of locations along the route corresponding to the change in field of view of the one or more sensors of the autonomous vehicle resulting from one or more static and dynamic obstructions at the location, wherein the dynamic obstructions include other vehicles.
In addition to one or more of the features described herein, the second set of locations and the first set of locations have one or more locations in common.
In addition to one or more of the features described herein, the method also includes determining the change in field of view of the one or more sensors of the autonomous vehicle at two or more grid points at each of the second set of locations.
In addition to one or more of the features described herein, the method also includes estimating a degree of occlusion at each of the two or more grid points and providing the degree of occlusion at each of the two or more grid points at each of the second set of locations as the dynamic cost component. The estimating the degree of occlusion includes obtaining a harmonic mean.
In addition to one or more of the features described herein, the optimizing the first cost function and the optimizing the second cost function include performing an algorithmic cost minimization process.
In addition to one or more of the features described herein, the method also includes determining the change in field of view of the one or more sensors of the autonomous vehicle at two or more grid points at each of the first set of locations.
In addition to one or more of the features described herein, the method also includes estimating a degree of occlusion at each of the two or more grid points and providing the degree of occlusion at each of the two or more grid points at each of the first set of locations as the static cost component. The estimating the degree of occlusion includes obtaining a harmonic mean.
In another exemplary embodiment, a system to control an autonomous vehicle to travel from an origin to a destination includes a memory device to store a map, and a controller to determine a route between the origin and the destination. The controller also determines an initial path along the route by optimizing a first cost function, the first cost function including a static cost component at a first set of locations along the route, and the static cost component at each location among the first set of locations along the route corresponding to a change in field of view of one or more sensors of the autonomous vehicle resulting from one or more static obstructions at the location that are indicated on the map. The controller further controls the autonomous vehicle to begin the travel on the route along the initial path.
In addition to one or more of the features described herein, the controller dynamically modifies the initial path in real time during the travel.
In addition to one or more of the features described herein, the controller modifies the initial path by optimizing a second cost function in real time.
In addition to one or more of the features described herein, the controller optimizes the second cost function by using a dynamic cost component at a second set of locations along the route, the dynamic cost component at each location among the second set of locations along the route corresponding to the change in field of view of the one or more sensors of the autonomous vehicle resulting from one or more static and dynamic obstructions at the location, and the dynamic obstructions including other vehicles.
In addition to one or more of the features described herein, the second set of locations and the first set of locations have one or more locations in common.
In addition to one or more of the features described herein, the controller determines the change in field of view of the one or more sensors of the autonomous vehicle at two or more grid points at each of the second set of locations.
In addition to one or more of the features described herein, the controller estimates a degree of occlusion at each of the two or more grid points and provide the degree of occlusion at each of the two or more grid points at each of the second set of locations as the dynamic cost component, and estimating the degree of occlusion includes obtaining a harmonic mean.
In addition to one or more of the features described herein, the controller optimizes the first cost function and optimize the second cost function by performing an algorithmic cost minimization process.
In addition to one or more of the features described herein, the controller determines the change in field of view of the one or more sensors of the autonomous vehicle at two or more grid points at each of the first set of locations.
In addition to one or more of the features described herein, the controller estimates a degree of occlusion at each of the two or more grid points and to provide the degree of occlusion at each of the two or more grid points at each of the first set of locations as the static cost component, and estimating the degree of occlusion includes obtaining a harmonic mean.
The above features and advantages, and other features and advantages of the disclosure are readily apparent from the following detailed description when taken in connection with the accompanying drawings.
Other features, advantages and details appear, by way of example only, in the following detailed description, the detailed description referring to the drawings in which:
The following description is merely exemplary in nature and is not intended to limit the present disclosure, its application or uses. It should be understood that throughout the drawings, corresponding reference numerals indicate like or corresponding parts and features.
As previously noted, autonomous driving involves planning a static route that the autonomous vehicle will take and a dynamic trajectory that defines specific path points and velocity along that route. The static route provides a lane-level path from the origin to the destination without considering the presence of any other vehicles. This static route is then modified during the travel to consider dynamic objects on the road using a real-time trajectory planner. Both static and dynamic planning use a map that indicates roads, direction of travel permitted on the roads, lane lines, and other information that facilitates autonomously traversing between an origin and a destination. The route plan may indicate the lanes to be used to reach the designated destination and the speed along each part of the route, for example. The trajectory plan may specify a more detailed position and velocity for the autonomous vehicle along the route (e.g., centered between the lane lines, to the right of the lane). Generally, a cost function with several cost components is optimized to determine the trajectory plan (e.g., path, speed). An exemplary cost component may be the distance to other vehicles. That is, the cost increases as the distance to other vehicles decreases. Thus, a path in the center of a center lane or to the right in a right lane may be determined based on optimizing the cost function.
The cost function may use a number of other cost components to optimize the path and vehicle operation along the route to the destination. In addition, the cost function may be used to optimize the path at two different stages. Prior to traversing the route, the nominal path points (i.e., the center line of the lanes in the route) may be adjusted by optimizing the cost function based on map information. During traversal of the route, in real time, the initial route plan may be updated by optimizing the cost function periodically or at irregular intervals based on an event or particular location, for example.
Embodiments of the systems and methods detailed herein add effective field of view (eFOV) as a cost component to the cost function to provide predictive and reactive field-of-view-based planning for autonomous driving. Predictive field-of-view-based planning refers to considering eFOV as part of the cost function analysis prior to traversing the route. Reactive field-of-view-based planning refers to considering eFOV as part of the cost function analysis during traversal of the route. Predictive field-of-view-based planning is performed by considering static obstructions (e.g., buildings, billboards, fences, intersection geometry) that are indicated along the route on the map. Reactive field-of-view-based planning is performed dynamically during the drive along the route by considering static and dynamic obstructions (e.g., other vehicles, pedestrians) encountered along the route.
Generally, according to one or more embodiments, in both predictive field-of-view-based planning (i.e., the pre-travel route planning) and reactive field-of-view-based planning (i.e., the during-travel trajectory planning), one of the cost optimization goals is to maximize eFOV (i.e., minimize occlusions for the sensors of the autonomous vehicle). Both predictive and reactive field-of-view-based planning use the estimation of degree of occlusion (DOO) as the cost component introduced into the cost optimization process according to one or more embodiments. The DOO and, specifically, decrease in DOO, corresponds with an increase in eFOV. Thus, an estimation of DOO, obtained as detailed herein, is representative of eFOV in the cost function.
In accordance with an exemplary embodiment,
The vehicle 100 also includes a controller 120. The controller 120 may control one or more aspects of the operation of the vehicle 100 based on information from the sensors 110. According to one or more exemplary embodiments, the controller 120 performs predictive field-of-view-based planning to determine an initial path 420 (
As previously noted, the trajectory planning includes optimizing a cost function. That is, a set of cost components are considered and a known process of cost function minimization is implemented. Exemplary cost components may include lane keeping (i.e., cost increases as the vehicle 100 departs from the lane 430 (
At block 325, to perform predictive field-of-view-based planning according to one or more embodiments, the processes include estimating DOO at locations of interest along the route 210 based on static obstructions 220 indicated on the map 200. This is further discussed with reference to
During the trip, the processes include optimizing the cost function in real time at block 350. As part of reactive field-of-view-based planning, the cost function includes a cost component, obtained from block 355, for second locations of interest. At block 355, the processes include estimating DOO at locations of interest based on static obstructions 220 and dynamic obstructions such as other vehicles 140. This is further discussed with reference to
The optimization of the cost function (at block 320) at all locations of interest along the route 210, which may include a cost component indicating estimates of DOO (at block 325) at first locations of interest as part of predictive field-of-view-based planning, is performed altogether for the entire route 210. This results in the initial path 420 being determined prior to the vehicle 100 traversing the route 210. However, optimization of the cost function (at block 350) at all locations of interest along the route 210, which may include a cost component indicating estimates of DOO (at block 355) at second locations of interest as part of reactive field-of-view-based planning, is performed piecemeal, in real time, as each location of interest is approached by the vehicle 100. The first locations of interest and the second locations of interest may be different, the same, or may overlap. Based on the optimized cost function, at block 350, modifying the initial path 420 at a given location along the route 210 may be performed in real time, at block 360. Reaching the destination D, at block 370, ends the process flow of the method 300.
Once the DOO corresponding with each grid point 405 is estimated, the position of the grid point 405 and corresponding DOO may be provided as a cost component (from block 325 to block 320). The cost function minimization that occurs at block 320 considers the cost component associated with DOO at each of the grid points 405 (from block 325), as well as other cost components such as deviation from the initial path 420, steering cost (i.e., how much steering is needed to follow a set of grid points 405). The result of the optimization of the cost function is the initial path 420, indicated in
Only intersection points 505 that are relevant to the route 201 mapped for the vehicle 100 are used. For example, assuming that driving on the right side of the road is legal, X1, X2, and X3 all relate to lanes 430 at which or from which potential colliding vehicles 140 with the vehicle 100 could be. However, the intersection point 505x represents a lane 430 in which any vehicle 140 should be travelling away from the vehicle 100 represented by the grid point 405. For a time period that represents a planning horizon T in seconds (e.g., 5-6 seconds), the DOO corresponding with the exemplary grid point 405 shown in
In EQ. 1, v1, v2, and v3 are the nominal speeds in the respective lanes 430. These nominal speeds (e.g., speed limit) are listed in the map 200. As
At block 610, a check is done of whether the location of interest that the vehicle 100 is approaching is also a second location of interest. As previously noted, for explanatory purposes, second locations of interest are a reference to locations at which reactive field-of-view-based planning is needed. That is, the check at block 610 determines if the cost component associated with DOO may have changed from the cost component provided (from block 325,
If the location of interest is also a second location of interest, according to the check at block 610, then a process flow similar to the one described with reference to
While the above disclosure has been described with reference to exemplary embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from its scope. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the disclosure without departing from the essential scope thereof. Therefore, it is intended that the present disclosure not be limited to the particular embodiments disclosed, but will include all embodiments falling within the scope thereof.