CHOREOGRAPHER SYSTEM FOR AUTONOMOUS OFF-ROAD VEHICLES

BACKGROUND

The disclosure relates generally to off-road vehicle guidance systems, and more particularly to a choreographer system for off-road vehicles.

Off-road vehicles, such as agricultural tractors, are typically manually guided through a field (e.g., via input from a driver). That is, an operator controls the steering and speed of an off-road vehicle from within a cab. Recent developments integrating GPS-based navigation systems into agricultural vehicle control systems have enabled automatic or semi-automatic steering modes. For example, some agricultural vehicles may include a control system configured to automatically direct the vehicle to follow a path between, over, or adjacent to rows in a field. The strip formed by the width of the vehicle and/or implement as the vehicle follows the path may be referred to as a swath or path.

In some cases, multiple off-road vehicles may be operating in the same geographic area or field, such as an agricultural field. The off-road vehicles may be towing agricultural implements of the same type or of different types, such as planters, harvesters, fertilizer application equipment, and so on. It may be useful to provide for improved coordination of multiple off-road vehicles.

BRIEF DESCRIPTION

Certain embodiments commensurate in scope with the originally claimed subject matter are summarized below. These embodiments are not intended to limit the scope of the claimed subject matter, but rather these embodiments are intended only to provide a brief summary of possible forms of the disclosure. Indeed, the disclosure may encompass a variety of forms that may be similar to or different from the embodiments set forth below.

In one embodiment, system for off-road vehicles includes a processor. The processor is configured to execute a choreographer system to derive a future first segment for a first off-road vehicle path, and to derive a future second segment for a second off-road vehicle path. The processor is further configured to execute the choreographer system to derive the existence of one or more potential issues based on the first segment and on the second segment, and to communicate the one or more potential issues to a path planning system, wherein the path planning system is configured to solve the one or more potential issues by deriving one or more vehicle plans.

In another embodiment, a method for operating off-road vehicles in a field includes deriving, via a choreographer system, a future first segment for a first off-road vehicle path, and deriving, via the choreographer system, a future second segment for a second off-road vehicle path. The method further includes deriving, via the choreographer system, the existence of one or more potential issues based on the first segment and on the second segment, and communicating, via the choreographer system, the one or more potential issues to a path planning system, wherein the path planning system is configured to solve the one or more potential issues by deriving one or more vehicle plans.

In a further embodiment, a tangible, non-transitory computer readable medium includes instructions configured to derive a future first segment for a first off-road vehicle path, and to derive a future second segment for a second off-road vehicle path. The instructions are further configured to derive the existence of one or more potential issues based on the first segment and on the second segment, and to communicate the one or more potential issues to a path planning system, wherein the path planning system is configured to solve the one or more potential issues by deriving one or more vehicle plans.

DRAWINGS

These and other features, aspects, and advantages of the present disclosure will become better understood when the following detailed description is read with reference to the accompanying drawings in which like characters represent like parts throughout the drawings, wherein:

FIG. 1 is a schematic diagram of an embodiment of off-road vehicles and agricultural implements within an agricultural field;

FIG. 2 is a schematic diagram of an embodiment of a control systems that may be utilized to control the off-road vehicles of FIG. 1;

FIG. 3 is bock diagram illustrating an embodiment of a modular path planning system for planning operations of the off-road vehicles of FIG. 1, and a choreographer system;

FIG. 4 is a top-down view of an embodiment of a segment of a planned off-road vehicle path that may be planned via the modular path planning system of FIG. 3;

FIG. 5 is the figure illustrates an embodiment of a field that may have been used as input into the modular path planning system of FIG. 3 and subsequently partitioned;

FIG. 6 illustrates embodiments of three time periods that may be predicted or otherwise derived by the choreographer system of FIG. 3; and

FIG. 7 is a flow chart of an embodiment of process for deriving vehicle plan via the modular path planning system of FIG. 3.

DETAILED DESCRIPTION

The techniques described herein provide for a choreographer system suitable for more efficiently enabling multiple off-road vehicle operations in, for example, an agricultural field. When implementing path plans for multiple off-road vehicles in the agricultural field there may be times when a sequence of events needs to be managed to avoid problems in the execution of the plans. Problems such as collisions due to intersecting vehicle paths may be managed through timing of when vehicles traverse specific path segments. The techniques described herein include a choreographer system that can modify the plan execution to manage these exceptions, resulting in more efficient field operations.

Turning now to FIG. 1, the figure is a schematic diagram of an embodiment of a first off-road vehicle 10 and an agricultural implement 12, and a second off-road vehicle 14 and an agricultural implement 16 within an agricultural field 18. The off-road vehicles 10 and 14 (e.g., tractor or other prime mover) are configured to tow, push, or otherwise propel the agricultural implements 12 and 16 throughout the field 18 along a direction of travel 20 and 22, respectively. In certain embodiments, the off-road vehicles 10, 14 are directed (e.g., via an automated system) along a path 24 and a path 26 to traverse the field 18 along substantially parallel rows 28. Path 24 starts at location 30 and ends at location 32, while path 26 starts at location 34 and ends at location 36. While two off-road vehicles 10, 14 are shown, it is to be understood that, in certain examples, a single off-road vehicle 10 or 12 may be used, and in other examples, three or more off-road vehicles 10 and/or 12 may be used. The agricultural implements 12, 14 may include any variety of implements and/or vehicle attachments including row planters, fertilizer equipment, headers for harvesters, booms for sprayers, etc.

As the vehicles 10, 14 tow the implements 12, 16 through the field 18 along paths 24, 26, the width of the implements 12, 16 each covers a respective swath through the field 18, as described in further detail below. In alternative embodiments, the off-road vehicles 10, 14 may be directed to traverse the field 18 along other routes (e.g., along a spiral path, etc.). As will be appreciated, the agricultural implements 12, 16 may be any suitable implement for performing agricultural operations throughout the field 18. For example, in certain embodiments, the agricultural implements 12, 16 may be a tillage tool, a fertilizer application tool, a seeding or planting tool, or a harvesting tool, among others. While the agricultural implements 12, 16 are propelled by the off-road vehicle 10, 14 in the illustrated embodiment, it should be appreciated that in alternative embodiments, the agricultural implements 12, 16 may be integrated within the off-road vehicles 10, 14.

In certain embodiments, the off-road vehicles 10, 14 are configured to operate autonomously (e.g., without an operator present in a cab of the off-road vehicle 10, 14). Accordingly, an autonomous system may direct the off-road vehicles 10 towing agricultural implement 12 and the off-road vehicle 14 towing agricultural implement 16 along the paths 24, 26 throughout the field 18 without direct control by an operator. In such embodiments, the autonomous off-road vehicles 10 and/or 14 may be configured to communicate with a base station 38. For example, in some embodiments, the off-road vehicles 10, 14 may receive plan data from the base station 38. The plan data may partition the field 18 into one or more partitions as described in more detail below, and may assign the off-road vehicles 10, 14 to the various partitions. Likewise, the plan data may derive certain swaths for each of the off-road vehicles 10, 14 to follow.

Furthermore, in certain embodiments, the off-road vehicles 10, 14 may be configured to operate semi-autonomously. For example, an operator within the cab may direct the off-road vehicle 10 and/or 14 to starting positions 30, 34 within or proximate to the field 18, and an automated system may direct the off-road vehicle 10 and/or 14 along the selected/planned route (e.g., the path 24, 26) through the field 18 while operator(s) monitors operation of the off-road vehicles 10, 14 and/or the implements 12, 16.

During multi-vehicle operations, a choreographer system is provided. The choreographer system may be included as one of the subsystems of a modular path planning system. The choreographer system's primary function is to avoid certain exceptions/problems such as, 1) vehicle 10, 14 and/or implement 12, 16 collisions at path intersections; 2) vehicle 10, 14 and/or implement 12, 16 collisions when travelling in opposite directions on adjacent swaths; 3) vehicle 10, 14 and/or implement 12, 16 collisions when travelling on adjacent paths in the same direction, such as when over-taking one another.

In certain embodiments, the path planning system will periodically call (or call on an event trigger) the choreographer system to validate or otherwise check current operations in the field 18 for any of the aforementioned exceptions. The choreographer system will then look at the current planned path horizon for each vehicle 10, 14. The current planned path horizon may include a currently assigned swath, a headland path to the next assigned swath, and the next assigned swath. The choreographer will break this path horizon into segments, and then based on the current vehicle speed and planned speeds, it will estimate a time at which the vehicle will be at each node between segments on this path horizon. The choreographer will then look for the following things to identify the potential problems mentioned above: 1) overlapping time periods when vehicles (e.g., 10, 14) will be present on intersecting path segments; 2) overlapping time periods when vehicles will be on adjacent path segments travelling in opposite directions; and 3) overlapping time periods when vehicles will be on adjacent path segments. The path segment size may be scaled appropriately depending on the following distance of multiple vehicles, vehicle size, implement size, and so on. If the choreographer system derives the existence of one or more of these aforementioned three situations, the choreographer system will indicate that there is a problem with the current mission plan. The mission planning and optimization system may then implement a number of different solutions to resolve the problem. Unlike a proximity detector, which detects proximity between vehicles 10, 14 and is purely reactive, the choreographer system may provide for a predictive problem resolution approach that better integrates with future planning yet is able to respond to current operations.

FIG. 2 is a schematic diagram of an embodiment of an automated field operations system 100 that may be utilized to control the off-road vehicles 10 and/or 14 of FIG. 1. In the illustrated embodiment, the automated field operations system 100 includes a primary control system 102 (e.g., mounted on the off-road vehicles 10, 14), and the off-road vehicles 10, 14 includes a first transceiver 104 configured to establish a wireless communication link with a second transceiver 106 of the base station 38. The base station 38 may be remove or may be included in the vehicles 10 and/or 14. As will be appreciated, the first and second transceivers 102, 104 may operate at any suitable frequency range within the electromagnetic spectrum. For example, in certain embodiments, the transceivers 102, 104 may broadcast and receive radio waves within a frequency range of about 250 MHz to about 10 GHz. In addition, the first and second transceivers 102, 104 may utilize any suitable communication protocol, such as a standard protocol (e.g., Wi-Fi, Bluetooth, etc.) or a proprietary protocol.

In the illustrated embodiment, the off-road vehicles 10, 14 includes a spatial locating device 108, which is mounted to the off-road vehicles 10, 14 and configured to determine positions of the off-road vehicles 10, 14. As will be appreciated, the spatial locating device 108 may include any suitable system configured to determine the positions of the off-road vehicles 10, 14, such as a global positioning system (GPS) or Global Navigation Satellite System (GLONASS), for example. The spatial locating device 108 is in communication with the primary vehicle control system 102. The spatial locating device 108 may include a Global Navigation Satellite System (GNSS) receiver 110 configured to communicate with two or more satellites in orbit (e.g., GPS, GLONASS, Galileo, BeiDou, etc.) to determine the location, heading, speed, etc. of the vehicles 10, 14. The receiver 110 may include one or more micro-processors 112, a memory component 114, an input/output system 116, a power supply 118, and radio circuitry 120. The processors 112 may run software stored on the memory component 114 to compute the positions of the vehicle 10, 14. Based on the computed position over time, the processor 112 may also determine vehicle 10, 14 heading, speed, etc. In some embodiments, the spatial locating device 108 may include a three-dimensional imaging system, such as a Lidar, which may be used in conjunction with the GNSS and configured to assist in determining a desired path and determining the vehicle's position and orientation relative to the path. Additionally, the spatial locating device 108 may include an augmentation system, such as Real Time Kinematic (RTK) or Wide Area Augmentation System (WAAS) to improve the accuracy of the spatial locating device 108.

In certain embodiments, the spatial locating device 108 may be configured to determine the positions of the off-road vehicle 10, 14 relative to a fixed point within the field (e.g., via a fixed radio transceiver 106). Accordingly, the spatial locating device 108 may be configured to determine the positions of the off-road vehicle 10, 14 relative to a fixed global coordinate system (e.g., via the GPS) or a fixed local coordinate system. In certain embodiments, the first transceiver 104 is configured to broadcast a signal indicative of the position of the off-road vehicles 10, 14 to the transceiver 106 of the base station 38. Based on the determined locations of the off-road vehicle 10, 14, the control system 102 may determine (e.g., via a processor) the relative proximity to one or more rows or swaths, one or more boundaries, one or more headlands, etc. Furthermore, the control system 102 may determine a planned path through points or curves received from the base station 38, and/or a correction path from the vehicle's current position back to the planned paths 24, 26.

In the illustrated embodiment, the off-road vehicles 10, 14 include a steering control system 122 configured to control direction of movement of the off-road vehicles 10, 14 and a speed control system 124 configured to control a speed of the off-road vehicles 10, 14. Furthermore, the control system 102 includes a controller 126 communicatively coupled to the first transceiver 104, to the spatial locating device 108, to the steering control system 122, and to the speed control system 124. In certain embodiments, the controller 126 may control the steering control system 122 and the speed control system 124 to guide the off-road vehicles 10, 14 along the planned path.

In further embodiments, the controller 126 is configured to control the steering control system 122 and the speed control system 124 to guide the off-road vehicles 10, 14 based on a path or paths planned via a modular path planning system. A correction path controller 128 is configured to receive the current vehicle state (e.g., position, velocity, heading, curvature, etc.), the planned path, and an estimated delay time of the control loop (e.g., to estimate the state of the vehicle when the vehicle actuates changes in the control signals). The correction path controller 128 may operate continuously to determine whether the vehicles 10, 14 are off the planned path. If so, the correction path controller 128 determines an error (e.g., a distance between the planned path and the current vehicle 10, 14 position or a heading error) and generates a continuous path for the vehicle 10, 14 to follow to a point along the planned path.

The correction path controller 128 includes a processor, such as the illustrated microprocessor 130, and a memory device 132. The correction path controller 128 may also include one or more storage devices and/or other suitable components. The processor 130 may be used to execute software, such as software for generating a correction path, and so forth. The processor 130 may include multiple microprocessors, one or more “general-purpose” microprocessors, one or more special-purpose microprocessors, and/or one or more application specific integrated circuits (ASICS), or some combination thereof. For example, the processor 130 may include one or more reduced instruction set (RISC) processors. The memory device 132 may include a volatile memory, such as random access memory (RAM), and/or a nonvolatile memory, such as ROM. The memory device 132 may store a variety of information and may be used for various purposes. For example, the memory device 132 may store processor-executable instructions (e.g., firmware or software) for the processor 130 to execute, such as instructions for generating a correction path. The storage device(s) (e.g., nonvolatile storage) may include ROM, flash memory, a hard drive, or any other suitable optical, magnetic, or solid-state storage medium, or a combination thereof. The storage device(s) may store data (e.g., target points, the planned path, etc.), instructions (e.g., software or firmware for controlling the off-road vehicle, etc.), and any other suitable data. In some embodiments, the tasks of the correction path controller 128 and the primary controller 126 may be performed by a single controller (e.g., the primary controller 126).

In certain embodiments, the controller 126 is an electronic controller having electrical circuitry configured to process data from the transceiver 104, the spatial locating device 108, the correction path controller 128, or a combination thereof, among other components of the off-road vehicles 10, 14. In the illustrated embodiment, the controller 126 includes a processor, such as the illustrated microprocessor 134, and a memory device 136. The controller 126 may also include one or more storage devices and/or other suitable components. The processor 134 may be used to execute software, such as software for controlling the off-road vehicles 10, 14 and so forth. Moreover, the processor 134 may include multiple microprocessors, one or more “general-purpose” microprocessors, one or more special-purpose microprocessors, and/or one or more application specific integrated circuits (ASICS), or some combination thereof. For example, the processor 134 may include one or more reduced instruction set (RISC) processors.

The memory device 136 may include a volatile memory, such as random access memory (RAM), and/or a nonvolatile memory, such as ROM. The memory device 136 may store a variety of information and may be used for various purposes. For example, the memory device 136 may store processor-executable instructions (e.g., firmware or software) for the processor 134 to execute, such as instructions for controlling the off-road vehicles 10, 14. The storage device(s) 136 (e.g., nonvolatile storage) may include read-only memory (ROM), flash memory, a hard drive, or any other suitable optical, magnetic, or solid-state storage medium, or a combination thereof. The storage device(s) 136 may store data (e.g., target points, vehicle paths, etc.), instructions (e.g., software or firmware for controlling the off-road vehicle, etc.), and any other suitable data.

In the illustrated embodiment, the steering control system 122 includes a wheel angle control system 138, a differential braking system 140, and a torque vectoring system 142. The wheel angle control system 138 may automatically rotate one or more wheels or tracks of the off-road vehicles 10, 14 (e.g., via hydraulic actuators) to steer the off-road vehicles 10, 14 along the determined path through the field 18. By way of example, the wheel angle control system 138 may rotate front wheels/tracks, rear wheels/tracks, and/or intermediate wheels/tracks of the off-road vehicles 10, 14, either individually or in groups. The differential braking system 140 may independently vary the braking force on each lateral side of the off-road vehicles 10, 14 to direct the off-road vehicles 10, 14 along the determined paths through the field. Similarly, the torque vectoring 142 may differentially apply torque from the engine to wheels and/or tracks on each lateral side of the off-road vehicles 10, 14, thereby directing the off-road vehicle 10, 14 along the determined path through the field. While the illustrated steering control system 122 includes the wheel angle control system 138, the differential braking system 140, and the torque vectoring system 142, it should be appreciated that alternative embodiments may include one or two of these systems, in any suitable combination. Further embodiments may include a steering control system having other and/or additional systems to facilitate directing the off-road vehicles 10, 14 along the determined path through the field (e.g., an articulated steering system, etc.).

In the illustrated embodiment, the speed control system 124 includes an engine output control system 144, a transmission control system 146, and a braking control system 148. The engine output control system 144 is configured to vary the output of the engine to control the speed of the off-road vehicles 10, 14. For example, the engine output control system 144 may vary a throttle setting of the engine, a fuel/air mixture of the engine, a timing of the engine, or a combination thereof, and/or other suitable engine parameters to control engine output. In addition, the transmission control system 146 may adjust gear selection within a transmission or adjust a continuously variable transmission (CVT) to control the speed of the off-road vehicles 10, 14. Furthermore, the braking control system 148 may adjust braking force, thereby controlling the speed of the off-road vehicles 10, 14. While the illustrated speed control system 124 includes the engine output control system 144, the transmission control system 146, and the braking control system 148, it should be appreciated that alternative embodiments may include one or two of these systems, in any suitable combination. Further embodiments may include a speed control system 124 having other and/or additional systems to facilitate adjusting the speed of the off-road vehicle.

As previously discussed, the off-road vehicles 10, 14 are configured to communicate with the base station 38 via the transceivers 104, 106. In the illustrated embodiment, the base station 38 includes a controller 150 communicatively coupled to the base station transceiver 106. The controller 150 is configured to output commands and/or data to the off-road vehicles 10, 14. For example, the controller 150 may provide a series of points or curves to the off-road vehicle 10, 14 via the transceivers 104, 106, from which the vehicles 10, 14 generate planned paths.

In certain embodiments, the controller 150 is an electronic controller having electrical circuitry configured to process data from certain components of the base station 38 (e.g., the transceiver 106). In the illustrated embodiment, the controller 150 includes a processor, such as the illustrated microprocessor 152, and a memory device 154. The processor 152 may be used to execute software, such as software for providing commands and/or data to the off-road vehicle controller 126, and so forth. Moreover, the processor 152 may include multiple microprocessors, one or more “general-purpose” microprocessors, one or more special-purpose microprocessors, and/or one or more application specific integrated circuits (ASICS), or some combination thereof. For example, the processor 152 may include one or more reduced instruction set (RISC) processors. The memory device 154 may include a volatile memory, such as random access memory (RAM), and/or a nonvolatile memory, such as read-only memory (ROM). The memory device 154 may store a variety of information and may be used for various purposes. For example, the memory device 154 may store processor-executable instructions (e.g., firmware or software) for the processor 152 to execute, such as instructions for providing commands and/or data to the off-road vehicle controller 126.

In the illustrated embodiment, the base station 38 includes a user interface 156 communicatively coupled to the controller 150. The user interface 156 is configured to present data from the off-road vehicles 10, 14 and/or the agricultural implements 12, 16 to an operator (e.g., data associated with operation of the off-road vehicles, data associated with operation of the agricultural implements, etc.). The user interface 156 is also configured to enable an operator to control certain functions of the off-road vehicles 10, 14 (e.g., starting and stopping the off-road vehicles, instructing the off-road vehicles to follow a selected/planned route through the field, etc.). In the illustrated embodiment, the user interface includes a display 158 configured to present information to the operator, such as the position of the off-road vehicles 10, 14 within the field, the speed of the off-road vehicle, and the path of the off-road vehicle, among other data. In addition, the user interface 156 (e.g., via the display 158, via an audio system, etc.) is configured to alert an operator if the vehicle 10 and/or 14 substantially diverges from the planned path. The user interface 156 may include keyboard(s), touchscreen(s), joysticks, buttons, voice control systems, and so on, suitable for receiving user inputs.

In the illustrated embodiment, the base station 38 includes a storage device 160 communicatively coupled to the controller 150. The storage device 160 (e.g., nonvolatile storage) may include read-only memory (ROM), flash memory, a hard drive, or any other suitable optical, magnetic, or solid-state storage medium, or a combination thereof. The storage devices 160 may store data (e.g., field maps, planned paths or vehicle routes, etc.), instructions (e.g., software or firmware for commanding the off-road vehicle, etc.), and any other suitable data.

While the off-road vehicle control system 102 includes the off-road vehicle controller 126 and the correction path controller 128, it should be appreciated that in alternative embodiments, the control system 102 may include the base station controller 150. For example, in certain embodiments, control functions of the control system 102 may be distributed between the off-road vehicle controller 126, the correction path controller 128, and the base station controller 150. In further embodiments, the base station controller 150 may perform a substantial portion of the control functions of the control system 102. For example, in certain embodiments, the transceiver 104 may output signals to the transceiver 106. In such embodiments, the transceiver 104 may output corresponding signals to the base station controller 150, and the base station controller 150 may determine the appropriate action based on the current state of the off-road vehicles 10, 14 and output a signal indicative of the appropriate action. For example, the controller 150 may output a signal to the user interface 156, instructing the user interface 156 to alert the operator of the off-road vehicle's divergence from the planned path. In addition, the controller 150 may output instructions to the off-road vehicle controller 126 (e.g., via the transceivers 104 and 106), instructing the off-road vehicles 10, 14 and/or the agricultural implements 12, 16 to perform certain operations (e.g., steering control and/or speed control) to guide the off-road vehicles 10, 14 along the planned path or to guide the off-road vehicles 10, 14 back to the planned path.

Also shown are a modular path planning system 162, which will be described if further detail below, that may include a choreographer system 163. The modular path planning system 162 or certain systems of the modular path planning system 162, including the choreographer system 163, may be disposed in the vehicles 10, 14, in the base station 38, or in a combination thereof. The modular path planning system 162 may be used to derive a plan that may include for example, swaths and partitions, swath connections, point-to-point plans, and so on, suitable for enabling one or more vehicles, such as the off-road vehicles 10, 14, to simultaneously operate in the field 18 in a more efficient manner. When implementing path plans for multiple off-road vehicles 10, 14 in the agricultural field 18 there may be times when a sequence of events needs to be managed to avoid problems in the execution of the plans. Problems such as collisions due to intersecting vehicle 10, 14 paths 24, 26 may be managed through timing of when the vehicle 10, 14 traverse specific path segments (e.g., portions of the path). The choreographer system 163 that can modify the plan execution to manage these exceptions, resulting in more efficient field 18 operations, as further described below.

FIG. 3 is bock diagram illustrating an embodiment of the modular path planning system 162 operatively coupled with the choreographer system 163. In the illustrated embodiment, the modular path planning system 162 includes various subsystems, more specifically, a plan requestor and executor system 164, a swath and partition generator system 166, the choreographer system 163, a swath connector system 170, a point-to-point planner system 172, and a traveling salesman problem (TSP) solver system 174. The system 162 and subsystems 164, 166, 163, 170, 172, and/or 174 may be implemented as computer code or instructions stored in the devices 132, 136, 154, 160 and executable via the processors 130, 134, 152. The subsystems 164, 166, 163, 170, 172, and/or 174 may execute concurrently or sequentially with respect to each other, and may communicate with each other with a variety of techniques, including interprocess communication (IPC) techniques, shared memory, message passing, socket communications, message ques, mapped files, and so on.

In operations, the modular path planning system 162 may interface with various external systems 176, such as controllers (e.g., controllers 126, 128, 150), user interfaces (e.g., user interface 156), and other systems (e.g., weather forecasting systems, dispatch systems). For example, the operator of the off-road vehicles 10, 14 may enter certain information (e.g., as part of a plan request 178) into the plan requestor and executor system 164, such as type of off-road vehicles to be used (e.g., tractor, combine, and so on), manufacturer, vehicle model, vehicle year, vehicle effective width, vehicle turning radius, type of agricultural implements 12, 16 to be used (e.g., planter, fertilizer, seeder, etc.), implement manufacturer, implement year, implement effective width, implement turning radius, and the like.

Other inputs that may be part of the plan request 178 may include a map of the field 18, desired area(s) to cover in the field 18, obstacle boundaries, field 18 boundaries, and so on. Operation information included in the plan request 178 may include the type of operation(s) to be performed (e.g., planting, seeding, tilling, fertilizing, etc.) and details of the operation (e.g., type of seed, type of fertilizer, date and time of day for operations, etc.). Accordingly, the plan request 178 may be transmitted, detailing the aforementioned information.

The modular path planning system 162 may use the plan request 178 as input to derive one or more vehicle plans 180 via the systems 164, 166, 163, 170, 172, and/or 174, as further described below. The vehicle plan(s) 180 may include inter-vehicle coordination suitable for enabling multiple vehicles to operate in the same field, e.g., field 18. The vehicles 10, 14 may then use the vehicle plan(s) 180 to drive and operate in the field 18. For example, the vehicle plan(s) 180 may include partitions, swaths, paths, path segments, and/or driving points to be observed by the vehicles 10, 14. Changes, such as a vehicle 10, 14 undergoing undesired maintenance, weather conditions, obstacle changes (e.g., pond overflowing), and the like, may result in updated plan(s) 182. The updated plan(s) 182 may include a simple reassignment of a swath to another vehicle, may include new swaths to follow, may include evasive actions to take, or may include a complete new vehicle plan. Accordingly, the vehicles 10, 14 may be operated with up-to-date information and may thus cover the field 18 in a more efficient manner.

In order to derive the vehicle plans 180, 182, the modular path planning system 162 may use the plan request 178 to communicate with the swath and partition generator system 166 and the swath connector system 170. The swath partition generator system 166 may apply the plan request 178 to divide the field 18 into one or more partitions and may establish relationships between the partitions, (e.g., Partition A bounds Partition B, Partition B bounds partitions C and D, partition C bounds partition D, Partition D does not bound any partition). In one embodiment, the partitions and their bounding relationships form a directed graph where all the edges are directed from one vertex to another (meaning partition bounds cannot form a loop).

Partitions are comprised of swaths, where a single vehicle/implement may be assigned to a single swath. A swath includes a width at least as wide as the width of the vehicle and/or agricultural implement that will be assigned the swath. The swath and partition generator system 166 receives the information contained in the plan request 178 and creates well-defined swaths and headlands in the field 18. A continuous curvature path is called a swath, and may consist of line segments, arcs, or clothoids. Groups of similar swaths are called swath partitions. The created swaths may include headland/cleanup paths, interior rows, or obstacle cleanup paths and may be grouped into partitions. Swaths are typically assigned to one vehicle, or assigned to several candidate vehicles (for example, when several vehicles have the same coverage width). A partition hierarchy is also generated, which describes how the partitions are related relative to an outward-to-inward spatial relationship; that is, a directed graph that represents to some degree ordering of operations.

Each outer and obstacle boundary in the field 18 has 0 or more headland/cleanup-laps associated with it and the swaths in each lap comprise a single partition. Swaths generated in the interior of the field 18 make up the remaining partitions. For each headland/clean-up partition all the swaths are assigned to a single vehicle, or a list of candidate vehicles. For inner partitions the swaths are staggered on a per vehicle basis. Once the partitions and their associated swaths are created, the information from the original plan request 178 and the partitions are sent to the swath connector system 170.

The swath connector system 170 may receive the plan request 178 and derived swath and partitions 184 and then examines cost constraints in the plan request information 178. Cost constraints may include fuel costs, cost of navigating muddy ground versus rocky ground, seed costs, fertilizer costs, equipment wear and tear costs, travel to starting areas, time, accuracy of operation, and so on. With this information, the operations desired dictate a relative ordering (outward-to-inward or inward-to-outward) relationship between the partitions so as to optimize operations.

Once the order of partitions 184 is established, potential swath connections are evaluated; that is, the paths between swath endpoints. A potential swath connection is a path from the end of one swath to an end of another swath such that the vehicle/implement does not collide with any obstacles. Each potential swath connection is assigned a cost based on a weighted average of several subcosts. The subcosts may include time of travel along the path, length of path, length of path not on a road, acceleration/deceleration requirements for travel along the path, amount of trim-back on swath, and so on.

The swath connector system 170 may then formulate the swath connector problem as a traveling salesperson (TSP) problem, where each “city” in the TSP is represented as an end-point of the swath. The purpose of the TSP is to connect the end-points of each swath in the lowest-cost ordering, per the costs listed above. Connections 186 between the swath end-points are made by use of calls to the point-to-point planner system 172, which generates continuous curvature paths (clothoids) 188 between two points. More specifically, point-to-point planner system 172 receives a map, vehicle information, a start pose (position and orientation and curvature) and an end pose (position and orientation ad curvature), and creates a continuous curvature plan between the two poses considering the vehicle 10, 14, information (turn radius, speed, vehicle size) and map (drivable areas, obstacles, and so on).

Once the swath connection problem is formulated as a TSP problem, the problem is then passed to the TSP solver system 174. The TSP solver system 174 may apply a variety of TSP techniques such as Christofides' algorithm, nearest neighbor algorithm, bitonic tour, branch-and-bound algorithms, and so on, to solve the TSP problem either exactly or approximately. Once the TSP solver system 174 returns a solution (e.g., a path from start city to end city), the solution is checked for validity, for example to ensure that all “cities” are presently visited. If more optimization or updated costs are to be calculated, the TSP solver system 174 may be called repeatedly with different initial conditions or costs, until satisfaction criteria are met, thus finding an optimal or near-optimal solution.

The choreographer system 163 may “choreograph” the vehicles 10, 14 to avoid vehicle 10, 14 and/or agricultural implement 12, 16 issues. The choreographer system's will be called to avoid: 1) vehicle 10, 14 and/or implement 12, 16 collisions at any path intersections; 2) vehicle 10, 14 and/or implement 12, 16 collisions when travelling in opposite directions on adjacent swaths; 3) vehicle 10, 14 and/or implement 12, 16 collisions when travelling on adjacent paths in the same direction, such as when over-taking one another. In certain embodiments, the path planning system 162 will periodically call (or call on an event trigger) the choreographer system 163 to validate or otherwise check current operations in the field 18 for any of the aforementioned exceptions. The choreographer system 163 will then look at the current planned path horizon for each vehicle 10, 14. The current planned path horizon may include a currently assigned swath, a headland path to the next assigned swath, and the next assigned swath. The choreographer will break this path horizon into segments, and then based on the current vehicle speed and planned speeds, it will estimate a time at which the vehicle will be at each node between segments on this path horizon. The choreographer will then look for the following things to identify the potential problems mentioned above: 1) overlapping time periods when vehicles (e.g., 10, 14) and/or implements (e.g., 12, 16) will be present on intersecting path segments; 2) overlapping time periods when vehicles (e.g., 10, 14) and/or implements (e.g., 12, 16) will be on adjacent path segments travelling in opposite directions; and 3) overlapping time periods when vehicles (e.g., 10, 14) and/or implements (e.g., 12, 16) will be on adjacent path segments. The path segment size may be scaled appropriately depending on the following distance of multiple vehicles, vehicle size, implement size, and so on. If the choreographer system 163 derives the existence of one or more of these aforementioned three situations, the choreographer system 163 will indicate that there is a problem with the current mission plan. The path planning system 162 may then implement a number of different solutions to resolve the problem.

For example, if the choreographer system 163 derives that there will be overlapping time periods when vehicles (e.g., 10, 14) and/or implements (e.g., 12, 16) will be present on intersecting path segments, the path planning system 162 may slow down one of the vehicles 10, 14 to let the other vehicle 10, 14 pass ahead of it. Further, the path planning system 162 may also determine when to stop one of the vehicles to let the other vehicle pass ahead of it. In order to determine which vehicle 10, 14 to stop, a vehicle hierarchy may be used. For example, smaller vehicles may be stopped first, but the hierarchy may also depend on the type of operation (e.g., planting, seeding, harvesting, fertilizing, etc.), on a vehicle type, a vehicle ID, an agricultural implement type, and so on.

If the choreographer system 163 derives that there will be overlapping time periods when vehicles (e.g., 10, 14) and/or implements (e.g., 12, 16) will be on adjacent path segments travelling in opposite directions, the path planning system 162 may call the swath connector 170 to assign a new next swath to one of the two vehicles. If the choreographer system 163 derives that there will be overlapping time periods when vehicles (e.g., 10, 14) and/or implements (e.g., 12, 16) will be on adjacent path segments, moving geofences may be used and/or a proximity monitor may be enabled monitor and alert if the vehicles 10, 14 enter areas surrounding the vehicles 10, 14 and then take appropriate action (e.g., increase a separation distance from each other).

The choreographer system 163 may also be called from the swath connector 170 when it creates a plan. The swath connector 170 may create a global area coverage plan (e.g., to cover the entire field 18). The swath connector 170 may call the choreographer system 163 to check the current plan over a certain horizon (field partition, % of field area, absolute area, etc.) for the same problems identified above, and in the same manner as mentioned above, but this process may cover a longer vehicle path (longer than current swath+headland connection+next swath).

Upon the desired level of optimality, the swath connector system 170 returns the path plan 180 for each vehicle 10, 14. The path plan 180 is delivered to the plan requestor and executor system 164, which may then provide the path plan 180 to external systems 176. Likewise, changes to the plan 180 may be similarly computed via the systems 164, 166, 163, 170, 172, 174, resulting in the updated plan 182.

It may be beneficial to illustrate certain path segments, such as an example segment 200 illustrated in FIG. 4, and partitions illustrated in FIG. 5. More specifically, FIG. 4 is a top-down view of a segment 200 of a planned off-road vehicle path 180 and/or 182. In the illustrated embodiment, the path segment 200 is a path through an end-of-row turn that connects the end of one row to the beginning of another row, while avoiding contact with a boundary 202 (e.g., a fence).

In the present embodiment, the path segment 200 includes five discrete clothoid segments 204, 206, 208, 210, 212 that combine to form a continuous path from a starting point 214 to an end point 216. Clothoid segments, also referred to as Euler spirals, are defined as curves with a curvature that changes linearly along the curve length. Put another way, the rate of curvature c for a given clothoid is constant. Clothoids are C2 continuous and may take into account physical constraints, such as minimum turning radius and maximum turning rate of the vehicle. A clothoid may be defined as a curve having a curvature that varies linearly with curve length. The velocity ({dot over (x)}(s), {dot over (y)}(s)) at a distance s along the clothoid, may be described by the following equations:

$\begin{matrix} \dot{x} (s) = \cos (\frac{σ}{2} s^{2} + κ_{i} s + q_{i}), & (1) \\ \dot{y} (s) = \sin (\frac{σ}{2} s^{2} + κ_{i} s + q_{i}), & (2) \end{matrix}$

where q_iis the initial heading, κ_iis the initial curvature, and σ is the rate of change of the curvature with respect to s, the distance along the clothoid. The curvature as a function of s may be described by:

κ=σs+κ_i, (3)

The heading as a function of s may be described by:

$\begin{matrix} q = \frac{σ}{2} s^{2} + κ_{i} s + q_{i} . & (4) \end{matrix}$

Thus, the position (x(s), y(s)) may be described by:

$\begin{matrix} x (s) = x_{i} + \int_{0}^{s} \cos (\frac{σ}{2} τ^{2} + κ_{i} τ + q_{i}) d τ, and & (5) \\ y (s) = y_{i} + \int_{0}^{s} \sin (\frac{σ}{2} τ^{2} + κ_{i} τ + q_{i}) d τ, & (6) \end{matrix}$

where τ is path length. Thus, a clothoid segment may be based on initial position, x_iand y_i, initial heading, q_i, initial curvature, κ_i, rate of curvature, σ, and path length, s. The capabilities of the vehicles 10, 14 and/or agricultural implements 12, 16 (e.g., minimum turning radius, maximum turning rate) may be taken into account when determining the parameters of the clothoid segments 204, 206, 208, 210, 212. For example, the maximum turning rate (e.g., maximum slew rate) of the vehicle may be proportional to a maximum value for rate of curvature σ. The minimum turning radius may determine a maximum value for curvature κ. It should be appreciated that segments having a constant curvature are clothoids with a rate of curvature σ of zero. Similarly, straight line segments are clothoid segments in which both the curvature κ and the rate of curvature σ are zero. Thus, straight line segments, segments of constant curvature, and segments having a constant rate of curvature may all be defined as clothoid segments. Accordingly, the path segment 200 of five clothoid segments 204, 206, 208, 210, 212 may be parameterized in a single vector:

z=[x
₀
,y
₀
,q
₀,κ₀,σ₀,s₀,x₁,y₁,q₁,κ₁,σ₁,s₁, . . . ,x₄,y₄,q₄,κ₄,σ₄,s₄]^T (7)

The off-road vehicle control system 102, the base station controller 150, and/or the modular path planning system 162 may adjust one or more of the initial position x; y; the initial heading q; the initial curvature κ; the rate of curvature σ, and the path length s of each segment 204, 206, 108, 210, 212, such that the path segment 200 is C2 continuous. The path segment 200 is C2 continuous when the final position, the final heading, and the final curvature at the end of one segment (e.g., the first segment 204) matches the initial position, the initial heading, and the initial curvature at the start of the proceeding segment (e.g., the second segment 206).

If the vehicle 10 or 14 is at the starting point 214, then the current vehicle state may be used as the initial position, initial heading, and initial curvature of the first clothoid segment 204. Conversely, the parameters of the last segment 212 may be adjusted such that the final position, final heading, and final curvature of the last segment 212 match the final position, final heading, and final curvature of the desired exit state. As previously discussed, the parameters of each segment 204, 206, 208, 210, 212 are adjusted in such a way that the rate of curvature σ does not exceed the vehicle's maximum turning rate, and/or the curvature κ does not fall below the minimum turning radius of the vehicle. Additionally, a minimum segment length (e.g., 10 centimeters) may be used such that the path lengths of each segment 204, 206, 208, 210, 212 remains above the minimum segment length.

The path segment 200 may also be configured to avoid boundaries 202 when generating clothoid segments 204, 206, 208, 210, 212 of the path segment 200 and adjusting the parameters of the segments. In some applications, the implement 12, 16 may be wider than the vehicles 10, 14. In such embodiments, the path segment 200 may include a buffer region 218 on either side of the path segment 200 that may be as wide as or wider than the implements 12 or 16, such that the implements 12 or 16 may be capable of traversing the path segment 200 without coming into contact with the boundary 202.

Turning now to FIG. 5, the figure illustrates an embodiment of a field 220 that may have been used as input as part of the information included in the plan request 178 and subsequently partitioned. The modular path planning system 162 may then partition the field 220 into one or more partitions, among other derived features. In the illustrated embodiment, the field 220 includes an L-shape. The modular path planning system 162 may derive that the field 220 would be more efficiently operated on by respecting multiple partitions. For example, an exterior boundary 222 may be derived. The exterior boundary 222 may define outer bounds of the field 220. Headland partitions 224, 226 may then be derived, suitable for turning around the field 220. Interior partition 228 are also derived, having various rows 230. Inside of the interior partition 228 a headlands obstacle partition 232 is derived, which surrounds an obstacle 234, such as a pond. A second interior partition 236 is also derived, having rows 238 perpendicular to rows 230.

As mentioned previously, an order of operations for the partitions shown may also be derived. For example, a cost function may determine that it may be more efficient, when planting, to plant inside of partitions 228 first, followed by a planting inside partition 236 second. Accordingly, the vehicle plan 180, 182 may follow certain headlands and row orientations in a desired planting order. When harvesting, the order may be different, thus resulting in a harvesting order. Indeed, different operations (e.g., planting, seeding, fertilizing, harvesting), different implements 12, 16, different implement settings, different vehicle 10, 14 types, and so on, may result in different order of partition operations. By deriving multiple partitions, partition types, partition order of operations, swaths, and so on, the techniques described herein may improve planning operations, resulting in a more efficient and productive vehicle plan(s) 180, 182.

FIG. 6 illustrates embodiments of three time periods 240, 242, 244 that may be predicted or otherwise derived by the choreographer system 163. As depicted in predicted time period 240, a directional path segment 246 for a first path may intersect a directional path segment 248 for a second path. The time period 240 and the path segments 246, 248 may be predicted by extrapolating current vehicle 10, 14 spatial locations into the future based on speed, terrain, current operation being performed (e.g., planting, seeding, fertilizing, tillage, etc.), and so on. In order to prevent the predicted time period 240 from occurring, the path planning system 162 may slow down one of the vehicles 10, 14 to let the other vehicle 10, 14 pass ahead of it. The path planning system 162 may also determine when to stop one of the vehicles to let the other vehicle pass ahead of it. In order to determine which vehicle 10, 14 to stop, a vehicle hierarchy may be used.

Predicted time period 242 includes a directional path segment 250 for a first path being adjacent at some future time to a directional path segment 252 for a second path, where the directional path segments 250, 252 have opposite directions. The time period 242 and the path segments 250, 252 may be predicted by extrapolating current vehicle 10, 14 spatial locations into the future based on speed, terrain, current operation being performed (e.g., planting, seeding, fertilizing, tillage, etc.), and so on. In order to prevent the predicted time period 242 from occurring, the path planning system 162 may call the swath connector 170 to assign a new next swath to one of the two vehicles.

Predicted time period 244 includes a directional path segment 254 for a first path being adjacent at some future time to a directional path segment 256 for a second path, where the directional path segments 254, 256 have the same direction. The time period 244 and the path segments 254, 256 may be predicted by extrapolating current vehicle 10, 14 spatial locations into the future based on speed, terrain, current operation being performed (e.g., planting, seeding, fertilizing, etc.), and so on. In order to prevent the predicted time period 244 from occurring, the path planning system 162 may creating moving geofences (e.g., geofence that moves with its respective vehicle 10, 14) may be used and/or a proximity monitor may be enabled monitor and alert if the vehicles 10, 14 enter areas surrounding the vehicles 10, 14 and then take appropriate action (e.g., increase a separation distance from each other). Accordingly, the choreographer system's derivations may be applied to derive future time periods and actions may be performed to more reliably and efficiently operate multiple vehicles 10, 14 in the field 18.

FIG. 7 is a flow chart of an embodiment of a process 300 suitable for deriving certain issues during multi-vehicle operations in agricultural fields and for solving the issues. The process 300 may be implemented as computer code or instructions, for example, included in the modular path planning system 162 and/or the choreographer system 163 and executable via the processors 150 and/or 102. In the depicted embodiment, the process 300 may call (block 302) the choreographer system 163. As mentioned previously, the choreographer system 163 may be called from the swath connector system 170 when the swath connector system 170 creates a plan, or it may be called based on certain event triggers, such as time-based triggers (every second, minute, 5 minutes, etc.), location triggers (e.g., when vehicles 10, 14 are within certain distance of each other), and so on.

The process 300 may then provide current plan horizon(s) 304 as input to the choreographer system 163, which may then use the current plan horizon(s) 304 to derive (block 306) future time period segments 308. For each vehicle 10, 14, the current planned path horizon 304 may include a currently assigned swath, a headland path to the next assigned swath, and the next assigned swath. The choreographer system 308 will break the current path horizon(s) 304 into segments 308, and then based on the current vehicle speed and planned speeds, it will estimate a time at which the vehicle will be at each node or point between segments 308.

The process 300 may then identify (block 310) potential issues 312, such as those described above with respect to time periods 240, 242, and/or 242 shown in FIG. 6. The potential issues 312 may include: 1) vehicle 10, 14 and/or implement 12, 16 collisions at path intersections; 2) vehicle 10, 14 and/or implement 12, 16 collisions when travelling in opposite directions on adjacent swaths; 3) vehicle 10, 14 and/or implement 12, 16 collisions when travelling on adjacent paths in the same direction, such as when over-taking one another. It is to be noted that the word “implement” as used herein may include any attachment that is placed on the vehicles 10 or 14, including towed attachments, pushed attachments, headers for harvesters, booms for sprayers, etc.

The process 300 may then communicate (block 314) the potential issues 312. For example, the choreographer system 163 may communicate with the plan requestor and executor system 164, the swath connector 170 and/or the swath and partition generator 166 the one or more potential issues 312. The process 300 may then solve (block 316) the one or more potential issues 312. For example, if the derived potential issues 112 includes overlapping time periods when vehicles (e.g., 10, 14) and/or implements (e.g., 12, 16) will be present on intersecting path segments, the path planning system 162 may slow down one of the vehicles 10, 14 to let the other vehicle 10, 14 pass ahead of it. Further, the path planning system 162 may also determine when to stop one of the vehicles to let the other vehicle pass ahead of it. In order to determine which vehicle 10, 14 to stop, a vehicle hierarchy may be used. For example, smaller vehicles may be stopped first, but the hierarchy may also depend on the type of operation (e.g., planting, seeding, harvesting, fertilizing, etc.), on a vehicle type, a vehicle ID, an agricultural implement type, and so on.

If the potential issues 312 include overlapping time periods when vehicles (e.g., 10, 14) and/or implements (e.g., 12, 16) will be on adjacent path segments travelling in opposite directions, the path planning system 162 may call the swath connector 170 to assign a new next swath to one of the two vehicles. If the potential issues 312 include overlapping time periods when vehicles (e.g., 10, 14) and/or implements (e.g., 12, 16) will be on adjacent path segments travelling in the same direction, moving geofences may be used and/or a proximity monitor may be enabled monitor and alert if the vehicles 10, 14 enter areas surrounding the vehicles 10, 14 and then take appropriate action (e.g., increase a separation distance from each other). The solution to the potential issues 312 may then be incorporated into the plan(s) 180, 182, and the vehicles 10, 14 may then follow the plan(s) plan(s) 180, 182 and thus avoid the potential issues 312.

While only certain features of the disclosure have been illustrated and described herein, many modifications and changes will occur to those skilled in the art. It is, therefore, to be understood that the appended claims are intended to cover all such modifications and changes as fall within the true spirit of the disclosure.

CHOREOGRAPHER SYSTEM FOR AUTONOMOUS OFF-ROAD VEHICLES

Information

Publication Number

Date Filed

Date Published

Inventors

Original Assignees

CPC

International Classifications

Abstract

Description

Claims