Patent Application
20240263421
This disclosure relates generally to a digging machine, and more particularly to systems and methods of controlling the digging machine under remote control operation and under autonomous operation.
Machines are used for moving material between locations or onto transport vehicles at a work site. For example, rope shovels, mining shovels, hydraulic shovels, and hydraulic excavators may move material using rotational movement while material carrying machines, such as haul trucks, may be loaded with the material, and then move the material by driving alone a path or road. Machines using such rotational capabilities may dig material with a material engaging work implement at a dig site and rotate the work implement to a dump or truck load site at which the work implement is dumped or unloaded into a material carrying machine such as a haul truck. The material carrying machine may then be propelled to another location such as a truck dump site at which the material is dumped or discharged from the machine. The process of moving material may be repeated numerous times in order to move the desired amount of material. Furthermore, digging machines may be operated in an autonomous, semi-autonomous, or manual manner to perform these tasks in response to commands generated as part of a work plan for the machines. When operating autonomously or semi-autonomously, the work plan may be generated by a planning system that designates the dig and dump or truck load locations.
U.S. Pat. No. 5,065,326 to Sahm (“the ′326 patent”) discloses a control system and method that automatically controls a work implement of an excavating machine to perform a complete excavation work cycle. The control system monitors the position of the work implement and the forces exerted on the work implement, and actuates to the work implement according to predetermined position and force setpoints. However, the control system and method of the ′326 patent may not receive non-line-of-sight operator inputs to define the dig and dump regions, and also may not sufficiently monitor the autonomous digging and dumping in order to alert a remote operator to one or more alert or warning conditions.
Aspects of the present disclosure may solve one or more of the problems set forth above and/or other problems in the art. The scope of the disclosure, however, is not defined by the ability to solve any specific problem.
According to one aspect of the present disclosure, a control system for a machine may include a machine body, a linkage assembly, an implement coupled to the machine body via the linkage assembly, an operation console remote from the machine, and a controller in communication with the operation console. The controller may be configured to receive one or more signals from the operation console in order to operate the machine in a remote control mode and define a dig region and a dump region. The controller may also be configured to operate the machine in an autonomous mode to dig material from the dig region and dump the material to the dump region.
In another aspect of the present disclosure, a method of operating a machine with an implement may include when operating in a remote control mode, defining a dig region and defining a dump region. The method also includes initiating an autonomous digging and dumping mode. When operating in the autonomous digging and dumping mode, monitoring the machine for one or more notification conditions, and when one or more notification conditions are detected, ending the autonomous digging and dumping mode and returning to the remote control mode.
In yet another aspect of the present disclosure, a digging system may include a digging machine and an operator station that is remote from the digging machine. The digging machine may include a machine body, a linkage assembly, a bucket movably coupled to the machine body via the linkage assembly, one or more cameras or sensors, and a controller. The operator station may be in communication with the one or more cameras or sensors and the controller. The controller may be configured to, in a remote operation mode, receive signals from the operator station to define a dig region and a dump region. The controller may further be configured to, in an autonomous operation mode, control the machine body, the linkage assembly, and the bucket to dig material from the dig region and to dump material to the dump region.
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate various exemplary embodiments, and together with the description serve to explain the principles of the disclosed embodiments.
Reference will now be made in detail to specific embodiments or features, examples of which are illustrated in the accompanying drawings. Generally, corresponding or similar reference numbers will be used, when possible, throughout the drawings to refer to the same or corresponding parts. Features in the drawings may not be drawn to scale, but may rather be drawn to highlight different aspects of the disclosure. In this disclosure, relative terms, such as, for example, “about,” “generally,” and “substantially” are used to indicate a possible variation of ±10% in a stated numeric value.
As discussed in detail below, one or more operators at remote control site 104 may remotely operate machine 102, for example, via a non-line-of sight (“non-LOS”) control or via a line-of-sight (“LOS”) control (e.g., remote from machine 102 but within sight of machine 102). Moreover, the one or more operators at remote control site 104 may perform one or more operations to define a dig region. The one or more operators may also define a dumping region (e.g., haul truck 106, a pile 114, etc.). Based on the operator input, machine 102 may autonomously perform a dig operation. As discussed in detail below, the dig operation may continue for a predetermined amount of time, a predetermined number of digs, until a predetermined amount (e.g., volume, weight, etc.) of material has been moved, until there are one or more error or warning conditions, other notification(s), etc.
Machine 102 includes an undercarriage 116, for example, including one or more ground engaging elements or tracks 118. Machine 102 also includes a machine body 120, which may be rotatable relative to undercarriage 116. Moreover, machine 102 includes a linkage assembly 130, for example, coupling a work implement or bucket 132 to machine body 120. Linkage assembly 130 may include one or more booms and sticks, such that linkage assembly 130, and thus bucket 132, is movable relative to machine body 120. Linkage assembly 130 and/or bucket 132 may be movable via respective hydraulic cylinder(s). As discussed here, various aspects of machine 102 may be movable or otherwise controlled, both via remote operator control and via autonomous control, to perform various digging or earth moving operations. Furthermore, machine 102 may include one or more cameras 134, for example, positioned in a forward position of machine body 120 (e.g., with bucket 132 and/or linkage assembly 130 in a field of view), which may help the remote operator control, for example, in non-LOS control.
Additionally, machine 102 includes a controller 152, for example, in communication with operator station 122 via wireless communication system 124. In these aspects, as shown in
For example, as shown in
Controller 152 may include control logic and/or instructions for both remote operator control and autonomous control. Alternatively or additionally, in some aspects, controller 152 may receive one or more inputs to control autonomous control operation, for example, autonomous control inputs from an autonomy or autonomous control electronic control module. Although not shown, the autonomy or autonomous control electronic control module may be incorporated in controller 152 or may be separate from controller 152. Additionally, machine 102 may include a plurality of sensors in communication with controller 152. For example, machine 102 may include a machine body sensor 154, for example, configured to detect and/or provide information regarding the position of machine body 120 relative to undercarriage 116 and/or the pitch and/or roll of machine body 120, for example, relative to an inertial frame. Machine 102 may also include one or more linkage assembly sensors 156, for example, configured to detect and/or provide information regarding the position of one or more portions of linkage assembly 130 relative to machine body 120. Additionally, machine 102 may include one or more bucket sensors 158, for example, configured to detect and/or provide information regarding the position and/or orientation of one or more portions of bucket 132, for example, relative to linkage assembly 130 and/or machine body 120. One or more of machine body sensor 154, linkage assembly sensor(s) 156, and/or bucket sensor(s) may be inertial measurement units (IMUs), hydraulic cylinder displacement sensors, joint angular sensors, or other appropriate sensors to measure the position and/or orientation of respective portions of machine 102.
In these aspects, controller 152 may also be in communication with one or more control units in control system 150. For example, controller 152 may be in communication with a machine body control unit 160, for example, configured to control a position of machine body 120 relative to undercarriage 116. Controller 152 may be in communication with a linkage assembly control unit 162, for example, configured to control a position of one or more portions of linkage assembly 130 relative to machine body 120. Additionally, controller 152 may be in communication with a bucket control unit 164, for example, configured to control a position of one or more portions of bucket 132 relative to linkage assembly 130 and/or machine body 120.
In these aspects, control system 150 and controller 152 may help to control the position and/or operation of machine 102, for example, machine body 120, linkage assembly 130, and/or bucket 132, when operating in a remote control operation mode and/or when operating in an autonomous control mode. For example, control system 150 may receive inputs and emit outputs to control the operation of machine body 120, linkage assembly 130, and/or bucket 132 to dig or otherwise move material from work site 110, for example, to one or more haul trucks 106, when operating in the remote control operation mode and/or when operating in the autonomous control mode. Furthermore, as discussed herein, control system 150 may receive one or more inputs or signals when operating in the remote control operation mode in order to establish operating parameters for when operating in the autonomous control mode.
Although not shown, controller 152 may be coupled to or include one or more memory units, which may contain instructions for controller 152 to help control a position or movement of one or more of machine body 120, linkage assembly 130, and/or bucket 132. Controller 152 may be a separate controller on machine 102, or may be integrated into a central vehicle controller (e.g., a main power controller, an operation controller, etc.). Alternatively, controller 152 may be integrated into one or more of control or management systems or modules (e.g., for operating the engine) of machine 102, or another dedicated control module on machine 102. In one aspect, machine 102 may be an electrohydraulic excavator, and controller 152 may control one or more electrical switches or valves in order to control one or more hydraulic cylinders or electrical elements in order to operate machine 102.
Controller 152 may include one or more microprocessors. For example, controller 152 may embody a single microprocessor or multiple microprocessors. The one or more microprocessors of controller 152 may be configured to perform any of the operations mentioned herein. For example, controller 152 may include a memory, a secondary storage device, a processor, such as a central processing unit or any other means for accomplishing a task consistent with the present disclosure. The memory or secondary storage device associated with controller 152 may be non-transitory computer-readable media that store data and/or software routines that may assist controller 152 in performing its functions, such as the functions of process or method 300 of
As shown in
Moreover, the one or more operators may control machine body 120, linkage assembly 130, and/or bucket 132 such that a portion (e.g., the heel) of bucket 132 contacts the material on work site 110 at another side (e.g., a right side) of dig region 200, for example, point 20 in
In these aspects, controller 152 may store edge or end position 202 and edge or end position 204 of dig region 200, for example, into the memory of controller 152 or another memory (not shown) in control system 150. For example, as mentioned, controller 152 may include an autonomy or autonomous control electronic control module, which may store edge or end positions 202 and 204. Alternatively, although not shown, the autonomy or autonomous control electronic control module may be separate from controller 152. In any of these aspects, controller 152 may also interpolate or extrapolate dig region 200, for example, including a plurality of dig paths 206 that span dig region 200. For example, dig region 200 may have a dig width, and bucket 132 may have a bucket width. In this aspect, controller 152 may determine a number of dig paths 206 based on the dig width divided by the bucket width.
As shown in
Moreover, the one or more operators may control machine body 120, linkage assembly 130, and/or bucket 132 such that a portion of bucket 132 is at or above another end (e.g., a rear portion) of bed 108 of haul truck 106, for example, point 40 in
In these aspects, controller 152 may store one edge or end position 212 and another edge or end position 214 of dump region 210, for example, into the memory of controller 152 or another memory (not shown) in control system 150. Controller 152 may also interpolate or extrapolate dump region 210, for example, including a plurality of dump positions that span dump region 210. For example, dig region 200 may have a dump length. In this aspect, for example, based on the number of dig paths 216, controller 152 may determine a number of loads or dumps. Controller 152 may determine a bucket position for the plurality of the dumps, such that consecutive dumps are spaced along the length of dump region 210. In other aspects, controller 152 may estimate a total volume or weight of material delivered to dump region 210, for example, based on material properties of the material at dig site 112, the size of bucket 132, the dig depth, etc. In these aspects, controller 152 may emit or output one or more warnings or notifications if machine 102 has delivered a weight or volume of material to dump region 210 that is at or approaching a maximum weight or volume of material that may be transported by haul truck 106. In these aspects, emitting or outputting the warning or notification may include suspending the autonomous digging operation and returning to remote operator control mode. For example, controller 152 may position bucket 132 in a lowered position (e.g., with a portion of bucket 132 on or near a ground surface), and await signal(s) from operator station 122. Also, controller 152 may output a signal to operator station 122, for example, to signal the one or more operators that machine 102 is now in a remote control operation mode. Alternatively, if dump region 210 is pile 114, then controller 152 may not monitor the weight or volume of material delivered to dump region 210.
Once machine 102 is positioned at the dig site, in a step 304, the operator(s) may define the dig region. As mentioned above and as shown in
Next, in a step 306, the operator(s) may define the dump region. As mentioned above and as shown in
The operator(s) may position bucket 132 in second edge or end position 214 and store the position with one or more button-presses or other inputs. In some aspects, steps 304 and 306 may include the operator(s) remotely controlling machine 102 to perform one dig (e.g., at dig site 112) and one dump (e.g., into bed 108 of haul truck 106).
In some aspects, in an optional step 308, the operator(s) may define one or more parameters for the material and/or dig paths 206. For example, in some aspects the operator(s) may input or select (e.g., from a set of predefined options) the type or composition of material (e.g., dirt or soil, gravel, sand, etc.) being moved by machine 102. The operator(s) may input a digging depth (e.g., approximately 1 meter to approximately 5 meters) and/or a digging length (e.g., approximately 0.5 meters to approximately 2 meters). In some aspects, the operator(s) may input or select a material angle of repose. Alternatively or additionally, as mentioned above, the operator(s) may perform one dig, and controller 152 may store the depth (e.g., when machine 102 is digging from a bench height and a floor of dig site 112 is exposed) and/or length of the dig to be used for corresponding digs during the autonomous operation. The operator(s) may input or select whether the loads are being dumped into a haul truck (e.g., haul truck 106) or into a pile (e.g., pile 114). In some aspects, the operator(s) may input or select the number of loads and/or the number of loads per minute (or otherwise input a load rate or frequency) to be moved by machine 102, for example, from dig region 200 to dump region 210. For example, the operator(s) may input or select a number of loads that may fit within or otherwise be movable by haul truck 106. In some aspects, the operator(s) may input or select a total payload (e.g., weight) that may be supported and transported by haul truck 106). In some aspects, the operator(s) may input or select a type and/or size of haul truck 106, and controller 152 may store the volume and/or weight of material that may be supported and transported by haul truck 106.
Next, in a step 310, the operator(s) may activate autonomous digging and dumping such that machine 102 (e.g., via controller 152 and control system 150) performs autonomous digging and dumping. The one or more operators may enter one or more inputs for machine 102 to enter an autonomous operation mode. For example, the operator(s) may signal to enter the autonomous operation mode via a user interface at operator station 122. For example, the one or more operators may depress a button or otherwise select the autonomous digging mode.
As mentioned, controller 152 may signal one or more of machine body control unit 160, linkage assembly control unit 162, and/or bucket control unit 164 to control the position and operation of machine 102 and bucket 132 to dig at dig region 200 (e.g., within the boundaries of dig region 200) and to dump at dump region 210 (e.g., within the boundaries of dump region 210). Controller 152 may signal one or more of machine body control unit 160, linkage assembly control unit 162, and/or bucket control unit 164 to perform a plurality of dig cycles, for example, including the plurality of dig paths 206.
During the autonomous digging and dumping, method 300 may include an optional step 312 that includes monitoring the digging and dumping for one or more error or warning conditions (e.g., a notification). For example, optional step 312 may include monitoring the volume or weight of material being delivered to dump region 210. In this aspect, controller 152 may receive one or more signals indicative of the weight or volume of material in bucket 132 after each dig path 206. Alternatively or additionally, controller 152 may estimate the weight or volume of material delivered to dump region 210 based on, for example, a size of bucket 132 and the material being dug. If the measured or estimated weight or volume of material delivered to dump region 210 is approaching a maximum weight or volume, then controller 152 may suspend autonomous operations. For example, controller 152 may suspend autonomous operations, such that the full or approximately full haul truck 106 may be replaced with a new, empty haul truck. In this aspect, controller 152 may prompt the operator(s) to re-define dump region 210, for example, relative to the position of the bed of the new haul truck. If machine 102 is in the middle of digging dig region 200, then controller 152 may suspend autonomous operations with bucket 132 being full, such that the material may be delivered to the next haul truck 106 or other dump region 210 by the operator(s) in the remote control operation mode.
In some aspects, controller 152 may detect that dig region 200 does not have sufficient material (e.g., to fill bucket 132). In these aspects, controller 152 may signal operator station 122 with a warning or notification, as the one or more operators may wish to re-define dig region 200, reposition machine 102, etc.
In other aspects, step 312 may include monitoring the position or orientation of undercarriage 116 and/or machine body 120. For example, if undercarriage 116 and/or machine body 120 change position or orientation (e.g., as detected by one or more IMUs, a global positioning system, or other sensor), machine 102 may be pitching or sliding relative to dig site 112. In these aspects, if controller 152 detects pitching, sliding, or other changes in position or orientation, controller 152 may output a warning or notification, for example, to operator station 122 for the one or more operators to check the autonomous operation. Alternatively or additionally, if controller 152 detects pitching, sliding, or other changes in position or orientation, controller 152 may suspend the autonomous operation, for example, with bucket 132 empty, such that the one or more operators can evaluate the positioning of machine 102. The one or more operators may reposition machine 102, for example, via remote control, and/or may reinitiate the autonomous operation. Moreover, in some aspects, if controller 152 detects a risk of machine 102 tipping. For example, if end position 214 of dump region 210 is too far from machine 102, there may be a risk of machine 102 tipping when bucket 132 is full. In this aspect, controller 152 may output a warning or notification to operator station 122 and/or may suspend the autonomous operation.
Next, in a step 314, method 300 includes ending the autonomous digging and dumping. Ending the autonomous digging and dumping may include re-entering operator remote control mode, for example, for the operator(s) to reposition or reorient machine 102 and/or to define new dig and/or dump regions such that the steps of method 300 may be repeated for the new dig and/or dump regions. Step 314 may correspond to machine 102 completing the plurality of dig paths 206 in dig region 200. Alternatively, as discussed above, controller 152 may detect one or more conditions that necessitate suspending or temporarily ending the autonomous operation. In either aspect, step 314 may include controller 152 signaling operator station 122, for example, to display, emit, or otherwise provide a signal or notification that machine 102 is in the remote control operation mode. Additionally, in the remote control operation mode, the one or more operators may survey dig site 112, for example, using one or more cameras 134 or sensors on machine 102. The one or more operators may clean up dig site 112, for example, by controlling machine body 120, linkage assembly 130, and/or bucket 132 to perform one or more digs. Cleaning up dig site 112 may be necessary due to material shifting during the autonomous digging or other features or contours of dig site 112.
The present disclosure may be applicable in systems and methods for controlling an implement on a machine, such as, e.g., bucket 132 on machine 102. During operation, control system 150, including controller 152, may alternate between a remote control operation mode and an autonomous mode. As discussed above, the operator(s) (e.g., at operator station 122 in remote control site 104) may navigate or otherwise position machine 102 to dig site 112 on work site 110 (e.g., in step 302). The operator(s) may define dig region 200 (e.g., in step 304), and may also define dump region 210 (e.g., in step 306). In some aspects, the operator(s) may define one or more parameters for the material at the dig region (e.g., in step 308) or other aspects of the work site 110. Based on this information, control system 150, including controller 152, may perform autonomous digging and dumping (e.g., in step 310). Control system 150, including controller 152, may monitoring the autonomous digging and dumping for one or more error or warning conditions (e.g., in step 312). If such conditions arise, controller 152 may provide one or more signals to operator station 122 and/or pause or suspend the autonomous digging and dumping operation, for example, returning to a remote operation mode. Furthermore, once the autonomous digging and dumping has been performed, for example, completing dig paths 206 in dig region 200, control system 150 may end the autonomous digging and dumping (e.g., in step 314).
In these aspects, various systems and methods discussed herein may allow for one or more remote operators (e.g., via a non-LOS remote control system or via a LOS remote control system) to navigate machine 102 and also to select dig region 200 and dump region 210. Additionally, the various systems and methods discussed herein may allow for the digging and dumping (e.g., multiple dig and dump cycles) to be conducted autonomously. Accordingly, the systems and methods discussed herein may alleviate the tedium or monotony of cycle loading (e.g., the multiple dig and dump cycles) for the remote operator(s). Moreover, the systems and methods discussed herein may allow for the remote operator(s) to be more productive and/or efficient, for example, overseeing multiple machines simultaneously (e.g., on work site 110 or another work site). In some aspects, the systems and methods discussed herein may allow for the remote operator(s) to take a break or otherwise reduce the overall stress of operating a digging machine. Furthermore, the systems and methods discussed herein may allow for the autonomous operation mode to return to the remote control operation mode under various conditions. Returning to the remote control operation mode may help to allow for the remote operator(s) to perform any additional tasks, such as, identifying a new dig or dump region (e.g., a bed of a new haul truck 106), clean-up of any debris or material at dig site 112, etc.
It will be apparent to those skilled in the art that various modifications and variations can be made to the disclosed system without departing from the scope of the disclosure. Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. It may be intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
