AUTONOMOUS SENSOR KIT FOR CONTROLLING OPERATIONS OF POWER MACHINES

BACKGROUND

This disclosure is directed toward power machines. More particularly, the present disclosure is directed to power machines that are configured to operate in whole or in part in an autonomous mode or be controlled by an operator in a remote-controlled mode. Power machines, for the purposes of this disclosure, include any type of machine that generates power to accomplish a particular task or a variety of tasks using a work element. One type of power machine is a work vehicle. Work vehicles are generally self-propelled vehicles that have a work element, such as a lift arm (although some work vehicles can have other work elements) that can be manipulated to perform a work function. Work vehicles include loaders (including mini-loaders), excavators, utility vehicles, mowers, tractors (including compact tractors), and trenchers, to name a few examples.

The discussion above is merely provided for general background information and is not intended to be used as an aid in determining the scope of the claimed subject matter.

SUMMARY OF THE DISCLOSURE

Some embodiments of the disclosure are directed to improvements in controlling operation of power machines in an autonomous mode or a remote operator-controlled mode. As described in greater detail herein, a power machine may include a control system that utilizes data collected by a sensor system. The sensor system may include one or more sensor groups, with each sensor group including a corresponding set of sensors arranged for installation on a particular area of the power machine (or component thereof). In this way, for example, a user may install a sensor group on a particular area of a power machine to enable autonomous or remote control of the power machine in response to signals provided by the installed sensor group. In some cases, different sensor sets (e.g., of different types, or as included in different sensor groups) can be selectively used to inform control of power machine operations in autonomous and in remote-controlled (or other operator-controlled) modes.

According to some aspects of the disclosure, a power machine is provided. The power machine may include a frame. The power machine may also include a power source and tractive assemblies supported by the frame. The power machine may also include a work element arranged to operate at a front side of the frame. The power machine may also include a sensor system that includes a plurality of sensor groups, the plurality of sensor groups including one or more of: a rear sensor group arranged at a rear side of the frame, including a rear radar system and a rear camera system; a side sensor group arranged at a lateral side of the frame, including a side camera system; or a front sensor group arranged at the front side of the frame, including a front radar system and a front camera system.

According to some aspects of the disclosure, a method is provided for equipping a power machine for autonomous operation. The method may include installing, as part of a sensor kit, a rear sensor group that is configured to be secured at a rear end of a frame of the power machine and includes a rear radar system and a rear camera system. The method may include installing, as part of the sensor kit, a side sensor group that is configured to be secured at a lateral side of the frame and includes a side camera system. The method may include installing, as part of the sensor kit, a front sensor group that is configured to be secured at a front end of the frame and includes a front radar system and a front camera system.

According to some aspects of the disclosure, method is provided for controlling operations of a power machine that includes a tractive system and a work element. The method may include receiving, with one or more electronic control devices, a selection of an autonomous mode or an operator-controlled mode. The method may include, with the one or more electronic control devices, receiving data from a sensor system that includes a plurality of sensor groups including one or more of: a rear sensor group arranged at a rear side of the frame, a side sensor group arranged at a lateral side of the frame, or a front sensor group arranged at the front side of the frame. The method may include, with the one or more electronic control devices, controlling the tractive system or the work element according to the received selection; where, in the autonomous mode, the one or more electronic control devices may determine commands to control the tractive system or the work element based on signals from a first sensor set of the sensor system that includes a rear radar system of the rear sensor group and a front radar system of the front sensor group; and where, in the operator-controlled mode, the one or more electronic control devices may transmit visual data from a second sensor set of the sensor system that includes camera systems for display to an operator; and determine commands to control the tractive system or the work element based on command signals received from the operator in response to the transmitted visual data.

This Summary and the Abstract are provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. The Summary and the Abstract are not intended to identify key features or essential features of the claimed subject matter, nor are they intended to be used as an aid in determining the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

The following drawings are provided to help illustrate various features of non-limiting examples of the disclosure and are not intended to limit the scope of the disclosure or exclude alternative implementations.

FIG. 1 is a block diagram illustrating functional systems of a representative power machine on which embodiments of the present disclosure can be practiced.

FIG. 2 is a perspective view showing generally a front of a power machine on which embodiments disclosed in this specification can be advantageously practiced.

FIG. 3 is a perspective view showing generally a back of the power machine shown in FIG. 2.

FIG. 4 is a block diagram schematic illustration of a power system of a power machine.

FIG. 5 is a schematic block diagram of a power machine and sensor groups according to some configurations.

FIG. 6 schematically illustrates example detection fields associated with one or more sensors of a power machine according to some configurations.

FIG. 7 is a flowchart of a method of controlling operations of a power machine that includes a tractive system and a work element according to some configurations.

FIG. 8 illustrates an example rear layout for a rear sensor group of a power machine according to some configurations.

FIG. 9 illustrates an example side layout for a lateral side sensor group of a power machine according to some configurations.

FIG. 10 illustrates an example front layout for a front sensor group of the power machine according to some configurations, with a work element attached to a lift arm structure.

FIG. 11 illustrates the front layout for a portion of a front sensor group of the power machine according to some configurations, with the work element detached from the lift arm structure.

DETAILED DESCRIPTION OF THE PRESENT DISCLOSURE

The concepts disclosed in this discussion are described and illustrated by referring to exemplary embodiments. These concepts, however, are not limited in their application to the details of construction and the arrangement of components in the illustrative embodiments and are capable of being practiced or being carried out in various other ways. The terminology in this document is used for the purpose of description and should not be regarded as limiting. Words such as “including,” “comprising,” and “having” and variations thereof as used herein are meant to encompass the items listed thereafter, equivalents thereof, as well as additional items.

Some embodiments of the disclosure are directed to improvements in controlling operation of power machines through an autonomous mode or a remote-controlled mode, as facilitated in particular by the arrangement of particular groups of sensors or operation based on particular sensor sets. As described in greater detail herein, a power machine may include a control system that utilizes data collected by a sensor system (e.g., a sensor system with one or more distinct sensor packages, each with one or more sensor devices configured to gather particular types of data regarding the power machine and its surroundings). In some examples, the sensor system may include one or more sensor groups, with each sensor group arranged for installation on a particular area of the power machine (or component thereof). In this way, for example, a user may install one or more sensor groups on corresponding area(s) of a power machine to easily and reliably enable autonomous or operator-controlled (e.g., remote-controlled) operation of the power machine based on signals provided by the installed sensor group. Further, in some examples, different respective sets of sensors can be used for different modes of operation, including autonomous (e.g., automatic) or operator-controlled modes to allow operation with or without input from human operators.

These concepts can be practiced on various power machines, as will be described below. A representative power machine on which the embodiments can be practiced is illustrated in diagram form in FIG. 1 and one example of such a power machine is illustrated in FIGS. 2-3 and described below before any embodiments are disclosed. For the sake of brevity, only one power machine is illustrated and discussed as being a representative power machine. However, as mentioned above, the embodiments below can be practiced on any of a number of power machines, including power machines of different types from the representative power machine shown in FIGS. 2-3. Power machines, for the purposes of this discussion, include a frame, at least one work element, and a power source that can provide power to the work element to accomplish a work task. One type of power machine is a self-propelled work vehicle. Self-propelled work vehicles are a class of power machines that include a frame, work element, and a power source that can provide power to the work element. At least one of the work elements is a motive system for moving the power machine under power.

FIG. 1 is a block diagram that illustrates the basic systems of a power machine 100, which can be any of a number of different types of power machines, upon which the embodiments discussed below can be advantageously incorporated. The block diagram of FIG. 1 identifies various systems on power machine 100 and the relationship between various components and systems. As mentioned above, at the most basic level, power machines for the purposes of this discussion include a frame, a power source, and a work element. The power machine 100 has a frame 110, a power source 120, and a work element 130. Because power machine 100 shown in FIG. 1 is a self-propelled work vehicle, it also has tractive elements 140, which are themselves work elements provided to move the power machine over a support surface and an operator station 150 that provides an operating position for controlling the work elements of the power machine. A control system 160 is provided to interact with the other systems to perform various work tasks at least in part in response to control signals provided by an operator. For example, the control system 160 can be an integrated or distributed architecture of one or more processor devices and one or more memories that are collectively configured to receive operator input or other input signals (e.g., sensor data) and to output commands accordingly for power machine operations.

Certain work vehicles have work elements that can perform a dedicated task. For example, some work vehicles have a lift arm to which an implement such as a bucket is attached such as by a pinning arrangement. The work element, i.e., the lift arm can be manipulated to position the implement to perform the task. The implement, in some instances can be positioned relative to the work element, such as by rotating a bucket relative to a lift arm, to further position the implement. Under normal operation of such a work vehicle, the bucket is intended to be attached and under use. Such work vehicles may be able to accept other implements by disassembling the implement/work element combination and reassembling another implement in place of the original bucket. Other work vehicles, however, are intended to be used with a wide variety of implements and have an implement interface such as implement interface 170 shown in FIG. 1. At its most basic, implement interface 170 is a connection mechanism between the frame 110 or a work element 130 and an implement, which can be as simple as a connection point for attaching an implement directly to the frame 110 or a work element 130 or more complex, as discussed below.

On some power machines, implement interface 170 can include an implement carrier, which is a physical structure movably attached to a work element. The implement carrier has engagement features and locking features to accept and secure any of a number of different implements to the work element. One characteristic of such an implement carrier is that once an implement is attached to it, it is fixed to the implement (i.e., not movable with respect to the implement) and when the implement carrier is moved with respect to the work element, the implement moves with the implement carrier. The term implement carrier as used herein is not merely a pivotal connection point, but rather a dedicated device specifically intended to accept and be secured to various different implements. The implement carrier itself is mountable to a work element 130 such as a lift arm or the frame 110. Implement interface 170 can also include one or more power sources for providing power to one or more work elements on an implement. Some power machines can have a plurality of work element with implement interfaces, each of which may, but need not, have an implement carrier for receiving implements. Some other power machines can have a work element with a plurality of implement interfaces so that a single work element can accept a plurality of implements simultaneously. Each of these implement interfaces can, but need not, have an implement carrier.

Frame 110 includes a physical structure that can support various other components that are attached thereto or positioned thereon. The frame 110 can include any number of individual components. Some power machines have frames that are rigid. That is, no part of the frame is movable with respect to another part of the frame. Other power machines have at least one portion that can move with respect to another portion of the frame. For example, excavators can have an upper frame portion that rotates with respect to a lower frame portion. Other work vehicles have articulated frames such that one portion of the frame pivots with respect to another portion for accomplishing steering functions.

Frame 110 supports the power source 120, which is configured to provide power to one or more work elements 130 including the one or more tractive elements 140, as well as, in some instances, providing power for use by an attached implement via implement interface 170. Power from the power source 120 can be provided directly to any of the work elements 130, tractive elements 140, and implement interfaces 170. Alternatively, power from the power source 120 can be provided to a control system 160, which in turn selectively provides power to the elements that are capable of using it to perform a work function. Power sources for power machines typically include an engine such as an internal combustion engine and a power conversion system such as a mechanical transmission or a hydraulic system that is configured to convert the output from an engine into a form of power that is usable by a work element. Other types of power sources can be incorporated into power machines, including electrical sources or a combination of power sources, known generally as hybrid power sources.

FIG. 1 shows a single work element designated as work element 130, but various power machines can have any number of work elements. Work elements are typically attached to the frame of the power machine and movable with respect to the frame when performing a work task. In some embodiments, as also discussed above, work elements can include lift arm assemblies. In some embodiments, work elements can include mower decks or other similar equipment. In addition, tractive elements 140 are a special case of work element in that their work function is generally to move the power machine 100 over a support surface. Tractive elements 140 are shown separate from the work element 130 because many power machines have additional work elements besides tractive elements, although that is not always the case. Power machines can have any number of tractive elements, some or all of which can receive power from the power source 120 to propel the power machine 100. Tractive elements can be, for example, track assemblies, wheels attached to an axle, and the like. Tractive elements can be mounted to the frame such that movement of the tractive element is limited to rotation about an axle (so that steering is accomplished by a skidding action) or, alternatively, pivotally mounted to the frame to accomplish steering by pivoting the tractive element with respect to the frame.

Power machine 100 includes an operator station 150 that includes an operating position from which an operator can control operation of the power machine. In some power machines, the operator station 150 is defined by an enclosed or partially enclosed cab. Some power machines on which the disclosed embodiments may be practiced may not have a cab or an operator compartment of the type described above. For example, a walk behind loader may not have a cab or an operator compartment, but rather an operating position that serves as an operator station from which the power machine is properly operated. More broadly, power machines other than work vehicles may have operator stations that are not necessarily similar to the operating positions and operator compartments referenced above. Further, some power machines such as power machine 100 and others, whether or not they have operator compartments or operator positions, may be capable of being operated remotely (i.e., from a remotely located operator station) instead of or in addition to an operator station adjacent or on the power machine. This can include applications where at least some of the operator-controlled functions of the power machine can be operated from an operating position associated with an implement that is coupled to the power machine. Alternatively, with some power machines, a remote-control device can be provided (i.e., remote from both of the power machine and any implement to which is it coupled) that is capable of controlling at least some of the operator-controlled functions on the power machine.

FIGS. 2-3 illustrate a loader 200, which is one particular example of a power machine of the type illustrated in FIG. 1 where the embodiments discussed below can be advantageously employed. Loader 200 is a skid-steer loader, which is a loader that has tractive elements (in this case, four wheels) that are mounted to the frame of the loader via rigid axles. Here the phrase “rigid axles” refers to the fact that the skid-steer loader 200 does not have any tractive elements that can be rotated or steered to help the loader accomplish a turn. Instead, a skid-steer loader has a drive system that independently powers one or more tractive elements on each side of the loader so that by providing differing tractive signals to each side, the machine will tend to skid over a support surface. These varying signals can even include powering tractive element(s) on one side of the loader to move the loader in a forward direction and powering tractive element(s) on another side of the loader to mode the loader in a reverse direction so that the loader will turn about a radius centered within the footprint of the loader itself. The term “skid-steer” has traditionally referred to loaders that have skid steering as described above with wheels as tractive elements. However, it should be noted that many track loaders also accomplish turns via skidding and are technically skid-steer loaders, even though they do not have wheels. For the purposes of this discussion, unless noted otherwise, the term skid-steer should not be seen as limiting the scope of the discussion to those loaders with wheels as tractive elements. Correspondingly, although some example power machines discussed herein are presented as skid-steer power machines, some embodiments disclosed herein can be implemented on a variety of other power machines. For example, some embodiments can be implemented on compact loaders or compact excavators that do not accomplish turns via skid-steer control.

Loader 200 is one particular example of the power machine 100 illustrated broadly in FIG. 1 and discussed above. To that end, features of loader 200 described below include reference numbers that are generally similar to those used in FIG. 1. For example, loader 200 is described as having a frame 210, just as power machine 100 has a frame 110. Skid-steer loader 200 is described herein to provide a reference for understanding one environment on which the embodiments described below related to track assemblies and mounting elements for mounting the track assemblies to a power machine may be practiced. The loader 200 should not be considered limiting especially as to the description of features that loader 200 may have described herein that are not essential to the disclosed embodiments and thus may or may not be included in power machines other than loader 200 upon which the embodiments disclosed below may be advantageously practiced. Unless specifically noted otherwise, embodiments disclosed below can be practiced on a variety of power machines, with the loader 200 being only one of those power machines. For example, some or all of the concepts discussed below can be practiced on many other types of work vehicles such as various other loaders, excavators, trenchers, and dozers, to name but a few examples.

Loader 200 includes frame 210 that supports a power system 220, the power system being capable of generating or otherwise providing power for operating various functions on the power machine. Power system 220 is shown in block diagram form, but is located within the frame 210. Frame 210 also supports a work element in the form of a lift arm assembly 230 that is powered by the power system 220 and that can perform various work tasks. As loader 200 is a work vehicle, frame 210 also supports a traction system 240, which is also powered by power system 220 and can propel the power machine over a support surface. The lift arm assembly 230 in turn supports an implement interface 270, which includes an implement carrier 272 that can receive and secure various implements to the loader 200 for performing various work tasks and power couplers 274, to which an implement can be coupled for selectively providing power to an implement that might be connected to the loader. Power couplers 274 can provide sources of hydraulic or electric power or both. The loader 200 includes a cab 250 that defines an operator station 255 from which an operator can manipulate various control devices 260 to cause the power machine to perform various work functions. Cab 250 can be pivoted back about an axis that extends through mounts 254 to provide access to power system components as needed for maintenance and repair.

The operator station 255 includes an operator seat 258 and a plurality of operation input devices, including control levers 260 that an operator can manipulate to control various machine functions. Operator input devices can include buttons, switches, levers, sliders, pedals and the like that can be stand-alone devices such as hand operated levers or foot pedals or incorporated into hand grips or display panels, including programmable input devices. Actuation of operator input devices can generate signals in the form of electrical signals, hydraulic signals, or mechanical signals. Signals generated in response to operator input devices are provided to various components on the power machine for controlling various functions on the power machine. Among the functions that are controlled via operator input devices on power machine 200 include control of the tractive elements 219, the lift arm assembly 230, the implement carrier 272, and providing signals to any implement that may be operably coupled to the implement.

Loaders can include human-machine interfaces including display devices that are provided in the cab 250 to give indications of information relatable to the operation of the power machines in a form that can be sensed by an operator, such as, for example audible or visual indications. Audible indications can be made in the form of buzzers, bells, and the like or via verbal communication. Visual indications can be made in the form of graphs, lights, icons, gauges, alphanumeric characters, and the like. Displays can provide dedicated indications, such as warning lights or gauges, or dynamic to provide programmable information, including programmable display devices such as monitors of various sizes and capabilities. Display devices can provide diagnostic information, troubleshooting information, instructional information, and various other types of information that assists an operator with operation of the power machine or an implement coupled to the power machine. Other information that may be useful for an operator can also be provided. Other power machines, such walk behind loaders may not have a cab nor an operator compartment, nor a seat. The operator position on such loaders is generally defined relative to a position where an operator is best suited to manipulate operator input devices.

Various power machines that can include or interact with the embodiments discussed below can have various different frame components that support various work elements. The elements of frame 210 discussed herein are provided for illustrative purposes and frame 210 is not the only type of frame that a power machine on which the embodiments can be practiced can employ. Frame 210 of loader 200 includes an undercarriage or lower portion 211 of the frame and a mainframe or upper portion 212 of the frame that is supported by the undercarriage. The mainframe 212 of loader 200, in some embodiments is attached to the undercarriage 211 such as with fasteners or by welding the undercarriage to the mainframe. Alternatively, the mainframe and undercarriage can be integrally formed. Mainframe 212 includes a pair of upright portions 214A and 214B located on either side and toward the rear of the mainframe that support lift arm assembly 230 and to which the lift arm assembly 230 is pivotally attached. The lift arm assembly 230 is illustratively pinned to each of the upright portions 214A and 214B. The combination of mounting features on the upright portions 214A and 214B and the lift arm assembly 230 and mounting hardware (including pins used to pin the lift arm assembly to the mainframe 212) are collectively referred to as joints 216A and 216B (one is located on each of the upright portions 214) for the purposes of this discussion. Joints 216A and 216B are aligned along an axis 218 so that the lift arm assembly is capable of pivoting, as discussed below, with respect to the frame 210 about axis 218. Other power machines may not include upright portions on either side of the frame or may not have a lift arm assembly that is mountable to upright portions on either side and toward the rear of the frame. For example, some power machines may have a single arm, mounted to a single side of the power machine or to a front or rear end of the power machine. Other machines can have a plurality of work elements, including a plurality of lift arms, each of which is mounted to the machine in its own configuration. Frame 210 also supports a pair of tractive elements in the form of wheels 219A-D on either side of the loader 200.

The lift arm assembly 230 shown in FIGS. 2-3 is one example of many different types of lift arm assemblies that can be attached to a power machine such as loader 200 or other power machines on which embodiments of the present discussion can be practiced. The lift arm assembly 230 is what is known as a vertical lift arm, meaning that the lift arm assembly 230 is moveable (i.e., the lift arm assembly can be raised and lowered) under control of the loader 200 with respect to the frame 210 along a lift path 237 that forms a generally vertical path. Other lift arm assemblies can have different geometries and can be coupled to the frame of a loader in various ways to provide lift paths that differ from the radial path of lift arm assembly 230. For example, some lift paths on other loaders provide a radial lift path. Other lift arm assemblies can have an extendable or telescoping portion. Other power machines can have a plurality of lift arm assemblies attached to their frames, with each lift arm assembly being independent of the other(s). Unless specifically stated otherwise, none of the inventive concepts set forth in this discussion are limited by the type or number of lift arm assemblies that are coupled to a particular power machine.

The lift arm assembly 230 has a pair of lift arms 234 that are disposed on opposing sides of the frame 210. A first end 232A of each of the lift arms 234 is pivotally coupled to the power machine at joints 216 and a second end 232B of each of the lift arms is positioned forward of the frame 210 when in a lowered position as shown in FIG. 2. Joints 216 are located toward a rear of the loader 200 so that the lift arms extend along the sides of the frame 210. The lift path 237 is defined by the path of travel of the second end 232B of the lift arms 234 as the lift arm assembly 230 is moved between a minimum and maximum height.

Each of the lift arms 234 has a first portion 234A of each lift arm 234 is pivotally coupled to the frame 210 at one of the joints 216 and the second portion 234B extends from its connection to the first portion 234A to the second end 232B of the lift arm assembly 230. The lift arms 234 are each coupled to a cross member 236 that is attached to the first portions 234A. Cross member 236 provides increased structural stability to the lift arm assembly 230. A pair of actuators 238, which on loader 200 are hydraulic cylinders configured to receive pressurized fluid from power system 220, are pivotally coupled to both the frame 210 and the lift arms 234 at pivotable joints 238A and 238B, respectively, on either side of the loader 200. The actuators 238 are sometimes referred to individually and collectively as lift cylinders. Actuation (i.e., extension and retraction) of the actuators 238 cause the lift arm assembly 230 to pivot about joints 216 and thereby be raised and lowered along a fixed path illustrated by arrow 237. Each of a pair of control links 217 are pivotally mounted to the frame 210 and one of the lift arms 232 on either side of the frame 210. The control links 217 help to define the fixed lift path of the lift arm assembly 230.

Some lift arms, most notably lift arms on excavators but also possible on loaders, may have portions that are controllable to pivot with respect to another segment instead of moving in concert (i.e., along a pre-determined path) as is the case in the lift arm assembly 230 shown in FIG. 2. Some power machines have lift arm assemblies with a single lift arm, such as is known in excavators or even some loaders and other power machines. Other power machines can have a plurality of lift arm assemblies, each being independent of the other(s).

An implement interface 270 is provided proximal to a second end 232B of the lift arm assembly 230. The implement interface 270 includes an implement carrier 272 that is capable of accepting and securing a variety of different implements to the lift arm assembly 230. Such implements have a complementary machine interface that is configured to be engaged with the implement carrier 272. The implement carrier 272 is pivotally mounted at the second end 232B of the arm 234. Implement carrier actuators 235 are operably coupled the lift arm assembly 230 and the implement carrier 272 and are operable to rotate the implement carrier with respect to the lift arm assembly. Implement carrier actuators 235 are illustratively hydraulic cylinders and often known as tilt cylinders.

By having an implement carrier capable of being attached to a plurality of different implements, changing from one implement to another can be accomplished with relative ease. For example, machines with implement carriers can provide an actuator between the implement carrier and the lift arm assembly, so that removing or attaching an implement does not involve removing or attaching an actuator from the implement or removing or attaching the implement from the lift arm assembly. The implement carrier 272 provides a mounting structure for easily attaching an implement to the lift arm (or other portion of a power machine) that a lift arm assembly without an implement carrier does not have.

Some power machines can have implements or implement like devices attached to it such as by being pinned to a lift arm with a tilt actuator also coupled directly to the implement or implement type structure. A common example of such an implement that is rotatably pinned to a lift arm is a bucket, with one or more tilt cylinders being attached to a bracket that is fixed directly onto the bucket such as by welding or with fasteners. Such a power machine does not have an implement carrier, but rather has a direct connection between a lift arm and an implement.

The implement interface 270 also includes an implement power source 274 available for connection to an implement on the lift arm assembly 230. The implement power source 274 includes pressurized hydraulic fluid port to which an implement can be removably coupled. The pressurized hydraulic fluid port selectively provides pressurized hydraulic fluid for powering one or more functions or actuators on an implement. The implement power source can also include an electrical power source for powering electrical actuators or an electronic controller on an implement. The implement power source 274 also exemplarily includes electrical conduits that are in communication with a data bus on the excavator 200 to allow communication between a controller on an implement and electronic devices on the loader 200.

Frame 210 supports and generally encloses the power system 220 so that the various components of the power system 220 are not visible in FIGS. 2-3. FIG. 4 includes, among other things, a diagram of various components of the power system 220. Power system 220 includes one or more power sources 222 that are capable of generating or storing power for use on various machine functions. On power machine 200, the power system 220 includes an internal combustion engine. Other power machines can include electric generators, rechargeable batteries, various other power sources or any combination of power sources that can provide power for given power machine components. The power system 220 also includes a power conversion system 224, which is operably coupled to the power source 222. Power conversion system 224 is, in turn, coupled to one or more actuators 226, which can perform a function on the power machine. Power conversion systems in various power machines can include various components, including mechanical transmissions, hydraulic systems, and the like. The power conversion system 224 of power machine 200 includes a pair of hydrostatic drive pumps 224A and 224B, which are selectively controllable to provide a power signal to drive motors 226A and 226B. The drive motors 226A and 226B in turn are each operably coupled to axles, with drive motor 226A being coupled to axles 228A and 228B and drive motor 226B being coupled to axles 228C and 228D. The axles 228A-D are in turn coupled to tractive elements 219A-D, respectively. The drive pumps 224A and 224B can be mechanically, hydraulic, or electrically coupled to operator input devices to receive actuation signals for controlling the drive pumps.

The arrangement of drive pumps, motors, and axles in power machine 200 is but one example of an arrangement of these components. As discussed above, power machine 200 is a skid-steer loader and thus tractive elements on each side of the power machine are controlled together via the output of a single hydraulic pump, either through a single drive motor as in power machine 200 or with individual drive motors. Various other configurations and combinations of hydraulic drive pumps and motors can be employed as may be advantageous.

The power conversion system 224 of power machine 200 also includes a hydraulic implement pump 224C, which is also operably coupled to the power source 222. The hydraulic implement pump 224C is operably coupled to work actuator circuit 238C. Work actuator circuit 238C includes lift cylinders 238 and tilt cylinders 235 as well as control logic to control actuation thereof. The control logic selectively allows, in response to operator inputs, for actuation of the lift cylinders or tilt cylinders. In some machines, the work actuator circuit 238C also includes control logic to selectively provide a pressurized hydraulic fluid to an attached implement. The control logic of power machine 200 includes an open center, 3 spool valve in a series arrangement. The spools are arranged to give priority to the lift cylinders, then the tilt cylinders, and then pressurized fluid to an attached implement.

The description of power machine 100 and loader 200 above is provided for illustrative purposes, to provide illustrative environments on which the embodiments discussed below can be practiced. While the embodiments discussed can be practiced on a power machine such as is generally described by the power machine 100 shown in the block diagram of FIG. 1 and more particularly on a loader such as track loader 200, unless otherwise noted or recited, the concepts discussed below are not intended to be limited in their application to the environments specifically described above.

FIG. 5 is a schematic block diagram of a power machine 500, which can be any of a number of different types of power machines (e.g., wheeled or tracked skid-steer loaders, an excavator, a telescopic handler, a mower, etc.), including any of the types generally discussed above. In the illustrated example of FIG. 5, the power machine 500 may include a frame 505, one or more tractive assemblies 510, a work element configured as a lift arm structure 515, where the lift arm structure 515 is coupled to an implement 520, and one or more sensor groups 525. In some configurations, the power machine 500 or any of the sensor groups 525A through 525D (collectively, the sensor group(s) 525) may include additional, fewer, or different components than illustrated in FIG. 5, in various arrangements. As one example, in some configurations, the power machine 500 may also include an antenna or other remote communication system (not shown), one or more processor devices (not shown), etc.

Each of the sensor groups 525 may include one or more sensors 530. The sensor(s) 530 may be variously configured to collect or detect data associated with the power machine 500 (e.g., data associated with performing an operation with the power machine 500 or data regarding a position of the power machine 500 relative to unknown or known objects). In some configurations, the sensor(s) 530 may include a radar device, a camera, an ultrasonic sensor or device, or the like, of various generally known types. Alternatively, or in addition, the sensor(s) 530 may include other sensors, such as, e.g., a torque sensor, a position sensor, an angle sensor, a speed sensor, an acceleration sensor, an accelerometer, a temperature sensor, a gyrometer, an inertial measurement unit (IMU), or the like. As further discussed below, particular sensor groups 525 may be configured to be secured at particular locations on the power machine 500 (e.g., particular sides) to collectively provide particular combinations of sensor ranges and sensor types for optimized operation in either a remote operation mode or an autonomous operation mode.

In some configurations, the power machine 500 may include a positioning system, such as, e.g., one or more components for interoperation with a global navigation satellite system (“GNSS”) (e.g., a global positioning system (“GPS”)). In some embodiments, the power machine 500 can include one or more components related to implementing or leveraging GPS or other positioning data. In some configurations, one or more of the sensor groups 525 can accordingly include one or more antennas configured to receive GPS data (e.g., one or more corresponding receivers). In order to improve accuracy, the positioning system may include at least two antennas. Generally, the components of the positioning system may be mounted to a variety of components of the power machine 500 using a variety of known brackets or other mounting equipment (e.g., to secure a GPS receiver to the frame 505, the lift arm structure 515, the implement 520, or another component of the power machine 500).

In some configurations, the sensor groups 525 (including the sensors 530 thereof) may be implemented as a sensor kit for equipping the power machine 500 for autonomous operation. Sensor groups 525 of a sensor kit may be installed on the power machine 500 in a collective arrangement to allow the power machine 500 to leverage data collected by the sensor kit for performing autonomous operations or supporting remote operator control. Thus, power machines of a variety of configurations can be readily equipped with a suite of sensors for improved control during manufacturing, or as part of a retrofit or other service operation.

In some configurations, each sensor 530 or sensor group 525 of a sensor kit may be specifically configured or calibrated for a designated area of the power machine 500. For instance, a sensor kit including a rear sensor group may include one or more radar devices and cameras that are specifically configured for collecting data associated with a rear operational area relative to the power machine 500 (e.g., a zone of potential travel behind the power machine 500). In some configurations, a rear operational area may be an operational area that is opposite the implement 520.

Some sensor kits may include a rear sensor group 525D that may be secured at a rear side 545 of the frame 505, one or more lateral side sensor groups 525A, 525C that may be secured at a lateral side 550, 550 of the frame 505, a front sensor group 525B that may be secured at a front side 540 of the frame 505, or a combination thereof. For instance, a sensor kit may sometimes include a single sensor group (e.g., the rear sensor group 525D, one or more of the lateral side sensor groups 525A, 525C, or the front sensor group 525B). Alternatively, in some configurations, a sensor kit may have complete peripheral sensor coverage of the power machine 500 by utilizing multiple sensor groups 525 (e.g., the rear sensor group 525D and the front sensor group 525B, or front, rear, and lateral side sensor groups 525B, 525D, 525A, 525C).

As used herein, for convenience in presentation, “front” and related directional terms are used flexibly to indicate directions relative to an operational area of an implement on a lift arm or other designated work element. Thus, for example, a “front” side of a frame can be defined based on a side of the frame at which a lift arm may operate an implement (e.g., when fully lowered or otherwise at a home position), and that front side can indicate a corresponding opposite “back” or “rear” side, etc. However, such a designated “front” or “rear” side may not necessarily correspond with a direction of forward travel for the power machine or with any other particular direction relative to an operator's point of view.

Although the sensors 530 are shown as distinct structures in FIG. 5, in some configurations, one or more of the sensors 530 may be implemented by or integrated within a single component of the power machine 500 itself. Configuration of the sensors 530 for individual or collective installation as modular components may be particularly beneficial in some cases (e.g., as implemented in a sensor kit for retrofitting or manufacturing power machines of various types). For example, dual rear-facing radar devices of a rear sensor group (as illustrated in FIG. 8) may be mounted to or integrated into the same mounting bracket on the rear side of the power machine 500. In this regard, a variety of mounting brackets or other hardware can be used to secure particular sensors 530 to particular components of the power machine 500, including with a variety of known structures.

FIG. 5 illustrates each of the sensor groups 525 as including three sensors 530. In some configurations, a particular sensor group 525 may include additional or fewer sensors 530. For example, the rear sensor group 525D may include two sensors 530 in some examples or four sensors 530 in other examples (not shown).

Generally, the sensor groups 525 may be arranged for installation on a particular area of the power machine 500 (e.g., on a particular side of the power machine 500). For example, as illustrated schematically in FIG. 5, the lateral side sensor group 525A may be a first side sensor group arranged at the first lateral side 550 of the frame 505, the front sensor group 525B may be a front side sensor group arranged at the front side 540 of the frame 505, the lateral side sensor group 525C may be a second side sensor group arranged at the second lateral side 555 of the frame 505, and the rear sensor group 525D may be a rear sensor group arranged at the rear side 545 of the frame 505. Thus, for example, the sensors 530 of the sensor groups 525A-525D, respectively, are generally configured to be secured (e.g., collectively) to the particular side 550, 545, 555, 540 of the power machine 500.

As illustrated by dashed arrows in FIG. 5, the sensor(s) 530 may be positioned or otherwise coupled to the power machine 500 at various locations, including on (or to) the frame 505, the lift arm structure 515, the implement 520, or the like. In some configurations, one or more of the sensors 530 may be positioned at an interface between two components of the power machine 500. For example, one of the sensors 530 of the sensor groups 525 can be secured between the lift arm structure 515 and the implement 520 (e.g., on an implement carrier or tilt assembly). In the illustrated example, the sensors 530 included in the rear sensor group 525D are positioned on the frame 505, although other structures can also be used (e.g., rear tailgates, etc.). In contrast, the sensors 530 included in the front sensor group 525B are positioned on different areas of the power machine, including the frame 505, the lift arm structure 515, and an interface between the lift arm structure 515 and the implement 520.

As also noted above, the sensor(s) 530 may variously include radar devices, cameras (e.g., infrared or visual spectrum cameras), ultrasonic sensors, or other sensors of different types. In some examples, particular sensor groups 525 may include particular types of sensors, to provide improved collective sensing for particular operations or operational modes. As one example, a rear sensor group may include a rear radar system (e.g., one or more radar devices) and a rear camera system (e.g., one or more cameras). As another example, a side sensor group may include a side ultrasonic system (e.g., one or more ultrasonic sensors) and a side camera system (e.g., one or more cameras). As yet another example, a front sensor group may include a front radar system (e.g., one or more radar devices) and a front camera system (e.g., one or more cameras). In some configurations, a sensor group may include multiple sensors of the same type.

Alternatively, in some configurations, the sensors 530 may be the same type of sensor such that each sensor group 525 includes the same type of sensor. For example, in some configurations, each sensor 530 may be a radar device. In some cases, different types of sensors may be advantageous over other types of sensors. For instance, a first type of sensor may have improved sensing capabilities or reliability over a second type of sensor based on, e.g., environmental or work site factors. As one example, radar devices may perform better in dusty environments than ultrasonic devices as dust may impact the ability of the ultrasonic devices to reliably detect an object. As another example, radar devices may perform better when exposed to vibrations than ultrasonic devices, which may sense the vibrations. For instance, such vibrations may be caused by the power machine 500 itself or by a ground surface being traversed by the power machine 500 during a roading operation.

Accordingly, in some configurations, the type of sensors utilized by the methods and systems described herein may be based on an intended or anticipated application or environment of the power machine 500. While some examples described herein may utilize various combinations of different sensor types, it should be understood that any combination of sensor types may be implemented by the technology disclosed herein, including a configuration where the same sensor type is utilized.

In some configurations, sensors 530 from different sensor groups 525 may be used together for performing a particular operation with the power machine 500. For instance, a sensor from a first sensor group and a sensor from a second different sensor group may each collect data that is used by the power machine 500 for controlling or performing a particular operation. For example, when traveling under autonomous control, the power machine 500 may utilize a set of radar or ultrasonic devices from multiple sensor groups, arranged for cooperative or independent operation. In contrast, when traveling under remote control, the power machine 500 may utilize a set of cameras from multiple sensor groups, alone or in combination with one or more radar or ultrasonic devices. Alternatively, or in addition, in some configurations, sensors 530 from the same sensor group 525 may be used together for performing a particular operation with the power machine 500.

In some examples, different sensors 530 can face in different directions. In this regard, for example, a “front-facing” sensor (e.g., radar device, ultrasonic device, or camera) is a sensor with a detection field that extends in a frontward direction to include a front-to-rear centerline of an associated power machine or has a detection axis (as further discussed below) that extends in a frontward direction with a deviation of less than 45 degrees From parallel to the front-to-rear centerline. A corresponding definition also applies to “rear-facing” (replacing the frontward direction with a rearward direction). Similarly, a “side-facing” sensor may have a detection field that extends transverse to a front-to-rear centerline of a power machine to include a lateral axis that extends through the detection device (e.g., camera) perpendicular to the front-to-rear centerline or has a detection axis that extends transverse to the front-to-rear centerline with a deviation of 45 degrees or more from parallel with the front-to-rear centerline.

To efficiently provide sensor coverage for a variety of operations, sensors 530 of different particular types can be included in particular sensor groups 525 for the power machine 500, with particular sensors 530 facing particular directions. For example, as further discussed below, ultrasonic or other (e.g., non-radar) systems can be used to detect objects in areas along a lateral side of the power machine 500 and radar systems (alone or with supplemental ultrasonics or other sensors) can be used to detect objects in areas to a front or rear of the power machine 500. In various embodiments of the present disclosure, radar devices or systems may be utilized to detect objects in a front direction or a rear direction of the power machine 500. To detect objects lateral to the power machine 500, ultrasonics devices or other (e.g., non-radar) systems may be utilized, particularly when an object (including an individual) approaches the power machine 500 from either side or when the power machine 500 is conducting a turn or skidding operation. Radar devices or systems benefit from increased range relative to ultrasonic-based solutions. Accordingly, the use of radar in the front side or rear side of the power machine 500 may facilitate these primary directions, especially as these primary directions of travel may be conducted at higher velocities relative to turns or skids of the power machine 500, where additional ultrasonic sensor data may be utilized (e.g., when in close proximity to the power machine 500).

In some configurations, particular sets of sensors 530 can be arranged with overlapping or generally complementary coverage to provide improved overall sensor capabilities for the power machine 500. For instance, overlapping coverage may provide for improved sensing accuracy, such as, e.g., by facilitating object detection verification and enhanced object detection quality or confidence level. In some instances, the technology disclosed herein may utilize or otherwise implement sensor fusion techniques or technology, such as, e.g., for handling instances where the same object is detected by two different sensors 530 having an overlapping detection field. In some examples, an amount of overlap between detection fields may be higher for primary directions of travel (e.g., forward travel and reverse travel) than for other directions (e.g., directions extending from a lateral side of a power machine). For example, FIG. 6 schematically illustrates example detection fields associated with one or more sensors 530 of the power machine 500 according to some configurations.

As used herein, a detection field generally refers to an area or region extending outward from the detection device (e.g., the sensor(s) 530 in which the detection device is rated to detect or collect data. Generally, for radar and ultrasonic devices, a detection field can be represented by a conical volume that expands from the device about a central detection axis. Similarly, for cameras, a detection field can be represented by a field of view as defined by an imaging sensor and lens assembly, with a corresponding optical (detection) axis.

With reference to FIG. 6, the power machine 500 may include a rear sensor group (e.g., the rear sensor group 525D of FIG. 5) including a rear radar system and a rear camera system. In particular, the rear radar system may include a first rear radar device 610 and a second rear radar device 615 (e.g., oriented to be rear-facing sensors). The rear camera system may include a rear camera 620 (e.g., a rear-facing visible light or infrared camera).

As illustrated in FIG. 6, the first rear radar device 610 may collect data for a first rear detection field 625, the second rear radar device 615 may collect data for a second rear detection field 630, and the rear camera 620 may collect data for a third rear detection field 635. In particular, the first and second radar devices 610 and 615 may have detection axes that are angled away from a front-to-rear direction (represented in FIG. 6 by the double-arrowed line 626) between the front side 540 and the rear side 545 of the frame 505, but may still provide overlapping radar coverage over part of the respective detection ranges. The rear camera 620 may provide a field of view (e.g., the third rear detection field 635) that is overlapping (e.g., generally coextensive) with the first and second rear radar devices 610, 615 and can thus provide sensor data to supplement or replace sensor data from the first and second rear radar devices 610, 615 in some modes of operation. While the first rear radar device 610, the second rear radar device 615, and the rear camera 620 are generally rear facing, other configurations are readily envisioned. For example, in some specific embodiments, the first rear radar device 610 and the second rear radar device 615 are angled slightly up and outward relative to the front-to-rear centerline (whereby the radars face away from each other) (e.g., as illustrated in FIG. 8). Such an embodiment may position the rear camera 620 high on the rear surface of the power machine 500 and angled downward.

Alternatively, or in addition, in some configurations, the rear sensor group may include an ultrasonic device (e.g., as a sensor 530 of the rear sensor group). The ultrasonic device may collect data for another rear detection field (e.g., a fourth rear detection field). In some configurations, the fourth rear detection field may overlap with the first rear detection field 625 and the second rear detection field 630. Accordingly, in some configurations, the ultrasonic device included in the rear sensor group may have overlapped spatial coverage with the first and second rear radar devices 610 and 615. Alternatively, or in addition, in some configurations, the ultrasonic device may have overlapped spatial coverage with another detection field, such as, e.g., the third detection field 635 of the rear camera 620.

The power machine 500 may also include a first side sensor group arranged at a first lateral side of the frame 505 (e.g., the lateral side sensor group 525A arranged at the first side 550 in FIG. 5) and a second side sensor group arranged at a second lateral side of the frame 505 (e.g., the lateral side sensor group 525C arranged at the third side 555 in FIG. 5). The first side sensor group may include a first side ultrasonic system and a first side camera system. The second side sensor group may include a second side ultrasonic system and a second side camera system. The first side ultrasonic system may include a first side ultrasonic device 640 that may collect data for a first lateral side detection field 645. The second side ultrasonic system may include a second side ultrasonic device 650 that may collect data for a second lateral side detection field 655.

In some examples, a camera system for a lateral side sensor group can include a front-facing camera, a side-facing camera, or a combination thereof. For example, the first side camera system of FIG. 6 may include a front-facing camera and a side-facing camera, and the second side camera system may also include a front-facing camera and a side-facing camera. Thus, for example, some side cameras can provide front-facing views of operations or obstacles can be provided relative to a main direction of travel or main work area of the implement 520, to guide particular travel or work operations. Further, in some cases, other side cameras can then provide side-facing views for improved overall situational awareness (and other benefits).

The power machine 500 may also include a front side sensor group arranged at the front side 540 of the frame 505 (e.g., the front sensor group 525B of FIG. 5). The front side sensor group may include, for example, a first front radar device 660, a second front radar device 665, and a front camera 670. The first front radar device 660 may collect data for a first front detection field 675. The second front radar device 665 may collect data for a second side detection field 680. The front camera 670 may collect data for a third side detection field 685. As one example, the front camera 670 may be arranged to acquire images of the implement 520 or load carried by the implement 520 (e.g., a bucket and the contents thereof), a work machine path ahead of the implement 520. A radar device can be similarly employed. In some configurations, the front side sensor group may include a front ultrasonic device arranged to detect whether an implement is attached to a lift arm (e.g., the lift arm structure 515) of the power machine 500.

As illustrated in the example of FIG. 6, in some configurations, the sensors 530 may be arranged on the power machine 500 such that a detection field of one sensor overlaps in particular ways with one or more other sensors. Accordingly, in some configurations, one or more of the sensor groups or sensors included therein may provide improved object detection and classification functionality (via image analysis using overlapping detection fields). Example improved sensor arrangements, and corresponding object detection and classification techniques, are described in greater detail in U.S. application Ser. No. 17/868,186, titled Systems and Methods for Obstacle Detection for a Power Machine, which is incorporated herein by reference.

As one example, with respect to the rear sensor group of FIG. 6, the first rear detection field 625 overlaps with the third rear detection field 635 and the second rear detection field 630 overlaps with the third rear detection field 635. As another example, with respect to the front sensor group of FIG. 6, the first front detection field 675 overlaps with the third front detection field 685 and the second front detection field 680 overlaps with the third front detection field 685. Accordingly, in some configurations, the rear, side, and front sensor groups may collectively provide ultrasonic or radar sensing over: first, second, and third overlapped front-facing detection fields (e.g., the first front detection field 675, the second front detection field 680, and the third front detection field 685); first and second opposing side-facing detection fields (e.g., the first side detection field 645 and the second side detection field 655), and first and second overlapped rear-facing detection fields (e.g., the first rear detection field 625 and the second rear detection field 630). In some configurations, as also generally discussed above, the side-facing (or other) detection fields may be provided by side ultrasonic system(s).

In some configurations, the power machine 500 may include a control system (e.g., the control system 160) configured to implement different modes of operation for the power machine 500, including an autonomous mode and a remote operator-controlled mode. When the autonomous mode is activated, the control system may control one or more components of the power machine 500 autonomously. As one example, in response to activation of the autonomous mode, the control system may autonomously control the tractive assemblies 510, the workgroup to effect movement of the implement 520, or another component of the power machine 500 based on signals from a sensor group 525, such as, e.g., the rear radar system, the side ultrasonic system, the front radar system, or another sensor or detection system of the power machine 500.

In some specific configurations, the power machine 500, in response to an operator input, may operate in a basic remote operator-controlled mode whereby the control system deactivates the radar or ultrasonic sensor packages and merely transmits videos/images from the cameras to a remote device and in response to receiving control signals from the remote device operates the power machine 500 accordingly. In a more advanced remote operator-controlled mode, the control system operates in accordance with the above functionality but activates the radar or ultrasonic sensor packages. In addition to receiving control signals from the remote device, the power machine control circuitry may also process data from these additional sensors and possibly override remote operator inputs where the remote operator inputs will lead to an impact with an unknown or known object, for example. In yet other configurations, the power machine 500 may operate in a (semi-)autonomous or automatic work mode whereby the control system executes one or more tasks without direct operator input. In such a configuration, visual-based sensors of the power machine sensor suite may be deactivated and sensor signals from the ultrasonic or radar-based sensors may be relied upon to travel in a work area and to conduct the one or more tasks. In a more advanced (semi-) autonomous or automatic work mode, in addition to the ultrasonic or radar-based sensors, visual-based sensors may be activated continually (or intermittently) to facilitate visual-based object detection and identification.

In contrast, when the operator-controlled mode is activated, the control system may transmit visual data from the rear, side, and front camera systems for display to an operator (via a remote user device or display device). The control system may then control the tractive assemblies 510, the lift arm structure 515, the implement 520, or another component of the power machine 500 based on command signals received from the remote operator in response to the transmitted visual data. In this regard, however, a remote-controlled mode may in some cases still include some autonomous operations. For example, when the operator-controlled mode is activated, the control system may control the tractive assemblies 510, the lift arm structure 515, the implement 520, or another component of the power machine 500 based on signals from a sensor set to perform object avoidance or otherwise supplement, modify, or override commands from a remote operator. In some examples, the power machine 500 can transition between autonomous and remote-controlled modes automatically or based on particular operator input. For example, when in autonomous mode, when an object is detected and the control system is unable to reroute around the object, the power machine 500 may revert to remote operator-controlled mode and may hold a position until an operator manually addresses the situation.

While FIG. 6 is described with reference to particular sensor arrangements and particular sensor types, it should be understood that sensor arrangements or sensor types may be implemented differently than is expressly illustrated in FIG. 6 or expressly discussed above. For instance, the rear sensor group may include only the rear radar system and omit the rear camera system. Alternatively, or in addition, the rear camera 620 may be replaced with a radar device (e.g., a third rear radar device). As another example, the side sensor group(s) (e.g., the first side sensor group or the second side sensor group) may include a radar system in place of (or as a supplement to) the ultrasonic system, the camera system, or a combination thereof. For example, one or more of the first side ultrasonic device 640, the second side ultrasonic device 650, or the camera(s) of the camera system may be omitted, or replaced with a radar device (e.g., as one or more side radar devices) or with a different type of non-radar sensor (e.g., known light detection and ranging (LIDAR) devices, etc.). As yet another example, the front sensor group may include only a front radar system (e.g., including the first front radar device 660 or the second front radar device 665). As such, in some configurations, the front camera system (e.g., the front camera 670) may be omitted or replaced with a radar device (e.g., a third front radar device).

As a further example, FIG. 7 is a flowchart of a method 700 of controlling operations of a power machine that includes a tractive system and a work element, such as, e.g., the power machine 500. As illustrated in FIG. 7, the method 700 includes receiving, with one or more electronic control devices, a selection of an autonomous mode or an operator-controlled mode (at block 705). In some configurations, the one or more electronic control devices may receive the selection as operator or user input. As one example, an operator may interact with an operator input of the power machine 500 to indicate a mode selection (e.g., an autonomous mode, an operator-controlled mode, a normal mode, etc.). In response to receiving the selection, the one or more electronic control devices may control the tractive assembly 510 or the work element (e.g., the lift arm structure 515 or the implement 520) according to the received selection (at block 710).

When operating in an autonomous mode, the one or more electronic control devices can control the tractive assembly 510 or the work element (e.g., the lift arm structure 515 or the implement 520) based on signals from a first sensor set that includes a rear radar system and a front radar system (at block 715). In some cases, no camera data may be used in an autonomous mode (e.g., with object detection implemented using radar or ultrasonic devices).

When operating in an operator-controlled mode, the one or more electronic control devices may transmit visual data from a second sensor set that includes camera systems, for display to an operator (at block 720). For example, image data can be wirelessly transmitted to a mobile control device of various known types, to display still pictures or videos to an operator to assist in remote control of the power machine 500. The one or more electronic control devices can then determine commands to control the tractive assembly 510 or the work element (e.g., the lift arm structure 515 or the implement 520) based on command signals received from the operator in response to the transmitted visual data (at block 725). For example, a mobile control device as noted above can be used to wirelessly transmit commands to the power machine 500, to be implemented via electronic or other signals to appropriate actuators.

In some configurations, when operating such an operator-controlled mode, the one or more electronic control devices may determine the commands to control the tractive assembly 510 or the work element (e.g., the lift arm structure 515 or the implement 520) further based on signals from a first sensor set that is different from the second sensor set. For example, in response to the signals from the first sensor set as discussed above (e.g., with radar or ultrasonic devices), the one or more electronic control devices may automatically slow, stop, or divert the power machine 500 to avoid a detected obstacle even when these commands are contrary to the operator inputs.

In some examples, different sets of sensors for different modes of operation can be distributed across multiple sensor groups, including with sensors of different sets within the same sensor group or with certain sensors included in multiple sensor sets. For example, with reference again to FIG. 5, the operations of the method 700 as discussed above can be implemented with a first set of sensors that includes radar or ultrasonic devices within each of the sensor groups 525, and with a second set of sensors that includes cameras in the sensor groups 525B, 525D and ultrasonic devices in the lateral side sensor groups 525A, 525C. In some configurations, the operations of the method 700 as discussed above can be implemented with the same type of sensors (as opposed to various combinations of different sensor types). For instance, in some configurations, the method 700 may be implemented with an arrangement including only radar devices.

FIGS. 8-11 illustrate example component layouts for the power machine 500 according to some configurations. In some examples, the components illustrated in FIGS. 8-11 may correspond to particular implementations of the sensor groups 525 of FIG. 5. In other examples, however, other configurations are possible, including with additional, fewer, or different sensor components or groups than expressly shown.

In particular, the power machine as shown in FIGS. 8-11 is configured as an example of a power machine with a lift arm structure that is described in greater detail in U.S. Provisional Application No. 63/401,451, titled Lift Arm Arrangements for Power Machines. In other examples, however, other configurations are possible, including for power machines with the same or different lift arm structures.

In particular, FIG. 8 illustrates an example rear layout 800 for a rear portion of the power machine 500 (e.g., the rear side 545 of the frame 505) according to some configurations. As illustrated in the example of FIG. 8, the rear layout 800 of the power machine 500 may include multiple taillights 802, a camera 805, a first radar device 820, a second radar device 825, a first antenna 830, and a second antenna 835. In some configurations, the camera 805, the first radar device 820, the second radar device 825, the first antenna 830, and the second antenna 835 may be included as components of a rear sensor group for the power machine 500. In some configurations, the rear layout 800 of FIG. 8 may include additional, fewer, or different sensor components or groups than expressly described or shown. For example, in some configurations, the camera 805 may be omitted or replaced with a radar device (e.g., a third radar device), configured to perform functionality of the replaced component.

The camera 805 may be secured at a top portion of the frame 505 and may generally be oriented as a rear-facing camera. In some specific configurations, the camera 805 may be angled downward. Accordingly, the camera 805 may acquire visual data for an area rearward of the power machine 500 (e.g., for remote display to an operator when the power machine 500 is operated in a remote-controlled mode). Accordingly, in some configurations, the camera 805 may be activated to acquire or provide visual data when the power machine 500 is in an operator-controlled mode (e.g., a remote-controlled mode). In some configurations, the camera 805 may be deactivated, so as not to acquire or provide visual data when the power machine 500 is in another mode (e.g., an automatic or other autonomous mode). In some alternative automatic/autonomous modes of operation, the rear-facing camera may be activated when tractive elements are operated in reverse. In such a case, the camera data may be utilized by controller circuitry to identify objects (independently of or in conjunction with data from radar 820 or radar 825 indicative of an object).

The first radar device 820 and the second radar device 825 may be arranged on a middle or lower portion of the frame 505, with the first radar device 820 is positioned laterally opposite the second radar device 825. As illustrated in FIG. 8, the first radar device 820 and the second radar device 825 are positioned at a same (or substantially similar) height on the rear portion of the frame 505. In some configurations, the first radar device 820 and the second radar device 825 may be positioned as different heights on the rear portion of the frame 505.

As generally discussed above, the first radar device 820 and the second radar device 825 can be rear-facing devices and can accordingly collect radar data associated with a rearward environment for the power machine 500. In some embodiments, the first radar device 820 and the second radar device 825 can be activated when the power machine 500 is operating in an autonomous mode and deactivated when the power machine 500 is operating in another mode (e.g., a remote-controlled mode). Similarly, in some embodiments, the first and second radar devices 825 can be separately activated, or can be activated in combination with different sets of other sensors in particular operational modes. In some specific remote operation modes, the first or second radar devices 820, 825 may be activated when the tractive elements of the power machine 500 are operated in reverse. In such a case, controller circuitry of the power machine 500 may monitor radar data and inhibit reverse operation (as commanded by a remote operator) where a collision is imminent.

As noted above, in some instances, the power machine 500 may include or utilize a positioning system, such as, e.g., GPS. The first antenna 810 and the second antenna 815 may be included as part of or support the positioning system of the power machine 500. As illustrated in FIG. 8, the rear sensor system may in particular include the first antenna 830 and the second antenna 835, to improve positional accuracy. Generally, the antennae 810, 815 can be positioned at various locations, although rearward, laterally separated, and generally elevated locations may be particularly useful in some examples. As such, for example, the first antenna 810 and the second antenna 815 may be arranged on a top portion of the frame 505, on a rear area of the power machine 500, with the antennae 810, 815 spaced laterally from each other by a full width of the frame 505 of the power machine 500.

FIG. 9 illustrates an example side layout 900 for a lateral side portion of the power machine 500 (e.g., the first side 550 or the third side 555 of the frame 505) according to some configurations. As illustrated in the example of FIG. 8, the side layout 900 of the power machine 500 may include a side light 902, a first camera 905, the second antenna 835, an ultrasonic device 915, and a second camera 910. In some configurations, the first camera 905, the second antenna 835, the ultrasonic device 915, and the second camera 910 may be included as components of a side sensor group for the power machine 500. In some configurations, the side layout 900 of FIG. 9 may include additional, fewer, or different sensor components or groups than expressly described or shown. For example, in some configurations, one or more of the first camera 905, the ultrasonic device 915, or the second camera 910 may be omitted or replaced with a radar device or other sensor configured to generally perform similar functionality as the replaced component (e.g., to provide the same or similar degrees of sensor overlap, etc.).

As illustrated in FIG. 9, the first camera 905 may be supported on the frame of the power machine 500, and the second camera 920 may be supported on the lift arm structure 515 of the power machine 500. Correspondingly, for example, the first camera 905 can be a side facing camera and the second camera 920 can be a front-facing camera. However, other configurations are possible in other examples. Generally, the first camera 905 and the second camera 920 may provide visual data for display to an operator of the power machine 500 (e.g., when the power machine 500 is operated in an operator-controlled mode). Accordingly, in some configurations, the first camera 905 and the second camera 920 may be activated when the power machine 500 is in an operator-controlled mode (e.g., a remote-controlled mode). In some configurations, the first camera 905 and the second camera 920 may also be deactivated when the power machine 500 is in another mode (e.g., an autonomous mode). In some examples, however, one or more of the cameras 905, 920 can be activated during autonomous operations (e.g., to allow object detection via electronic image analysis).

The ultrasonic device 915 may be arranged on the frame 505 of the power machine 500 and may be a side-facing device. Accordingly, for example, the ultrasonic device 915 may collect data associated with a side environment with respect to the power machine 500. This may be useful, in particular, for close proximity object detection in a wide range of environments. In some embodiments, the ultrasonic device 915 may be activated when the power machine 500 is operating in an autonomous mode and deactivated when the power machine 500 is operating in another mode (e.g., a remote-controlled mode). Alternatively, or in addition, in some configurations, the ultrasonic device 915 may be enabled when the power machine 500 is operating in an autonomous mode and another mode (e.g., a remote-controlled mode). Such a side-facing ultrasonic device 915 may be desirable to detect objects lateral to the power machine 500. For example, a human approaching the power machine 500 from the side or where a skidding operation of the power machine 500 brings a rear-quarter panel into close proximity with an unknown or known object.

FIG. 10 illustrates an example front layout 1000 for a front portion of the power machine 500 (e.g., the front side 540 of the frame 505) according to some examples, with the implement 520 configured as an example bucket that is attached to the lift arm structure 515 at a pivotable implement carrier. As illustrated in the example of FIG. 10, the front layout 1000 of the power machine 500 may include multiple head lights 1002, a camera 1005, and a radar device 1010. In some configurations, the camera 1005 and the radar device 1010 may be included as components of a front sensor group for the power machine 500. In some configurations, the front layout 1000 of FIG. 10 may include additional, fewer, or different sensor components or groups than expressly described or shown. For example, in some configurations, the camera 1005 may be omitted or replaced with a radar device configured to perform functionality of the camera 1105.

In some cases, the camera 1005 may be arranged on a top portion of the lift arm structure 515 or a tilt assembly of the implement 520 and may function as a front-facing camera. Accordingly, the camera 1005 may provide visual data for display to an operator of the power machine 500 (e.g., when the power machine 500 is operated in a remote-controlled mode). For example, the camera 1005 may provide visual data of the implement 520 (e.g., a bucket, as shown, and the contents therein). Relatedly, in some configurations, the camera 1005 may be activated when the power machine 500 is in a remote-controlled mode (e.g., a remote-controlled mode). In some configurations, the camera 1005 may be deactivated when the power machine 500 is in another mode (e.g., an autonomous mode). In some examples, a different sensor (e.g., a radar or ultrasonic device) can be similarly arranged to similarly monitor the contents or other general state of the implement 520.

In some cases, the radar device 1010 may be arranged on a middle or lower portion of the lift arm structure 515 (e.g., at or near an interface connection between the lift arm structure 515 and the implement 520). For example, the radar device 1010 can be included on an implement carrier or, as shown in the example of FIG. 10, a tilt assembly for the implement 520. Accordingly, the radar device 1010 may collect radar data associated with a front work environment in relation to the power machine 500. In some configurations, the radar device 1010 may be activated when the power machine 500 is operating in an autonomous mode and deactivated when the power machine 500 is operating in another mode (e.g., a remote-controlled mode). In an alternative embodiment, one or more radar devices may be mounted in a substantially forward-facing direction on one or more of the lift arms of the power machine 500. In some specific implementations, one or more radar devices may be mounted outboard on each of the lift arms. These radar devices may be mounted with a small outboard angle relative to a longitudinal axis of the power machine 500.

In some instances, the lift arm structure 515 or the implement 520 may block part of a field of detection of the radar device 1010 or the camera 1005, such as, e.g., when the lift arm structure 515 or the implement 520 is lifted. In such instances, the front work environment (or a portion thereof) may be blocked (e.g., not monitored or visible). Accordingly, in some configurations, the front sensor group of the power machine 500 may include an additional sensor 530 (e.g., a radar device). The radar device (or other sensor 530) may be arranged on a front portion of the power machine 500 such that the radar device may collect data associated with the front work environment (or portion thereof), including in situations when the lift arm structure 515 or the implement 520 is lifted. In some examples, the radar device may be mounted to the frame 505 or other portion of the power machine 500 where a detection field of the radar device is unobstructed by the lift arm structure 515 or the implement 520 when the lift arm structure 515 or the implement 520 is in a lifted position.

In some examples, the front sensor group may include two additional sensors (e.g., the first front radar device 660 and the second front radar device 665 of FIG. 6). As noted above, with respect to FIG. 6, the front sensor group of the power machine 500 may include the first front radar device 660 having the first front detection field 675 and the second front radar device 665 having the second side detection field 680. As illustrated in FIG. 6, the first and second front detection fields 675 and 680 may provide unobstructed detection of the front work environment of the power machine 500 when the lift arm structure 515 or the implement 520 is in a lifted position.

In some configurations, the control system of the power machine 500 may prioritize data used to control the power machine 500 based on a position of the lift arm structure 515 or the implement 520, and, in some instances, based on a present operation or work task of the power machine 500. For example, the control system may prioritize data from the first and second front radar devices 660, 665 over the data from the camera 1005 or the radar device 1010 depending on whether the lift arm structure 515 or the implement 520 is lifted. As one specific example, when the lift arm structure 515 or the implement 520 is in a lifted position and a present operation of the power machine 500 includes moving in a forward direction, the control system may prioritize data collected by the first front radar device 660 or the second front radar device 665 for controlling movement of the power machine 500 in the forward direction, detecting objects within the front work environment of the power machine 500 as the power machine 500 moves forward, etc.

FIG. 11 illustrates an example of the front layout 1000 with the implement 520 detached from the lift arm structure 515, according to some configurations. As illustrated in the example of FIG. 11, the front layout 1000 of the power machine 500 may include an ultrasonic device 1105. In some configurations, the front layout 1000 of FIG. 11 may include additional, fewer, or different sensor components or groups than expressly described or shown. For example, in some configurations, the ultrasonic device 1105 may be omitted or replaced with a radar device configured to perform functionality of the ultrasonic device 1105.

The ultrasonic device 1105 may be positioned on or near an implement carrier (or another interface for attaching the implement 520 to the lift arm structure 515). In some configurations, the ultrasonic device 1105 may collect data indicating whether the implement 520 is properly aligned with or attached to the lift arm structure 515. Alternatively, or in addition, in some configurations, the ultrasonic device 1105 may collect data describing the implement 520, such as, e.g., data for identifying a type of implement, an orientation of the implement 520, etc. In some configurations, the ultrasonic device 1105 may be enabled when the power machine 500 is operating in an autonomous mode, a remote-controlled mode, or other modes. In an autonomous mode, signals from the ultrasonic device 1105 may facilitate de-coupling of an implement by providing controller circuitry of the power machine 500 with relative positioning between the implement carrier and an implement.

In some embodiments, aspects of the technology disclosed herein, including computerized implementations of methods according to the technology disclosed herein, can be implemented as a system, method, apparatus, or article of manufacture using standard programming or engineering techniques to produce software, firmware, hardware, or any combination thereof to control a processor device (e.g., a serial or parallel general purpose or specialized processor chip, a single-or multi-core chip, a microprocessor, a field programmable gate array, any variety of combinations of a control unit, arithmetic logic unit, and processor register, and so on), a computer (e.g., a processor device operatively coupled to a memory), or another electronically operated controller to implement aspects detailed herein. Accordingly, for example, embodiments of the technology disclosed herein can be implemented as a set of instructions, tangibly embodied on a non-transitory computer-readable media, such that a processor device can implement the instructions based upon reading the instructions from the computer-readable media. Some embodiments of the technology disclosed herein can include (or utilize) a control device such as an automation device, a special purpose or general purpose computer including various computer hardware, software, firmware, and so on, consistent with the discussion below. As specific examples, a control device can include a processor, a microcontroller, a field-programmable gate array, a programmable logic controller, logic gates etc., and other typical components that are known in the art for implementation of appropriate functionality (e.g., memory, communication systems, power sources, user interfaces and other inputs, etc.).

The term “article of manufacture” as used herein is intended to encompass a computer program accessible from any computer-readable device, carrier (e.g., non-transitory signals), or media (e.g., non-transitory media). For example, computer-readable media can include but are not limited to magnetic storage devices (e.g., hard disk, floppy disk, magnetic strips, and so on), optical disks (e.g., compact disk (CD), digital versatile disk (DVD), and so on), smart cards, and flash memory devices (e.g., card, stick, and so on). Additionally, it should be appreciated that a carrier wave can be employed to carry computer-readable electronic data such as those used in transmitting and receiving electronic mail or in accessing a network such as the Internet or a local area network (LAN). Those skilled in the art will recognize that many modifications may be made to these configurations without departing from the scope or spirit of the claimed subject matter.

Certain operations of methods according to the technology disclosed herein, or of systems executing those methods, may be represented schematically in the FIGS. or otherwise discussed herein. Unless otherwise specified or limited, representation in the FIGS. of particular operations in particular spatial order may not necessarily require those operations to be executed in a particular sequence corresponding to the particular spatial order. Correspondingly, certain operations represented in the FIGS., or otherwise disclosed herein, can be executed in different orders than are expressly illustrated or described, as appropriate for particular embodiments of the technology disclosed herein. Further, in some embodiments, certain operations can be executed in parallel, including by dedicated parallel processing devices, or separate computing devices configured to interoperate as part of a large system.

As used herein in the context of computer implementation, unless otherwise specified or limited, the terms “component,” “system,” “module,” “block,” and the like are intended to encompass part or all of computer-related systems that include hardware, software, a combination of hardware and software, or software in execution. For example, a component may be, but is not limited to being, a processor device, a process being executed (or executable) by a processor device, an object, an executable, a thread of execution, a computer program, or a computer. By way of illustration, both an application running on a computer and the computer can be a component. One or more components (or system, module, and so on) may reside within a process or thread of execution, may be localized on one computer, may be distributed between two or more computers or other processor devices, or may be included within another component (or system, module, and so on).

Also as used herein, unless otherwise expressly limited or defined, the term “automatic” refers to operations (or systems, etc.) that are at least partly dependent on electronic application of computer algorithms for decision-making without human intervention. In this regard, for example, “automatic travel” refers to travel of a power machine or other vehicle in which at least some decisions regarding steering, speed, distance, or other travel parameters are made without direct intervention by a human operator. Relatedly, the term “autonomous,” unless otherwise expressly limited or defined, refers to a subset of automatic operations (or systems, etc.) that control a power machine with no real-time input from a human operator. Thus, for example, automatic operation of a power machine can be controlled by a real-time combination of computer and human decision making (e.g., with divided control relative to speed, travel path, workgroup operations, etc.). In contrast, speed, travel path, and workgroup operations can all be under real-time control only of computer algorithms during autonomous operation of a power machine. In this regard, however, it should be understood that operator input may sometimes be received to start, stop, interrupt, or define boundary parameters (e.g., top speed) for autonomous travel or other autonomous operations.

Also as used herein, unless otherwise limited or defined, “or” indicates a non-exclusive list of components or operations that can be present in any variety of combinations, rather than an exclusive list of components that can be present only as alternatives to each other. For example, a list of “A, B, or C” indicates options of: A; B; C; A and B; A and C; B and C; and A, B, and C. Correspondingly, the term “or” as used herein is intended to indicate exclusive alternatives only when preceded by terms of exclusivity, such as “either,” “one of,” “only one of,” or “exactly one of.” Further, a list preceded by “one or more” (and variations thereon) and including “or” to separate listed elements indicates options of one or more of any or all of the listed elements. For example, the phrases “one or more of A, B, or C” and “at least one of A, B, or C” indicate options of: one or more A; one or more B; one or more C; one or more A and one or more B; one or more B and one or more C; one or more A and one or more C; and one or more of each of A, B, and C. Similarly, a list preceded by “a plurality of” (and variations thereon) and including “or” to separate listed elements indicates options of multiple instances of any or all of the listed elements. For example, the phrases “a plurality of A, B, or C” and “two or more of A, B, or C” indicate options of: A and B; B and C; A and C; and A, B, and C. In general, the term “or” as used herein only indicates exclusive alternatives (e.g., “one or the other but not both”) when preceded by terms of exclusivity, such as “either,” “one of,” “only one of,” or “exactly one of.”

Although the technology disclosed herein has been described by referring to preferred embodiments, workers skilled in the art will recognize that changes may be made in form and detail without departing from the scope of the discussion.

AUTONOMOUS SENSOR KIT FOR CONTROLLING OPERATIONS OF POWER MACHINES

Information

Publication Number

Date Filed

Date Published

Inventors

Original Assignees

CPC

International Classifications

Abstract

Description

Claims

CROSS-REFERENCE TO RELATED APPLICATIONS

Provisional Applications (1)