Monitoring System for a Machine

TECHNICAL FIELD

The disclosure generally relates to monitoring systems and, more particularly, to a system for monitoring or reporting the operation of a machine in connection with a portable computing device.

BACKGROUND

Machines including excavators, loaders, dozers, motor graders, haul trucks, and other types of equipment are used to perform a variety of tasks. When training a new operator, an instructor will often ride along with the new operator in the cab of the machine. To do so, machines may be equipped with a “jump-seat” for use by the instructor or for use by other personnel. These seats take up additional space in the operator cab and may be relatively uncomfortable.

Regardless of whether training a new operator or managing an experience operator, it is often desirable to monitor the performance of an operator and provide feedback regarding their performance. In many cases, the feedback is more beneficial if it is provided shortly after completing a task or an operating shift. Locating an instructor or other personnel within the cab may create a crowded operating environment within the cab and also requires dedication of the instructor or other personnel to a single operator. Improved manners of monitoring operator performance and potentially providing feedback are thus desirable.

U.S. Patent Publication No. 2010/0179844 discloses a system for managing a fleet of material handling vehicles. A communications interface is provided on each vehicle to transmit data regarding the operation of the vehicle. The data is transmitted to a communications system that is connected to a computer system. The computer system processes the data and presents reports on a screen of the computer system. The reports may be configured in many different manners.

The foregoing background discussion is intended solely to aid the reader. It is not intended to limit the innovations described herein, nor to limit or expand the prior art discussed. Thus, the foregoing discussion should not be taken to indicate that any particular element of a prior system is unsuitable for use with the innovations described herein, nor is it intended to indicate that any element is essential in implementing the innovations described herein. The implementations and application of the innovations described herein are defined by the appended claims.

SUMMARY

In one aspect, a system for reporting operator performance of a machine at a work site includes a plurality of sensors for generating signals indicative of operating conditions of the machine and a communications interface for communicating with a system remote from the machine. A portable computing device is removably mounted on the machine and has a processor and a data storage system including a user identification data code unique to a machine operator. A controller is configured to communicate with components of the portable computing device, determine an identity of the machine operator based upon the user identification data code, receive signals from the plurality of sensors, and determine the operating conditions of the machine based upon the signals received from the plurality of sensors. The controller is further configured to determine performance data generated as a result of operation of the machine, generate a performance report based at least in part upon the performance data, and provide the performance report to at least one of the portable computing device and the system remote from the machine.

In another aspect, a method for reporting operator performance of a machine at a work site includes removably mounting on the machine a portable computing device having a processor and a data storage system including a user identification data code unique to a machine operator. The method further includes communicating with components of the portable computing device, determining an identity of the machine operator based upon the user identification data code, receiving signals from a plurality of sensors for generating signals indicative of operating conditions of the machine, and determining the operating conditions of the machine based upon the signals received from the plurality of sensors. Performance data generated as a result of operation of the machine is determined, a performance report based at least in part upon the performance data is generated, and the performance report is provided to at least one of the portable computing device and a system remote from the machine.

In still another aspect, a method of monitoring operator performance of a machine at a work site includes removably mounting on the machine a portable computing device having a portable device controller, receiving signals from a plurality of sensors operatively associated with the machine indicative of the operating conditions of the machine, and determining the operating conditions of the machine based upon the signals. The method further includes transmitting a plurality of machine signals to a machine instrument array indicative of the operating conditions of the machine and, for each operating condition, the machine instrument array having a predetermined configuration, transmitting visual image signals from a visual image sensor indicative of the point of view relative to the machine to a remote console, and displaying point of view images at the remote console based upon the visual image signals. The method still further includes determining a plurality of console signals indicative of the operating conditions of the machine and defining a configuration of a console instrument array of the remote console and, for each of the plurality of operating conditions, the configuration of the console instrument array being generally identically to the predetermined configuration of the machine instrument array. The plurality of console signals are transmitted to the console instrument array of the remote console.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a side elevational view of a motor grader;

FIG. 2 is a schematic representation of an instrument array of a machine together with an instrument array of a remote control console;

FIG. 3 is a front view of a portion of an operator cab including an instrument array;

FIG. 4 is a schematic representation of a portable computing device;

FIG. 5 is an example of images displayed on a display of the machine;

FIG. 6 is a view similar to FIG. 5 but with different images displayed based on a first machine operation;

FIG. 7 is a view similar to FIG. 5 but with still different images displayed based on a second machine operation;

FIG. 8 is a block diagram of a control system of a machine with a portable computing device mounted thereat;

FIG. 9 is a block diagram of a control system of a machine with a portable computing device mounted at a remote control console;

FIG. 10 is a flowchart illustrating a process for operating with an operator in the machine or in a remote control console; and

FIG. 11 is a flowchart illustrating a process for reporting operator performance of the machine.

DETAILED DESCRIPTION

FIG. 1 is a diagrammatic illustration of machine 10 such as a motor grader that may be used in accordance with an embodiment of the disclosure. The motor grader includes a frame 11 and a prime mover such as an engine 12. A set of front wheels 13 may be operatively connected to the frame 11 generally adjacent a front end of the motor grader and two sets of rear wheels 14 may be operatively connected to the frame 11 generally adjacent a rear end of the motor grader. In an alternate embodiment, only a single set of rear wheels 14 may be provided. One or both sets of rear wheels 14 may be powered by a power transfer mechanism (not shown) operatively connected to the engine 12. The power transfer mechanism may be any desired type of drive system including a hydrostatic propulsion system, an electric drive system or a mechanical drive system. An operator cab 15 may be mounted on the frame 11 and may include various controls, gauges, displays and other mechanisms used by an operator.

A work implement such as a blade or moldboard 20 extends downward from the frame 11. The moldboard 20 may be mounted on a blade tilt adjustment mechanism 21 that is supported by a rotatable circle assembly 22 operatively connected to the blade tilt adjustment mechanism 21. A variety of hydraulic cylinders or actuators may be provided for controlling the position of the moldboard 20. For example, circle assembly 22 may be supported by a pair of blade lift actuators 23 (with only one visible in FIG. 1). Adjustment of the blade lift actuators 23 allows the height of rotatable circle assembly 22, and hence the height of moldboard 20, to be adjusted. Blade lift actuators 23 may be moved independently or in combination with each other. A center shift cylinder 24 may be provided to shift the circle assembly 22 from side-to-side. A blade tip cylinder 25 may be provided to control the angle between an edge of the moldboard 20 and the ground. One or more side shift cylinders (not shown) may be provided to control lateral movement of the moldboard 20 relative to the circle assembly 22. The circle assembly 22 may include a mechanism such as gear teeth to allow rotation of the moldboard 20. Other manners of positioning and controlling the moldboard 20 may be utilized if desired.

The machine 10 may be equipped with a plurality of sensors or sensing devices that gather data from various components and systems and generate signals that are directly or indirectly indicative of the performance and operating conditions of the machine. The sensors may generate signals indicative of operating conditions of the machine. Sensors may be associated with, for example, the engine 12, a transmission (not shown), a torque converter (not shown), the front wheels 13, the rear wheels 14, the operator cab 15, the moldboard 20, various actuators such as the blade lift actuators 23, the center shift cylinder 24, the blade tip cylinder 25, fluid supplies (not shown), operator input devices, a parking brake and/or other systems and components of machine 10. These sensors may automatically gather real-time data such as the operation of engine 12, the position of and load on the work implement, fluid pressure, flow rate, temperature, contamination level, and/or viscosity, fluid (i.e., fuel, oil, water, etc.) consumption rates, electric current and/or voltage levels, loading levels (e.g., payload value, percent of maximum allowable payload limit, payload history, payload distribution, etc.), transmission output ratio, and other desired information.

In addition, various sensors may be associated with the machine 10 that may be used to determine machine travel characteristics (e.g., speed, acceleration, torque, slip rate, etc.) as well as the position and orientation of machine 10. For example, an accelerometer 27 may be provided on the machine 10 to provide an acceleration signal indicative of measured acceleration of the machine 10 relative to a gravity reference. In one example, the accelerometer 27 may provide measurements in six degrees of freedom (i.e., fore-aft, lateral, and vertical directions as well as pitch, roll and yaw). In some circumstances, it may be desirable to position the accelerometer 27 generally adjacent operator cab 15 so that movement sensed by the accelerometer somewhat matches movement sensed by an operator.

Still further, a pitch rate sensor 28 (e.g., a gyroscope) may be provided on the machine 10. The pitch rate sensor 28 may be used to provide a pitch rate signal indicative of a pitch rate of the machine 10. As the machine 10 moves, the pitch rate will be indicative of the rate of change of the pitch angle of the machine. The pitch rate sensor 28 may also be used to determine the pitch and roll of the machine 10.

A position sensor 29 may sense a position of the machine 10. The position sensor 29 may include a plurality of individual sensors that cooperate to provide signals to controller 51 to indicate the position of the machine 10. The controller 51 may determine the position of the machine 10 as well as its orientation (i.e., the direction machine 10 is facing). In some instances, the position sensor 29 may be used to determine the pitch and roll of the machine 10. The position sensor 29 may be a series of global positioning system sensors, an odometer or other wheel rotation-sensing sensor, a perception based system or may use other systems such as lasers to determine the position of machine 10.

As depicted schematically in FIG. 2, the operator cab 15 may include an instrument array 30 including one or more display devices and one or more input devices. Each display device may function as a machine display device for displaying images related to the operating conditions of the machine. More specifically, machine 10 may include a plurality of fixed or stationary display devices such as gauges 31 and displays 32 that are permanently mounted within the operator cab 15. As referred to herein, a gauge 31 may have a fixed functionality such that it always reflects or measures the same function (e.g., a fuel gauge, a temperature gauge, a pressure gauge). As referred to herein, a display 32 may have a modifiable or changeable functionality such that it may reflect or measure different functions (e.g., a backup camera display, machine speed, engine speed, or machine event warning display center). Each of the displays 32 may include a computer screen or some other type of display upon which an image such as a computer-generated image may be displayed. Examples of the images displayed on a machine display may include a circular dial, a color-coded indicator, a graph or any other image for conveying information.

Each input device may function as a machine input device for providing commands or data input such as inputting information, changing operations, and issuing commands to the machine 10 and to a remote system 120. More specifically, machine 10 may include a plurality of fixed or stationary input devices such as buttons, knobs, dials, levers, joysticks, and other controls that are permanently mounted within the operator cab 15. One or more of the stationary input devices may have a fixed functionality (referred to herein as a fixed function input device 33) such that they are always used to control the same function (e.g., on/off switches, cab temperature controls, pedals, radio controls). One or more of the stationary input devices may have a modifiable or changeable functionality (referred to herein as a modifiable function input device 34) such that they may be modified to control the input of different functions (e.g., cab temperature, machine lights, or engagement of auto blade position set features). Examples of modifiable function input devices 34 include a touch screen display with a computer-generated image, a knob adjacent a computer display, or any other desired input device.

FIG. 3 depicts a portion of an interior of operator cab 15. Operator cab 15 may include a seat 35 with an adjacent instrument array 30. The instrument array 30 may include a centrally positioned display 32 and a rear view camera display 36 positioned above the display 32. The rear view camera display 36 may be used to display other information if desired. The instrument array 30 may also include a plurality of fixed function input devices 33 in the form of switches and knobs. Still further, the instrument array may include a plurality of joysticks 37 that may function as fixed function or variable or modifiable function input devices. A portable computing device 100 may be removably mounted in operator cab 15 as described below. The portable computing device 100 may form a portion of the instrument array 30.

A control system 50 may be provided to control the operation of the machine 10. The control system 50, as shown generally by an arrow in FIG. 1 indicating association with the machine 10, may include an electronic control module such as controller 51. The controller 51 may receive operator input command signals and control the operation of the various systems of the machine 10. The control system 50 may include one or more input devices to control the machine 10 and one or more sensors to provide data and other input signals representative of various operating conditions of the machine 10.

The controller 51 is shown in FIG. 1 adjacent the operator cab 15 but may be mounted at any convenient location on machine 10. The controller 51 may be an electronic controller that operates in a logical fashion to perform operations, execute control algorithms, store and retrieve data and other desired operations. The controller 51 may include or access memory, secondary storage devices, processors, and any other components for running an application. The memory and secondary storage devices may be in the form of read-only memory (ROM) or random access memory (RAM) or integrated circuitry that is accessible by the controller. Various other circuits may be associated with the controller such as power supply circuitry, signal conditioning circuitry, driver circuitry, and other types of circuitry.

The controller 51 may be a single controller or may include more than one controller disposed to control various functions and/or features of the machine 10. In one embodiment depicted in FIG. 8, the controller 51 may include a machine controller 52 for controlling aspects of machine 10, an engine controller 53 for controller aspects of engine 12, and an implement controller 54 for controlling aspects of the work implement. In another embodiment, machine controller 52 may control aspects of the machine 10, the engine 12, and the work implement. The term “controller” is meant to be used in its broadest sense to include one or more controllers and/or microprocessors that may be associated with the machine 10 and that may cooperate in controlling various functions and operations of the machine. The functionality of the controller 51 may be implemented in hardware and/or software without regard to the functionality. The controller 51 may rely on one or more data maps relating to the operating conditions of the machine 10 that may be stored in the memory of controller. Each of these maps may include a collection of data in the form of tables, graphs, and/or equations. The controller 51 may use the data maps to maximize the efficiency of the machine 10.

As depicted in FIG. 2, a portable computing device 100 may be removably mountable on the machine. The portable computing device 100 may include a central processing unit 101 (FIG. 4), a data storage system 102 such as memory and/or a secondary storage device, and other components for running an application. The central processing unit 101, the data storage system 102, and other aspects of the portable computing device 100 may act as a portable device controller 110 that interacts with machine controller 52 as a component of the controller 51. The portable computing device 100 may also include a display 103, a communications interface such as a wireless interface 104, a camera 105, a microphone 106, a global positioning sensor 107, and one or more input devices 108. If desired, the portable computing device 100 may be removably mounted within operator cab 15 within a docking device or station. As a result, the display 103 may function as a portable device display relative to machine 10. The docking device may function as a machine mounting location 109 for removably mounting the portable computing device 100 thereat to provide power to charge the portable computing device 100 as well as provide data connections to machine 10.

The display 103 may be configured as a touch screen to also operate as a portable device input. The wireless interface 104 may provide a communications channel between the machine controller 52 and the portable computing device 100 as well as between the controller 51 and a remote system 120. In one embodiment, the wireless communication between the machine controller 52 and the portable computing device 100 may be via a Bluetooth® communications system or protocol. Other communications systems and protocols are contemplated. In an alternate embodiment, a wired communications interface (not shown) may be provided to effect the connection between the machine controller 52 and the portable computing device 100. Communication between the machine controller 52 and the portable computing device 100 may be achieved by a wireless connection that occurs once the portable computing device is within a predetermined distance of the machine 10. In another embodiment, communication between the machine controller 52 and the portable computing device 100 may be achieved by a wired connection upon mounting the portable computing device in or on the machine 10. In other words, portable computing device 100 may be configured to communicate with the machine controller wirelessly or through a wired connection.

Components of the portable computing device 100 may supplement or replace some of the components of machine 10. For example, the functionality of controller 51 may be distributed so that certain functions are performed by the machine controller 52 and other functions are performed by the portable computing device 100. In addition, sensors, displays and input devices of the portable computing device 100 may supplement or replace some of the sensors, displays, gauges, and input devices of machine 10. For example and as described in further detail below, the display 103 of the portable computing device 100 may be used to supplement or replace displays permanently mounted on the machine 10. Similarly, the wireless interface 104 of the portable computing device 100 may also be used to supplement or replace a wireless interface of the machine 10.

The camera 105 of the portable computing device 100 may be used to capture images of the work site or the machine 10. These images may be stored in the machine controller 52, in the portable computing device 100, or in a remote system 120 wirelessly connected to the controller 51. Further, the portable computing device 100 may be mounted within operator cab 15 so that camera 105 may be used to monitor actions of an operator such as tracking an operator's eyes to monitor for potential signs of fatigue. Microphone 106 of portable computing device 100 may be used to receive voice commands from an operator and provide the voice commands to the controller 51. The controller may thus be able to operate components or control certain aspects of the operation of machine 10 based upon those voice commands and thus reduce the need for additional input devices. The microphone 106 may also be used to communicate or record notes regarding the operations at the work site or the operation of the machine 10 such as noted hazards or required maintenance items. The global positioning sensor 107 of the portable computing device 100 may used to supplement or replace the sensors of position sensor 29, if desired. As a result, the display 103 and other components of the portable computing device 100 may supplement or combine with the machine display devices and the machine input devices to form a machine instrument array made up of the instrument array 30 and the portable computing device.

In an alternative embodiment, the portable computing device 100 may control and operate all aspects of the machine 10 and thus the portable computing device 100 may function as the controller 51. Regardless of whether the connection between the machine controller 52 and the portable computing device 100 is wired or wireless and regardless of whether controller 51 includes machine controller 52, aspects of the control system 50 may be located remote from the machine 10 and controller 51 may communicate wirelessly to remote system 120 that forms a part of the control system 50.

The portable computing device 100 may store as data codes a plurality of user or operator preferences such as the temperature of the operator cab 15, the settings of the seat within the operator cab 15, input device sensitivity, and desired display images based upon certain operating conditions. Upon docking the portable computing device 100 or moving it within a predetermined range in the case of a wireless connection, the machine controller 52 and the portable computing device may communicate so that controller 51 will operate with the user preferences saved within the portable computing device. The data codes may further include a user identification data code that is specific or unique to each machine operator. As a result, upon docking the portable computing device 100, the controller 51 may determine the identity of the machine operator based upon the user identification data code.

The controller 51 may be configured so that machine 10 is inoperable without portable computing device 100. For example, portable computing device 100 may have saved therein data codes such as authorization keys that prevent the operation of the machine controller 52 without such authorization keys. By requiring appropriate authorization keys, it may be possible to limit an operator's use to only certain types or models of machines 10. More specifically, each operator may be assigned a specific portable computing device 100 and that portable computing device may be set up so as to limit an operator to operate only certain types or models of machines. Still further, the portable computing device 100 may limit an operator to operating one or more specific machines.

The controller 51 may be further configured to limit the operation of machine 10 based upon data codes in the form of limited access keys stored within portable computing device 100. The limited access keys may be used to limit the types of operations performed with the machine 10 as well as limit the time or location at which the machine is operated. For example, a limited access key may be used to prevent an inexperienced operator from performing certain operations (e.g., to prevent operation without sufficient training) In another example, an operator may be prohibited from clearing certain types of data such as performance data as well as fault or error codes. The limited access keys may further be used to prevent a machine 10 from being moved outside of a predetermined area (e.g., outside of a global positioning system defined fence) or from being moved or performing certain operations outside of a predetermined range of hours or times.

Machine operators may be provided with one set or type of limited access keys while maintenance personnel may be provided with a different set or type of limited access keys. As a result, maintenance personnel may be prevented from performing certain operations such as moving a machine or operating certain implements and operators may be prevented from accessing certain components or panels of the machine that should only be accessed by maintenance personnel. Through such limited access keys, a work site manager or machine owner may be able to match the use and access to the machines 10 based upon the ability and authorization of the various personnel. Still further, a record of such use and access may be stored within the portable computing device 100 and/or communicated wirelessly to remote system 120.

As described above, the operator cab 15 may include a plurality of fixed or stationary display devices such as gauges 31 and displays 32 that are permanently mounted within the operator cab 15 as part of the machine instrument array. Other display devices may be removably mounted on the machine 10 such as within the operator cab 15. In one example, if the portable computing device 100 is removably positioned within the operator cab 15, the display 103 of the portable computing device may be used as an additional or portable display within the operator cab. As with the displays 32 that are fixed within the operator cab 15, the display 103 of a portable computing device 100 may be used to display any type of image including those that resemble gauges. The gauges 31, the displays 32, and the display 103 of the portable computing device 100 may interact to form a display system such as the instrument array 30 of machine 10.

As also described above, the operator cab 15 may include a plurality of fixed or stationary input devices such as fixed function input devices 33 and modifiable function input devices 34 that are permanently mounted within the operator cab 15. Other input devices may be removably mounted on the machine 10 such as within the operator cab 15. If the portable computing device 100 is removably positioned within the operator cab 15 and the display 103 includes touch screen functionality, a portion of the display may be used as an input device with modifiable functionality. Still further, the portable computing device 100 may include other components such as microphone 106 that may be used as input devices to control the operation of machine 10. The fixed function input devices 33, the modifiable function input devices 34, and aspects of the portable computing device 100 may interact to form an input system of machine 10.

The controller 51 may be configured to control the images displayed on each of the modifiable displays and gauges. The controller 51 may display different images depending on the operation being performed and the operating conditions of the machine 10. In one example, upon starting machine 10, the displays may indicate the machine status, the temperature of various fluids, and the fuel level. After reporting and confirming the initial levels, it may be desirable to only display this information upon a material change, at predetermined time intervals, or upon the request of an operator.

The controller 51 may change the images being displayed based upon the operation being performed. For example, in FIG. 5, a display 32 is depicted for basic operation of machine 10. The display 32 depicts various images such as a map 38, a speedometer 39, a fuel gauge image 40, and a coolant temperature gauge image 41. The display 32 may also include a gear selection image 42, a differential fluid temperature image 43, an engine oil temperature image 44, and a transmission oil temperature image 45. The display may further identify the name of the operator and an identification code at 46, the number of hours since the engine was last serviced at 47, the status of machine lights at 48.

When the motor grader is performing a grading operation, it may be desirable to only depict some of the images displayed in FIG. 5. As shown in FIG. 6, the map 38 has been replaced by an image 49 from a grade control system and the coolant temperature gauge image 41 has been removed. The speedometer 39, the fuel gauge image 40, the differential fluid temperature image 43, the engine oil temperature image 44, and the transmission oil temperature image 45 have all been moved to be in line along an upper row of the display 32. With this configuration, the operator may be primarily focused on the image 49 from the grade control system.

In another operation such as the motor grader operating on a road, different images may be displayed as depicted in FIG. 7. In such case, the speedometer 39 and images 56 from cameras depicting the area around the motor grader may be emphasized in the display 32.

If an emergency alert or alarm occurs, the controller 51 may display an image on display 32 related to the emergency alert or alarm as well as instructions as to the appropriate steps to be taken or operations to be performed. For example, if one or more of the fluid temperatures exceeds a predetermined threshold, the controller 51 may generate a warning for the operator on the display 32.

Although described above with respect to a motor grader, the present disclosure may be applicable to many other types of machines. For example, when operating a wheel loader, controller 51 may display an indication of how close the wheels are to slipping. This information may be used by an operator to maximize or increase the performance of the wheel loader. In another example, when loading an excavator or a wheel loader, the controller 51 may display mass or weight of the load in the work implement of the machine.

A visual image system 55 such as a camera system may be provided for generating visual images indicative of a point of view relative to the machine 10. The visual image system 55 may include a plurality of visual image sensors such as cameras (not shown) for generating visual image signals. The visual image signals may be transmitted wirelessly to a system remote from machine 10. In doing so, the visual image signals may be processed to some extent by controller 51 at machine 10 and subsequently transmitted to remote system 120 or transmitted to the remote system and processed by the remote system. A plurality of cameras (not shown) of the visual image system 55 may be positioned to capture different views that an operator would have from within the operator cab 15 of machine 10. In an alternate embodiment, a plurality of cameras (not shown) may be positioned to provide a point of view including the machine 10 and a portion of the work site at which the machine is operating.

An example of a remote system may be a remote control console 130 schematically depicted adjacent instrument array 30 in FIG. 2. For example, the remote control console 130 may include an instrument array such as a console instrument array 131, a point of view display 132, and a console controller 133. The console instrument array 131 may include one or more display devices and one or more input devices. Each display device may function as a console display device for displaying images related to the operating conditions of the machine. As with the machine instrument array described above, console instrument array 131 may include a plurality of fixed or stationary display devices such as gauges 134 and displays 135 that are permanently mounted at the remote control console 130. The gauges 134 of remote control console 130 may also have a fixed functionality such that they always reflect or measure the same function. The displays 135 of the remote control console 130 may also have a modifiable or changeable functionality such that they may reflect or measure different functions. Each of the displays 135 may include a computer screen or some other type of display upon which an image such as a computer-generated image may be displayed as described above with respect to displays 32.

Each input device may function as a console input device for providing commands or data input such as inputting information, changing operations, and issuing commands to the machine 10 through the remote control console 130. As with the machine instrument array described above, console instrument array 131 may include a plurality of fixed or stationary input devices such as buttons, knobs, dials, levers, joysticks, and other controls that are permanently mounted the remote control console 130. Fixed function input devices 136 of the remote control console 130 may have a fixed functionality such that they are always used to control the same function. The modifiable function input devices 137 may have a modifiable or changeable functionality such that they may be modified to control the input of different functions. The fixed function input devices 136 and the modifiable function input devices 137 may be similar to the respective fixed function input device 33 and the modifiable function input device 34 as described above.

The point of view display 132 may include one or more displays that are positioned adjacent the console instrument array 131 in a manner as desired such as for efficient operation of the remote control console 130 by an operator. In doing so, the point of view display 132 may include a plurality of displays or areas on a single display upon which visual images are displayed to depict a point of view relative to machine 10. For example, a first point of view display 140 may be centered above the console instrument array 131, and a pair of second point of view displays 141 positioned on opposite sides of the first point of view display. The first point of view display 140 may be used to display images in front of machine 10 while the second point of view displays 141 may be used to display images to the right and left of the machine. If desired, additional images of other points of view may be displayed such as behind the machine 10 or views of other aspects of the machine such as the blade.

In addition to configuring the point of view display 132 in a manner similar to that which an operator would see when operating machine 10, the console instrument array 131 of remote control console 130 may also be configured generally identically to the instrument array 30 within the operator cab 15. By positioning the various display devices and the input devices in generally the same location on the console instrument array 131, the transition of an operator moving from operating machine 10 from within the operator cab 15 to operating machine 10 from remote control console 130 may be simplified. In one example, the console display devices and console input devices of the console instrument array 131 may be generally identical in configuration and position to each of the machine display devices and the machine input devices of instrument array 30. More specifically, the console instrument array 131 may be configured so that it includes gauges 134 and displays 135 that generally match the gauges 31 and displays 32 of the machine instrument array. Similarly, fixed function input devices 136 and modifiable function input devices 137 may be provided on console instrument array 130 so that they generally match the fixed function input devices 33 and the modifiable function input devices 34 of the instrument array 30.

In an alternate embodiment, rather than build console instrument arrays 131 that match the instrument array 30 of each type of machine 10, some or all of the fixed function components of the instrument array 30 may be replaced on console instrument array 131 by images on displays 135 of the console instrument array. For example, controller 51 may generate images on the displays 135 of console instrument array 131 that are generally identical to some or all of the gauges 31 of the instrument array 30. In instances in which the images are not generally identically configured, it may still be desirable to position the images of the gauges 31 on the displays 135 in generally identical positions as compared to those on the instrument array 30.

Similarly, some or all of the fixed function input devices 33 may be replaced by modifiable function input devices 137 on console instrument array 131 in the form of images on a touch screen display or a display associated with another input device. In instances in which the input devices or images thereof are not generally identically configured, it may still be desirable to position the input devices on the console instrument array 131 in generally identical positions to those on the instrument array 30.

The console controller 133 may form a portion of controller 51 to assist in the operation of machine 10. In doing so, the console controller 133 may interact with the machine controller 52 as well as any of the other controllers associated with the machine 10 to control various functions and/or features of the machine 10.

Still further, the remote control console 130 may include a console mounting location 138 for removably mounting the portable computing device 100 thereat. As with the instrument array 30, components of the portable computing device 100 may supplement or replace some of the components of remote control console 130. For example, the display 103 of portable computing device 100 may supplement or replace aspects of the console instrument array 131. In addition, the functionality of controller 51 may be distributed so that certain functions are performed by the machine controller 52 (as well as other controllers on the machine 10), and other functions may be performed by the portable device controller 110 of portable computing device 100, and still other functions may be performed by the console controller 133. As with mounting the portable computing device 100 on the machine 10, sensors, displays and input devices of the portable computing device 100 may supplement or replace some of the sensors, displays, gauges, and input devices of remote control console 130 upon mounting the portable computing device on the remote control console. As a result, upon operatively mounting the portable computing device 100 at the console mounting location 138, the display devices of the console instrument array 131 may be configured to display information related to operation of the machine and the console input devices may be configured to receive input commands from an operator.

Through such a configuration, upon operatively mounting the portable computing device 100 at machine mounting location 109 of machine 10, the controller 51 may operate to include the machine controller 52, any other controllers on machine 10, and the portable device controller 110. The controller 51 may be configured to receive signals from the plurality of sensors of the machine 10 and determine the operating conditions of the machine based upon the signals.

For each operating condition of the machine 10, a predetermined configuration of the instrument array 30 of the machine 10 may be established or determined. If desired, a plurality of possible predetermined configurations may be stored in data maps of the controller 51. Once the operating conditions of the machine are determined, the controller 51 may transmit a plurality of signals (i.e., machine signals) to the machine instrument array indicative of the operating conditions of the machine 10. In instances in which the instrument array 30 includes displays 32, the signals may reflect the image and location of each display as well as the value of the operating condition displayed.

Upon operatively mounting the portable computing device 100 at the console mounting location 138 of remote control console 130, the controller 51 may operate to include the machine controller 52, any other controllers on machine 10, the portable device controller 110, and the console controller 133. As in the situation in which the portable computing device 100 is operatively mounted at machine, the controller 51 may receive signals from the plurality of sensors of the machine 10 and determine the operating conditions of the machine based upon the signals. The controller 51 may also transmit visual image signals from one or more visual image sensors associated with the machine 10, process the signals and display point of view images on the point of view display 132 at the remote control console 130 based upon the visual image signals.

Based upon the operating conditions of the machine 10 and the predetermined configuration of the machine instrument array for those operating conditions when the portable computing device 100 is mounted at machine 10, controller 51 may generate and transmit a plurality of signals (i.e., console signals) to the console instrument array 131. The signals may be indicative of the operating conditions of the machine and define a configuration of the console instrument array so that, for each operating condition, the configuration of the console instrument array is generally identically to the predetermined configuration of the machine instrument array. The controller 51 may further be configured to transmit signals from the remote control console 130 to the machine 10 to direct the machine to perform desired operations.

By configuring the machine 10 and the remote control console 130 to receive a portable computing device 100 having a display 103, the display forms a portion of the machine display of the machine instrument array upon mounting the portable computing device at the machine 10 and forms a portion of the console display of the console instrument array upon mounting the portable computing device at the remote control console 130. The controller 51 is configured to determine a plurality of images to be displayed based at least in part upon a set of operating conditions of the machine 10, and is further configured to display the plurality of images on the machine display upon mounting the portable device display 100 at the machine, and to display the plurality of images on the console display upon mounting the portable device display at the remote control console 130.

In instances in which the instrument array 30 further include a gauge 31 at a predetermined location on the instrument array, the controller 51 may be configured to display an image of a gauge 31 on the console display at a location on the console instrument array 131 generally corresponding to the predetermined location. In instances in which both the instrument array 30 and the console instrument array 131 include a gauge 31 and the console instrument array 131 further includes a console gauge, the machine gauge and the console gauge may be generally identically configured. Still further, in instances in which the instrument array 30 further include an input device, the console instrument array 131 may include a console input device that is generally identical to the machine input device. For example, if each of the instrument array 30 and the console instrument array 131 includes a variable or modifiable function input device, the controller 51 may be configured to determine a plurality of images to be displayed generally adjacent each modifiable function input device based at least in part upon a set of operating conditions of the machine. The controller 51 may be further configured to display the plurality of images generally adjacent the modifiable function input device 34 of the instrument array 30 upon mounting the portable computing device 100 at the machine 10, and to display the plurality of images generally adjacent the modifiable function input device 137 of the console instrument array 131 upon mounting the portable computing device at the remote control console 130.

FIG. 8 depicts a control diagram upon operatively mounting the portable computing device 100 on machine 10. As depicted, the machine controller 52, the engine controller 53, the implement controller 54, and the portable device controller 110 may interact as controller 51. Controller 51 may receive, at node 60, data codes that may be stored within a data storage system 102 such as memory or a secondary storage device of portable computing device 100. In some embodiments, it may possible for the data codes to be stored within a remote system 120 and transferred to controller 51 such as by the portable computing device 100. The data codes may include user preferences for setting aspects of the machine 10 that may be set by an operator such as the temperature of the operator cab 15, setting of the seat within the operator cab, and input device sensitivity. By storing the user preferences on the portable computing device 100, each machine 10 may be configured to an operator's desired characteristics upon docking the portable computing device 100 within the machine. As a result, the amount of time necessary to set up a machine 10 when being used by a new operator may be minimized.

Additional data codes may include authorization keys so that an operator may only use those machines for which proper authorization has been received. Still further, the data codes may include limited access keys so that only certain types of operations may be performed by an operator or other personnel (e.g., a mechanic) based upon the stored limited access keys.

The controller 51 may generate various output signals based upon the data codes received at node 60, the sensor signals received at node 61, and the signals from the input devices received at node 62. At node 65, the controller 51 may generate output signals that are transmitted to the gauges 31. In other words, the controller 51 may generate signals that are transmitted to the display devices in which the function is fixed. At node 66, the controller 51 may generate signals or images that are transmitted to the displays 32. More specifically, based upon the user preferences, the operating conditions of the machine 10, and any input commands from an operator, the controller 51 may determine the information to be displayed on the displays 32. In doing so, the controller 51 may also determine which images should be displayed on the display 103 of the portable computing device 100. At node 67, the controller 51 may generate signals or images that are transmitted to displays associated with the modifiable function input devices 34 including any associated with portable computing device 100. At node 68, the controller 51 may generate operating commands to control the operation of machine 10.

FIG. 9 depicts a control diagram upon operatively mounting the portable computing device 100 on remote control console 130. FIG. 9 is similar to FIG. 8 and like reference numbers are used to depict like components and nodes. As depicted, the machine controller 52, the engine controller 53, the implement controller 54, the portable device controller 110, and the console controller 133 may interact as controller 51. Functions of controller 51 may be distributed as desired amongst the various sub-controllers. Controller 51 may receive, at node 60, a plurality of data codes as described above.

At node 61, the controller 51 may receive signals from the various sensors associated with various aspects of machine 10 as well as any sensors associated with portable computing device 100. At node 62, the controller 51 may receive signals from various input devices associated with remote control console 130 and portable computing device 100. At node 63, the controller may receive signals from visual image system 55 indicative of a point of view relative to the machine 10.

The controller 51 may generate various output signals based upon the data codes received at node 60, the sensor signals received at node 61, the signals from the input devices received at node 62, and the visual image signals received at node 63. At node 65, the controller 51 may generate output signals that are transmitted to the gauges 134 of the remote control console 130. In other words, the controller 51 may generate signals that are transmitted to the display devices in which the function is fixed. At node 66, the controller 51 may generate signals or images that are transmitted to the displays 135. More specifically, based upon the user preferences, the operating conditions of the machine 10, and any input commands from an operator, the controller 51 may determine the information to be displayed on the displays 135 of remote control console 130. In addition, the controller 51 may determine which images should be displayed on the display 103 of the portable computing device 100. At node 67, the controller 51 may generate signals or images that are transmitted to displays associated with the modifiable function input devices 137 including any associated with portable computing device 100. At node 68, the controller 51 may generate operating commands to control the operation of machine 10. At node 69, the controller 51 may generate images to be displayed on the point of view display 132.

FIG. 10 depicts a flowchart of a process for operating machine 10 alternatively with either with an operator located in the operator cab 15 or located at a remote control console 130. At stage 160, data codes may be entered or loaded into portable computing device 100. The data codes may limit the types of machines that the owner of the portable computing device 100 may operate, may limit the types of operations that may be performed by the machine 10, and may set user preferences including desired images to be displayed based upon certain operating conditions encountered by machine 10. At decision stage 161, the controller 51 may determine whether the portable computing device is removably mounted or docked at machine 10. If the portable computing device is docked at machine 10, data codes may be transferred at stage 162 from the portable computing device to the machine controller 52.

At stage 163, the controller 51 may receive data from the various sensors associated with the machine 10 and portable computing device 100. At stage 164, the controller 51 may determine the operating conditions of the machine 10 based upon the data received from the sensors at stage 163. At stage 165, the controller 51 may determine based at least in part on the operating conditions of the machine which images should be displayed on the displays 32 of the machine and which images should be displayed on the display 103 of the portable computing device 100. As such, the instrument array 30 may have a predetermined configuration for each operating condition.

More specifically, for some operating conditions, the predetermined configurations may be identical and for other operating conditions, the predetermined conditions may be different. As an example, when machine 10 is performing certain tasks or operations, it may be desirable to show certain operating conditions of the machine 10 such as the pressure associated with certain of the hydraulic cylinders of the machine. In other situations, it may be desirable to depict the position of blade relative to a ground reference. At stage 166, the controller 51 may transmit a plurality of machine signals to the instrument array 30 to configure the instrument array so that it will reflect the operating conditions of the machine 10.

If any of the input devices are configured as modifiable function input devices 34, the controller 51 may determine at stage 167 the function of each modifiable function input device. In addition, the controller 51 may generate at stage 168 a signal to display an image generally adjacent to such modifiable function input devices 34 so as to communicate the functionality to an operator within operator cab 15.

At stage 169, the controller 51 may receive various input signals from an operator. At stage 170, the controller may generate operating commands to operate the machine 10 as desired based upon the data codes, signals received from the sensors, and the input from the operator. At stage 171, the controller 51 may transmit the operating commands to the various machine components to operate machine 10 as desired.

If the portable computing device 100 was not removably mounted or docked at machine 10 at decision stage 161, the controller 51 may determine at decision stage 172 whether the portable computing device 100 is docked at the remote control console 130. If the portable computing device 100 is not docked at either the machine 10 or the remote control console 130, the controller 51 may enter a standby mode in which it waits for the portable computing device to be docked in either the machine 10 or the remote control console 130.

If the portable computing device 100 is removably mounted or docked at the remote control console 130, the machine 10 may be operated from the remote control console in generally the same manner as if the operator were located in the operator cab 15. In doing so, it may be desirable for the console instrument array 131 to generally be configured in the same manner as the instrument array 30 of machine 10 and for the console instrument array 131 to operate generally in the same manner as instrument array 30. Upon mounting the portable computing device 100 at remote control console 130, data codes may be transferred at stage 173 from the portable computing device to the console controller 133. In an alternate embodiment, the data codes may be transferred to other portions of controller 51.

At stage 174, the controller 51 may receive data from the various sensors associated with the machine 10 and portable computing device 100. At stage 175, the controller 51 may determine the operating conditions of the machine 10 based upon the data received from the sensors at stage 163. At stage 176, visual image signals indicative of the point of view relative to the machine 10 may be transmitted from machine 10 to the remote control console 130. At stage 177, point of view images may be generated and displayed on the point of view display 132 based upon the visual image signals received from the visual image system 55 on machine 10.

At stage 178, the controller 51 may determine based at least in part on the operating conditions of the machine 10 which images should be displayed on the displays 135 of the remote control console 130 and which images should be displayed on the display 103 of the portable computing device 100. As such, the instrument array 30 may have a predetermined configuration for each operating condition. In one embodiment, it may be desirable for the images to be displayed on the displays 135 of the remote control console 130 to be generally identical to those displayed on the displays 32 of the machine 10 for the same set of operating conditions. In other embodiments, the images displayed on the displays 135 may not be generally identical to those displayed on displays 32 of machine 10 but it may still be desirable for the images to be in generally identical positions. In that manner, it may be easier for an operator familiar with operating machine 10 to operate the machine through remote control console 130. In other words, if an operator is familiar with the configuration and location of the various displays on the machine 10, remote operation of the machine will likely be simplified if the various displays are similarly configured and located on the remote control console 130. In the absence of similarly configured images, remote operation may still be simplified if the location of the images remains generally consistent.

At stage 179, the controller 51 may transmit a plurality of console signals to the console instrument array 131 to configure the console instrument array so that it will reflect the operating conditions of the machine 10.

If any of the input devices are configured as modifiable function input devices 137, the controller 51 may determine at stage 180 the function of each modifiable function input device. In addition, the controller 51 may generate at stage 181 a signal to display an image generally adjacent to such modifiable function input devices 137 so as to communicate the functionality to an operator at the remote control console 130.

At stage 182, the controller 51 may receive various input signals from an operator. At stage 183, the controller may generate operating commands to operate the machine 10 as desired based upon the data codes, signals received from the sensors, and the input from the operator. At stage 184, the controller 51 may transmit the operating commands from the remote control console 130 to the machine controller 51 to operate the various machine components of machine 10 as desired.

In an alternate process, the process set forth in FIG. 10 may be modified to function as a remote monitoring system to monitor operator performance of a machine 10 at a work site. More specifically, an operator may operate machine 10 from within operator cab 15 as set forth in stages 160-171. However, an instructor or other personnel may be positioned at a remote console (not shown) similar to remote control console 130 having a second portable computing device mounted therein. The remote console may operate in accordance with stages 173-179 so that the instructor may have the same point of view as the operator within the operator cab 15. In one process, the instructor may not have the ability to control the machine remotely and may communicate with the operator to provide instructions and other input. In another instance, such as if the remote console is configured as remote control console 130, the instructor may be able to override commands from the operator in which case the process may also follow stages 180-184.

An alternate monitoring system utilizing a portable computing device 100 including a system for reporting the performance of an operator of a machine at a work site is set forth in FIG. 11. At stage 260, data codes including a user identification code unique to a machine operator may be entered or loaded into portable computing device 100. A separate or unique user identification code may be issued to each machine operator and stored within the portable computing device 100 assigned to each machine operator.

In one embodiment, the machine operator may use the portable computing device at stage 261 to inspect the machine 10. In doing so, images of components to be inspected may be displayed on the display 103 of the portable computing device. Controller 51 may be configured to provide step-by-step instructions or images through the display 103 to guide the machine operator through the inspection process and provide directions to locate the components to be inspected. If desired, the instructions may also or alternatively be provided through speakers (not shown) associated with the portable computing device 100 or through headphones (not shown) worn by the machine operator.

The machine 10 may be provided with bar codes or radio-frequency identification (RFID) tags or integrated circuits at the various locations to be inspected prior to the operation of the machine 10. The portable computing device 100 may be configured to read the bar codes or RFID tags as the machine operator performs the inspection process. The machine operator may be required to enter data upon reading each bar code or RFID tag such as by entering data through display 103, another input device, or by taking pictures of the relevant component of the machine 10 with camera 105 of the portable computing device 100. The controller 51 may store the inspection data on the portable computing device 100 and/or may transmit the inspection data to a remote system 120 at stage 262. As part of the inspection process, the machine operator may be able to note components identified during the inspection that are of concern or may require future maintenance. Still further, the machine operator may be able to contact maintenance personnel or a maintenance department directly with such concerns or to identify components that will require maintenance.

If the machine inspection is not completed at decision stage 263, the controller 51 may be configured to prevent or lock out operation of the machine 10 until the inspection process is completed. If the machine inspection is completed at decision stage 263, instructions in the form of desired tasks or operations may be loaded at stage 264 into portable computing device 100 to provide instructions to the machine operator. In an alternate process, the instructions may be loaded earlier in the process such as during a planning meeting between the machine operator and a supervisor.

At stage 265, the machine operator may inspect the work site. For example, the machine operator may walk around the work site or move about the work site in a vehicle prior to beginning any material moving operations. At stage 266, the machine operator may capture images of the work site with camera 105 of portable computing device 100. To assist in communicating to the machine operator the desired operations to be performed, the portable computing device may display images of the existing work site on the display 103 and display or overlay at stage 266 a virtual image of the desired or final configuration of the work site on top of the image of the existing work site. In other words, the display 103 may be used to depict the work site as it exists before any material moving process as well as the desired topography as a result of the material moving process.

At stage 267, the portable computing device 100 may be docked with machine 10. Such docking may occur wirelessly by moving the portable computing device 100 within a predetermined distance of the machine 10 or through a wired connection by physically connecting the portable computing device to an appropriate docking connection or interface (not shown) of the machine. The controller 51 may determine at stage 268 the identity of the machine operator based on the user identification data code either permanently or temporarily stored on the portable computing device 100. At stage 269, the machine operator may use the machine 10 to perform various tasks such as material moving operations.

While performing the tasks, the controller 51 may, at stage 270, receive and record data from the various sensors associated with the machine 10. This data may be used to generate performance data that is thus generated as a result of operation of the machine 10. In other words, the controller 51 may record the position and performance of the machine 10 as the machine operator performs a plurality of tasks or desired operations. For example, the controller 51 may record the position of the machine 10 relative to the work site as well as the position of the machine relative to a gravity reference (e.g., pitch and roll) as well as the velocity and acceleration of the machine. The controller may also record the position of various work implements such as blade together with the actuators or hydraulic cylinders and other components associated with the work implement. Still further, the controller 51 may store predetermined ranges of acceptable machine positions and determine instances in which the machine 10 is positioned outside of the predetermined ranges. The controller 51 may also determine the distance of the machine 10 from any keep-out areas at the work site and record instances in which the machine was within a predetermined distance from a keep-out area. Such keep-out areas may include static areas such as obstacles at the work site or dynamic areas including the position of the movable objects such as machines and personnel. Efficiency data may also be recorded such as the number of passes that the operator made while moving an amount of material as well as any instances of traction device slippage such as slippage of rear wheels 14 or tracks (not shown).

If the machine operator has not completed the desired tasks at decision stage 271, the operator may continue to perform the tasks at stage 269 and the data recorded at stage 270. If the tasks have been completed at decision stage 271, the machine operator may, at stage 272, remove the portable computing device 100 from machine 10.

At stage 273, the machine operator may re-inspect the work site. In doing so, the machine operator may use the camera 105 of the portable computing device 100 to capture an image or images of the work site after the material moving operation is completed. These images may be stored as a visual data component of the performance data. At stage 274, the performance data received from the sensors at stage 270 together with any pre-operation work site images recorded at stage 265 and post-operation images of the work site recorded at stage 273 may be reported or transmitted to remote system 120. At stage 275, a performance report may be generated at the remote system 120. The performance report may include a summary of the performance of the machine operator, specific details of the performance as well as instructions for an operator to improve performance.

If desired, a first performance report may be provided to the portable computing device 100 for viewing by the machine operator and a second performance report may be provided to a system remote from the machine 10 for viewing by a manager, an instructor or other personnel. The first and second reports may contain identical information or one may contain more detail or be formatted differently than the other. Still further, it may be desirable to provide certain aspects of the performance reports of other machine operators or generate additional performance reports to allow machine operators to compare their performance with that of their peers.

At stage 274, the performance report may be provided to the portable computing device 100. If desired, the performance report may also include suggestions to or instructions for the operator on how to improve future performance. The performance report may also be provided to other systems remote from the machine 10 if desired.

If desired, controller 51 may be configured to transmit or report data to the remote system 120 during the operation of the machine 10 rather than waiting until the operation of the machine 10 is completed. In other examples, the controller 51 may be configured to transmit or report data only after a predetermined event or amount of time has elapsed. Through such a configuration, repeated actions that may cause a safety concern may trigger a report to the remote system 120 as well as feedback from a manager, an instructor or an automated system.

INDUSTRIAL APPLICABILITY

The industrial applicability of the system described herein will be readily appreciated from the foregoing discussion. The foregoing discussion is applicable to machines that may be operated by an operator. Machine 10 may be configured to receive a portable computing device 100 thereat. Performance data may be received and stored during the operation of the machine 10. The performance data may be based at least in part upon the operating conditions of the machine 10. The performance data may be transmitted to a remote system 120 and one or more performance reports generated. The performance reports may be reported to the machine operator through portable computing device 100 and/or to other personnel through remote system 120. The performance reports may include any information relative to the operator's performance as well as advice or instructions to improve the operator's performance.

In another aspect, a remote console may be provided. The remote console may removably receive a portable computing device 100 thereat. Signals may be transferred from the machine 10 to the remote console so that personnel remote from the machine may monitor the performance of the machine operator. The remote personnel may provide real-time feedback to the machine operator while observing the machine operation.

It will be appreciated that the foregoing description provides examples of the disclosed system and technique. However, it is contemplated that other implementations of the disclosure may differ in detail from the foregoing examples. All references to the disclosure or examples thereof are intended to reference the particular example being discussed at that point and are not intended to imply any limitation as to the scope of the disclosure more generally. All language of distinction and disparagement with respect to certain features is intended to indicate a lack of preference for those features, but not to exclude such from the scope of the disclosure entirely unless otherwise indicated.

Recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein. All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context.

Accordingly, this disclosure includes all modifications and equivalents of the subject matter recited in the claims appended hereto as permitted by applicable law. Moreover, any combination of the above-described elements in all possible variations thereof is encompassed by the disclosure unless otherwise indicated herein or otherwise clearly contradicted by context.

Monitoring System for a Machine

Information

Publication Number

Date Filed

Date Published

Inventors

Original Assignees

CPC

US Classifications

International Classifications

Abstract

Description

Claims