Patent Application
20140240506
This patent disclosure relates generally to remote monitoring of machines and, more particularly, to a display system for monitoring information relating to the operation of such machines.
Machines used in industries such as mining, construction and farming can operate in a variety of worksite environments. Such machines can include excavators, loaders, dozers, haul trucks, backhoes, motor graders, material handlers and the like. In some of these worksite environments, such as worksites in areas with extreme environmental conditions or in very remote areas, it may be desirable to remotely monitor and operate the machines as opposed to having an onboard operator. Moreover, in some circumstances depending on the tasks or jobs performed by the machines, it may be more efficient and cost effective to have a single operator monitor and operate multiple machines from a remote location.
In situations where a single operator is monitoring and operating multiple machines from a remote location, the machines may be configured to be operable semi-autonomously. More specifically, each machine may be operable in a remotely controlled mode where the remote operator directly controls one or more functions of the machine and in an autonomous operation mode where the machine is operating automatically. When remotely monitoring and controlling multiple machines, the remote operator may be called upon to simultaneously track multiple data streams associated with the multiple machines. This has the potential to overwhelm the operator with too much information leading to mental fatigue and a resultant loss in efficiency and/or productivity.
In one aspect, the disclosure describes a display system for presenting information relating to the remote monitoring of machines. The display includes a display screen configured to include a main section adapted to display video image data relating to a selected monitored machine. A first side section is arranged along a first edge of the display screen and adapted to display respective video image data associated with each of a plurality of monitored machines. A second side section is arranged along a second edge of the display screen and adapted to display video image data relating to a worksite in which the selected monitored machine and the plurality of monitored machines are operating.
In another aspect, the disclosure describes a display system for presenting information relating to the remote monitoring of machines. The display system includes a display screen configured to include a main section adapted to display a plurality of video image data sets each corresponding to a different point of view relative to a selected monitored machine. A first side section is arranged along a first edge of the display screen and adapted to display, for each of a plurality of monitored machines, a plurality of image data sets each image data set corresponding to a different point of view relative to a respective one of the plurality of monitored machines.
In yet another aspect, the disclosure describes a method for displaying information relating to the remote monitoring and operation of machines. The method includes the step of displaying a plurality of video image data sets each corresponding to a different point of view relative to a selected monitored machine in a main section of a display screen. For each of a plurality of monitored machines, a plurality of image data sets, each image data set corresponding to a different point of view relative to a respective one of the plurality of monitored machines, are displayed in a first side section arranged along a first edge of the display screen. Video image data relating to a worksite in which the selected monitored machine and the plurality of monitored machines are operating is displayed in a second side section arranged along a second edge of the display screen.
This disclosure generally relates to the remote monitoring of machines. More specifically, certain disclosed embodiments provide a display system for presenting information relating to the remote monitoring of a plurality of machines. Referring to
While the display system of the present disclosure is described in connection with a tractor, the arrangement disclosed herein has universal applicability in various other types of machines as well. In this regard, the term “machine” may refer to any machine that performs some type of operation associated with an industry such as mining, construction, farming, transportation, or any other industry known in the art. For example, the machine may be an earth-moving machine, such as a wheel loader, track loader, bulldozer, excavator, dump truck, backhoe, motor grader, material handler or the like.
A controller 22, shown schematically in
The controller 22 may be a single controller or may include more than one controller disposed to control various functions and/or features of the machine 10. The term “controller” is meant to be used in its broadest sense to include one or more controllers and/or microprocessors that may be associated with the machine 10 and that may cooperate in controlling various functions and operations of the machine. The functionality of the controller 22 may be implemented in hardware and/or software without regard to the functionality. The controller 22 may rely on one or more data maps relating to the operating conditions of the machine 10 that may be stored in the memory of controller 22. Each of these maps may include a collection of data in the form of tables, graphs, and/or equations. The controller 22 may use the data maps to maximize the efficiency of the machine 10.
The machine 10 may be equipped with a plurality of sensors or sensing devices that gather data from various components and systems and generate signals that are directly or indirectly indicative of various machine parameters associated with the performance and operating conditions of the machine. Sensors may be associated with, for example, the engine 16, a transmission, the tracks 14, various actuators such as the actuators for the front and rear work implements 18, 20, fluid supplies, a parking brake and/or other systems and components of machine 10. These sensors may automatically gather real-time data such as the operation of engine 16, the position of and load on the work implements 18, 20, fluid pressure, flow rate, temperature, contamination level, and/or viscosity, fluid (i.e., fuel, oil, water, etc.) levels and consumption rates, electric current and/or voltage levels, engagement status of the parking brake, loading levels (e.g., payload value, percent of maximum allowable payload limit, payload history, payload distribution, etc.), transmission output ratio, and other desired information.
In addition, various sensors may be associated with the machine 10 that may be used to determine machine travel characteristics (e.g., speed, acceleration, torque, slip rate, etc.) as well as the position and orientation of machine 10. For example, a pitch rate sensor 24 (e.g., a gyroscope) may be provided on the machine 10. The pitch rate sensor 24 may be used to provide a pitch rate signal indicative of a pitch rate of the machine 10. As the machine 10 moves, the pitch rate will be indicative of the rate of change of the pitch angle of the machine. The pitch rate sensor 24 may also be used to determine the pitch angle and roll of the machine 10. Still further, an accelerometer may be provided on the machine 10 to provide an acceleration signal indicative of measured acceleration of the machine 10 relative to a gravity reference. In one example, the accelerometer may provide measurements in six degrees of freedom (i.e., fore-aft, lateral, and vertical directions as well as pitch, roll and yaw).
A position sensor 26 may also be provided to sense a position of the machine 10. The position sensor 26 may include a plurality of individual sensors that cooperate to indicate the position of the machine 10. The controller 22 may determine the position of the machine 10 as well as its orientation (i.e., the direction machine 10 is facing) based on information provided by the position sensor 26. In some instances, the position sensor 26 may be used to determine the pitch and roll of the machine 10. The position sensor 26 may be a series of global positioning system sensors, an odometer or other wheel rotation-sensing sensor, a perception based system or may use other systems such as lasers to determine the position of machine 10.
A camera system 28 may be provided for generating video image data relating to the machine 10. More specifically, the camera system 28 may be configured to provide video image data corresponding to at least one point of view relative to the machine 10. To this end, the camera system 28 may include one or more video cameras supported on the machine. For example, a plurality of cameras may be positioned to capture different views that an operator would have from within the operator cab 13 of the machine 10. The video cameras may be mounted at any suitable location on the machine 10. For example, as shown schematically in
One or more site video cameras may also be provided to produce video image data relating to a worksite in which the machine is operating. One of the site video cameras may be configured as a machine tracking site video camera 36 that tracks the machine 10, using, for example, data provided by the position sensor on the machine 26, and moves automatically to follow the machine 10 as it moves within the worksite. One or more additional site video cameras 38 may be arranged to provide a broader view of the worksite. In this respect, each of the site video cameras 36, 38 may produce a video image data set that corresponds to a different point of view relative to the workplace.
For enabling remote monitoring and/or operation of the machine, a remote operator console 40 (see
The remote operator console 40 may include one or more input devices 42 for providing commands or data input such as inputting information, changing operations, and issuing commands to one or more of the machine 10 through the remote operator console 40. The input devices 42 may include buttons, knobs, dials, levers, joysticks or the like. One or more of the input devices may have a fixed functionality such that they are always used to control the same function. Additionally, one or more of the input devices 42 may have a modifiable or changeable functionality such that they may be modified to control the input of different functions. Examples of modifiable function input devices 42 include a touch screen display with a computer-generated image, a knob, switch, button, pedal or joystick adjacent a computer display, or any other desired input device.
The remote operator console 40 may further include a display system 44 for presenting information to the remote operator relating to the remote monitoring of the plurality of machines 10. The information displayed may include the video image data produced by the camera systems 28 associated with each of the machines. To this end, the camera system 28 of each machine 10 may be configured to communicate its various video image data sets to the controller 22, which then communicates the data on to the remote operator console 40 or the camera system 28 may be configured to communicate directly with the remote operator console 40. In doing so, the video image data sets may be processed to some extent by the controller 22 at the machine 10 or processed by the remote operator console 40. The display system 44 may include a display screen 46 for displaying the images. The display screen 46 may comprise a liquid crystal device, a device based on light-emitting diodes or any other suitable display technology.
Referring to
The display screen 46 may also include a first side section 52 that may be adapted to display video image data 54 associated with each of a plurality of monitored machines 10, such as is produced by one or more of the video cameras 30, 32, 34 that may be provided on the respective machines 10. The first side section 52 may be arranged along a first edge 56 of the display screen 46, such as, for example, the right side edge of the display screen 46 as shown in the embodiment of
The display screen 46 may further include a second side section 60 that is adapted to display video image data 62 relating to the worksite in the which the plurality of machines 10 are operating, such as may be produced by the one or more of the site cameras 36, 38 that may be provided. The second side section 60 may be arranged along a second edge 64 of the display screen 46. For example, as shown in
When the operator at the remote operator console switches the selection for display in the main section 48 from one machine to another, the video image data relating to the previously controlled machine will shrink down and move from the main section 48 to the first side section 52. Additionally, the image data relating to the newly selected machine 10 will move from the first side section 52 to the main section 48. The video image data 62 produced by the site video cameras 36, 38 can be displayed on the main section 48 in situations where none of the machines 10 are selected by an operator at the remote operator console 40 for display in the main section. Alternatively, the main section 48 may remain blank when no machine 10 is selected for display in the main section 48. The display system may further be configured such that, when desired by an operator, the main section 48 may be expanded to take up the entire or nearly the entire display screen 46 with the first side section 52 and the second side section 60 being removed or overlaid by the main section 48.
In addition to providing video image data 50, 54, the main section 48 and the first side section 52 may be configured to display machine parameter information 66 relating to each of the plurality of machines 10 in communication with the remote operator console 40. More specifically, machine parameter information 66 regarding the selected monitored machine 10 may be displayed in the main section 48 while machine parameter information 66 relating to each of the plurality of other machines 10 may be displayed in the first side section 52. In the embodiment of
An alternative embodiment of a layout for the display screen 46 is shown in
The present disclosure is applicable to the remote monitoring of any type of machine. The disclosure is particularly applicable to the remote monitoring of a plurality of machines by a single operator at a remote operator console. The display system of the present disclosure can present information about both a selected one of the machines and the rest of the machines being monitored by the operator in quickly understandable layout. The layout of the display screen presents this information in a manner that avoids overloading the operator with too much information and can thereby help prevent operator mental fatigue. Thus, the monitoring of the machines through the operator console may be more efficient and productive.
It will be appreciated that the foregoing description provides examples of the disclosed system and technique. However, it is contemplated that other implementations of the disclosure may differ in detail from the foregoing examples. All references to the disclosure or examples thereof are intended to reference the particular example being discussed at that point and are not intended to imply any limitation as to the scope of the disclosure more generally. All language of distinction and disparagement with respect to certain features is intended to indicate a lack of preference for those features, but not to exclude such from the scope of the disclosure entirely unless otherwise indicated.
Recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein. All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context.
