Patent Application
20120249342
The present disclosure is directed to a display system and, more particularly, a system for displaying within a mobile machine the machine's environmental surroundings.
Mobile machines such as haul trucks, excavators, motor graders, backhoes, water trucks, and other large equipment are utilized at a common worksite to accomplish a variety of tasks. At these worksites, because of the size of these machines, lack of visibility, slow response time, and difficulty of operation, operators should be keenly aware of their surroundings. Specifically, each operator should be aware of the location of stationary objects at the worksite, road conditions, facilities, and other mobile machines in the same vicinity. Based on the speed of a particular machine, and its size and response profile, the operator of the machine should respond differently to each encountered obstacle in order to avoid collision and damage to the machine, the objects at the worksite, and the other mobile machines. In some situations, however, there may be insufficient warning for the operator to adequately maneuver the machine away from damaging encounters.
One way to help reduce the likelihood of damaging encounters is disclosed in U.S. Patent Publication No. 2009/0259401 of Edwards et al. that published Oct. 15, 2009 (the '401 publication). Specifically, the '401 publication discloses a collision avoidance system that includes an obstacle sensor such as a motion detector, an RFID detector, a GPS tracking system, a LIDAR device, a RADAR device, or a Sonar Device; a camera; and a display such as a monitor, an LCD screen, or a plasma screen located within a cab of a machine. The display shows captured images from the motion detector of obstacles on a visual representation of a worksite (i.e., on an electronic map). The display can operate in a mixed mode, where a first portion of the display is devoted to the map with the obstacles shown on the map, a second portion is devoted to images from the camera, and a third portion is devoted to status information. By using the collision avoidance system, a machine operator may be more aware of machine surroundings and better able to avoid collision with the obstacles.
Although the collision avoidance system of the '401 publication may help to avoid obstacle collision, it may be less than optimal. In particular, the display disclosed in the '401 publication may be unable to simultaneously show obstacle information obtained from multiple sources in an overlapping manner on the electronic map. Without this ability, some knowledge regarding the obstacles could be lost and/or misinterpreted.
The disclosed machine display system is directed to overcoming one or more of the problems set forth above and/or other problems of the prior art.
One aspect of the present disclosure is directed to a display system for a machine operating at a worksite. The display system may include an obstacle detection device configured to detect objects within a distance of the mobile machine, and a locating system configured to track objects at the worksite. The display system may also include a display located within the mobile machine, and a controller in communication with the obstacle detection device, the locating system, and the display. The controller may be configured to cause information from the obstacle detection device and the locating system to be simultaneously shown in an overlapping manner on a common portion of the display.
Another aspect of the present disclosure is directed to a method of machine display. The method may include detecting objects at a worksite within a distance of a mobile machine, and tracking positions of objects at the worksite. The method may further include simultaneously displaying information associated with detected objects and tracked objects in an overlapping manner on a common portion of a display device within the mobile machine.
Machine 12 may embody any type of machine. For example, machine 12 may embody a mobile machine such as the haul truck depicted in
In one embodiment, machine 12 may be equipped with short range sensors 18, medium range sensors 18, and/or long range sensors 18 located at different positions around body 14 of machine 12. Each sensor 18 may be a device that detects and ranges objects, for example a LIDAR (light detection and ranging) device, a RADAR (radio detection and ranging) device, a SONAR (sound navigation and ranging) device, a camera device, or another device known in the art. In one example, sensor 18 may include an emitter that emits a detection beam to a particular zone within a detection range around machine 12, and an associated receiver that receives a reflection of that detection beam. Based on characteristics of the reflected beam, a distance and a direction from an actual sensing location of sensor 18 on machine 12 to a portion of the sensed object within the particular zone may be determined. Sensor 18 may then generate a signal corresponding to the distance, direction, size, and/or shape of the object, and communicate the signal to an onboard controller 26 for subsequent conditioning and presentation on a display 28 within operator station 20.
Operator station 20 may house portions of a machine display system 30 that include, among other things, locating device 24, controller 26, and display 28. Display 28 may be located proximate an operator seat (not shown) and be configured to show information relating to known and unknown obstacles within the detection range of machine 12. As will be explained in more detail below, operator station 20 may also include means for receiving input from an operator regarding how the information should be displayed. In one embodiment, display 28, itself, may include hardware and or software that enables the input to be received from the operator of machine 12. In another embodiment, a separate input device (not shown), for example a keyboard, a mouse, a light stick, or another input device known in the art may be included within operator station 20 and communicatively coupled with controller 26 and/or display 28 for receipt of operator input.
One or more locating devices 24 may be associated with machine 12 and other known object, for example other mobile machines and stationary facilities, at worksite 10. Locating devices 24 may cooperate with the components of GNSS 22 and/or another tracking system (e.g., an Inertial Reference System (IRS), a local tracking system, or another known locating system) to determine a position of machine 12 and the other known objects at worksite 10 and to generate corresponding signals indicative thereof. Locating device 24 may be in communication with controller 26 to convey signals indicative of the received or determined positional information and identification of the tracked object(s) for further processing. Controller 26, as will be described in more detail below, may then selectively cause a representation of machine 12 and the other known objects to be shown overlaid at their relative positions on an electronic representation of worksite 10 within display 28 of machine 12.
Controller 26 may embody a single microprocessor or multiple microprocessors that include a means for monitoring the location of machine 12 and the other known and unknown objects at worksite 10, and for displaying information regarding characteristics of machine 12 and the objects within operator station 20. For example, controller 26 may include a memory, a secondary storage device, a clock, and a processor, such as a central processing unit or any other means for accomplishing a task consistent with the present disclosure. Numerous commercially available microprocessors can be configured to perform the functions of controller 26. It should be appreciated that controller 26 could readily embody a general machine controller capable of controlling numerous other machine functions. Various other known circuits may be associated with controller 26, including signal-conditioning circuitry, communication circuitry, and other appropriate circuitry. Controller 26 may be further communicatively coupled with an external computer system, instead of or in addition to including a computer system, as desired.
Display 28 may be any appropriate type of device that provides a graphics user interface (GUI) for presentation of machine and object locations and/or other information to operators of machine 12. For example, display 28 may be a computer console or cab-mounted monitor, an LCD screen, a plasma screen, or another similar device that receives instructions from controller 26 and displays corresponding information. It is contemplated that display 28 may also be configured to receive input from the operator regarding desired modes and/or display functionality, for example by way of a touch screen interface, if desired.
As shown in the particular embodiment of
First screen portion 32a may be configured to show a representation of machine 12 in its environment at worksite 10, for example in an electronic map of worksite 10. Specifically, first screen portion 32a may be configured to show a representation of machine 12 relative to the terrain of worksite 10 and/or the locations of other known objects at worksite 10 that are being tracked via GNSS 22/locating device 24 (referring to
Second screen portion 32b may have properties similar to first screen portion 32a. In particular, second screen portion 32b may also be configured to show a representation of machine 12 in its environment at worksite 10. Specifically, second screen portion 32b may be configured to show machine 12 relative to particular zones around machine 12 that are scanned by obstacle detection sensors 18 (referring to
Input area 34 may allow the operator of machine 12 to provide instructions regarding display preferences. Specifically, input area 34 may allow the operator to direct how many portions should be provided within screen area 32 and what information should be displayed within each portion. For example, the operator may choose to divide screen area 32 into first and second portions 32a, 32b, as shown in
Controller 26 may be configured to correlate information obtained via GNSS 22/locating device 24 and obstacle detection sensors 18. In particular, controller 26 may be configured to compare signals received from GNSS 22/locating devices 24 with signals received from obstacle detection sensors 18 to determine if the unknown obstacles detected within zones 48-58 by sensors 18 correspond with the known obstacles being tracked by GNSS 22/locating devices 24. When signals from GNSS 22/locating devices 24 indicate a position of a known object within one of zones 48-58 at about the same time that obstacle detection sensors 18 indicate that an unknown object of similar characteristic to the known object is within the same one of zones 48-58, controller 26 may conclude that the unknown object detected by sensors 18 is, in fact, the known object being tracked by GNSS 22/locating devices 24. In this situation, controller 26 may both highlight the corresponding one of zones 48-58 and generate a representation of the known object at the correct position within that zone. In addition, controller 26 may be configured to record and track via GNSS 22/locating device 24 a position of an unknown object detected by obstacle detection sensors 18, when the position of the detected object does not correspond with the position of a known object already being tracked by GNSS 22. Based on this information, controller 26 may then be configured to update the map of worksite 10 to include the positional information of the newly tracked object. In some embodiments, the operator may be able to input information as to the identification of the newly tracked object at the time of detection, if desired.
The disclosed machine display system finds potential application within any mobile machine at any worksite where it is desirable to display within the machine an electronic representation of the machine's surrounding environment at the worksite. The disclosed machine display system may be capable of simultaneously displaying overlapping images of information obtained via an obstacle detection sensor and via a GNSS/locating device. By allowing the simultaneous display of this overlapping information, an operator of the associated machine may be able to correlate the information obtained from the different sources and make decisions that are more informed. In addition, the disclosed machine display system may be capable of automatically correlating the information and utilizing information from one source as input to the other source for enhanced obstacle detection and tracking.
It will be apparent to those skilled in the art that various modifications and variations can be made to the machine display system of the present disclosure. Other embodiments will be apparent to those skilled in the art from consideration of the specification and practice of the machine display system disclosed herein. It is intended that the specification and examples be considered as exemplary only, with a true scope being indicated by the following claims and their equivalents.
