The present disclosure relates generally to an automated load response system and, more particularly, to a system for automatically determining when a machine has lost a load, and then responding accordingly.
Machines such as, for example, on and off-highway haul trucks, railway cars, and other types of heavy equipment are used to perform a variety of tasks. Some of these tasks involve carrying large, awkward, loose, and/or heavy loads and, because of the size of the machines and/or poor visibility provided to operators of the machines, these tasks can be difficult to complete safely and effectively. In fact, it is not uncommon for part of the load to fall from the machine completely unnoticed by the operator. This problem can be exacerbated when the machine is remotely or autonomously controlled. A lost load can result in low productivity and efficiency, as well as potential obstruction and/or contamination of roadways, railways, or waterways. The lost load can also damage other machines and harm living organisms in the immediate vicinity.
One way to minimize the affect of a lost load may be to automatically detect a lost load in the path of an autonomous vehicle and then automatically reroute the autonomous vehicle around the lost load. An example of this strategy is described in U.S. Pat. No. 5,610,815 (the '815 patent) issued to Gudat et al. on Mar. 11, 1997. The '815 patent describes an autonomous vehicle equipped with a front mounted laser scanner and a vehicle positioning and navigation system. The laser scanner transmits an infra-red beam pulse in a known angular direction. The time for the beam to reflect off an object and return to the scanner is used to calculate the distance to the object in this angular direction from the scanner. An image comprising many pulses is represented by data pixels, each pixel having a range value and an angle value (cylindrical coordinates), both values associated with the object's location. The vehicle positioning and navigation system projects the vehicle's path into the plane of the image, and filters out pixels not in the vehicle's path. The remaining pixel data is transformed from cylindrical to Cartesian coordinates. Each transformed pixel has a road height value and a horizontal range value, both values associated with the object's location. This data is used to fit a curve to the height at the center of the road, the curve representing the expected road height value at each horizontal range value. When the actual road height values are sufficiently different from the expected road height values, the vehicle positioning and navigation system reroutes the vehicle to avoid the detected object, now classified as an obstacle.
Although the laser scanner of the '815 patent may detect a lost load in the path of an autonomous vehicle, it may do little to detect a load lost by the autonomous vehicle itself. Furthermore, though the vehicle positioning and navigation system of the '815 patent may identify a lost load, it may do little to identify damage to a road surface caused by a lost load. In addition, though the vehicle positioning and navigation system of the '815 patent may reroute the autonomous vehicle to avoid a lost load, it may provide little warning to other machines to avoid the geographical location of the lost load, or arrange for cleanup, salvage, or repair necessitated by the lost load.
The disclosed system is directed to overcoming one or more of the problems set forth above.
In one aspect, the present disclosure is directed to an automated lost load response system for a first machine. The response system includes a first sensor situated to detect the contour of the road surface forward of the first machine, and a second sensor situated to detect the contour of the road surface rearward of the first machine. Each of the first and second sensors generates a signal corresponding to its detection of the contour of the road surface. The response system also includes a controller that is in communication with both sensors. The controller is configured to provide a load loss warning based on the signals.
In another aspect, the present disclosure is directed to a method of responding to a load lost from a first machine. The method includes detecting a contour of a road surface before and after the first machine passes over it. Additionally, the method includes detecting a difference between the contour of the road surface before and after the first machine passed over it. The method also includes providing a potential load loss warning based on the detection of a difference between the contours.
Worksite 14 may be, for example, a mine site, a landfill, a quarry, a construction site, or another type of worksite known in the art. Road surface 16 may or may not embody a surface that machine 10 regularly traverses. For example, road surface 16 may be a gravel road, a quarry floor, a track bed, a concrete bridge, or another type of surface that machine 10 may traverse.
Load material 12 may embody a material that is large, awkward, loose, heavy, valuable, dangerous, or possesses another attribute making knowledge of its loss desirable. For example, load material 12 may be scrap, lumber, rock, ore, garbage, precious metal, or another similar material. As illustrated in
Container 18 may embody any partially or fully enclosed structure temporarily or permanently attached to machine 10 that can be used to support and retain load material 12. For example, container 18 may be an open bed without sides, a flat-floor bed with one or more sides, a dual-slope bed with one or more sides, a van, a reefer, or another partially or fully enclosed structure, which may support and retain load material 12.
A controller 20 may be associated with machine 10 to gather information about load material 12 and/or road surface 16. Controller 20 may include means for monitoring, recording, storing, indexing, processing, and/or communicating information. These means may include, for example, a memory, one or more data storage devices, a central processing unit, and/or another component that may be used to run the disclosed applications. Furthermore, although aspects of the present disclosure may be described generally as being stored in memory, one skilled in the art will appreciate that these aspects can be stored on or read from different types of computer program products or computer-readable media such as computer chips and secondary storage devices, including hard disks, floppy disks, optical media, CD-ROM, or other forms of RAM or ROM.
Controller 20 may be attached to machine 10 and communicate with a pose device 24 to determine the geographical location and orientation of machine 10 relative to a fixed coordinate system G (referring to
Pose device 24 may determine the geographical location of point OT and the orientation of coordinate system T (i.e. machine 10) relative to a local reference point, a coordinate system associated with worksite 14, a coordinate system associated with Earth, or another type of fixed 3-D coordinate system. Pose device 24 may include a locating device 31 (not shown) to determine the geographical location of point OT and an orientation device 32 (not shown) to determine the orientation of coordinate system T. Pose device 24 may also include a controller 34 (not shown) to filter and then communicate these determinations to controller 20.
Locating device 31 may receive and analyze high-frequency, low power radio or laser signals from multiple locations to triangulate a relative geographical location. For example, locating device 31 may embody an electronic receiver configured to communicate with one or more satellites, or a local radio or laser transmitting system to determine a relative 3-D geographical location of point OT. Alternatively, locating device 31 may embody an Inertial Reference Unit (IRU), odometric or dead-reckoning positioning device, or another known locating device operable to receive or determine a relative 3-D geographical location of point OT. Locating device 31 may communicate the geographical location of point OT to controller 34, which may filter the geographical location of point OT into spatial coordinates in coordinate system G.
Orientation device 32 may include laser-level sensors, tilt sensors, inclinometers, or other known devices operable to determine a relative pitch and a relative roll of coordinate system T. Orientation device 32 may also include a radio direction finder, a gyrocompass, a fluxgate compass, or another known device operable to determine a relative yaw of coordinate system T. Orientation device 32 may communicate the information regarding the pitch, roll, and yaw of coordinate system T to controller 34, which may filter the information into rotations about point OG in coordinate system G.
Controller 34 may include means for monitoring, recording, storing, indexing, processing, and/or communicating information. These means may include, for example, a memory, one or more data storage devices, a central processing unit, and/or another component that may filter information provided by locating device 31 and orientation device 32, and then communicate a signal indicative of this filtered information to controller 20. This signal may, for example, include the geographical location of point OT in spatial coordinates in coordinate system G: XG(OT)=[−bT1−bT2−bT3], where OT is the geographical location communicated by locating device 31. The signal may also include the rotations of coordinate system T about point OG in coordinate system G: AG(RT)=[pt yt rt], where RT is the information communicated by orientation device 32.
Additionally,
As previously discussed, each sensor 28 may determine the geographical location of points on road surface 16 relative to itself. In particular, each sensor 28 may determine spatial coordinates in the form XSP=[ρ θ] of points on road surface 16. For example, each sensor 28 may embody a LIDAR (light detection and ranging) device, a RADAR, (radio detection and ranging) device, a SONAR (sound navigation and ranging) device, or another type of device that may determine the range and direction to points on road surface 16. Sensor 28 may use a beam pulse to measure the distance between itself and a point E on road surface 16. The sensor 28 may have an emitter, which may emit a brief beam pulse 40. Sensor 28 may also have a receiver, which may receive the beam pulse 40. Sensor 28 may measure the time between the beam pulse 40 emission and reception. This measured time may be the time beam pulse 40 took to travel to, reflect off, and return from point E. Sensor 28 may convert the measured time into a distance. This distance may be spatial coordinate ρ. The beam pulse emission may be at an angle varied between 0 degrees and 180 degrees. This angle may be spatial coordinate θ. Sensor 28 may communicate a signal including the geographical locations of several points E, each with different spatial coordinates θ, to controller 20. For example, this signal may include the geographical locations of n points E in coordinate system S:
each row representing one point.
Referring to
Controller 20 may relate, by time location, the determinations of pose device 24 and each sensor 28. Controller 20 may then relate points E in coordinate system S to their locations in other coordinate systems. In particular, controller 20 may relate points E in coordinate system S in polar form to their locations in coordinate system S in Cartesian form. The relation between coordinate system S in polar form (i.e. XSP) and coordinate system S in cartesian form (i.e. XS) may be as follows:
where each row represents one point.
Controller 20 may further relate points E in coordinate system S in Cartesian form to their locations in coordinate system T. The relation between coordinate system S in Cartesian form and coordinate system T may be as follows:
and represents the translational transform from coordinate system S in Cartesian form to coordinate system T.
Controller 20 may further relate points E in coordinate system T to their locations in coordinate system G. The relation between coordinate system T and coordinate system G may be as follows:
and represents the translational transform from coordinate system T to coordinate system G.
Controller 20 may temporarily store the coordinate system G location of points E in a map 42 (referring to
Controller 20 may filter extraneous point E* out of map 42 using a height (g3 coordinate) threshold. A straight height threshold may be meaningless since road surface 16 is not necessarily flat. Hence, the threshold may be referenced against the expected height, as predicted by third order fits along vectors parallel to axis vectors xG and yG, at the g1 and g2 coordinates, respectively, of each point. In this manner, points on a hill of road surface 16 may be retained in map 42 since the actual height and the height expectation, as predicted by the third order fits should match very closely. But, extraneous point E* may be removed from map 42, and replaced by the height expectation, since its actual height may barely reflect the height expectation, as predicted by the third order fits.
One or more sensors 44 may be attached to machine 10 and situated to detect material falling from container 18. Sensor 44 may be attached to a side 46 of container 18, a rear portion 38 of container 18, or another surface of container 18 or machine 10, to detect material falling from container 18. Each sensor 44 may embody a device that detects and ranges objects. For example, sensor 44 may be a LIDAR (light detection and ranging) device, RADAR (radio detection and ranging) device, SONAR (sound navigation and ranging) device, or another type of device which may detect and range objects.
Each sensor 44 may include an emitter, which may emit a detection beam, and a receiver, which may receive the detection beam. The emitter may vary the direction of the emitted detection beam so it repeatedly traverses a set of predetermined angles forming a field of detection. The receiver may locate the angle and range the distance of objects within the field by analyzing the angle and time at which it receives detection beams. Sensor 44 may then generate a signal corresponding to the detection and location of objects within the field of detection, which sensor 44 may communicate to controller 20.
Controller 20 may receive the signal from sensor 44 and analyze it to determine the presence of unexpected objects. Specifically, controller 20 may convert the signal generated by sensor 44 into a function (hereafter “the generated function”) representing the location of surface points of objects on or near machine 10 at regular time intervals. Controller 20 may automatically compare the generated function to a stored function representing the location of expected surface points, for example, the sides of container 18 or another part of machine 10. By this comparison (hereafter “the comparison of functions”), controller 20 may determine the presence of an unexpected object when the functions are not equivalent. For example, the unexpected object might be load material 12a, an open door, or an improperly located machine 10 part.
Controller 20 may further analyze the comparison of functions to determine additional information about the unexpected object. Where the two functions are not equivalent, the generated function may represent the location of points above (not on) road surface 16 at time τ. Controller 20 may compare these points to points stored in map 42. A point that is sensed by both sensors 28 and 44 may be an extraneous point E*. Therefore, controller 20 may remove the extraneous point E* from map 42, and replace it with the height expectation, as predicted by the third order fits.
Map 48, electronic in form, may be stored in the memory of controller 20. Map 48 may be updated in real time to reflect the geographical and time locations of points on road surface 16. It may also be used to reroute machine 10 or another machine to avoid a potentially hazardous area of road surface 16. Alternatively, map 48 may be stored in an offboard system, which may be continually referenced and updated by any machine functioning within a certain geographical area (i.e. within worksite 14).
Controller 20 may determine one set 50 of height differences is potentially hazardous by analyzing the points, which are in the area of set 50, on road surface 16 stored in map 42. In particular, controller 20 may calculate the gradient of the gradient at points on road surface 16. If the magnitude of this gradient of the gradient exceeds a certain established value at a certain established number of nearby points, set 50 may be deemed potentially hazardous. Furthermore, if the direction of the gradient of the gradient has a component opposite to the direction of axis vector zG, set 50 may be categorized as an obstruction to road surface 16. Otherwise set 50 may be categorized as concave damage to road surface 16. Alternatively, controller 20 may calculate the size of set 50 by integrating the height differences Δ over the g1 and g2 boundaries of set 50. This size may represent an amount of load material 12 lost by machine 10 between time τ0 and time τ. If the amount exceeds a certain established value, set 50 may be deemed potentially hazardous.
Controller 20 may determine and respond to the cause of set 50 by interacting with additional devices illustrated in
Controller 20 may activate warning devices and transmit data to other machines and/or systems based on the determination of the cause of set 50. For example, a transmission device 54 may be used to transmit a warning of a potential lost load to another machine, or a request for maintenance to service facility 58; an operator warning device 62 may be activated to warn an operator of a potential lost load; or an external warning device 64 may be activated to warn machine 10's immediate vicinity of a potential lost load.
Transmission device 54 may be associated with controller 20 to transmit and/or receive data, warnings, and/or instructions through a communications link to another machine 56, a service facility 58, a satellite 60, or another offboard system, based on the signals generated by sensor 28, pose device 24, clock device 30, load sensor 52, sensor 44, and/or another onboard and/or offboard device. Transmission device 54 may include hardware and/or software that enables transmission device 54 to send and/or receive data messages through a communication link. The communications may include satellite, cellular, infrared, radio, and/or other types of wireless communication that enable transmission device 54 to wirelessly exchange information with offboard systems. Alternatively, the communications may include electrical, optical, and/or other types of wired communication that enable transmission device 54 to exchange information with offboard systems.
An operator warning device 62 may be associated with controller 20 and situated to alert a machine operator of load material 12b. Means of alert may include generating sound, light, smell, or another change to working conditions detectable by a machine operator. For example, operator warning device 62 may embody a warning lamp; alarm; horn; head-up display; odorant or tissue-irritating substance dispenser; or another device operable to warn a machine operator of load material 12b.
An external warning device 64 may, alternatively or additionally, be associated with controller 20 and situated to alert the immediate vicinity of machine 10 to load material 12b. Means of alert may include generating sound, light, smell, or another change to environment detectable by living things. For example, external warning device 64 may embody a warning lamp, alarm, horn, chemical repellent dispenser, or another device operable to warn a living thing of load material 12b.
An override device 66 may be associated with controller 20 and situated to provide a machine operator a means of interfacing with the automated lost load response system, allowing alterations to the system's function. For example, override device 66 may embody a foot pedal, joystick controller, lever, switch, button, wheel, or another interface device known in the art, and it may be used to disable, enable, or otherwise alter the functioning of the automated lost load response system.
The disclosed system may be useful for tasks involving the transportation of large, awkward, loose, and/or heavy loads. The system may detect information about a load material and/or a road surface, and transmit and/or report this information to interested persons and/or other systems. In particular, the disclosed system may be used to detect when material is lost from a container and warn another machine of the loss.
The disclosed system may, based on the signal generated by a sensor 28, the signal generated by pose device 24, the signal generated by clock device 30, and the processing of controller 20, determine the locations of points on road surface 16. Controller 20 may temporarily store these points in map 42. Controller 20 may then filter extraneous points out of map 42 using a height threshold. Alternatively, controller 20 may use a sensor 44 to determine which points stored in map 42 may represent points above (not on) road surface 16. In particular, controller 20 may generate a function representing the location of surface points of objects on or near machine 10. Controller 20 may automatically compare this generated function to a stored function corresponding to expected surface points, for example, container 18, or another part of machine 10. By this comparison, controller 20 may determine the presence of an unexpected object, for example, a load material 12a, an open door, or an improperly located machine 10 part. Controller 20 may filter points on this object out of map 42.
Controller 20 may compare the filtered map 42 to map 48, which permanently stores the locations of points on road surface 16. By this comparison, controller 20 may automatically determine which points in map 42 represent significant changes to the contours of road surface 16 (hereafter “changes”). By analyzing the gradient of the gradient at these points, controller 20 may automatically determine which contours of road surface 16 may be potentially hazardous to a machine 10. Alternatively, controller 20 may automatically determine which contours of road surface 16 may be potentially hazardous to a machine 10 by analyzing the volume changes at the points. Controller 20 may then store or buffer in preparation for transmission the locations of the contours of road surface 16 that may be potentially hazardous to a machine 10.
The disclosed system may also determine additional information about the changes. In particular, controller 20 may determine whether the changes represent load material 12b, damage to road surface 16 caused by load material 12a, or some other change that may be unrelated to load materials 12a and 12b. These determinations may aid controller 20, service facility 58, or another system, or person in efficiently and appropriately handling the changes. For example, the determinations may lead controller 20 to notify service facility 58 or another machine 56 of the location and amount of lost load material 12b; transmit a machine 10 or road surface 16 maintenance request to service facility 58; or notify an operator and/or other interested person to inspect and/or act upon machine 10 in some way.
The disclosed system may interface with additional onboard or offboard sensors to determine the additional information. For example, controller 20 may, based on the signal generated by load sensor 52, and the time of the changes, determine whether the changes represent load material 12b, damage to road surface 16 caused by load material 12a, or some other change that may be unrelated to load materials 12a and 12b. If there is a change in the weight of container 18 (filled with load material 12), then controller 20 may determine that a portion of load material 12 has been lost. Controller 20 may determine that because a portion of load material 12 has been lost, the changes may represent load material 12b or damage to road surface 16 caused by load material 12a, and record or buffer this information in preparation for transmission. Additionally, based on the signal generated by load sensor 52, controller 20 may determine and then store or buffer in preparation for transmission the weight of the load material lost. But, if there is no change in the weight of container 18 (filled with load material 12), then controller 20 may determine that the changes may be unrelated to load materials 12a and 12b, and then store or buffer this determination in preparation for transmission.
Controller 20 may also interface with other onboard or offboard systems to determine the additional information. For example, controller 20 may correlate the changes with information regarding weather, machine diagnostic data, operator maintained data regarding qualities of the transported load material 12, images of the changes, or other desirable information. This correlated information may be stored and buffered in preparation for transmission.
The disclosed system may use transmission device 54 to transmit to offboard systems the information that controller 20 has stored or buffered in preparation for transmission. Alternatively, based on the information that controller 20 has stored or buffered in preparation for transmission, controller 20 may activate operator warning device 62 and/or external warning device 64.
Sometimes, it may be necessary to disable the disclosed system, for example, to load or unload machine 10. Therefore, based on a signal generated by override device 66, controller 20 may temporarily prevent transmission of maintenance requests and activation of warning devices. Alternatively, based on information from other onboard or offboard systems, controller 20 may, without interfacing with an operator, temporarily prevent transmission of maintenance requests and activation of warning devices.
As previously discussed, the disclosed system may detect load material 12b. In particular, sensor 28b may detect load material 12b located rearward of machine 10. Though this detection may not necessitate rerouting of machine 10, it may aid controller 20, service facility 58, or another system, or a person in efficiently and appropriately handling load material 12b. Specifically, another machine 56 may be rerouted to avoid load material 12b.
Additionally, the disclosed system may detect damage to road surface 16 caused by load material 12a. In particular, sensor 28b may be oriented so as to enable a beam pulse emission to reflect off a point on concave portion 39 of road surface 16. Controller 20 may determine that concave portion 39 of road surface 16 represents changes to contours of road surface 16. Controller 20 may then use load sensors 52 to determine load material 12a was lost just before the changes to the contours of road surface 16 were detected. This determination may aid controller 20, service facility 58, another system, or a person in efficiently and appropriately handling load material 12a.
The disclosed system may transmit information about load materials 12a and/or 12b to a service facility 58, which may then arrange for cleanup, salvage, or repair necessitated by the lost load. Furthermore, the system's constant monitoring of potential lost loads and its ability to warn other machines, and the machine 10's immediate vicinity, of load material 12a, may allow machines and/or persons to avoid the geographical location of the lost load, thus preventing possible serious damage and/or injury.
It will be apparent to those skilled in the art that various modifications and variations can be made to the method and system of the present disclosure. Other embodiments of the method and system will be apparent to those skilled in the art from consideration of the specification and practice of the method and system disclosed herein. It is intended that the specification and examples be considered as exemplary only, with a true scope of the disclosure being indicated by the following claims and their equivalents.
Number | Name | Date | Kind |
---|---|---|---|
5018698 | Tobias | May 1991 | A |
5044820 | Prang | Sep 1991 | A |
5359666 | Nakayama et al. | Oct 1994 | A |
5375663 | Teach | Dec 1994 | A |
5414625 | Hattori | May 1995 | A |
5548516 | Gudat et al. | Aug 1996 | A |
5610815 | Gudat et al. | Mar 1997 | A |
5631658 | Gudat et al. | May 1997 | A |
6047234 | Cherveny et al. | Apr 2000 | A |
6091833 | Yasui et al. | Jul 2000 | A |
6201883 | Mizui | Mar 2001 | B1 |
6526352 | Breed et al. | Feb 2003 | B1 |
6608913 | Hinton et al. | Aug 2003 | B1 |
7068815 | Chang et al. | Jun 2006 | B2 |
7151996 | Stein | Dec 2006 | B2 |
7202776 | Breed | Apr 2007 | B2 |
Number | Date | Country |
---|---|---|
WO 9960335 | Nov 1999 | WO |
Number | Date | Country | |
---|---|---|---|
20090084173 A1 | Apr 2009 | US |