Patent Application
20250003202
The present application is based on and claims priority under 35 U.S.C. §119 to Japanese Patent Application No. 2023-108228, filed on Jun. 30, 2023, the contents of which are incorporated herein by reference in their entirety.
The present disclosure relates to a display device for an excavator, a display device for a work machine, and a monitor system of an excavator.
Conventionally, an excavator which performs autonomous control has been proposed. When an excavator performs autonomous control, a remote system which is provided with a management apparatus so as to identify the status of the excavator while staying at a distance has been proposed. In the remote system, a professional staff member can determine the state of the excavator performing autonomous control by the content displayed on the management apparatus.
According to an aspect of the present disclosure, there is provided a display device for an excavator including a control part configured to acquire information relating to an operation from the excavator when the excavator sequentially performs a plurality of the operations by autonomous control, and change a content to be displayed relating to the excavator, in response to recognizing that the operation of the excavator will be switched based on the information.
The system of the above-described conventional technology does not consider what kind of information is to be displayed on the display of the management apparatus. However, it is considered that the display of the management apparatus is to display information according to the operation of the excavator, so that the state of the excavator can be identified more easily.
In an aspect of the present invention, information according to the operation of the excavator performing autonomous control is displayed, so that it easier to identify the state of the excavator, thereby facilitating the management of the excavator performing autonomous control.
According to an embodiment of the present invention, the state of the excavator performing autonomous control can be easily identified, thereby facilitating the management of the excavator performing autonomous control.
Embodiments of the present invention will now be described with reference to the drawings. Further, the embodiments described below do not limit the invention, but are illustrative examples, and not all features or combinations thereof described in the embodiments are necessarily essential to the invention. Note that the same or corresponding configurations in the drawings may be denoted by the same or corresponding reference numerals, and explanations thereof may be omitted.
First, an outline of a remote management system SYS according to the first embodiment will be described with reference to
As illustrated in
The excavator 100, the fixed point measuring apparatus 400, and the management apparatus 200 are connected so as to transmit and receive data through a communication line NW. For example, the excavator 100 is capable of wireless communication.
The excavator 100 is configured to perform work by autonomous control at a work site.
One or more of the excavators 100 may be provided. In the remote management system SYS, the excavator 100 can provide information relating to the autonomous control of the excavator 100 to the management apparatus 200.
The fixed point measuring apparatus 400 is provided at the work site where the excavator 100 performs work, and is configured to detect the status at the work site.
One or more of the fixed point measuring apparatuses 400 may be provided. Thus, the remote management system SYS can provide information relating to the work site to the management apparatus 200 through the fixed point measuring apparatus 400.
The excavator 100 and the fixed point measuring apparatus 400 transmit information relating to the excavator 100 and information relating to the work site to the management apparatus 200 through the communication line NW. Therefore, the excavator 100 and the fixed point measuring apparatus 400 are provided with sensors that can recognize the position and shape of objects existing in the work site in three dimensions. For example, the excavator 100 (to be described later) is provided with a space recognition device S7, and the fixed point measuring apparatus 400 (to be described later) is provided with a space recognition device S41. Therefore, the excavator 100 and the fixed point measuring apparatus 400 can transmit the result of the three-dimensional measurement of the work site to the management apparatus 200.
The space recognition devices S7 and S41 may use LIDAR (light detection and ranging) to capture images of the work site. The LIDAR apparatus measures, for example, the distance between one million or more points within the monitoring range and the LIDAR apparatus. Note that the present embodiment is not limited to the method using a LIDAR apparatus, but may use any a space recognition device capable of measuring the distance between the device and an object. For example, a stereo camera may be used, or a distance measuring device such as a millimeter wave radar device may be used.
The management apparatus 200 is provided for managing a work site including the excavator 100 by a manager. The management apparatus 200 includes a display device D1.
The display device D1 displays a screen based on information transmitted from the excavator 100 and the fixed point measuring apparatus 400. That is, the display device D1 displays information relating to the work site, etc., so that the manager can confirm the status of the work site.
Thus, the management apparatus 200 can present the status of the work site including the excavator 100 in a multifaceted manner according to the detection results from the excavator 100 and the fixed point measuring apparatus 400. Note that the present embodiment does not limit the device for measuring the work site including the excavator 100 to the excavator 100 and the fixed point measuring apparatus 400, but the device may be another type of device such as a drone flying above the work site or a space recognition device that can be held by a user.
The present embodiment describes an example in which a display device for the excavator for monitoring the excavator 100 performing autonomous control is applied to the management apparatus 200. The present embodiment does not limit the display device for the excavator to the management apparatus 200, but the display device may be applied to other devices. For example, information may be displayed on a tablet terminal or the like held by a manager at a work site, or information may be displayed on a display device in the excavator 100.
Next, an outline of the excavator 100 according to the present embodiment will be described with reference to
The boom 4, the arm 5, and the bucket 6 constitute an excavation attachment which is an example of an attachment, and are hydraulically driven by a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9, respectively. A boom angle sensor S1 is attached to the boom 4, an arm angle sensor S2 is attached to the arm 5, and a bucket angle sensor S3 is attached to the bucket 6. The excavation attachment may be provided with a bucket tilt mechanism.
The boom angle sensor S1 detects the turning angle of the boom 4. In the present embodiment, the boom angle sensor S1 is an acceleration sensor and can detect a boom angle which is a turning angle of the boom 4 with respect to the upper turning body 3. The boom angle becomes a minimum angle when the boom 4 is lowered the most, for example, and increases as the boom 4 is raised.
The arm angle sensor S2 detects the turning angle of the arm 5. In the present embodiment, the arm angle sensor S2 is an acceleration sensor and can detect the arm angle, which is the turning angle of the arm 5 relative to the boom 4. The arm angle becomes the minimum angle when the arm 5 is most closed, for example, and increases as the arm 5 is opened.
The bucket angle sensor S3 detects the turning angle of the bucket 6. In the present embodiment, the bucket angle sensor S3 is an acceleration sensor and can detect the bucket angle which is the turning angle of the bucket 6 with respect to the arm 5. The bucket angle becomes the minimum angle when the bucket 6 is most closed, for example, and increases as the bucket 6 is opened.
The boom angle sensor S1, the arm angle sensor S2, and the bucket angle sensor S3 may be a potentiometer using a variable resistor, a stroke sensor for detecting the stroke amount of the corresponding hydraulic cylinder, or a rotary encoder for detecting the turning angle around the connecting pin. The boom angle sensor S1, the arm angle sensor S2, and the bucket angle sensor S3 constitute an attitude sensor for detecting the attitude of the excavation attachment.
The upper turning body 3 is equipped with a cabin 10 as a driver's cabin, an engine 11, a machine body inclination sensor S4, a turning angular speed sensor S5, an imaging device S6, a space recognition device S7, a positioning device S8, a communication device T1, and the like.
An excavator controller 30 is installed in the cabin 10. A driver's seat, an operation device, and the like are installed in the cabin 10.
The excavator controller 30 is an operation device for executing various operations. The excavator controller 30 is provided in the cabin 10, for example, and controls the driving of the excavator 100. The function of the excavator controller 30 may be implemented by any hardware, software, or a combination thereof. For example, the excavator controller 30 is configured to be mainly composed of a microcomputer including a memory device such as a CPU (Central Processing Unit), a RAM (Random Access Memory), a non-volatile auxiliary storage device such as a ROM (Read Only
Memory), and various input/output interfaces. The excavator controller 30 implements various functions by executing, on the CPU, various programs installed in the non-volatile auxiliary storage device, for example.
The engine 11 is a driving source of the excavator 100. In the present embodiment, the engine 11 is a diesel engine. The output shaft of the engine 11 is coupled to the respective input shafts of a main pump 14 and a pilot pump 15.
The machine body inclination sensor S4 is configured to detect the inclination of the upper turning body 3 with respect to a predetermined plane. In the present embodiment, the machine body inclination sensor S4 is an acceleration sensor for detecting an inclination angle around a front-to-rear axis and an inclination angle around a left-to-right axis of the upper turning body 3 with respect to the horizontal plane. The front-to-rear axis and the left-to-right axis of the upper turning body 3 are orthogonal to each other and pass through, for example, the excavator center point, which is a point on the turning axis of the excavator 100.
The turning angular speed sensor S5 is configured to detect the turning angular speed of the upper turning body 3. In the present embodiment, the turning angular speed sensor S5 is a gyro sensor. The turning angular speed sensor S5 may be a resolver or a rotary encoder. The turning angular speed sensor S5 may detect the turning speed. The turning speed may be calculated from the turning angular speed.
The imaging device S6 is configured to acquire an image of the area around the excavator 100. In the present embodiment, the imaging device S6 includes a front camera S6F for imaging the space in front of the excavator 100, a left camera S6L for imaging the space to the left of the excavator 100, a right camera S6R for imaging the space to the right of the excavator 100, and a rear camera S6B for imaging the space to the rear of the excavator 100.
The imaging device S6 is, for example, a monocular camera having an imaging element such as CCD or CMOS, and may output the captured image to the display device D1.
The front camera S6F is, for example, mounted on the roof of the cabin 10. The left camera S6L is mounted on the left edge of the upper surface of the upper turning body 3. The right camera S6R is mounted on the right edge of the upper surface of the upper turning body 3. The rear camera S6B is mounted on the rear end of the upper surface of the upper turning body 3.
In the present embodiment, by providing the imaging device S6 in the above-described arrangement, an object existing around the excavator 100 can be captured.
The space recognition device S7 is configured to recognize the state of the space around the excavator 100. The space recognition device S7 includes a rear space recognition device S7B for detecting the space behind the excavator 100, a left space recognition device S7L for detecting the space to the left of the excavator 100, a right space recognition device S7R for detecting the space to the right of the excavator 100, and a front space recognition device S7F for detecting the space in front of the excavator 100.
The space recognition device S7 may use LIDAR to detect objects existing around the excavator 100. LIDAR measures, for example, the distance between LIDAR and 1 million or more points within the monitoring range. Note that the present embodiment is not limited to the method using LIDAR, but may be a space recognition device capable of measuring the distance to an object. For example, a stereo camera may be used, or a distance measuring device such as a distance imaging camera or a millimeter wave radar may be used. When a millimeter wave radar or the like is used as the space recognition device S7, by transmitting a large number of signals (such as laser light) toward the object from the space recognition device S7 and receiving the reflected signals, the distance and direction of the object may be derived from the reflected signal.
The rear space recognition device S7B is attached to the rear end of the upper surface of the upper turning body 3. The left space recognition device S7L is attached to the left end of the upper surface of the upper turning body 3. The right space recognition device S7R is attached to the right end of the upper surface of the upper turning body 3. The front space recognition device S7F is attached to the front end of the upper surface of the cabin 10.
The space recognition device S7 may be configured to detect a predetermined object in a predetermined area set around the excavator 100. For example, the space recognition device S7 may have a person detection function configured to detect a person while distinguishing between a person and an object other than a person.
The positioning device S8 is configured to acquire information relating to the position of the excavator 100. In the present embodiment, the positioning device S8 is configured to measure the position and orientation of the excavator 100 in the reference coordinate system. Specifically, the positioning device S8 is a GNSS receiver incorporating an electronic compass to measure the latitude, longitude, and altitude of the current position of the excavator 100 and to measure the orientation of the excavator 100. The reference coordinate system according to the present embodiment is, for example, a world geodetic system. The world geodetic system is a three-dimensional orthogonal XYZ coordinate system in which the origin is placed at the center of gravity of the Earth, the X-axis is taken in the direction of the intersection of the Greenwich meridian and the equator, the Y-axis is taken in the direction of 90 degrees east longitude, and the Z-axis is taken in the direction of the north pole.
The communication device T1 is configured to control communication with devices outside the excavator 100. In the present embodiment, the communication device T1 is configured to control communication between the communication device T1 and devices outside the excavator 100 via a communication line NW (including a wireless communication network). The communication device T1 includes, for example, a mobile communication module compatible with a mobile communication standard such as LTE (Long Term Evolution), 4G (4th Generation), 5G (5th Generation), or a satellite communication module for connection to a satellite communication network.
The communication device T1 controls, for example, radio communication between an external GNSS (Global Navigation Satellite System) survey system and the excavator 100.
The drive system of the excavator 100 according to the present embodiment includes an engine 11, a regulator 13, a main pump 14, and a control valve unit 17. As described above, the hydraulic drive system of the excavator 100 according to the present embodiment includes hydraulic actuators such as traveling hydraulic motors 1L, 1R, a turning hydraulic motor 2A, the boom cylinder 7, the arm cylinder 8, and the bucket cylinder 9 for hydraulically driving the lower traveling body 1, the upper turning body 3, the boom 4, the arm 5, and the bucket 6, respectively.
The engine 11 is the main power source in the hydraulic drive system and is mounted on the rear of the upper turning body 3, for example. Specifically, the engine 11 rotates at a predetermined target rotational speed under direct or indirect control by the excavator controller 30 described later to drive the main pump 14 and the pilot pump 15. The engine 11 is, for example, a diesel engine fueled with diesel fuel.
The regulator 13 controls the discharge amount of the main pump 14. For example, the regulator 13 adjusts the angle (tilt angle) of the swash plate of the main pump 14 in response to a control instruction from the excavator controller 30. The regulator 13 includes, for example, regulators 13L and 13R as described later.
The main pump 14, similar to the engine 11, is mounted at the rear of the upper turning body 3, for example, and supplies hydraulic oil to the control valve unit 17 through a high-pressure hydraulic line. The main pump 14 is driven by the engine 11 as described above. The main pump 14 is, for example, a variable displacement hydraulic pump, and as described above, the stroke length of the piston is adjusted by adjusting the tilt angle of the swash plate by the regulator 13 under the control of the excavator controller 30 to control the discharge flow rate (discharge pressure). The main pump 14 includes, for example, main pumps 14L and 14R as described below.
The control valve unit 17 is a hydraulic control device for controlling a hydraulic system in the excavator 100. In the present embodiment, the control valve unit 17 includes control valves 171 to 176. The control valve 175 includes a control valve 175L and a control valve 175R, and the control valve 176 includes a control valve 176L and a control valve 176R. The control valve unit 17 is configured to selectively supply hydraulic oil discharged from the main pump 14 to one or more hydraulic actuators through the control valves 171 to 176. The control valves 171 to 176 control, for example, the flow rate of hydraulic oil flowing from the main pump 14 to the hydraulic actuator and the flow rate of hydraulic oil flowing from the hydraulic actuator to the hydraulic oil tank. The hydraulic actuator includes a boom cylinder 7, an arm cylinder 8, a bucket cylinder 9, traveling hydraulic motors 1L, 1R, and a turning hydraulic motor 2A. More specifically, the control valve 171 corresponds to the left traveling hydraulic motor 1L, the control valve 172 corresponds to the right traveling hydraulic motor 1R, and the control valve 173 corresponds to the turning hydraulic motor 2A. The control valve 174 corresponds to the bucket cylinder 9, the control valve 175 corresponds to the boom cylinder 7, and the control valve 176 corresponds to the arm cylinder 8.
The pilot pump 15 is an example of a pilot pressure generating device, and is configured to supply hydraulic oil to the hydraulic control device via the pilot line. In the present embodiment, the pilot pump 15 is a fixed capacity hydraulic pump. However, the pilot pressure generating device may be implemented by the main pump 14. That is, the main pump 14 may have a function of supplying hydraulic oil to the control valve unit 17 via the hydraulic oil line as well as a function of supplying hydraulic oil to various hydraulic control devices via the pilot line. In this case, the pilot pump 15 may be omitted.
The operation device 26 is a device used by an operator for operating an actuator. The actuator includes at least one of a hydraulic actuator and an electric actuator.
The discharge pressure sensor 28 is configured to detect the discharge pressure of the main pump 14. In the present embodiment, the discharge pressure sensor 28 outputs the detected value to the excavator controller 30.
The operation sensor 29 is configured to detect the operation contents of the operator using the operation device 26. In the present embodiment, the operation sensor 29 detects the operation direction and the operation amount of the operation device 26 corresponding to each of the actuators, and outputs the detected value to the excavator controller 30. In the present embodiment, the excavator controller 30 controls the opening area of the proportional valve 31 in accordance with the output of the operation sensor 29. The excavator controller 30 then supplies the hydraulic oil discharged by the pilot pump 15 to the pilot port of the corresponding control valve in the control valve unit 17. In principle, the pressure of the hydraulic oil supplied to each of the pilot ports (pilot pressure) is a pressure corresponding to the operating direction and the operating amount of the operation device 26 corresponding to each of the hydraulic actuators. Thus, the operation device 26 is configured to supply the hydraulic oil discharged from the pilot pump 15 to the pilot port of the corresponding control valve in the control valve unit 17.
The proportional valve 31, which functions as a control valve for machine control, is arranged in a pipeline connecting the pilot pump 15 and the pilot port of the control valve in the control valve unit 17, and is configured to change the flow path area of the pipeline. In the present embodiment, the proportional valve 31 operates in response to a control instruction output by the excavator controller 30. Therefore, the excavator controller 30 can supply the hydraulic oil discharged by the pilot pump 15 to the pilot port of the control valve in the control valve unit 17 via the proportional valve 31, independently of the operator's operation of the operation device 26.
This configuration allows the excavator controller 30 to operate the hydraulic actuator corresponding to the specific operation device 26 even when no operation is performed on the specific operation device 26.
For example, when the excavator controller 30 is set to perform autonomous control on the excavator 100, the excavator controller sets one or more of a target turning angle for the upper turning body 3, a target angle for each of the upper turning body 3, the boom 4, the arm 5, and the bucket 6, and a target rotational speed for the engine 11 based on the operation performed by autonomous control, and performs control to operate various configurations of the excavator 100.
For example, the excavator controller 30 outputs a control instruction to the regulator 13 as necessary to change the discharge amount of the main pump 14.
For example, the excavator controller 30 controls a machine guidance function that guides the manual operation of the excavator 100 by an operator through the operation device 26. The excavator controller 30 controls a machine control function that automatically supports the manual operation of the excavator 100 by an operator through the operation device 26.
A part of the function of the excavator controller 30 may be implemented by another controller (control device). That is, the functions of the excavator controller 30 may be implemented so as to be distributed over a plurality of controllers. For example, the machine guidance function and the machine control function may be implemented by an exclusive-use controller (control device).
The fixed point measuring apparatus 400 includes a communication device T3, a position information storage unit 450, an imaging device S40, a space recognition device S41, and a controller 430.
The communication device T3 is an interface for communicating with an external apparatus such as the management apparatus 200 through the communication line NW. The communication device T3 may be any device that can be connected to the communication line NW, and may be, for example, a mobile communication module supporting a mobile communication standard such as LTE, 4G, 5G, etc.
The position information storage unit 450 stores position information of the fixed point measuring apparatus 400. The position information is expressed in a reference coordinate system similar to the position information acquired by GNSS, for example. The reference coordinate system is, for example, the world geodetic system described above.
The imaging device S40 is configured to acquire an image of a work site where the excavator 100 is performing work. The imaging device S40 is, for example, a monocular camera having an imaging element such as a CCD or a CMOS.
The space recognition device S41 detects an object present in the work site where the excavator 100 is performing work. The space recognition device S41 uses, for example, LIDAR as described above.
The controller 430 controls the fixed point measuring apparatus 400. The controller 430 may implement the function thereof by, for example, any hardware or any combination of hardware and software. The controller 430 may be mainly composed of a computer including, for example, a processor device such as a CPU, a memory device such as a RAM (main storage device), and an auxiliary storage device such as a ROM. For example, the controller 430 loads a program installed in the auxiliary storage device into the memory device and executes the program on the CPU to implement various functions.
The management apparatus 200 includes a controller 230, a communication device T2, and a display device D1.
The communication device T2 is configured to control communication between the communication device T1 attached to the excavator 100 and the communication device T3 attached to the fixed point measuring apparatus 400.
The controller 230 is an arithmetic device for executing various kinds of arithmetic operations. In the present embodiment, the controller 230 is composed of a microcomputer including a CPU and a memory. Various functions of the controller 230 are implemented by the CPU executing a program stored in the memory.
As illustrated in
Next, each functional block of the controller 430 of the fixed point measuring apparatus 400, the excavator controller 30 of the excavator 100, and the controller 230 of the management apparatus 200 will be described.
Each functional block in the controller 430 of the fixed point measuring apparatus 400 will be described. Each functional block in the controller 430 is conceptual and does not necessarily need to be physically configured as illustrated. All or some of the functional blocks can be functionally or physically distributed and integrated in arbitrary units. All or some of the processing functions performed in each functional block is implemented by a program executed by a CPU. Alternatively, each functional block may be implemented as hardware by wired logic. The controller 430 includes a transmission control unit 431 (transmission control part) by implementing a program.
The transmission control unit 431 transmits image information by the imaging device S40, measurement information by the space recognition device S41, and position information stored in the position information storage unit 450 to the management apparatus 200 in association with each other. The transmission control unit 431 transmits image information and measurement information every predetermined time. For example, the transmission control unit 431 may transmit information every time the imaging device S40 performs imaging (for example, at less-than one second intervals) or every time the space recognition device S41 performs measurement (for example, at less-than one second intervals).
Each functional block in the excavator controller 30 of the excavator 100 will be described below. Each functional block in the excavator controller 30 is conceptual and does not necessarily need to be physically configured as illustrated. All or some of the functional blocks can be functionally or physically distributed and integrated in arbitrary units. All or some of the processing functions performed in each functional block is implemented by a program executed by a CPU. Alternatively, each functional block may be implemented as hardware by wired logic. The excavator controller 30 includes an acquiring unit 301, an excavator state identifying unit 302, an operation setting unit 303, a target trajectory generating unit 304, an autonomous control unit 305 (autonomous control part), a transmission control unit 306 (transmission control part), and an actuator driving unit 307 by implementing a program.
The excavator controller 30 according to the present embodiment implements the autonomous control of the excavator 100. The excavator controller 30 according to the present embodiment can control a plurality of kinds of work with respect to the excavator 100. The work according to the present embodiment are tasks assigned to the excavator 100 at the work site.
The excavator 100 according to the present embodiment can perform various kinds of work by controlling the excavator controller 30. Executable work includes, for example, excavation loading and leveling work. Note that the present embodiment illustrates an example of the work that the excavator 100 can perform, and is not limited to excavation loading and leveling work.
The work that the excavator 100 can perform in the present embodiment is implemented by one operation of the excavator 100 or a combination of a plurality of operations of the excavator 100. For example, when the work is excavation loading, it is necessary to perform the operations in the order of excavation place transition, excavation, lifting, and soil discharge. That is, a combination of these operations results in the work of the excavator 100. The excavation place transition is the operation of moving the bucket 6 to the excavation position, the excavation is the operation of inserting the bucket into the soil and lifting the soil, the lifting is the operation of lifting the soil and moving the soil to the discharge point, and the discharge is the operation of discharging the soil from the bucket 6 to a dump truck, etc. The operation of the excavator 100 in the present embodiment represents the control of dividing the work performed by the excavator 100 into one or more parts. That is, the work of the excavator 100 is implemented by combining one or more operations.
The acquiring unit 301 acquires signals from various detection devices provided in the excavator 100. For example, the acquiring unit 301 acquires the detection results of the boom angle sensor S1, the arm angle sensor S2, and the bucket angle sensor S3). The acquiring unit 301 acquires image information, measurement information, and position information.
The image information is information captured by the imaging device S6. The measurement information is information measured by the space recognition device S7. The position information is information indicating the position and orientation of the excavator 100 in the reference coordinate system measured by the positioning device S8.
The excavator state identifying unit 302 identifies the state of the excavator 100 based on the signal acquired by the acquiring unit 301. In the present embodiment, the state of the excavator 100 includes the position and orientation of the excavator 100 and the attachment state of the excavator 100 (for example, positions of the boom 4, the arm 5, and the bucket 6). The position of the excavator 100 is, for example, the position of the excavator 100 in the reference coordinate system (latitude, longitude, and altitude of the excavator 100 reference point). The excavator state identifying unit 302 identifies the position and orientation of the excavator 100 based on the output of the positioning device S8.
The attachment state (for example, the positions of the boom 4, the arm 5, and the bucket 6) can be identified from the detection result of the angle sensor (the boom angle sensor S1, the arm angle sensor S2, and the bucket angle sensor S3) and the sizes of each of the boom 4, the arm 5, and the bucket 6.
The operation setting unit 303 sequentially sets the operation performed by the excavator 100 according to the work set as autonomous control. In the present embodiment, the operation setting unit 303 sets the next operation for implementing the work when the excavator 100 completes one operation. By repeating the setting, the work combining a plurality of operations can be implemented. Note that the setting of the work for the excavator 100 is performed by the manager or the like in advance, and the description thereof is omitted.
The target trajectory generating unit 304 generates the target trajectory of the excavator 100 for performing the operation set by the operation setting unit 303.
For example, the target trajectory for performing the set operation is generated based on the data on the target work surface stored in the nonvolatile storage device (not illustrated) of the excavator 100. The target trajectory generating unit 304 may generate the target trajectory for performing the set operation based on the information relating to the topography around the excavator 100 recognized by the space recognition device S7.
In order to move along the target trajectory generated by the target trajectory generating unit 304, the autonomous control unit 305 determines the operation content of the operation element (hydraulic actuator) and generates an operation signal corresponding to the determined operation content to implement the autonomous operation function of the excavator 100.
The transmission control unit 306 performs control for transmitting various kinds of information to the management apparatus 200 via the communication device T1.
For example, the transmission control unit 306 controls the transmission, to the management apparatus 200, of image information imaged by the imaging device S6, measurement information detected by the space recognition device S7, position information indicating the position and orientation of the excavator 100, and state information indicating the attachment state.
Further, the transmission control unit 306 controls the transmission of information relating to the operation executed by the excavator 100 to the management apparatus 200. In the present embodiment, the transmission control unit 306 controls the transmission of information relating to the operation to the management apparatus 200, including the information relating to the operation currently set in the excavator 100 and items corresponding to the operation. Further, when autonomous control is set in the excavator 100, the transmission control unit 306 may transmit the type of work set in the excavator 100 to the management apparatus 200.
The type of work set in the excavator 100 may be, for example, excavation loading or leveling work.
The operation set in the excavator 100 at present is the operation constituting the work, for example, excavation place transition, excavation, lifting, soil discharge, etc. Note that the operation of the excavator 100 described in the present embodiment is indicated as an example and is different according to the embodiment. That is, the work performed by the excavator 100 and the operation constituting the work are different according to the type of work machine, the implementation of the program of the work machine, or the request of the management side managing the work machine.
The items corresponding to the operation are, for example, items indicating the current state that the manager wants to confirm while the excavator 100 is performing the operation and items indicating the goal of the operation. In the present embodiment, the parameters (for example, numerical information) in the items are transmitted to the management apparatus 200. The items indicating the current state include, for example, the number of remaining excavations, the actual turning angle, and the like. The items indicating the target of the operation include, for example, the target turning angle.
Further, the transmission control unit 306 controls to transmit information relating to the current status of the excavator 100 to be confirmed to the management apparatus 200 regardless of the operation. In the present embodiment, the current status includes, for example, the driving mode, the traveling mode, the amount of urea water remaining, the amount of fuel remaining, the water temperature, and the oil temperature of the excavator 100.
Transmission by the transmission control unit 306 is performed every predetermined time. The predetermined time may be any time, but is set to a time interval at which the status changed by the work of the excavator 100 can be recognized. For example, the transmission control unit 306 may transmit the above information at an interval of 1 second.
The actuator driving unit 307 is configured to drive an actuator mounted on the excavator 100. In the present embodiment, the actuator driving unit 307 generates and outputs an operation signal for each of the plurality of solenoid valves included in the proportional valve 31 based on the operation signal generated by the autonomous control unit 305.
Each solenoid valve receiving the operation signal increases or decreases the pilot pressure acting on the pilot port of the corresponding control valve in the control valve unit 17. As a result, the hydraulic actuator corresponding to each control valve operates at a speed corresponding to the stroke amount of the control valve.
Each functional block in the controller 230 of the management apparatus 200 will be described below. Each functional block in the controller 230 is conceptual and does not necessarily need to be physically configured as illustrated. All or some of the functional blocks can be functionally or physically distributed and integrated in arbitrary units. All or some of the of the processing functions performed in each functional block is implemented by a program executed by a CPU. Alternatively, each functional block may be implemented as hardware by wired logic. The controller 230 includes a reception control unit 231 (reception control part), an operation determining unit 232, a screen generating unit 233, and a display control unit 234 (display control part) by implementing a program.
The reception control unit 231 controls to receive various kinds of information from the fixed point measuring apparatus 400 and the excavator 100 through the communication device T2.
For example, the reception control unit 231 receives image information, measurement information, and position information from the fixed point measuring apparatus 400 (an example of a device existing around the excavator 100).
The reception control unit 231 also receives image information, measurement information, position information, and state information from the excavator 100. The image information is information captured by the imaging device S6. The measurement information is information measured by the space recognition device S7. The position information is information indicating the position and orientation of the excavator 100 in the reference coordinate system measured by the positioning device S8. The state information is information indicating the attachment state (for example, positions of the boom 4, the arm 5, and the bucket 6).
Further, the reception control unit 231 receives (acquires) information relating to the operation executed by the excavator 100 from the excavator 100 performing autonomous control. The information relating to the operation includes information relating to the operation currently set in the excavator 100 and items corresponding to the operation. When the operation of the excavator 100 is started, the type of the work to be started (for example, excavation loading or leveling work, etc.) may be included.
Further, the reception control unit 231 receives information relating to the current status of the excavator 100 to be confirmed from the excavator 100 regardless of the operation.
The operation determining unit 232 determines whether or not the operation of the excavator 100 has been switched based on the information received by the reception control unit 231.
The screen generating unit 233 generates a display screen to be displayed on the display device D1. The screen generating unit 233 according to the present embodiment generates a display screen for managing the excavator 100 under autonomous control. The display screen is updated according to the current operation of the excavator 100.
When it is determined by the operation determining unit 232 that the operation has been switched, the screen generating unit 233 changes the contents displayed with respect to the excavator 100. That is, the screen generating unit 233 generates a display screen indicating the contents suitable for the operation of the excavator 100.
When the work of the excavator 100 is excavation loading, the excavator 100 repeats the operation of excavation place transition, excavation, lifting, and soil discharge. When an abnormal status occurs during the operation as it is repeated, the operation is switched to standby. The screen generating unit 233 of the management apparatus 200 of the present embodiment generates a display screen so as to switch in accordance with these operations. Next, the operation of the excavator 100 will be described.
For example, when the operation is the excavation place transition, the excavator 100 identifies the excavation place and performs autonomous control to control the attachment so that the bucket 6 moves to the excavation place. During the operation, the screen generating unit 233 may generate a display screen in which, for example, the ground shape, the target turning angle, and the actual turning angle are indicated as items corresponding to the operation.
When the operation is excavating, the excavator 100 controls excavating against the ground. During the operation, the screen generating unit 233 may generate a display screen indicating, for example, the ground shape of the excavation place, remaining work time, and remaining number of excavations as items corresponding to the operation.
When the operation is lifting, the excavator 100 controls lifting of the excavated soil. During the operation, the screen generating unit 233 may generate a display screen indicating, for example, the soil shape, target turning angle, and actual turning angle of the loading platform of the dump truck as items corresponding to the operation.
When the operation is soil discharge, the excavator 100 controls soil discharge from the lifted attachment. During the operation, the screen generating unit 233 may generate a display screen indicating, for example, the loading weight of the dump truck at the soil discharge destination as an item corresponding to the operation. Further, a display screen indicating the position and attitude of the dump truck at the soil discharge destination and the empty space of the loading platform of the dump truck may be generated.
The items corresponding to the above-described operation are indicated as an example, and may be different according to the embodiment in managing the excavator or the request of the manager.
In the present embodiment, the case where the work of the excavator 100 is excavating and loading will be described. However, the present embodiment does not limit the work of the excavator to excavating and loading, and may be, for example, leveling work. In the leveling work, the excavator 100 continuously performs the leveling work. In this case, the screen generating unit 233 may generate a display screen including, for example, work accuracy, calculation result of the tip of the bucket 6, etc., as items corresponding to the leveling work. Next, the screen generated by the screen generating unit 233 will be described.
The bird's eye view image display field 504 indicates a bird's eye view image including the surroundings of the excavator 100. The screen generating unit 233 generates a bird's eye view image indicated in the bird's eye view image display field 504 based on the image information of the imaging device S6 received from the excavator 100 and the image information of the imaging device S40 received from the fixed point measuring apparatus 400. An icon 541 representing the excavator 100 is indicated in the bird's eye view image. The icon 541 is, for example, an image previously held by the management apparatus 200 in accordance with the model of the excavator 100. The bird's eye view image displayed in the bird's eye view image display field 504 may be a still image or a moving image.
The bird's eye view image is displayed so as to include the surrounding status of the excavator 100. Therefore, the bird's eye view image includes the excavation place 542 and the dump truck 543. Thus, when the display screen is displayed, the manager can recognize the positional relationship between the excavator 100, the excavation place 542, and the dump truck 543.
The excavation place display field 501 indicates the soil shape of the excavation place to be excavated by the excavator 100. The screen generating unit 233 generates an image representing the soil shape indicated in the excavation place display field 501 based on the device measuring the excavation place, for example, the measurement information of the space recognition device S7 received from the excavator 100 or the measurement information of the space recognition device S41 received from the fixed point measuring apparatus 400. In the present embodiment, a soil shape illustrating the heights of the soil existing in the work site in different colors is generated as an image based on a predetermined height (for example, the surface of the ground) as the reference. Thus, the depth (height) of the excavation place 511 can be indicated in the excavation place display field 501. The height indicator 512 indicates the correspondence between the depth (height) and the color. The depth (height) may indicate a scale unit. Thus, the depth of the excavation place 511 can be recognized by the color displayed in the excavation place display field 501.
The excavator status display field 502 indicates information relating to the current status of the excavator 100. For example, the excavator status display field 502 indicates a three-dimensional model 321 representing the current shape of the excavator 100 based on the received state information. Thus, the manager can recognize the current operation performed by the excavator 100. Further, the excavator status display field 502 displays information relating to the current status of the excavator 100 that changes regardless of the operation. For example, an icon 522 indicating the current driving mode of the excavator 100, an icon 523 indicating the current traveling mode of the excavator 100, an indicator 524 indicating the amount of urea water remaining in the excavator 100, an indicator 525 indicating the temperature of the cooling water in the excavator 100, an indicator 526 indicating the amount of fuel remaining in the excavator 100, and an indicator 527 indicating the temperature of the working oil flowing through the hydraulic drive system of the excavator 100 are indicated. Thus, the screen generating unit 233 generates an image indicated in the excavator status display field 502 based on state information, information relating to the status of the excavator 100, and the like.
The dump truck loading platform display field 503 indicates the soil shape of the loading platform of the dump truck to be the target where soil is discharged by the excavator 100. The screen generating unit 233 generates an image representing the soil shape indicated by the area 531 indicating the loading platform of the dump truck in the dump truck loading platform display field 503 based on the device measuring the loading platform of the dump truck, for example, the measurement information of the space recognition device S7 received from the excavator 100 or the measurement information of the space recognition device S41 received from the fixed point measuring apparatus 400. In the present embodiment, a soil shape in which the respective heights of the soil existing on the loading platform are indicated by colors is generated as an image based on a predetermined height (e.g., the bottom surface of the loading platform) as the reference. Thus, the height of the soil loaded on the loading platform of the dump truck can be indicated in the dump truck loading platform display field 503. The height indicator 532 indicates the correspondence between the height and the color. The height may indicate a scale unit. Thus, the height of the soil loaded on the loading platform of the dump truck can be recognized.
The work display field 505 displays information relating to the work set for the excavator 100 to perform autonomous control. Specifically, a plurality of operations constituting the work are indicated according to the order in which the operations are performed.
In the example illustrated in
The operation item display field 506 illustrates information relating to the current operation of the excavator 100. Specifically, the operation item display field 506 illustrates the current parameters (e.g., numerical information) of each item corresponding to the current operation of the excavator 100.
In the example illustrated in
The calculation of the parameters indicated in these items may be performed by the management apparatus 200 or the excavator 100. For example, the management apparatus 200 may calculate the parameters indicated in this item based on the work plan of the excavator 100 stored in a nonvolatile storage device (not illustrated) and the information received from the excavator 100. That is, the calculation of the “remaining work time” and the “remaining number of excavations” illustrated in
In the display screen, a display field corresponding to the current operation may be indicated by a display mode different from other display fields. In the example illustrated in
The display control unit 234 displays the display screen generated by the screen generating unit 233 on the display device D1.
In the present embodiment, various kinds of information of the display screen displayed by the display control unit 234 may be updated according to the information received from the excavator 100. For example, the information displayed in the excavator status display field 502 and the shape of the three-dimensional model 321 may be updated according to the received status of the excavator 100 and the state information. Further, the parameters displayed in the operation item display field 506 or the bird's eye view image or the like indicated in the bird's eye view image display field 504 may be updated according to the received information.
In the present embodiment, when it is determined that the operation has been switched by the operation determining unit 232, a display screen in which the contents indicated by the screen generating unit 233 are changed is generated. In response, the display control unit 234 displays the display screen in which the contents are changed on the display device D1. As a result, the displayed contents are switched according to the operation of the excavator 100.
For example, when the operation of the excavator 100 is switched from “excavation” to “lifting”, the display control unit 234 switches the display screen. Next, the display screen when the operation of the excavator 100 is switched from “excavation” to “lifting” will be described.
The display screen illustrated in
The display of the work display field 605 illustrated in
In this way, the screen generating unit 233 indicates the order of the plurality of operations performed by the excavator 100 in the work display field 605. When it is recognized that the operation of the excavator 100 is switched, the screen generating unit 233 switches the display of the work display field 605 so that the operation performed by the excavator 100 can be recognized. Thus, the manager can recognize the operation currently performed by the excavator 100. Therefore, the manager can easily identify the current status of the excavator 100.
In the operation item display field 606, the items displayed are switched according to the operation currently performed by the excavator 100. Specifically, in the operation item display field 606, “target turning angle” and “actual turning angle” are indicated as items corresponding to “lifting”. “Target turning angle” indicates an angle determined according to the target trajectory, and “actual turning angle” indicates an angle detected by the angle sensor.
When it is recognized that the operation of the excavator 100 is switched, a screen generating unit 233 generates a display screen in which a display field related to the operation after switching is changed. Thus, the manager can identify the detailed state in the current operation of the excavator 100. At that time, the screen generating unit 233 also changes the display of the display field related to other operation (“excavation”) other than the operation after switching. In the present embodiment, the screen generating unit 233 generates a display image in which the display field of items related to the other operation (“excavation”) is hidden. As a result, the display of the display field not related to the current operation of the excavator 100 is prevented, and, therefore, the manager can easily confirm the display field related to the current operation, and thus can easily recognize the current state of the excavator 100.
Thus, when it is recognized that the operation of the excavator 100 is switched, the screen generating unit 233 starts the display of the item corresponding to the operation after the switch, and generates the display screen so as to hide the item of the other operation. Thus, by confirming the content displayed on the display screen, the manager can easily identify the state related to the current operation of the excavator 100, and can reduce the burden in confirming.
When it is recognized that the operation of the excavator 100 is switched, the display of the excavator status display field 502 by the screen generating unit 233 is maintained. That is, the display of the status of the excavator 100 that changes regardless of the operation of the excavator 100 is maintained even when the operation is switched. As a result, with respect to the information that changes regardless of the operation of the excavator 100, the manager can easily recognize the status of the excavator 100 because the confirming method does not change according to the operation.
Further, in the display screen illustrated in
The above-described embodiment illustrates an example of a display screen and is not limited to the display screen described above. Therefore, other modes of the display screen will be described in the modified example 1.
In the above-described embodiment, an example in which the operation item display field is switched according to the operation of the excavator 100 has been described. On the other hand, in the modified example 1, an example in which the display field is displayed for all operations will be described.
The display screen illustrated in
The dump truck loading platform display field 703 is grayed out because this display field does not correspond to the operation (excavation) currently being performed by the excavator 100.
Similarly, the turning place transition/lifting item display field 706 and the soil discharge item display field 708 are grayed out because these display fields do not correspond to the operation (excavation) currently being performed by the excavator 100.
The excavation place display field 501 is highlighted because this display field corresponds to the operation (excavation) currently being performed by the excavator 100. Specifically, the frame line of the excavation place display field 501 is thicker than other frame lines.
The excavation item display field 707 corresponds to the operation (excavation) currently performed by the excavator 100, and, therefore, the excavation item display field 707 is highlighted. Specifically, the frame line of the excavation item display field 707 is thicker than other frame lines.
In the example illustrated in
When the operation of the excavator 100 is switched from “excavation” to “lifting”, the display control unit 234 switches the display screen. Next, the display screen when the operation of the excavator 100 is switched from “excavation” to “lifting” will be described.
The excavation place display field 801 is grayed out because this display field does not correspond to the operation (lifting) currently performed by the excavator 100.
The dump truck loading platform display field 603 is highlighted because this display field corresponds to the operation (lifting) currently performed by the excavator 100. Specifically, the frame line of the dump truck loading platform display field 703 is thicker than the other frame lines.
Similarly, the turning place transition/lifting item display field 806 is highlighted because this display field corresponds to the operation (lifting) currently performed by the excavator 100. Specifically, the frame line of the turning place transition/lifting item display field 806 is thicker than other frame lines.
The excavation item display field 807 and the soil discharge item display field 808 are grayed out because these display field do not correspond to the operation (lifting) currently performed by the excavator 100.
Thus, when the operation of the excavator 100 is recognized to be switched, the screen generating unit 233 generates a display screen in which the display field corresponding to the operation after the switching is emphasized compared with the display field corresponding to the other operation. In the present embodiment, as the emphasized display, the display mode of the frame line of the display field is changed. The method for changing the display mode is not limited to the example of changing the frame line to be thick, and the color of the frame line may be changed. As a result, the manager can easily view the display field corresponding to the operation after switching, and thus can easily confirm the status of the current operation of the excavator 100.
When the operation of the excavator 100 is recognized to be switched, the screen generating unit 233 generates a display screen changed to gray out for the display field not corresponding to the operation after the switch. Thus, it becomes difficult to view the display field not related to the operation after the switched, that is, it becomes easier to view the display field corresponding to the operation compared with the display field not corresponding to the operation. Thus, it becomes easy to confirm the status of the current operation of the excavator 100.
In modified example 1, as a change of the display method of the display field, the case where the display mode of the frame line of the display field is changed and the display field is grayed out has been explained. However, modified example 1 does not limit the change mode of the display field to the change of the display mode of the frame line and the gray out of the display field. In modified example 2, the case where the display field is enlarged and reduced in size will be described.
The excavation place display field 901 is a display field that does not correspond to the operation (lifting) currently performed by the excavator 100, and, therefore, the excavation place display field 901 is grayed out and reduced in size.
The dump truck loading platform display field 903 is a display field that corresponds to the operation (lifting) currently performed by the excavator 100, and, therefore, the dump truck loading platform display field 903 is enlarged from a reduced state (such as the excavation place display field 901) with a thicker frame line.
Similarly, the turning place transition/lifting item display field 906 is a display field that corresponds to the operation (lifting) currently performed by the excavator 100, and, therefore, the turning place transition/lifting item display field 906 is enlarged from a reduced state (similar to the display fields 907, 908) with a thicker frame line.
The excavation item display field 907 and the soil discharge item display field 908 are display fields that do not correspond to the operation (lifting) currently performed by the excavator 100, and, therefore, the excavation item display field 907 and the soil discharge item display field 908 are reduced after being grayed out.
Thus, when it is recognized that the operation of the excavator 100 is to be switched, the screen generating unit 233 generates an enlarged display screen by thickening the display field corresponding to the operation after the switching compared with the display field corresponding to the other operation.
When it is recognized that the operation of the excavator 100 is to be switched, the screen generating unit 233 generates a display screen in which the display field not corresponding to the operation after the switch is grayed out and reduced. Thus, it becomes difficult to view the display field not corresponding to the operation after the switch, that is, it becomes more easier to view the display field corresponding to the operation corresponding to the current operation compared with the display field not corresponding to the current operation. Thus, it becomes easy to confirm the status of the current operation of the excavator 100. In the present modified example, an example in which enlargement or reduction of the display field is combined with the change of the display mode of the frame line or the graying out of the display field has been described. However, the present modified example is not limited to the described combination, and the display field may only be enlarged or reduced.
In the above-described embodiments and modified examples, the management apparatus 200 switches the display contents according to the operation of the excavator 100 performing autonomous control. That is, by displaying the display suitable for the operation of the excavator 100, the manager who manages the excavator 100 by the management apparatus 200 can confirm the state according to the current operation of the excavator 100. That is, the manager can appropriately recognize the current state of the excavator 100, and, therefore, when an abnormality occurs in the excavator 100, the abnormality can be recognized immediately, thereby improving safety.
In the above-described embodiments and modified examples, a case where an excavator is used as an example of a work machine has been described. However, the configuration illustrated in the embodiments and modified examples is not limited to an example where an excavator is applied as a work machine, and may be applied to, for example, a crane, a forklift, etc. That is, in the embodiments and modified examples described above, the case of the management apparatus 200 for managing the excavator 100 has been described, but the management apparatus 200 can also be applied as a management apparatus (an example of a display device for a work machine) for managing the work machine.
Although the above described embodiments describe an example of a display device for an excavator, a display device for a work machine, and a monitoring system for an excavator, the present invention is not limited to the above embodiments. Various changes, modifications, substitutions, additions, deletions, and combinations are possible within the scope described in the claims, which also obviously fall within the technical scope of the present invention.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2023-108228 | Jun 2023 | JP | national |
