Patent Application
20200407952
The present disclosure relates to a hydraulic excavator and a system.
For a hydraulic excavator, Japanese Patent Laying-Open No. 2017-71982 (PTL 1) discloses attaching a boom angle sensor to a boom pin, a dipper stick angle sensor to a dipper stick pin, and a bucket angle sensor to a bucket link to sense values which are in turn used to calculate the position of the tip of a tooth of the bucket.
PTL 1: Japanese Patent Laying-Open No. 2017-71982
The configuration described in the above document necessitates attaching an expensive sensor to an axis of each of the boom, the dipper stick and the bucket in order to determine the posture of a work implement, which is disadvantageous in terms of cost. Further, when a sensor is attached to the work implement, water, soil, and the like adhering to the work implement may affect the sensor in durability.
Herein is disclosed a hydraulic excavator and a system allowing a simple configuration to be employed to determine the posture of a work implement accurately.
According to the present disclosure, there is provided a hydraulic excavator including a revolving unit, a work implement, an imaging device, and a controller. The work implement and the imaging device are attached to the revolving unit. The work implement operates on a prescribed operating plane. The imaging device captures an image of the work implement at an angle larger than 0° with respect to the operating plane. The controller determines a position of the work implement relative to the revolving unit based on a posture of the work implement in the image captured by the imaging device.
The present disclosure thus allows a simple configuration to be employed to determine the posture of a work implement accurately.
Hereinafter, an embodiment will be described with reference to the drawings. In the following description, identical components are identically denoted. Their names and functions are also identical. Accordingly, they will not be described repeatedly.
Revolving unit 3 is disposed on traveling apparatus 5 and supported by traveling apparatus 5. Revolving unit 3 can revolve about an axis of revolution RX with respect to traveling apparatus 5. Revolving unit 3 has a cab 4. An occupant (or operator) of hydraulic excavator 100 gets in cab 4 and operates hydraulic excavator 100. Cab 4 is provided with an operator's seat 4S where the operator sits. The operator can operate hydraulic excavator 100 in cab 4. The operator in cab 4 can operate work implement 2, operate revolving unit 3 to revolve it with respect to traveling apparatus 5, and operate traveling apparatus 5 to cause hydraulic excavator 100 to travel.
Revolving unit 3 has an engine compartment 9 accommodating an engine and a counterweight provided in a rear portion of revolving unit 3. In engine compartment 9 are disposed an engine, a hydraulic pump and so forth (not shown).
Revolving unit 3 is provided with a handrail 29 frontwardly of engine compartment 9. Handrail 29 is provided with an antenna 21. Antenna 21 is for example an antenna for GNSS (Global Navigation Satellite Systems). Antenna 21 has a first antenna 21A and a second antenna 21B provided on revolving unit 3 and spaced from each other in a vehicular widthwise direction.
Work implement 2 is supported by revolving unit 3. Work implement 2 has a boom 6, a dipper stick 7, and a bucket 8. Boom 6 is pivotably coupled to revolving unit 3. Dipper stick 7 is pivotably coupled to boom 6. Bucket 8 is pivotably coupled to dipper stick 7. Bucket 8 has a plurality of teeth. Bucket 8 has a distal end portion, which will be referred to as a tooth tip 8a.
Boom 6 has a proximal end portion coupled to revolving unit 3 via a boom pin 13. Dipper stick 7 has a proximal end portion coupled to a distal end portion of boom 6 via a dipper stick pin 14. Bucket 8 is coupled to a distal end portion of dipper stick 7 via a bucket pin 15.
Hydraulic excavator 100 has a variety of components, and in the present embodiment, their positional relationship will be described with work implement 2 serving as a reference.
Boom 6 of work implement 2 pivots with respect to revolving unit 3 about boom pin 13 provided at the proximal end portion of boom 6. When a specific portion of boom 6 which pivots with respect to revolving unit 3, for example, a distal end portion of boom 6 moves, it provides a locus in an arc. A plane including the arc is specified as an operating plane P. When hydraulic excavator 100 is seen in a plan view, operating plane P is represented as a straight line. The straight line extends in a direction, which is a fore/aft direction of main body 1 of hydraulic excavator 100 or revolving unit 3, and it is hereinafter also simply referred to as the fore/aft direction. A lateral direction (or vehicular widthwise direction) of main body 1 of hydraulic excavator 100 or a lateral direction of revolving unit 3 is orthogonal to the fore/aft direction in a plan view, and it is hereinafter also simply referred to as the lateral direction.
A side where work implement 2 protrudes from main body 1 of hydraulic excavator 100 in the fore/aft direction is the fore direction and a direction opposite to the fore direction is the aft direction. A right side and a left side of the lateral direction when one faces front are the right direction and the left direction, respectively.
The fore/aft direction refers to a fore/aft direction of an operator who sits at the operator's seat in cab 4. A direction in which the operator sitting at the operator's seat faces is defined as the fore direction and a direction behind the operator who sits at the operator's seat is defined as the aft direction. The lateral direction refers to a lateral direction of the operator who sits at the operator's seat. A right side and a left side when the operator sitting at the operator's seat faces front are defined as the right direction and the left direction, respectively.
Boom 6 is pivotable about boom pin 13. Dipper stick 7 is pivotable about dipper stick pin 14. Bucket 8 is pivotable about bucket pin 15. Dipper stick 7 and bucket 8 are each a movable member movable on the side of the distal end of boom 6. Boom pin 13, dipper stick pin 14, and bucket pin 15 extend in a direction orthogonal to operating plane P, i.e., in the lateral direction. Operating plane P is orthogonal to at least one (in the embodiment, all three) of axes that serve as centers about which boom 6, dipper stick 7, and bucket 8 pivot.
As has been set forth above, boom 6 pivots on operating plane P with respect to revolving unit 3. Similarly, dipper stick 7 pivots on operating plane P with respect to boom 6, and bucket 8 pivots on operating plane P with respect to dipper stick 7. Work implement 2 of the embodiment has its entirety operated on operating plane P. Tooth tip 8a of bucket 8 moves on operating plane P. Operating plane P is a vertical plane including a range in which work implement 2 is movable. Operating plane P intersects each of boom 6, dipper stick 7, and bucket 8. Operating plane P can be set at a center of boom 6, dipper stick 7, and bucket 8 in the lateral direction.
As shown in
Work implement 2 has a boom cylinder 10, a dipper stick cylinder 11, and a bucket cylinder 12. Boom cylinder 10 drives boom 6. Dipper stick cylinder 11 drives dipper stick 7. Bucket cylinder 12 drives bucket 8. Boom cylinder 10, dipper stick cylinder 11, and bucket cylinder 12 are each a hydraulic cylinder driven with hydraulic oil.
Bucket cylinder 12 is attached to dipper stick 7. As bucket cylinder 12 extends and contracts, bucket 8 pivots with respect to dipper stick 7. Work implement 2 has a bucket link. The bucket link couples bucket cylinder 12 and bucket 8 together. The bucket link has a first link member 16 and a second link member 17. First link member 16 and second link member 17 have their respective tips relatively rotatably coupled together via a bucket cylinder top pin 19. Bucket cylinder top pin 19 is coupled to a tip of bucket cylinder 12. Therefore, first link member 16 and second link member 17 are pinned to bucket cylinder 12.
First link member 16 has a proximal end rotatably coupled to dipper stick 7 via a first link pin 18 in a vicinity of bucket pin 15 located at the distal end portion of dipper stick 7. First link member 16 is pinned to dipper stick 7. Second link member 17 has a proximal end rotatably coupled via a second link pin 20 to a bracket located at a foot of bucket 8. Second link member 17 is pinned to bucket 8.
Hydraulic excavator 100 has an imaging device 50. Imaging device 50 in the embodiment is a monocular camera.
Imaging device 50 is attached to revolving unit 3. Imaging device 50 is attached to cab 4. Imaging device 50 is attached inside cab 4. Imaging device 50 is attached in a vicinity of an upper end of a left front pillar of cab 4. Imaging device 50 is disposed in an internal space of cab 4 in a vicinity of the left front pillar at a position away from work implement 2 in the lateral direction. Imaging device 50 is disposed apart from operating plane P of work implement 2 in the lateral direction. Imaging device 50 is disposed leftwardly of operating plane P.
A controller 60 is mounted on hydraulic excavator 100. Controller 60 will more specifically be described hereinafter.
In the embodiment, first link pin 18 and bucket cylinder top pin 19 are marked so that they are recognizable in an image captured by imaging device 50, and first link pin 18 is set as a feature point A and bucket cylinder top pin 19 is set as a feature point B. More specifically, a pin forming a feature point is entirely colored previously or colored so as to be outlined to highlight the feature point's geometrical line to provide the mark to thus allow the feature point to be recognizable in the captured image.
As shown in
An angle formed in a side view by a straight line passing through boom pin 13 and dipper stick pin 14 and a straight line passing through dipper stick pin 14 and bucket pin 15 is defined as dipper stick angle θa. Dipper stick angle θa is an angle of dipper stick 7 with respect to boom 6.
An angle formed in a side view by a straight line passing through dipper stick pin 14 and bucket pin 15 and a straight line passing through bucket pin 15 and tooth tip 8a is defined as bucket angle θk. Bucket angle θk is an angle of bucket 8 with respect to dipper stick 7.
A posture of work implement 2 on operating plane P is determined by a combination of boom angle θb, dipper stick angle θa, and bucket angle θk. For example, a position, or XY coordinates, on operating plane P of feature point A set on first link pin 18 located at the distal end portion of dipper stick 7 is determined by a combination of boom angle θb and dipper stick angle θa. A position, or XY coordinates, on operating plane P of feature point B set on bucket cylinder top pin 19 displacing as bucket 8 operates is determined by a combination of boom angle θb, dipper stick angle θa, and bucket angle θk.
Imaging device 50 is attached at a position at which the operating plane of work implement 2 is viewed in an oblique direction. Imaging device 50 captures an image of work implement 2 at an angle larger than 0° with respect to operating plane P. Work implement 2 and imaging device 50 are both attached to revolving unit 3, and even when hydraulic excavator 100 travels or revolves, imaging device 50 has a positional relationship unchanged with respect to operating plane P.
Imaging device 50 captures an image of work implement 2. Imaging device 50 images operating plane P of work implement 2. Imaging device 50 captures an image of work implement 2 moving on operating plane P. The image captured by imaging device 50 includes at least a portion of work implement 2.
Image processing unit 61 receives from imaging device (camera) 50 an image captured by imaging device 50. Image processing unit 61 subjects the received, captured image to image processing. Image processing unit 61 sets an orthogonal coordinate system on the captured image. Image processing unit 61 sets a U axis extending in a horizontal direction of the captured image and a V axis extending in a vertical direction of the captured image. The U axis and the V axis are orthogonal to each other. Image processing unit 61 sets a UV coordinate system in the captured image.
Feature point recognition unit 62 recognizes in the captured image a feature point set on work implement 2. Feature point recognition unit 62 determines a position of feature point A (or first link pin 18) in the captured image and a position of feature point B (or bucket cylinder top pin 19) in the captured image. More specifically, feature point recognition unit 62 determines feature point A's UV coordinate components and feature point B's UV coordinate components. Feature point recognition unit 62 thus determines a posture of work implement 2 in the captured image.
Angle extraction unit 63 determines a position of work implement 2 relative to revolving unit 3 based on the posture of work implement 2 in the captured image. More specifically, angle extraction unit 63 determines boom angle θb, dipper stick angle θa, and bucket angle θk. As has been set forth above, imaging device 50 assumes a position constantly held fixed relative to operating plane P regardless of how hydraulic excavator 100 travels and how revolving unit 3 revolves. Therefore, once the UV coordinate components of feature points A and B in the captured image have been determined, the XY coordinate components of feature points A and B on operating plane P are uniquely determined. It can be said that XY coordinate components on operating plane P is a function of UV coordinate components in the captured image.
Hydraulic excavator 100 before shipment includes an encoder 161 and an angle conversion unit 162. Encoder 161 is a general term for a boom angle sensor attached to boom pin 13, a dipper stick angle sensor attached to the dipper stick pin, and a bucket angle sensor attached to the bucket link. Instead of encoder 161, a potentiometer may be attached to work implement 2 to measure an angle. A stroke sensor that senses the stroke of the hydraulic cylinder may be attached to convert an amount of movement of the hydraulic cylinder into an angle.
Angle conversion unit 162 receives an electrical signal from encoder 161 and converts the electrical signal into boom angle θb, dipper stick angle θa, and bucket angle θk.
Recording unit 163 associates a posture of the work implement reflected in the captured image, more specifically, the coordinates of first link pin 18 (or feature point A) and those of bucket cylinder top pin 19 (or feature point B) in the captured image, with boom angle θb, dipper stick angle θa and bucket angle θk obtained when the image is captured, and thus records the coordinates and the angles. The feature points and the angles are recorded in recording unit 163 at a factory before hydraulic excavator 100 is shipped.
The captured image shown in
Hereinafter, based on a posture of work implement 2 in a captured image, that is, based on where feature points A and B are located in the captured image, a method for determining a position of the work implement relative to revolving unit 3, typically boom angle θb, dipper stick angle θa and bucket angle θk, will specifically be described.
Initially, encoder 161 shown in
With encoder 161 attached to work implement 2, boom 6 and dipper stick 7 are operated to move the position of feature point A in the captured image to position the feature point at one of the recording points shown in
Once bucket angle θk has been recorded for all of the eight recording points shown in
Work implement 2 is operated and the angles are recorded in this way for all of the recording points. After such recording is completed, encoder 161 is removed from work implement 2. A preparatory operation performed before hydraulic excavator 100 is shipped from a factory is thus completed.
When hydraulic excavator 100 is shipped from the factory, boom angle θb, dipper stick angle θa and bucket angle θk obtained when feature points A and B are located at each recording point in a captured image are stored in controller 60 (or recording unit 163).
After the shipment from the factory, when an image of work implement 2 is captured using imaging device 50 at a work site, feature point recognition unit 62 (see
Angle extraction unit 63 extracts boom angle θb and dipper stick angle θa corresponding to the closest recording point from recording unit 163, and sets them as the current boom angle θb and dipper stick angle θa.
Feature point recognition unit 62 also recognizes a position of feature point B in the captured image. Angle extraction unit 63 determines a recording point of the eight recording points shown in
Angle extraction unit 63 extracts bucket angle θk corresponding to the closest recording point from recording unit 163 and sets it as the current bucket angle θk.
Thus, boom angle θb, dipper stick angle θa, and bucket angle θk can be determined based on the positions of feature points A and B in the captured image. From boom angle θb, dipper stick angle θa and bucket angle θk thus determined, the XY coordinate components of feature points A and B on operating plane P can be determined, and a position of work implement 2 relative to revolving unit 3 can thus be determined.
A function and effect of the above embodiment will be described. In the embodiment, as shown in
Further, as shown in
Further, as shown in
Angle extraction unit 63 can use the information of the captured image and the angles of work implement 2 that are previously associated and thus stored to determine boom angle θb, dipper stick angle θa and bucket angle θk based on an image captured by imaging device 50. A simple configuration without an angle sensor can be employed to determine boom angle θb, dipper stick angle θa and bucket angle θk, as done in a conventional hydraulic excavator including an angle sensor.
Further, angle extraction unit 63 determines boom angle θb and dipper stick angle θa based on the posture of dipper stick 7 in the captured image, as shown in
Further, as shown in
Further, as shown in
Further, as shown in
Further, as shown in
In the above embodiment an example has been described in which angles of a work implement stored in association with recording points that are closest to the positions of feature points A and B in a captured image are set as the work implement's current angles. As shown in
A plurality of sets of data in which the positions of feature points A and B in a captured image are associated with a posture of work implement 2 may be recorded in recording unit 163 described above. For example, a position of bucket 8 in grading a horizontal land surface and that of bucket 8 when shaping a vicinity of a shoulder of a slope from a foot of the slope are significantly different and it is difficult to accommodate them within the same single angle of view of imaging device 50. Accordingly, when data obtained when imaging device 50 images a front side and data obtained when imaging device 50 images an obliquely upper side are previously recorded, and imaging device 50 can be angularly adjusted and appropriate data can be selected depending on the contents of the work of interest, the current posture of work implement 2 operating in a wide range can be determined accurately.
While feature point A described above is set on first link pin 18 located at the distal end portion of dipper stick 7, feature point A may instead be set on boom 6. In that case, boom angle θb can be determined from a position of feature point A in a captured image, and dipper stick angle θa and bucket angle θk can be determined based on the determined boom angle θb and a posture of bucket 8 to determine all of boom angle θb, dipper stick angle θa and bucket angle θk, similarly as done in the embodiment.
In the description of the above embodiment, hydraulic excavator 100 includes controller 60 and controller 60 mounted on hydraulic excavator 100 determines a relative position of work implement 2 by way of example. The controller that determines the relative position of work implement 2 may not be mounted on hydraulic excavator 100.
The presently disclosed embodiments are to be considered as illustrative in any respect and not restrictive. The scope of the present invention is not indicated by the above description but by the scope of the claims, and is intended to include meaning equivalent to the terms of the claims and any modifications within the scope.
1 main body, 2 work implement, 3 revolving unit, 4 cab, 5 traveling apparatus, 6 boom, 7 dipper stick, 8 bucket, 8a tooth tip, 10 boom cylinder, 11 dipper stick cylinder, 12 bucket cylinder, 13 boom pin, 14 dipper stick pin, 15 bucket pin, 16 first link member, 17 second link member, 18 first link pin, 19 bucket cylinder top pin, 20 second link pin, 50 imaging device, 60, 260 controller, 61 image processing unit, 62 feature point recognition unit, 63 angle extraction unit, 100 hydraulic excavator, 161 encoder, 162 angle conversion unit, 163 recording unit, A and B feature points, AX optical axis, P operating plane, RX axis of revolution.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2018-098160 | May 2018 | JP | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2019/011556 | 3/19/2019 | WO | 00 |
