Patent Application
20170051474
The present disclosure relates to a path detection system, and more particularly to a system and method for path detection of ground engaging teeth of a machine implement.
Machines, for example, a wheel loader, include an implement to perform a variety of tasks on a worksite. This implement may be embodied as a bucket for digging, collecting, and dumping material from one location to another at the worksite. The bucket may include ground engaging teeth provided at an edge of the bucket that contacts the ground. The ground engaging teeth are used to dig into and/or lift the material off the ground.
However, sometimes during operation, the ground engaging teeth may loosen due to usage. Further, the loose around engaging teeth may fall off the implement. Sometimes, these ground engaging teeth may mix with the material being loaded into a crusher. In such a situation, there may be chances of the crusher getting damaged if such stray ground engaging teeth are present in the material entering the crusher. This may result in high machine downtime, increased service and maintenance time, and high operational costs.
Image data from an image feed of the ground engaging teeth may be analyzed using computer vision algorithms to detect the ground engaging teeth during a dump operation. Accordingly, on-board cameras pointed at the ground engaging teeth may be provided as input to the computer vision algorithms to assess the number of the ground engaging teeth present on the bucket. However, such systems may generate false positives, inaccurately tracking the ground engaging teeth using complex methods that may not be robust and effective.
United States Published Application Number 2015/0149049 describes a process and tool for monitoring the status, health, and performance of wear parts used on earth working equipment. The process and tool allow the operator to optimize the performance of the earth working equipment. The tool has a clear line of site to the wear parts during use and may be integrated with a bucket or blade on the earth working equipment.
In one aspect of the present disclosure, a method for path detection associated with ground engaging teeth of an implement is provided. The method includes receiving, by a controller, an image feed of a dump operation being performed by the implement from an image capturing device. The image feed includes a plurality of frames. The method includes detecting, by the controller, the ground engaging teeth in each of the plurality of frames of the image feed by iteratively scanning a plurality of sections of each of the plurality of frames. The method includes analyzing, by the controller, a path traced by the ground engaging teeth over the plurality of frames, such that path information associated with the ground engaging teeth is any one of aggregated or discarded based on a movement of the ground engaging teeth over the plurality of frames. The method includes determining, by the controller, the path traced by the ground engaging teeth based on the aggregated path information. The method includes comparing, by the controller, the determined path traced by the ground engaging teeth with a pre-determined path. The method includes identifying, by the controller, if at least one of the ground engaging teeth is missing based on the comparison. The method includes providing, by the controller, a notification of the missing ground engaging teeth based on the identification.
Other features and aspects of this disclosure will be apparent from the following description and the accompanying drawings.
Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or the like parts. Also, corresponding or similar reference numbers will be used throughout the drawings to refer to the same or corresponding parts.
Referring to
The machine 10 also includes wheels 16 for the purpose of mobility. The powertrain may also include a torque converter, a transmission system inclusive of gearing, a drive shaft and other known drive links provided between the power source and the wheels 16 for the transmission of the motive power. Further, the machine 10 has an operator cabin 18 that houses controls for operating the machine 10.
As shown in FIG, 1, a linkage assembly 19 is attached to the frame 12 of the machine 10. The linkage assembly 19 includes a lift arm 20. An implement, such as a bucket 22, is pivotally coupled to the lift arm 20. It may be noted that the linkage assembly 19 and the implement of the machine 10 may vary based on the type of machine 10 or the type of operation or task required to be carried out by the machine 10.
During operation of the machine 10, the lift arm 20 and the bucket 22 may be moved to different positions in order to perform excavation and dumping tasks. The movement of the lift arm 20 and/or the bucket 22 is controlled by hydraulic and/or pneumatic cylinders 24, which are coupled to these parts. Accordingly, based on the movement of the lift arm 20 and the bucket 22, the machine 10 may perform different operations such as excavating, loading, and dumping.
An edge 26 of the bucket 22 that contacts a ground surface includes ground engaging teeth 28 provided thereon. The ground engaging teeth 28 extend outwards from the edge 26. The ground engaging teeth 28 may be used to dig into and/or scrape the ground surface to lift off a material 30. The ground engaging teeth 28 may minimize wear on the edge 26 of the bucket 22 so that the easily replaceable ground engaging teeth 28 may be re-welded whenever required, thereby increasing the life of the bucket 22. The present disclosure relates to a path detection system 32 (see
Referring to
Referring to
The controller 38 analyses the image feed on a frame by frame basis. The controller 38 makes use of an algorithm to assess each of the frames of the image feed. The controller 38 detects a presence of the ground engaging teeth 28 in the image teed. In one embodiment, the algorithm may be a machine learning algorithm. For example, a support vector machine using a histogram of oriented gradients as a feature vector may be utilized to perform the detection.
Using the machine learning algorithm, the controller 38 may iteratively scan multiple adjacent sections or areas of each frame. In other words, as shown in
A training dataset, which is aggregated or built for the purposes of training machine learning, may include large volumes of representative training data for detecting the presence of the ground engaging teeth 28. The training data may be divided into positive training set images and negative training set images. The positive training set images may include clean images, dirty images, and augmented images for example translated images, rotated images, and/or mirror images. The algorithm applies the trained support vector machine classifier to multiple frames of the image feed, such that the sliding window 40 iteratively progresses along the sections of the frame and derives the histogram of oriented gradient features. In one example, the training data may be stored in a database 42 (see
Further, the controller 38 may detect the presence of the ground engaging teeth 28 in each frame of the image feed by performing a raw detection of the ground engaging teeth 28 from the sliding window 40 detector and then cleaning the detection made, for example by using RANSAC. The controller 38 pools overlapping detections and further finds a highest scoring window inside each pooled location denoted by representative boxes 43 shown in
After detection of the ground engaging teeth 28, the controller 38 analyses a path traced by the ground engaging teeth 28 as the bucket 22 moves from one position to another over each of the frames of the image feed. Referring to
The controller 38 uses an algorithm to analyze the path traced by each of the around engaging teeth 28 over different frames of the image feed. By accumulating and analyzing the detections of the ground engaging teeth 28 over multiple frames throughout each of the dump operations, the controller 38 may algorithmically ascertain the path traced by the ground engaging teeth 28, developing a higher confidence in the number of the ground engaging teeth 28 present on the bucket 22 as the dump operation progresses. The path traced is expected to be a curved path 44. As shown in the accompanying figures, each of the eight ground engaging teeth 28 traces the curved path 44 over the multiple frames of the image feed. However, the path may vary based on the positioning of the image capturing device 34 on the machine 10.
More particularly, the algorithm for analyzing the path traced by each of the ground engaging teeth 28 may make use of a spatial-temporal trajectory filter that is applied by the controller 38 over the multiple frames of the image feed. After detecting the ground engaging teeth 28 in a given frame ‘n’, the controller 38 searches subsequent frames, say frame ‘n+1’ and frame ‘n+2’ for the closest detection of the presence of the ground engaging teeth 28 within a predetermined distance, for example approximately 100 pixels. If such detection is found by the controller 38, the controller 38 aggregates path information of the detection of the ground engaging teeth 28 that lie within the predetermined distance to that of an existing path. This existing path information is associated with the path traced by the given ground engaging teeth 28, in line with the movement of the ground engaging teeth 28 over the multiple frames of the image feed. Alternatively, if the ground engaging teeth 28 are detected outside of the predetermined distance in the subsequent frames, the controller 38 discards or rejects the path information of the given ground engaging teeth 28.
Accordingly, based on the aggregated path information of each of the ground engaging teeth 28 over the multiple frames of the image feed, the controller 38 determines the path that is traced by the ground engaging teeth 28 during the dump operation. It should be noted that the path information must be analyzed over multiple consecutive frames. The steps of path estimation described herein make use of simple filter settings. These steps are included on an exemplary basis and do not limit the scope of the present disclosure.
The controller 38 may then determine the path traced by the around engaging teeth 28 over the multiple frames based on the aggregated path information associated with each of the ground engaging teeth 28. As shown in
The controller 38 may additionally compare the path traced by each of the ground engaging teeth 28 with a predetermined path. For example, the predetermined path may be a previous instance of the path recorded. In one embodiment, the predetermined path may be retrieved from the database 42. Based on the comparison, the controller 38 identifies if one or more of the around engaging teeth 28 were either lost or distorted during the dump operation. For example, the controller 38 may count a number of the paths traced by the ground engaging teeth 28 to check if any of the ground engaging teeth 28 are missing. Alternatively, any other type of comparison may be used to identify if one or more of the ground engaging teeth 28 were lost during the dump operation.
If the controller 38 identifies that at least one of the ground engaging teeth 28 are missing, the controller 38 may send a command signal to the output device 46 to provide a notification to an operator. The output device 46 may optionally include any known visual and/or auditory device such as, a speaker, a screen, an alarm, a light emitting diode, and so on to provide the notification to the operator. The analyses performed by the controller 38 and/or the output provided by the controller 38 may be provided on a relatively close to real-time basis, factoring in a slight delay that is attributed to data capture associated with the subsequent frames of the image feed. Accordingly, if time taken to capture the subsequent data frames is defined by the predefined time period, for example approximately between 3 and 5 seconds, the delay in providing the output by the controller 38 may be appropriately defined on the basis of the predefined time period.
The controller 38 may embody a single microprocessor or multiple microprocessors. Numerous commercially available microprocessors can be configured to perform the functions of the controller 38. The controller 38 may include all the components required to run an application such as, for example, a memory, a secondary storage device, and a processor, such as a central processing unit or any other means known in the art. Various other known circuits may be associated with the controller 38, including power supply circuitry, signal-conditioning circuitry, solenoid driver circuitry, communication circuitry, and other appropriate circuitry.
The present disclosure relates to the system and method for path detection of the ground engaging teeth 28 associated with the implement of the machine 10.
Further, at step 58 the controller 38 compares the determined path traced by the ground engaging teeth 28 with the pre-determined path. At step 60, based on the comparison the controller 38 identifies if one or more of the ground engaging teeth 28 are missing. At step 62, the controller 38 provides the notification of the missing ground engaging teeth 28 if such ground engaging teeth 28 are identified.
The present disclosure provides a simple and cost effective solution that can be implemented easily for detecting the path traced by the ground engaging teeth 28. The system also notifies the operator if any of the ground engaging teeth 28 were lost during the dump operation so that corrective actions may be taken before the material 30 is introduced into the crusher. Further, the system may greatly reduce a number of false positives in the identification and detection of the presence of the ground engaging teeth 28 in the image feed.
While aspects of the present disclosure have been particularly shown and described with reference to the embodiments above, it will be understood by those skilled in the art that various additional embodiments may be contemplated by the modification of the disclosed machines, systems and methods without departing from the spirit and scope of what is disclosed. Such embodiments should be understood to fall within the scope of the present disclosure as determined based upon the claims and any equivalents thereof.
