The invention relates to a method and apparatus for controlling at least one spray head of a crop sprayer.
Crop spraying techniques are known where sensors are used to gather data about a field to be sprayed, the data is processed in order to identify foliage that is to be sprayed, and the spray heads of a crop sprayer are controlled in order to selectively spray the identified foliage. A typical application is the more targeted use of chemicals on weeds. This can result in a reduction of the amount of chemicals used on the field.
There is a need for alternative techniques for identifying foliage. Such alternatives may enable foliage to be identified with greater certainty or at lower cost.
According to a first aspect of the invention, there is provided a method of controlling at least one spray head of a crop sprayer, the method comprising the steps of:
In an embodiment, the method comprises determining whether to activate the first spray head in step (d) after determining in step (c) whether each of the sub-areas includes more than a defined threshold of pixels having defined color values corresponding to foliage to be sprayed.
In an embodiment, the method comprises determining whether to activate the spray head in step (d) after each determination in step (c) that an individual one of the first set of sub-areas includes more than a defined threshold of pixels having defined color values corresponding to foliage to be sprayed.
In an embodiment, the method further comprises:
In an embodiment, the method comprises, upon the spray head not having been activated:
In an embodiment the captured color image also corresponds to an area of the field to be potentially sprayed by a second spray head, and the method comprises:
In an embodiment, at least a portion of the second set of sub-areas correspond to both the first and second spray heads.
In an embodiment, the color values are in the CIELAB color space.
In an embodiment, the defined color values correspond to a volume within the CIELAB color space.
In an embodiment, the defined number of sub-areas is one sub-area.
In a second aspect, the invention provides apparatus for controlling at least one spray head of a crop sprayer:
In an embodiment, the controller is further configured to determine whether to activate the first spray head in step after determining whether each of the sub-areas includes more than a defined threshold of pixels having defined color values corresponding to foliage to be sprayed.
In an embodiment, the controller is further configured to determine whether to activate the spray head in after each determination that an individual one of the first set of sub-areas includes more than a defined threshold of pixels having defined color values corresponding to foliage to be sprayed.
In an embodiment, the controller is further configured to determine whether to activate the first spray head after:
In an embodiment, the controller is further configured to, upon the spray head not having been activated:
In an embodiment, the captured color image also corresponds to an area of the field to be potentially sprayed by a second spray head, and the controller is configured to:
In an embodiment, at least a portion of the second set of sub-areas correspond to both the first and second spray heads.
In an embodiment, the color values are in the CIELAB color space and the defined color values correspond to a volume within the CIELAB color space.
In an embodiment, the defined number of sub-areas is one sub-area.
In a third aspect, the invention provides a crop sprayer comprising a spray boom having a plurality of spray heads and apparatus as described above.
Further aspects of the invention, which should be considered in all its novel aspects, will become apparent to those skilled in the art upon reading of the following description which provides at least one example of a practical application of the invention.
Example embodiments of the invention will be described below by way of example only, and without intending to be limiting, with reference to the following drawings, in which:
Embodiments are disclosed of a method and apparatus for controlling at least one spray head of a crop sprayer as well as a crop sprayer incorporating the apparatus. In the preferred embodiment, the method and apparatus control all of the spray heads of the crop sprayer. In an example, the apparatus comprises one or more cameras that capture images as the crop sprayer traverses a field and one or more controllers that processes the captured images to determine whether or not to turn on associated spray heads based whether sub-areas of the images contain a defined number of pixels within a defined color range, for example a color range corresponding to foliage such as weeds.
While in the example, of
With reference to first camera node 110A, it will be apparent that it has a housing 118 mounted to two angled support arms 115A, 115B of a mounting bracket 114 that is used to attach camera node 110A to the spray boom 130 so that it is forward facing. The housing 118 has a camera window 116 and the camera itself is mounted to a printed circuit board (not shown) disposed within the housing with the camera so that the camera is aligned with camera window 116. The arms 115A hold the housing so that it is angled off the horizontal (in this example at an angle of 22 degrees) in order to enable the camera to capture a color image of an area of the field to be potentially sprayed as the crop sprayer traverses a field. When mounted to a boom arm of a typical crop sprayer, the bracket will position the camera window 116 around 800 mm to 1000 mm off the ground.
There is a first connector 112A on the right-hand side of housing 118 and, as better seen in
Spray heads 140A, 140B, 140C, 140D are mounted in order to draw fluid from pipe 150 mounted to spray boom 130. Pipe 150 is in fluid communication with a fluid reservoir (not shown) that supplies the spray heads 140.
Referring to
As indicated above,
Central node 315 is configured for two-way communication 315 with a user device, such as a smart phone, tablet, or computer. For example, to enable user device to update firmware in the central node 120 or camera node 110 or to change setting of the apparatus. The central node 120 can also communicate data to user device for use in diagnostics. As shown in
Referring to
Camera 420 can be any suitable digital camera. In most use cases a camera having a resolution of 640×480 pixels is suitable for mounting to a crop sprayer because at approximately 1 meter above the ground, each pixel corresponds to approximately 1.5 mm on the ground. It will be appreciated that the selected resolution impacts on cost of the camera and also on processing requirements and thus, potentially, on the cost of the processor. In an example, the target operating speed of the crop sprayer is 12-18 km/h and the apparatus captures images at a rate of ˜30 times per second which is also related to the selected resolution and the processing capacity.
Controller 410 communicates with the central node and the nozzles via input/output ports 430. In an example, for the nozzles the output port 430 is a relay circuit for turning solenoids on the respective nozzles on and off.
Referring to
At step 505, the processor 412 of a respective camera node 110 controls the camera 420 to capture a color image in the CIELAB color space of an area of the field to be potentially sprayed by a spray head 140 that is under the control of the respective camera node 110.
At step 515, the processor 412 divides the captured image into a set of first sub-areas. An illustrative example is provided in
As illustrated in
At step 520, the processor 412 begins iterating through the first sub-areas. At step 525, the processor 412 determines whether the current sub-area has more than a defined number of pixels that meet a test for being “Green” (that is, having defined color values corresponding to the color green). This involves, an inner iterative loop (not shown) of determining on a pixel-by-pixel basis whether the respective pixel corresponds to a range of color values that are treated as green by the apparatus. As indicated above, the image is captured in the CIELAB color space. One of the reasons for doing so is that subsequently determining whether a pixel is within a defined range of color values in the CIELAB color space is computationally straightforward. However, the apparatus and method can be adapted to work with other color spaces. An example of defined color values is shown in
After all the pixels have been processed it is determined whether there are enough “green” pixels to mark a sub-area as green at step 530. The pixel threshold is set based on a number of factors such as the number of pixels within a sub-area and the range of colors that are treated as being “green” (or whatever range of color values that corresponds to foliage to be sprayed). In some examples 2-4 pixels may be appropriate. In another example, a single pixel may be sufficient.
After a sub-area has been marked as green at step 535 or it is determined at step 525 that there are not more “green” pixels that the pixel threshold, it is determined at step 535 whether all of the first sub-areas have been processed. If not, the processor 412 iterates to the next sub-area (e.g. working from the left most column 621 in the top row 611 to the right most column 628 in the bottom row 614).
After all the first sub-areas have been processed, processor 412 proceeds to step 540 and divides the image into a second set of sub-areas arranged in overlapping relationship with at least a portion of the first set of sub-areas. An example of a second set of sub-areas is shown in
Processor 412 then carries out steps 545 to 560 for the second set of sub-areas; these being equivalent to steps 520 to 535 described above. That is, at step 545, processor 412 begins iterating through the sub-areas. At step 550, processor 412 determines whether a current sub-area has more that a threshold of pixels that are within the define color range and, if so, the sub-area is marked as green at step 555. Step 560 ensures all sub-areas are processed.
After all second sub-areas are processed, at step 565, the processor 412 determines for each spray head whether the number of “green” sub-areas correspond to the respective spray head is greater than or equal to a defined number of sub-areas for triggering spraying. In some examples, a single “green” sub-area triggers spraying. Depending on the outcome of step 565, the processor proceeds to step 570 of “Spray” or step 575 “Don't Spray”. In this example, on reaching a decision to spray, at step 570, and assuming the spray head is currently off, processor 412 activates the spray head for a defined period of time based on the speed of travel of the crop sprayer (for example 200 ms). Then at step 580, the processor 412 determines whether the apparatus is still in an on state and, if so, reverts back to step 510 to process the next capture image as indicated by process connector “X” 590, 505.
When in a subsequent iteration it is decided to spray at step 570, the processor 412 resets the off timer for the respective spray head. Accordingly, if at step 575, the decision is to not spray, processor 412 leaves the off timer running. As a result, the relevant spray head will remain on while the off timer is running and be turned off a defined period after the last green sub-area.
When it is determined at step 580 that the crop sprayer is no longer on, the process ends 585.
It will be appreciated that at step 565, because the sub-areas of middle column 824 of the second set of sub-areas, overlap the center line 833, they can contribute to both the first and second nozzle being turned on if they are found to have sufficient green pixels.
It will be appreciated that the above method 500 results in all of the sub-areas being processed. An advantage of proceeding in this way is that data can be captured to implement quality control and refine the settings. In other implementations, processing shortcuts may be implemented, for example by ending iteration loops for finding pixels having a defined color or sub-areas with a defined number of pixels having the defined colors as soon as the relevant threshold has been reached.
It will be appreciated that other arrangements of a second set of overlapping sub-areas can be employed, for example, an embodiment employing only the central column 824 will provide some improvement relative to just employing the first set of sub-areas. Other shapes of sub-areas than the rectangular array are also possible but the configuration described above is computationally efficient to implement.
Testing was conducted to determine the improvement offered by using a second set of sub-areas (referred to as the offset grid in Table 1 below). It can be seen that while using a single grid results in many detections, additionally using an offset grid improves detection of sub-areas that should be sprayed by approximately 3-5% depending on the threshold value for treating individual pixels as green.
The processed image 930 shows that in this example, the first array of sub-areas has 8 rows and 16 columns (8 for each half of the image), and the second, overlapping array has 7 rows and 14 columns. It will be seen that a region 931 of the processed image comprises sub-areas marked as to be sprayed which will cause both spray heads (nozzles) to be activated. This region 931 corresponds to the weed 925 that can be found in the original image.
In
The processed image 1030 shows that region 1031 of the processed image comprises sub-areas marked as to be sprayed. It will be observed that in this example, the only sub-area 1032 that corresponds to the right-hand side of the image is from the second array of overlapping sub-areas. That is, in this example, the second spray head would not have been activated if only the first array of sub-areas had been processed.
In
Unless the context clearly requires otherwise, throughout the description and the claims, the words “comprise”, “comprising”, and the like, are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense, that is to say, in the sense of “including, but not limited to”.
Reference to any prior art in this specification is not, and should not be taken as, an acknowledgement or any form of suggestion that that prior art forms part of the common general knowledge in the field of endeavor in any country in the world.
The invention may also be said broadly to consist in the parts, elements, characteristics and features referred to or indicated in the specification of the application, individually or collectively, in any or all combinations of two or more of said parts, elements, characteristics or features.
Aspects of the present invention have been described by way of example only and it should be appreciated that modifications and additions may be made thereto without departing from the scope thereof as defined herein.
Where in the foregoing description reference has been made to integers or components having known equivalents thereof, those integers are herein incorporated as if individually set forth.
It should be noted that various changes and modifications to the presently preferred embodiments described herein will be apparent to those skilled in the art. Such changes and modifications may be made without departing from the spirit and scope of the invention and without diminishing its attendant advantages. It is therefore intended that such changes and modifications be included within the present invention.
Number | Date | Country | Kind |
---|---|---|---|
2021901471 | May 2021 | AU | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/AU2022/050381 | 4/26/2022 | WO |