Patent Application
20210345567
The present invention claims priority from Australian Provisional Patent Application No. 2018903312 filed on 5 Sep. 2018, the entire contents of which are incorporated herein by reference.
The present invention relates to methods and systems for plant stress determination and automated irrigation management methods and systems based on plant stress determination.
Over agricultural fields, the standard meteorological observations—air temperature, relative humidity, solar radiation, wind speed—are used to estimate the Potential Evapotranspiration (PET), which constitutes the primary use of water in agriculture. PET is calculated conventionally by a Penman-Monteith approach based FAO56 model. PET can be estimated using only the meteorological input and assumes that there is no limitation of water in soil available for the evapotranspiration. Consequently, PET is regarded as the maximum possible water use by crops with given meteorological condition and often overestimates the actual consumption of water by the crops. PET can be converted to the actual evapotranspiration (AET) by adjusting it using crop growth indicator and water stress indicator. AET becomes close to PET as crops are in full growth and in active photosynthetic mode with minimal sign of water stress.
The present invention is an extension of the irrigation management systems and methods disclosed in our International Patent Application No. PCT/AU2018/050858, the full contents including description, claims and drawings of which publication are assumed to have been read and incorporated herein by reference to avoid repetition of description. This patent specification spatially derives soil moisture for any location within an irrigation district and the ability to map the soil moisture at any point in time and spatially across the irrigation district. The derivation is achieved using the monitored inputs of:
The specification discloses the use of input data from sensors to reflect the highly spatially variable nature of some of these parameters to, in-turn, derive the soil moisture which is also highly spatially variable in nature.
System identification techniques are used to produce an algorithm based on the above inputs and the use of soil moisture sensors at various locations in the field to derive the relationship and provide ongoing calibration.
It is not admitted that any of the information in this patent specification is common general knowledge, or that the person skilled in the art could be reasonably expected to ascertain or understand it, regard it as relevant, or combine it in any way before the priority date.
In one aspect of the invention there is provided a method of plant stress determination using a computer-based camera system having thermal imaging and visual imaging to capture foliage at close proximity of at least one plant to provide high resolution images/video thereof; analyzing both thermal and visual images/video therefrom to form a composite image; determining the thermal activity of the composite image/video and photosynthesis state of said at least one plant; and deriving the plant stress from said determination.
In a further aspect there is provided plant stress determination system including a computer-based camera system having thermal imaging and visual imaging to capture foliage at close proximity of at least one plant to provide high resolution images/video thereof; analyzing both thermal and visual images/video therefrom to form a composite image; determining the thermal activity of the composite image/video and photosynthesis state of said at least one plant; and deriving the plant stress from said determination.
Preferably said computer-based camera system is based on a smartphone. In one aspect said thermal imaging is captured by a thermal camera associated with said smartphone and said thermal imaging may be captured by a thermal camera associated with said smartphone.
In a preferred embodiment said thermal and visual images capture at least one or more of information related to time of capture, GPS location, accelerometer data, direction data and inclinometer data.
In a further aspect said composite image uses edge detection of said foliage to align/co-register the visual and thermal images/video and the aligned/co-registered thermal images/video may be processed using augmented reality techniques to provide image measurement.
In a further embodiment the plant stress determination further includes inputs from relevant derived evapotranspiration and soil moisture data from land to be irrigated to provide an irrigation schedule for an operator linked to a networked computer system overseeing said plant stress determination, said relevant derived evapotranspiration and soil moisture data from said land to be irrigated.
Another aspect the plant stress determination is calculated by said computer-based camera system and said thermal imaging and said visual imaging may be synchronized to be taken at the same time.
A further aspect the plant stress determination further includes inputs from a surface representing a non-transpiring leaf whose temperature is measured and a surface representing a transpiring leaf whose temperature is measured.
In yet a further aspect of the invention there is provided an irrigation management system to irrigate predetermined areas of an irrigation district, said irrigation management system including:
A still further aspect of the invention provides a method of scheduling irrigation of land including steps of determining plant stress using a computer-based camera system having thermal imaging and visual imaging to capture foliage at close proximity of at least one plant to provide high resolution images/video thereof; analyzing both thermal and visual images/video therefrom to form a composite image; determining the thermal activity of the composite image/video and photosynthesis state of said at least one plant; deriving the plant stress from said determination; wherein the plant stress determination further includes inputs from relevant derived evapotranspiration and soil moisture data from land to be irrigated to provide an irrigation schedule to an operator linked to a networked computer system overseeing said plant stress determination, said relevant derived evapotranspiration and soil moisture data from said land to be irrigated.
In yet a further aspect of the invention there is provided a method of gaining information on atmospheric conditions related to plant stress by using a thermal target that has:
A further aspect of the invention provides method of plant stress determination using a computer-based camera system having visible, near infrared, shortwave infrared, and thermal infrared imaging capability to capture foliage at close proximity of at least one plant to provide high resolution images/video thereof; analyzing selections of composites from said visible and infrared bands in images/video; determining the water stress, leaf water content, leaf pigment condition and photosynthetic activity from the composite image/video of said at least one plant; and deriving the plant stress from said determination.
Preferably said computer-based camera system is based on a smartphone. The near and shortwave infrared imaging can be captured by an infrared camera associated with said smartphone and said thermal imaging is captured by a thermal camera associated with said smartphone.
In a preferred embodiment said thermal, near/shortwave infrared and visual images capture at least one or more of information related to time of capture, GPS location, accelerometer data, direction data and inclinometer data.
The composite image uses edge detection of said foliage to align/co-register the visual and thermal images/video and the aligned/co-registered thermal images/video are processed using augmented reality techniques to provide image measurement.
The plant stress determination further includes inputs from relevant derived evapotranspiration and soil moisture data from land to be irrigated to provide an irrigation schedule for an operator linked to a networked computer system overseeing said plant stress determination, said relevant derived evapotranspiration and soil moisture data from said land to be irrigated.
An embodiment of the method and apparatus will now be described by way of example only with reference to the accompanying drawing in which:
Embodiments of the present invention seek to provide improvements of our earlier invention disclosed in International Patent Application No. PCT/AU2018/050858 by determining plant stress or water stress at the location. Plant stress (water stress) is a state where the plant is growing in non-ideal growth conditions that increase the demands made upon it. The effects of stress can lead to deficiencies in growth, crop yields, permanent damage or death if the stress exceeds the plant tolerance limits. Accordingly, accurate irrigation for plants will reduce the stress to maximise plant growth and health. In effect the soil moisture is indirectly an indicator of the plant stress. Plant stress is inversely a measure of the plant photosynthesis (i.e. productive growth). As the water stress builds in the plant the photosynthesis begins to decline. There is an optimal point of plant stress when the crop should be irrigated for water use efficiency and plant production. The plant stress may also vary for different stages of the plant growth cycle. For example, higher stress may encourage root penetration of the soil which in turn can improve plant growth.
The derived evapotranspiration 24 can then allow soil moisture 34 to be determined by the networked computer system. Soil moisture 34 is derived using the inputs of rainfall 14 from weather station 12, irrigation historical data 36 together with soil type 38 stored in the networked computer system. A representative soil moisture sensor 40 is monitored by the networked computer system to provide feedback to calibrate the algorithm used to derive the soil moisture 34. A typical commercially available soil moisture sensor 40 would be 1 metre long and have measurement probes every 10 cm along its length. These measurement probes will provide a better soil moisture analysis as moisture penetration is important as well as surface moisture.
The present invention enhances the operation of the disclosure in International Patent Application No. PCT/AU2018/050858 shown within dashed lines 10. The present invention uses the data from the derived soil moisture 34 as an input into the derivation 44 of plant/crop stress to further finetune irrigation schedules 42 provided by the networked computer system (not shown).
Foliage is the main organ of photosynthesis and transpiration in higher plants. The invention provides field measurement (ground point source) 46 of foliage temperature using a thermal imaging camera (not shown). The measurement is taken at a known GPS location at a point in time. The thermal image is taken by an infra-red camera that is coupled to a smartphone (not shown). The infra-red camera is coupled to the smartphone (directly, or via Bluetooth or other wireless communication). The smartphone is linked to the computer network system. Such a thermal imaging camera is commercially available from FLIR Systems, Inc. for attachment to Apple IOS or Android smartphones. The smartphone can provide the necessary GPS location and the date and time relating to the images taken. The smartphone will run an application for ordering water that has access to the wider data inputs and evapotranspiration 24, soil moisture 34 and plant/crop stress analytics 44. The smartphone will provide a dual image (thermal image 46 from the infra-red camera and visual image 50 from the smartphone camera) of the same view of the crop canopy. The typical use of the smartphone will be to take the images of the crop canopy while the smartphone is handheld with a typical view area size of 1 m×1 m with the smartphone held at a height of approximately 1 m above the ground. The resulting images will have pixel resolution of approximately a millimetre. The infra-red camera will firstly be used to calibrate satellite data 48 having coarse resolution and subject to infrequent satellite passes, typically—one to two weeks. Using known image processing techniques e.g. the open source imaging program ImageJ, the crop canopy image will be processed, and information gained on foliage size/density/colour within the particular view of the image that has been taken. The infra-red image data 46 (e.g. the radiated energy spectra) will also be ‘paired’ with the visual image 50 providing additional data 52 (e.g. thermal analysis) for specific foliage within the view. Of particular focus is the ‘pairing’ of the two images so that the thermal image of the leaf matter is known. The pairing will align or co-register the images. There is a direct correlation between the thermal status (temperature) of the leaf matter and the state of the plant's photosynthesis. The measure of the plant photosynthetic state is inversely proportional to the plant stress. This point source analysis at ground level provides a micro analysis of the crop canopy at close proximity. It allows for an instantaneous indirect measure of the status of the plant's photosynthesis.
The visual image will be analysed to determine the content of the crop canopy in the view using visualisation software. Of particular interest will be the identification of the leaf matter as opposed to other crop content such as stems, including leaf petioles, nodes and internodes, shoots and flowers. The smartphone will also be able to distinguish other non-plant content such as soil. The visual analysis will also be to determine the size of the components identified in the view and establish, for example, if the leaf matter is old growth or new growth.
Plant stress is the object function of soil moisture derivation and estimating the optimal time to irrigate a crop i.e. the irrigation schedule 42. The ground measurement of plant stress will be used within the system identification process to provide ongoing calibration of the irrigation scheduling system. Additional direct inputs to the derivation of plant stress at ground measurement will come from local weather data from weather station 12 such as ambient temperature, humidity, solar radiation, and wind speed. Time of day will also be relevant to the daily plant photosynthesis cycle. Crop type and crop growth stage will also be relevant to the plant stress. These inputs are best accessed via the derived evapotranspiration 24, but not exclusively.
The application for ordering water in the smartphone can also suggest parts of an irrigated area to take ground images to give best representation. For example, it would be beneficial to obtain ground measurements at a range of indicative soil moistures (e.g. low, medium and high). The application for water ordering will not only provide an estimated reading on the plant stress for the crop at that location/time, but will also be used in a broader knowledge base/AI system to refine and calibrate the plant stress prediction system for that location (parameters such as soil type, weather) but also for crop type/variety that in turn can be applied across a wider operation.
Historic data 54 (e.g. yields) will provide feedback and learning on the optimal plant stress levels at which the irrigation schedule 42 can achieve the most efficient water use and at the same time achieve maximum crop production.
The continuous learning and refinement can not only be applied to a specific farm but also used across similar farming enterprises using the same/similar crops and varieties. The platform provides for a wider knowledge base, calibration and learning. This process would employ computer-based data analytics and artificial intelligence algorithms.
The systems Identification techniques will be applied to develop a relationship for plant stress based on data inputs described, and also employing fine tuning and calibration via feedback from field measures such as the smartphone, soil moisture probes 40 and historic yield data 54. The system identification processes described in this invention rely directly on sensor data (e.g. rainfall 14, temperature 16) and input data (e.g. crop type 28, soil type 38) to obtain interim derived outputs (e.g. evapotranspiration 24, soil moisture 34) in the overall process of producing an optimal irrigation schedule 42. The inputs shown in (sensor and input data) to obtain the interim derived outputs are not limited to the shown structure. For example, temperature 16 may be a direct input to deriving plant stress.
An additional embodiment of this patent will be that the smartphone device will provide instant output of the plant stress for the crop captured in the view.
Alternatively, the prior art has described the use of thermal images taken at higher altitudes above the crop canopy using towers or drones/aircraft. These are known as air thermal imaging. Towers provide continuous thermal imaging at a point source. Drones/aircraft provide spatial thermal detail but periodically when the drone/aircraft passes over the crop canopy. Typically, the pixel resolution can be 5 cm for a drone.
Satellite thermal images have also been obtained at a greater altitude again. The higher the altitude of imaging the less resolution there is with detail in the images. Typically, the pixel resolution can be 30 m for a Landsat satellite and 250 m for a MODIS satellite. There is also an averaging associated with both types of thermal imaging. If, for example the crop is planted in rows the thermal image will be an average of the crop canopy and the exposed soil.
Obtaining images at a ground level, in accordance with the invention, overcomes the above resolution problems of both air thermal imaging and satellite thermal images. The analysis that will be undertaken in this process will be able to distinguish between crop canopy and exposed soil. As evapotranspiration occurs in the leaf, and thermal measurement of the leaf is an indication of the rate/level of evapotranspiration and accordingly, crop stress. Thermal measurement of foreign items in the field of view (e.g. stems, weeds or soil) would produce errors in this measurement as they are at a different temperature to that of the leaf. Thus, thermal measurement techniques of a lesser resolution (e.g. satellite/air) cannot distinguish leaf temperature accurately as they are averaging temperatures over a much larger area that includes the foreign items. Typically, the pixel resolution will be 1 mm. This fine resolution can distinguish between leaf matter and stems, including leaf petioles, nodes and internodes, shoots and flowers. The image processing will also be able to distinguish between old growth and new growth through learning techniques. The images will specifically focus on measuring the leaf matter of the crop canopy without interference from extraneous non-leaf based matter in order to measure the photosynthetic status of the plant. These image processing techniques are of a known science and in the public domain.
On the left hand side of
A switch 108 is coupled to circuitry 102 to provide the simultaneous operation of both the smartphone camera lens(es) 87 and the thermal imaging device 100 for photographing plants 70. A radar or ultrasonic sensor 110 may assist in obtaining measurement data of the plant foliage 70 in conjunction with the MEMS sensors of smartphone 86 and the augmented reality software 80 (
Smartphone 86 will capture data 114 (
In an additional embodiment, the camera system 84 may also include a near infrared and/or a shortwave infrared imaging device. Such a device may allow leaf water content information to be determined. The camera system 84 proposed takes advantages of both crop growth/vigour and water stress information measured simultaneously in the close range from the target crop. By combining visible, near infrared, and thermal infrared images along with on-site ancillary information such as local meteorological data, crop type/growth, and portable calibration target, the device can produce plant-by-plant estimates of crop water stress, crop vigour, and water consumption with a minimal reliance on empirical crop parameters.
Embodiments of the invention have been described above by way of non-limiting example only. In practice, a plurality of weather stations 12, flow gates, flow meters and soil moisture sensors 40 are scattered around the irrigation district to provide an extensive irrigation system. Variations and modifications to the embodiments may be made without departing from the scope of the invention.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2018903312 | Sep 2018 | AU | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/AU2019/050919 | 8/29/2019 | WO | 00 |
