Patent Application
20220076141
The present invention relates to a system and a method to predict insect attack risk.
In agriculture, insect attacks are one of the largest sources of damages for crops. By their nature, insect attacks appears to be unpredictable and generally different season by season, due to a number of constantly changing factors, including, but not limited to, weather pattern, crops growth and disposition.
Late discovery of insect attack is hence a serious issue, as available remedies may be ineffective to save crops. Especially, lack of efficient and timely monitoring is one of the main reasons why the use of biological pesticides is struggling to widespread due to its limited effectiveness in time.
The known monitoring and prediction techniques are based on experts (e.g. entomologists) who directly analyse the crops or, according to more recent solutions, remotely evaluate pictures to identify target insects and empirically estimate possible attacks.
The known techniques are proving to be inefficient since they require human interventions and skills to identify the specific target insects and show significant limitations in the capability of predict insect attacks.
The present invention addresses the problem of providing and insect attack risk prediction system which shows satisfying prediction performances.
According to a first object, the present invention relates to an insect attack prediction system defined by the appended independent claim 1. Particular embodiments of the system are described by the dependent claims 2-9.
In accordance with a second object, the present invention relates to an insect attack prediction method defined by the appended independent claim 10.
Further characteristics and advantages will be more apparent from the following description of a preferred embodiment and of its alternatives given as an example with reference to the enclosed drawings in which:
Particularly, the insect attack prediction system 100 as represented in
The memories of the processor apparatus 101 include instructions to configure the processor apparatus 101 so as to perform an insect attach prediction method. According to the example of
According to the described example, the sensor apparatus 102, the insect identification module 103 and the risk prediction module 104 are local modules, i.e. they are functionally associated to a specific area of interest. If the prediction system 100 is configured to serve a plurality of different areas of interest further sensor apparatuses 102, further insect identification modules 103 and further risk prediction modules 104 can be employed.
Moreover, the following additional software modules can be executed by the processor apparatus 101: a local knowledge module (L-KW) 105 and a knowledge configuring and updating module 106 (KW-CONF-UPDT).
Particularly, the sensing apparatus 102 comprises at least a digital camera 107 to take digital images. The digital camera 107 can be a photo-camera or a video-camera. More particularly, the digital camera 107 can be selected from the group: RGB camera, infrared camera, ultraviolet camera.
Moreover, the sensing apparatus 102 may comprise at least one meteorological sensor 108 configured to detect at least one meteorological quantity. As an example, the meteorological quantities can be selected from the group: temperature, humidity, pressure, moisture level, leaf hygrometer.
Furthermore, the sensing apparatus 102 can comprise at least one environmental sensor 109 configured to detect at least one environmental quantity. To the purpose of the present invention, an environmental quantity is a physical or chemical quantity indicative of environmental pollution and which is not identified as a standard meteorological quantity. As an example, the environmental quantities can be selected from the group: quality of air (e.g. carbon dioxide CO2 concentration, carbon monoxide CO concentration, Volatile Organic Compounds concentration, Ammonia concentration, etc.), luminosity, sound presence, sound level, long terms seasonal time, uses of pesticide.
The sensing apparatus 102 can also include suitable electronic circuits necessary to the conditioning of the signals provided by the sensors, their conversion into digital form, together with suitable software configured to acquire the measured quantities for following digital processing.
According to a specific embodiment, at least the digital camera 107 and/or other sensors of the sensing apparatus 102 can be housed in a trap device (not shown) for in situ capture of infesting insects. As an example, the trap device described in the Italian patent application document No. 102018000001753 can be employed in the prediction system 100. An employable trap device comprises a housing with an inner chamber provided with at least one opening towards the external environment and devices (such as a sticky paper) configured to capture and immobilize the insect. Moreover, the trap device may include devices configured to release substances (such as pheromone) suitable for attracting the insect in the inner chamber. The employed digital camera 107 is oriented to take digital images IM of the insects captured by the sticky paper. In area of interest, one or more trap devices can be installed.
The sensing apparatus 102 is configured to provide digital images IM and further electrical detected signals SOT (carrying the detected quantities), to the insect identification module 103.
The insect identification module 103 is configured to process at least one insect digital image IM to provide a presence data IPD representing the presence of the insect in an area of interest. Particularly, the insect identification module 103 is configured to identify an insect of a specific species (also called, target insect) from a digital image. More particularly, the insect identification module 103 is also configured to count the total number of insects present in a single image and/or in a plurality of images taken in subsequent times.
According to an embodiment, the insect identification module 103 comprises software instructions implementing a computer vision algorithm 300 (VIS) to extract feature values from a digital image IM and an insect classification algorithm 301 (CLSS). The computer vision algorithm 300 can be a known computer vision tool configured to elaborate the digital image IM to extract values of entomologic parameters.
As an example, the considered entomologic parameters include at least one parameter selected from the group: colour of the insect eyes, length of the insect and of the wings, colour of the tip of the wings, length and colour of the sting, spot on the abdomens. Particularly, the identification of the insect could be performed by evaluating the similarity between the entomologic reference parameters and the entomologic measured parameters, according to pre-established weights.
The insect classification algorithm 301 can be based on a comparison of pre-established entomologic parameters with the entomologic parameters as measured from the computer vision algorithm 300. The entomologic reference parameters are values (or value ranges) pre-established on the basis of a knowledge of the insect species in connection with the area of interest.
Furthermore, the insect classification algorithm 301 can also be based on the meteorological quantities (provided by the sensing apparatus 102) associated to the insect catch time, such as an example: interval of temperature during the catch and/or interval of humidity during the catch. Moreover, the insect classification algorithm 301 can also be based on the environmental quantities provided by the sensing apparatus 102.
A known classification algorithm can be used to implement insect classification algorithm 301. Particularly, the insect classification algorithm 301 can be a non-neural network based algorithm (e.g. a target optimization function) or a neural network based algorithm. As an example, the insect identification algorithm can include a Convolutional Neural Network (CNN).
It is noticed that a data set comprising a correlation between entomologic parameters, meteorological quantities, environmental quantities and corresponding insect identified species can be part of an insect behavioural knowledge data set. This insect behavioural knowledge data set can be stored in the local knowledge module 105, to be accessed by the knowledge configuring and updating module 106. Particularly, the knowledge configuring and updating module 106 is a software module responsible for the creation and propagation of the most updated knowledge on target insects to the local knowledge module 105 and the further local knowledge modules 105, when employed (as represented in
It is observed that an improvement (such as more effective set) of an insect behavioural knowledge data set (adopted by a specific insect identification modules 103) that could be useful for further insect identification modules 103 can be detected automatically by the knowledge configuring and updating module 106 taking into account user feedbacks and/or data concerning occurred attacks.
To this purpose the knowledge configuring and updating module 106 can be provided with a difference detection module 114 (DIFF-I), such as a software module configured to detect differences between insect behavioural knowledge data sets.
With reference to the insect classification algorithm 301, it is noticed that it can be fully defined by a classification algorithm definition data set comprising: a model typology (e.g. CNN model), configuration values (e.g. the values of the weights of the CNN) and variable types (i.e. the variable processed by the algorithm itself).
It is observed that the knowledge configuring and updating module 106 is a software module that allows configuring the specific employed insect classification algorithm 301 by defining and storing the classification algorithm definition data sets associated to one or more areas of interest. Moreover, each local knowledge module 105 stores the associated algorithm definition data set. As an example, the local knowledge module 105 is also responsible of the safe transition of parameters, information and data between the insect identification module 103 and the knowledge configuring and updating module 106 (as represented by a first communication link ULI).
According to a particular example, it possible to use a first classification algorithm 301 of the non-neural network type for a first operation period. This first classification algorithm 301 allows creating and updating an insect behavioural knowledge data set. Subsequently, taking into account the insect behavioural knowledge data set obtained in the first operation period, a second classification algorithm based, as an example, on neural network can be trained and employed to identify insects in a second operation period. In accordance with this embodiment, the second classification algorithm replaces the first classification algorithm.
Reference is now made to the risk prediction module 104. The risk prediction module 104 is configured to operate according to a mathematical prediction algorithm 302 (PRED) in order to estimate the risk of insect attack to the area of interest. The risk prediction module 104 processes the results provided by the insect identification module 103 (such as the presence data IPD or the counted insect number) according to the mathematical prediction algorithm 302.
Moreover, the risk prediction module 104 is configured to estimate the probability of attack by also processing at least one of the following quantities/data: at least one meteorological quantity, at least one environmental quantity and historical data for insect presence. Preferably, the risk prediction module 104 is configured to estimate the probability of attack by processing at least two of the following quantities/data: at least one meteorological quantity, at least one environmental quantity and historical data for insect presence. More preferably, all the above listed three quantity/data types (meteorological, environmental and historical) can be processed by the risk prediction module 104.
The meteorological quantities and the environmental quantities have been already defined. Particularly, a trend of the counted number of insects (e.g. an increasing gradient) and a trend of the meteorological quantity (e.g. particular conditions of temperature and humidity on a time interval) are useful in the prediction of insect attack.
The historical data for insect presence include, as an example, data on insect attacks to the specific area of interest occurred before the current period of time submitted to the risk estimation. The historical data can be short-term series (e.g. concerning the previous five years) or long-term series (e.g. concerning a time windows of more than five years).
Moreover, the historical data for insect presence can be provided together with related meteorological and environmental quantities. In addition, historical data for insect presence, relating to previously occurred insect attacks to other areas, different from the area of interest, can be taken into consideration.
Moreover, the risk prediction module 104 can also operate, preferably, basing on geographical data describing the area of interest. As an example, the geographical data refer to the presence of a natural barrier (such as, a hill of a mountain or a river) that could influence local conditions both positively or negatively.
It is noticed that the occurrence of an insect attack depends on many factors, partially described in entomology, such as, for example: the increase in the presence of insects (e.g. a fly) over time, a certain meteorological condition extended over time (temperature, humidity, etc.) geographical factors, etc.
The mathematical prediction algorithm 302 can be based on algorithms requiring Machine Learning; such algorithms can be neural network based (e.g. CNN) or on non-neural network based. As an example, non-neural network algorithms can include Logistic Regression or Random Forest algorithm.
The mathematical prediction algorithm 302 can be fully defined by a prediction algorithm definition data set comprising the following features: a model typology (e.g. CNN model), configuration values (e.g. the values of the weights of the CNN or other parameters) and variable types (i.e. the variable processed by the algorithm itself).
Moreover, according to a preferred embodiment, the employed mathematical prediction algorithm 302 is adaptive that is to say that the mathematical prediction algorithm 302 can be modified automatically, by modifying one or more of the features of the definition data set.
As an example of the adaptive functionality, the mathematical prediction algorithm 302 can be configured to process a first plurality of variables and, subsequently, can be automatically re-configured (i.e. re-trained) in order to process a second plurality of variables, comprising additional variables.
According to an embodiment, the machine learning of the employed mathematical prediction algorithm 302 can start from a predefined calculation situation, where the weights of estimating function variables are pre-established (e.g. provided by the knowledge configuring and updating module 106) to subsequently evolve, taking into consideration further data or results also obtained from other monitored areas of interest.
Particularly, an employable mathematical prediction algorithm 302 of the non-neural network type is the Logistic Regression in which the probability of an event P is calculated as:
In equation (2):
Xi are the values assumed by the variables (or descriptors) identified (e.g. gradient of increase of the flies, gradient of increase of temperature in the last periods, gradient of humidity, etc.);
Wi are the weights of the model developed by the machine learning related to the contribution of the individual variables for the computation of the attack probability.
β is y-intercept.
In accordance with the above description, in an initial stage the attack probability P can be estimated by expression (1) as trained with data known in the literature, and subsequently the prediction model can be reconfigured considering further data and/or additional variables (e.g. user's feedbacks, additional variables Xi) to calculate the probability of local attack with improved precision.
With reference to the particularly embodiment in which the mathematical prediction algorithm 302 is adaptive, it is further noticed that the risk prediction module 104 can be provided with a difference matching module 113 (DIFF), such as a software tool. The difference matching module 113 executes a difference matching algorithm which constantly compares a current prediction algorithm definition data set with previous defined prediction algorithm definition data set or data set referred to other areas of interest to detect possible difference.
As an example, a plurality of current variables (e.g. parameters time series) used in a current attack prediction is compared with a plurality of preceding variables (e.g. previous time series) used in previously performed attack predictions in order to identify possible changes. If changes are identified, the difference matching module 113 starts collecting data provided by the sensing apparatus 102, corresponding to the changed variables, and transfers them to the knowledge configuring and updating module 106 to evaluate the elaboration of new prediction algorithm definition data set. In
Advantageously, the risk prediction module 104 also comprises an alerting module 110 (ALRT-MOD) configured to generate an alerting signal SAL indicating predicted insect attacks to the area of interests. Particularly, the risk prediction module 104 provides a smart communication path between the insect identification module 103, the knowledge configuring and updating module 106 and/or other external communication devices (such as smart phones, or personal computers) associated to users (e.g. farmers) interested in being informed about possible insect attacks. More particularly, the alerting module 110 integrates with most common UC solution APIs (e.g. Cisco Webex, Amazon Chime, Microsoft Skype, etc.) to allow a natural language interaction between the machine and the operators responsible for the entire process. As an example, operators can use sentences like “what is the CO2 level now?” and get contextual and specific answers back.
In addition, the insect attack prediction apparatus 100 can be provided, with a smart maintenance module 110 (SMN) responsible to provide a smart way to keep the hardware fully functional. Particularly, the smart maintenance module 111 (e.g. a software tool) uses as input the information stored in the local knowledge module 105 that are relevant to the specific hardware used (e.g. type of sticky paper, type of pheromone, etc.) and correlates it with sensor information (e.g. VOC air measurements, temperature and humidity history, current coverage of the sticky paper, etc.) to provide information about maintenance needs. As an example, the smart maintenance module 111 could evaluate that the density of the pheromone is not sufficient to guarantee a good level of sexual attraction of the targeted insect and consequently trigger a notification to change it. Furthermore, the smart maintenance module 111 can evaluate the presence of too many insects on the sticky paper to run properly the identification process and, consequently, send a notification to replace the paper.
In an embodiment, the insect attack prediction apparatus 100 can be provided with a smart management module 112 (SMG) which is responsible to adapt the work parameters of the entire prediction system 100. For instance, if the number of insects caught in a specific timeframe is significantly greater than those captured in previous periods, the smart management module 112 (a software tool) may decide to change the sampling rate increasing its frequency. Furthermore, if specific anomalies (e.g. high level of CO2) are identified, the smart management module 112 may decide to change the sampling rate again and go back to original value until when anomalies disappear.
It is noticed that the prediction stem 100 can be also configured to predict attacks made by more than one insect species.
After a symbolic start step 201 the operation method 200 includes an insect identification step 202, which can be carried out by the insect identification module 103. In the insect identification step 202 one or more digital images IM provided by the camera 107 are processed according to the vision algorithm 300 and the insect classification algorithm 301 to provide a presence data IPD, representing the presence of identified insects in the area of interest for insect attack.
Particularly, the identification step 202 runs every T period of time (e.g. initial T being one hour). More particularly, a comparison of parameters extracted from the image IM with pre-established entomological parameters (having pre-established values) associated with specific insect species is performed. Moreover, the insect classification algorithm 301 may also take into consideration meteorological quantities and environmental quantities. This analysis compares information extracted from the sensor apparatus 102 with entomological behavioural parameters for every specific insect.
Particularly, the computer vision algorithm 300 also identifies relevant insect position on the catching sticky paper to create and take into account historical series. Thanks to this historical series generation and analysis process, insects identified in period T shall not be identified as new insects in interval T+1.
It is noticed that, advantageously, the measured quantities provided by the sensing apparatus 102 and the digital images IM are provided to the configuring and updating module 106 in order to allow the updating of the identification algorithms associated to other area of interests.
In a prediction step 203 (PRED-STP), which can be carried out by the risk prediction module 104, the results of the identification step 202, together with a current insect behavioural knowledge data set, is processed by the mathematical prediction algorithm 302 to estimate a risk of attack (e.g. a probability PRB) to the area of interest.
Particularly, the risk prediction module 104 runs with same frequency of the insect identification module 103 and gets information/data coming from the insect identification module 103 every T time. The prediction step 203 can provide, as an example, a gradient of insect presence based on the short-term history.
If the probability PRB of an insect attack is greater than a pre-established threshold the alerting signal SAL (e.g. “Insect X attack happening in area Y”) is generated and transmitted to the alerting module 110 to communicate it to the relevant users, in a communication step 204.
It is noticed that the prediction module 104 operates according to a multi-parameter approach according to which the mathematical prediction algorithm 302 it is based not only on the insect presence data IPD but also on ore more of acquired further quantities/data (such as, meteorological quantities, environmental quantities and/or historical data). Said multi-parameter approach is particularly efficient since it reproduces with good approximation the complex natural phenomena.
Moreover, the prediction method 200 can include an updating and distributing step 205 in which, if a richer and more effective insect behavioural knowledge data set is detected (i.e. by the difference detection module 114), this data set is stored in the relevant local knowledge module 105 and distributed to other relevant local knowledge modules 105. Also an updating of the mathematical prediction algorithm 302 can be performed by updating the prediction algorithm definition data set, when the difference module 113 detects this necessity.
Method 200 ends with a symbolic end step 206 (ED).
Advantages
The described system and method show several advantages over the prior art techniques.
The described prediction system and method based on a multi-parameter algorithm (i.e. not exclusively based on insect identification) allow automatic and precise evaluation of potential risk for the area covered by the analyser. Moreover, an automatic notification to relevant people of the attack probabilities can be performed by the described system. Also the particular insect identification method, not based only on entomological parameters, shows advantages in its efficiency.
Furthermore, the system and method as described above, also allow creating an insect behavioural knowledge data set which represents a structured knowledge base of the issue under analysis (i.e. insect attack pattern) that can be modified and adapted on the basis of real time observations.
In addition, the capability of the attack prediction algorithm to be adaptive makes the method able to automatically adapt to short and long term context changes. Particularly, the capability to learn and adapt to changing nature of the issue under analysis shown by the described method is particularly advantageous since during past years insect attacks have been very unpredictable due to the changes in the context (weather, pollutions, seasonal shifts, etc.).
| Number | Date | Country | Kind |
|---|---|---|---|
| 102019000002249 | Feb 2019 | IT | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/IB2020/050958 | 2/6/2020 | WO | 00 |
