This invention belongs to the field of veterinary, more specifically to bee infections and parasite control devices and methods.
Bees in most parts of the world are threatened by the Varroa destructor and Varroa jacobsin mite. The varroa mite is a harmful parasite, which is capable of destroying entire bee colonies in the course of a few years. Varroa mites weaken bees, deform their bodies and increase the chances of infection from other diseases. The European bee (Apis mellifera) has no natural protection against the varroa mite and is in danger in every stage of its life.
Bee varroatosis control methods include known chemical, mechanical and thermal treatment methods. In these methods, the mite is distinguished from bees due to its smaller size and mass, thanks to which it is more sensitive than bees against short-term exposure to chemicals and heat. Separated mites easily fall through small openings, through which bees cannot fit.
Harry E. Vanderpool (U.S. Pat. No. 6,702,645 Separating parasites from bees) suggests a device where bees and mites are exposed to a substance, which forces the mites to separate from the bees and separated in this way mites are directed, by an air stream, through a sieve. As an example of such a substance is used sugar powder. Unfortunately, the number of mites extracted from bees, using this method, in relation to those left with mites, is not very high.
A more effective way of parasites control could be using of a live video with image processing and means to kill each detected parasite. This idea has been used to kill fish parasites by Esben Beck (WO 2011115496 A1, Method and device for destroying parasites on fish), where parasites detected on live video frames are killed using high intensity pulses of light, which are automatically positioned to the detected parasite location.
As an alternative to selective destroying of parasites, damaged objects, for example fish fillet, can be detected and removed, if image processor finds a parasite (Takashi Okamoto US2007/0238147A1, JP 2007-286041 Method of detecting foreign matter, Ernest M. Reimer U.S. Pat. No. 6,061,086 Apparatus and method for automated visual inspection of objects).
Counting of mite infected bees on a video image is described in, Jasna Kralj. Parasite-host interactions between Varroa destructor Anderson and Trueman and Apis mellifera L. Influence of parasitism on flight behavior and on the loss of infested. Frankfurt am Main: Fachbereich Biologie and Informatik der Johann Wolfgang Goethe—Universität, 2004. Video-cameras are placed under and above the transparent pipes that connect the landing board to the hive entrance. The recordings were later examined, in order to count the number of varroa mite infected bees. This was very time consuming, as people had to examine all the recorded material. Also, due to the straight position, with the wings close to the body, that the bees must maintain, when passing through the pipes into the hive, it is not possible to achieve high detection rate of the mites on the back of the bee even by human operator.
Counting of bees by use of a camera above the landing board is described in the work of J. Campbell, L. Mummert and R. Sukthankar. Video Monitoring of Honey Bee Colonies at the Hive Entrance. ICPR Workshop on Visual Observation and Analysis of Animal and Insect Behavior, December 2008. The goal was to collect statistics of the bees flying in and out of the hive, and have not paid attention to the detection of varroa mites. There are many methods for detecting objects, including insects. The closest to this invention are:
In the described methods for insect detection, based on boundaries and histograms, it is assumed that it is possible to get a picture of the entire insect. Parasites, such as the varroa mite, can, however, be hidden between the host's body parts and, therefore, only a part of the parasite may be on a given image.
Common varroatosis control solutions are, in principle, unselective and utilize chemical and/or physical effects, which affect both the mites as well as the bees. The problem with the solutions is the similarity between the mite and bee organisms, due to which, the dosage interval that works against mites and does not affect the bee is quite narrow.
Common computer vision detection methods try to find parasites within the entire observed area, not taking into account the link between the parasite and host. Also, computer vision detection methods have not used specialized chambers.
To solve the described problems, this invention proposes two alternative devices to diagnose or control varroatosis, as well as the necessary method and software for parasite detection.
The first alternative device comprises one or more cameras, which are connected to one or more image processors with specific software. The one or more processors function is to recognize image of the bee from raw image and by applying image processing to said bee image and/or by displaying bee images taken with different orientation in relation to the bee to operator for assist by his decision, to determine the presence or absence of varroa mite on the body of one or more bees. Based on that, the device will do one or more action of the following list:
a) count the number of infected and non-infected bees,
b) separate infected bees from the non-infected bees,
c) kill the detected mite.
In order to carry out action b) or c), the device is equipped with actuators, which are connected to said processors. The actuators may be, for example, controllable gates, controllable air pumps and valves, positioning devices or a controllable heat source.
The cameras are positioned above the landing board and/or below the transparent landing board.
Above the landing board there are one or more nozzles, into which infected bees can be sucked by air stream. In order to create the airstream, an electromagnet impulse pump can be fit. The said nozzles are placed on moveable carriage, equipped with a positioning drive. On the carriage there may be a heat element, for example, a semiconductor laser, complete with the necessary optics, which kills the detected mites on the bees, leaving the bees untouched undamaged.
The device may contain comprise a further light source that could be controlled by the said image processor and should be capable of creating light impulses. Light impulses length is preferably in range from 0.1 microsecond to 0.2 seconds to expose the bee.
Under natural light, it is possible for the-said device to work without the extra (artificial) light source. The sun, in this case, shall be considered as light source creating reflections.
The alternative solution for diagnosing and controlling varroatosis in bees is disclosed a device which has comprises at least one observation chamber, at least one light source and at least one camera, which is connected to at least one image processor, with varroa mite detecting software. The said processor may also control actuators to govern the bees' movement to and from the observation chamber. The actuators may be, e.g.:
a) controllable gates, which enable the processor to inhibit or allow entrance of bees into the observation chamber or exit from there,
b) light sources to attract bees,
c) valves or pumps, to control the flow of air or narcotic gases into and out of the observation chamber.
Preferably the said light sources are controlled by the image processor. In addition to the normal on-off switching function, preferably at least one light source has impulse mode to generate light impulses which are synchronized with the frame frequency of the camera. Light impulses length is preferably in range from 1 microsecond to 0.2 seconds.
The said observation chambers are preferably large enough to allow the bee to turn around and/or spread its wings further away from its body.
Several cameras and/or mirrors may be positioned in the device in such a way as to be able to acquire images of the bee or bees under observation from at least 3 different observation points, preferably at view angles that differ from each other approximately by quotient of 360 degrees divided by the number of view directions.
The software of image processor(s) is able to detect a varroa mite on raw images, acquired from observation chambers, and, based on that is capable to:
a) count the number of infected and non-infected bees,
b) separate infected bees from non-infected other bees,
c) kill the detected mite.
In order to carry out b) and/or c), the device is supplied with actuators that are connected to the afore-mentioned one or more image processors. The actuators may be controllable gates, air or narcotic gas pumps and valves, positioning device or controllable heat source. The image processor can control the gates to inhibit or allow bees to enter or leave the observation chamber.
The image processor for both alternative devices comprises the image processing software that comprises a module with the function to detect mites in one or more pictures acquired of the same bee using the following algorithm:
The analysis of pixels, in order to determine the relation between the color of the given pixel and the characteristic color of a varroa mite, the analysis of pixels may be carried out in suitable ranges of the HSB color space or three-dimensional tables in the RGB color space, where for each color there is a rating indicating its similarity to the color of mites. The pixels of some colors may be found on both bees as well as mites. For such colors, it is suggested to assign lower ratings. For colors that are rare in bees, but are in mites, should have a higher rating.
The software also includes comprises a module, whose function is to determine the location of a bee's body zones on multiple images and a module, whose functions is to compose a combined rating according to detection ratings from same body areas from different raw images or count presence of different body parts in the images and to determine, based on combined rating or counting results the presence or absence of a mite.
In addition, the current invention provides the image processing software has a module, whose function is to determine the location of a bee's body zones in an image, and a module, whose function is to adjust the criterias for detecting a varroa mite, depending on the mite detecting body zone of the bee or exclude of the presence of a varroa mite in areas where the false positive detection according the algorithm is highly possible.
The bee image processing software has comprises a further module with the function to adjust various image processing parameter values, using images of bee or mite, where mite detection has been successfully performed by the image processing or with the help of an operator. The parameters can be limits of color range required when searching pixels having similar color values to reflection of light source from glossy body of the mite, rating limits for pixels in the surrounding region or the minimal pixel rating required for region rating in the surrounding region for mite presence determination.
In the second alternative of the device there are one or more observation chambers in the field-of-view of the cameras, supplied with controllable gates, through which bees can enter and exit the chambers. Preferably the said observation chambers are large enough to allow bees to move or turn around and/or allow their wings to be spread away from their abdomen. In addition, the device may comprise light sources, which are controlled by the processor, to provoke bees into moving in a certain way.
Both alternative devices have a step or other means in the field-of-view of the cameras, on which a bee, when climbing, moves its body in such a way as to allow for a better view of a mite in the gap between the thorax and abdomen. Both alternative devices have also actuator, controlled by image processor, in form of heating element that can be made as a source of collimated radiation in a way, that its radiation can be directed by the image processor to the position of the mite.
In addition to separating infected bees from the non-infected, this invention provides the possibility to detect and kill a mite that has separated from bees. For that, the device has an impulse light source, such as a gas discharge lamp, laser or light emitting diode. In order to transport bees to and from the observation chambers or observation areas, the invention opens the use valve-controlled air streams.
After the image processing have determined the location of the parts of a bee body on an image, this invention provides to count the number of times a specific body part has been presented and to continue processing of raw images of the bee, until the mite detection has been processed at enough different orientations to ensure a negative result, or to have reassurance based on different images from the same body zone in the case of a positive result. If multiple detections exist in the same body zone with a probability rating of over 50%, some statistical process is carried out, to calculate the combined rating, for example multiplying the contra-probabilities of detection obtained by different detections.
In addition to disclosed above devices, this invention provides a method of detecting parasites that may be used as for detecting varroa mites, so in various other fields, which require such image processing capabilities, the devices of which may differ from the ones described herein.
The method for parasite detection using image processing techniques on an image of the host goes through comprises the following steps:
A) The image of the host is searched for pixels whose color corresponds to the characteristic for the reflected light of the light source. This may be one or several adjacent pixels, which have a similar color values to the reflection of light from the glossy body of the parasite.
B) The said pixels are assigned o One or more surrounding regions of various sizes are assigned to the said pixels.
C) Color values of pixels in each assigned surrounding regions are analyzed to the similarity to the color value, common for a mite, and each surrounding region is assigned a region rating, indicating the probability that there is a parasite in the determined area of the body of host.
D) Based on surrounding region ratings it is determined, whether the host is or is not infected by a parasite.
With the implementation of this method several images of the host are acquired. Combined rating based on detection rates of several images yields more correct result than rating from a single image. As the host moves itself between the expositions, it is possible to watch the parasite and host on the images from different directions. When combining detection rating it is rational to determine location of different zones of host on different images and to use for combined rating region ratings for the same body zones from the different images.
The next subject of the current invention is a method for determination of location of body elements in an image of an insect comprising the following steps:
A) separating the image of the insect image background and thresholding the said image to binary image;
B) eroding the insect blob by number of pixels enough to remove legs;
C) dilating result of eroding by about the same number of pixels or preferably by slightly higher amount of pixels;
D) subtracting result of dilation from initial binary image; whereas the result of dilation obtained in point C) is considered as body core and result of subtraction in point D) is considered as legs. From determined legs antennas are selected as legs whose quotient of area to border length is less than certain criteria (e.g. 7). Head of the insect is considered to be in this side of body where antennas are determined or which is closer to center of gravity of all legs.
For application of the devices and methods corresponding to the current invention, the afore described software (computer programme product) stored in the memory of a processor of device, including software code portions, is provided. The software is adapted to perform the aforesaid methods, when the computer program is running in the image processor.
The device (
The device is placed on a bee reservoir in such a way that bees can enter the device through the entrance slot 324. A reservoir for non-infected bees is placed on top of the device at exit slot 326 and a reservoir for infected bees is placed at exit slot 328.
The said reservoirs are equipped with retaining valves and attracting lights to prevent bees from congregating around the electromagnet gates 315 and 316.
All eight sections work independently of each other. The following describes the working of the device using the example from the open section in
The attracting lights 321 and 322 are periodically switched on and off. When the lights are switched off, the bees move following the scent of the bees that previously passed there. In order to acquire images of good quality, the area under observation of camera 310 is illuminated by the gas discharge lamp 319 or LEDs 320, which are controlled by the processor 311 to generate short, high-intensity light impulses. The red light, which is invisible to bees, is used when a bee is being let into or out of the observation chamber, where one of the electromagnet gates 314, 315 or 316 is open. When determining the position of a bee's body parts and presence of a varroa mite, color images are used, exposed by the gas discharge lamps. Each image, by exposition of the each light source, is processed by the image processor 311.
While there is no bee in view of the camera 310, the processor holds the gate 314 open. Gates 315 and 316 are closed. When a bee enters from the slot 324 and reaches into the observation chamber, through open gate 314 and to field-of-view of the camera 310, it is determined by the presence of a motion detection in the image frames from the camera. Once a bee passes through gate 314, the processor 311 closes it. While all gates, 314, 315 and 316 are closed, the bee moves within the confines of the glass window 313 and barrier 317, seeking for escape. It is then, that the procedure for mite detection, as described in the next paragraph, is run. Once the procedure is finished, one of the exit gates is opened, depending on the result: 315 if the bee is infected or 316 if the bee is not infected. LED lights are used to acquire monochromatic image of the bee until it is detected that the bee has left the chamber via an opened gate. If a separated mite has been detected during varroa mite detection procedure, the gate 315 is opened, and after the bee has left, the gas discharge lamps 329 and 330 are activated, to kill the mite with the heat. When the observation chamber is empty, gate 314 is opened and the procedure begins again with next bee.
The detection of varroa mite by image processing is carried out according to the diagrams shown in
The relationships between the procedure blocks and data blocks are shown be the six figure numbers near each block, where the first three digits indicate the source block and the last three digits indicate the destination block. The run of the procedure between the figures is indicated by off page reference arrows R1 to R9. The procedure begins with the step “ACQUIRE IMAGE” (block 704,
A larger objects must be a bee and next the orientation of the bee's body and location of its parts (
If it is decided, in block 858 (
The focal axis of the ellipse divides the body blob in two halves. Now the focal axis is divided into nine equal sections (
If, in block 864, it is determined that the asymmetry of the image is smaller than a limit, it is necessary to determine whether the image represent back or underside of the bee.
For that, in block 868, the region of the abdomen is used and, using a threshold value (for example 110, if the luminosity range is 256), the image is thresholded into a binary picture and, in block 870, black blobs are eroded by five pixels. The black blobs after the said erosion are the cross-hatched areas in
Based on the accomplished determination of the orientation of the body, each pixel of the image is assigned a parameter, in block 876, indicating to which zone of the body it belongs. The zones are assigned two-digit numbers, indicating the position of the zone, where the first digit indicates the segment as shown in
The results, obtained in block 710 (
Now the detection of mites through the reflection method (block 714,
In
Further begins the mite detection by color signature method, according to block 718 (
The adjusted rating and the body zone number of color signature are saved in table 720, after which the program returns to block 816 (
The process proceeds to block 722 (
Due to the fact that various strains of bees have various body colors and that the colors of varroa mites may differ, this detection program works adaptively, collecting statistics, in blocks 730 and 734 (
If a small object is detected in block 706 (
The device (
The image processor in the camera assembly 123 observes the landing board. Enough images are acquired and saved of each bee arriving to the hive, to determine by image processing the presence or absence of varroa mite on the bee's body. The procedure for detecting the presence of a varroa mite is similar to one described with the first embodiment of the device. The images acquired by cameras above the landing board may contain multiple bees, in which case the procedure of detecting the location of the body zone and search for reflections and specific color signatures is run on each object that can be differentiated from the background and is of a bee's size. In order to adapt to changing illumination conditions, known in the art computer vision techniques are used. Because the movement of bees cannot be restricted, the device uses the result obtained from the images by the time a given bee reaches arena of the suction nozzles 117. If the bee is infected by varroa mite, the image processor calculates the direction and amount of movement required to move the sliding plate to the position, so that one of the nozzles 118 is positioned close above the bee, and issues the required command to the stepper motor 113 driver. When the chosen nozzle reaches the bee, the image processor activates the respective solenoid 105 driver. The said driver generates a current impulse, by which the selected solenoid's anchor and piston 107 move quickly up. This creates a vacuum in the respective cylinder 108 and the air current flowing into the respective nozzle forces the bee through the nozzle. When the current impulse ends, the spring 106 returns the piston 107 back to its initial, lower position. Air exits from the cylinder 108, closing the valve 118 and opening the valve 120, and the bee is forced through opening 121 into the respective collection gage. After that, the image processor resumes analyzing of remaining bees, saving and processing the subsequent images from the cameras. At the same time, the image processor transmits to the computer network data regarding the bees in the collection gages, so that the bee-keeper is informed of their occupation and can empty them in time.
The device's third embodiment comprises all the components from second embodiment, however the landing board is made from a transparent material and below it is located an extra, underside camera assembly, similar to previously described camera assembly, except that it is directed upwards from underneath of the landing board, giving a view of the bees' undersides. The camera assembly, placed as described in the second embodiment, shall be referred to, here, as the upside camera assembly. The device of the third embodiment works similarly to the device of the second embodiment, except that the image processor of the upside camera assembly has an additional connection to the image processor of the underside camera assembly. The image processor of the underside camera assembly observes and analyses images of bees, moving on the landing board, similarly to the image processor of the second embodiment and forwards the coordinates of infected bees to the upside camera assembly image processor, after which the upside camera assembly image processor takes over the observation of the identified bees. The separation of infected bees identified by both image processors is controlled by the upside camera assembly image processor, using the stepper motor and solenoids as previously described with the second embodiment.
The device (
The image processor in the camera assembly 223 observes the landing board similarly to the device in the second embodiment. Enough images are acquired and saved of each bee arriving to the hive, to determine by image processing the presence or absence of varroa mites on the bee's body. Bees infected by a varroa mite are observed until they reach the arena of laser modules 205. Then the image processor calculates the direction and amount of movement required to move the sliding plate 210 together with laser modules to the position that one of the laser diodes 205 is positioned close above the bee and issues the required command to the stepper motor 213 drive. When the chosen laser module reaches the bee, the image processor generates a command via the respective driver to the laser module to emit a low-power beam. The focused laser beam creates a dot, which is visible on the field-of-view of the camera and the image processor calculates the distance between the center of the dot and the position of the varroa mite. Correcting the position of the stepper motor and waiting for the progressive movement of the bee, until the calculated distance is minimal, the image processor activates the laser module on full power generating a impulse of radiation, that kills the varroa mite by heat.
The image processor then resumes saving and analyzing images from the cameras. At the same time, the image processor transmits data to the computer network regarding the analyzed bees and the killed varroa mites.
The device according to the fifth embodiment comprises of a main body 401 (
The device is placed on a container of bees, in such a way that the bees can only exit the container through the transparent channels 402, 403 and 404. Under control of the image processor, the infra-red emitters begin generating short light impulses, which are synchronized to the cameras. In order to prevent the infrared radiation from shining directly into the cameras, during the exposure, only two camera assemblies work at the same time: the front two (406 and 407) and then the back two (405 and 408) camera assemblies. Each image, exposed by the infra-red transmitters, is analyzed by the image processor, which detects the presence and location of the bees in the channels 402, 403 and 404. If a bee is in any channel completely within field of view of the cameras, the acquisition of color images begins by two cameras at a time, alternating the front and then the back cameras, with the gas discharge lamps 409 and 410 respectively exposing the images. The image processor analyses the images with the algorithm described in the first embodiment, with the reflection (
The result is that only the uninfected bees can get to the higher reservoir and the infected bees stay in the bottom reservoir. After the operation of the device is finished, the bees in the bottom reservoirs can by treated by known methods.
The device according to the sixth embodiment partly overlaps with the device of the fifth embodiment. The device comprises of a main body 501 (
In the channels 502, 503 and 504, there is a transversal rod 534, which forms a step, where a bee is forced to climb, allowing for a better view of the area between the thorax and abdomen.
The device is placed on a container of bees, in such a way that the bees can only exit the container through the transparent channels 502, 503 and 504. Under control of the image processor the lower electromagnet gates open and the infra-red emitters begin generating short infra-red light impulses, which are synchronized s to the cameras. The upper electromagnet gates 513-518 remain closed. In order to prevent infra-red radiation from shining directly into the cameras, the cameras work in alternating pairs, first the two in the front (506 and 507) and then the two at the back (505 and 508). Each picture exposed by the infra-red light is analyzed by the image processor, to determine if there is a bee in either of the channels 502, 503 or 504 and to determine its position. The position of the bee is determined by the center of gravity of the bee in the image. When a bee has entered a channel, the respective lower electromagnet gate is closed and the process waits until all the channels have a bee, or until the timeout of one minute is exceeded. After that the cameras begin taking color images alternating between the two in the front and the two in the back, exposed by the respective gas discharge lamps 510 and 509. The image processor analyses the obtained images as described in the first embodiment, using the reflection (
First, the image processor determines which laser module is in the correct orientation to kill the mite. If the mite is only seen by one camera, the laser with the same direction of optical axis as the camera is used. For example, a mite found by the camera module 506 will be killed by the laser module 533. As well, the discovered mite will be killed with the help of the camera module it was discovered by. If the mite is in view of multiple cameras, the camera is chosen, where the horizontal distance between the mite to the bee's center of gravity is minimal. In the following images, the image processor observes the movement of the bee as well as the movement of the identified object sharing the signs of a mite in relation to the position of the bee, in all the camera images. If the distance between the mite and the bee's center of gravity increases, another camera module may be chosen.
The arena of the laser beams include the transverse bar 534, in order to allow for the destroying of mites that are between the thorax and abdomen of a bee. When the bee approaches the arena of the laser beams, the image processor chose which laser to use and positions it, using the stepper motor 519, to the horizontal position of the mite sign. The chosen camera module acquires images, at the rate of 120 pictures per second, while synchronously to the camera the chosen laser module generates low-power impulses. The image processor detects, on each picture, the horizontal position of the dot from the laser impulses and the detected signs of the mite and determines the necessary correction and corrects by use of the stepper motor the position of the laser. By extrapolation the movement of the mite sign caused by advancing of the bee, the image processor determines the moment, when the laser beam should be at the mite position. At that moment the laser impulse of full power is generated to kill the mite by arising heat. If, through the movement of the bee, the position of the mite is moved out of view of the chosen camera, another camera should be chosen and accordingly to reposition the laser modules. If the bee passes over the arena of the laser beams, it cannot, due to the closed gates, exit the channel and, at some point, the bee will return and the procedure continues, as described above.
If a bee is infected by multiple mites, or if multiple infected bees are in the same channel, the mites are destroyed one at a time, as described above. When all the mites infecting a bee have been killed, the gates at the top of the channel are opened and the bee let free to exit into the upper reservoir.
The first embodiment of the method is illustrated in
The counting results of the third step form the rating. If this is smaller than 15, the result is negative. The following table indicates the probability of there being a mite in the given reflection, based on the results of the third step.
The second embodiment of method uses the RGB color system. It uses a three-dimensional table, where the color regions considered to be close to those of reflected light of the light source are marked. In the image to be analyzed, pixels whose RGB values belong to the marked region are selected and circumscribed by squares of side length 50 pixels. Each pixel in the squares is rated by its RGB value, based on the rating from the said three dimensional ratings table. Ratings of each pixel in the square are added together and the sum is used to determine the probability of there being a mite in the area. The following scale is used:
The above described device and method embodiments use software embedded in the image processor memory, which contain parts of software code with the described above modules.
The described device, in all six of its embodiment, can be produced industrially, to determine the number of varroa mites in a bee colony and decide on the need of control, or perform an effective control. The device, in its first, fifth or sixth embodiment should be used when preparing for wintering, or for treating colonies during winter, in which case the bees would have to be taken into a warmer room, temporarily interrupting the wintering. One device should be capable of selecting infected bees from the colony within one day (24 hours) and enough devices should be produced to allow companies offering varroatosis control services to use it. The device in its second, third and fourth embodiment should be used with colonies, completely treated from infection, living in regions where the risk of infection is high, to keep them safe. Various modification of this device could also be used in scientific research in the field of beekeeping.
Number | Date | Country | Kind |
---|---|---|---|
P201200005 | Mar 2013 | EE | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/EE2013/000004 | 3/27/2013 | WO | 00 |