Patent Application
20190166823
This invention relates to pest control, and more specifically to animal traps used in pest control.
In their natural environment, pest animals are often comingled with benign and beneficial animals that are of similar size and have similar behaviors. Conventional animal traps that target a pest animal risk accidental detention, killing or maiming non-pest animals. Examples are found in residential settings, industrial farming, and invasive species eradication.
Non-limiting examples of pest animals are rats, mice, raccoons, skunks, nutria, opossums and coyotes. Several of these pests are similar in size as common pets or livestock such as cats, pigs, chickens and dogs. Other animals that might coexist with these animals may be wild, but not considered pests such as squirrels. Given these possible similarities in body size and shape, physical barriers that selectively exclude benign or beneficial animals from the pest trap are difficult or impossible to design. In invasive species eradication, a natural ecosystem has been disrupted or is threatened by a pest species. In an example, Africanized honey bees and European honey bees are similar in body size, shape and behavior making it difficult to construct traps that differentiate between them.
It is desirable yet difficult to identify pests from non-pests for the purpose of animal traps and many investigators have proposed solutions. Examples include Meehan (U.S. Pat. No. 4,884,064) who describes a plurality of sensors for detecting presence of a pest. The sensors are to be unresponsive to animals that are of a different size than the pest. Guice et. al. (U.S. Pat. No. 6,653,971) describes a system for discriminating between harmful and non-harmful airborne biota. The proposed methods require directed energy beams and energy absorbent backstops to generate a return signal that may be used in a pest, non-pest classifier. Anderson et. al. (U.S. Pat. No. 6,796,081) utilizes a maze structure to protect against access by children, pets or non-target species. Kates (U.S. Pat. No. 7,286,056) describes an energy beam and receiver to detect presence of a pest when the beam is interrupted. Arlichson (U.S. Pat. No. 9,003,691) proposes a trap door cage utilizing a sensor to detect an animal in the trap and trigger the closing of a trap door. Kittelson (U.S. Pat. No. 9,439,412) proposes a non-lethal animal trap utilizing a microprocessor and motion sensor to detect objects within the trap and activate an electrically controlled latch to close the trap doors.
While these methods have been proposed, a practical, low cost, reliable and accurate method for excluding non-pest animals from a pest animal trap has remained elusive. It is clear that there is a need for an improvement.
This invention provides a method and a system for an animal trap that classifies animals approaching and entering the trap by general visual appearance. With this method and system, the trap is able to target pest animals with minimal harm to benign or beneficial animals despite similarities in size, shape, color and behaviors.
Digital images of animals in proximity to the trap are acquired by a camera and analyzed by a computer algorithm by convolving the digital image with a kernel to create a convolved feature map. The feature maps are further processed with a computer algorithm machine learning classifier to provide a confidence score for each animal. The animal confidence scores are then compared with threshold values and if the value is exceeded for an animal, then an algorithm categorizes the animal and selects trap action. If the animal is categorized as a pest animal, the trap control system detains or kills the animal. If the animal is categorized as a non-pest animal, the trap control system either deters the animal from entering the trap, or takes no action.
The system consists of the camera, the computer with software programs containing image convolution, classifier, and trap control algorithms, and the trap data acquisition and control hardware with a data acquisition and control bus, relays, latches, transformers, emitters and sensors to enable multiple trap response actions.
Several embodiments are included in the following description in conjunction with the drawings. All patents, patent applications, articles and publications referenced within the description are incorporated by reference in their entirety. To the extent that any inconsistencies or conflicts in definitions or use of terms exist between the references and the present application, the present application shall prevail.
To promote the understanding of the invention, specific embodiments will be described. While the invention is shown in only a few of its forms, it should be apparent to those skilled in the art that it is not so limited but is susceptible to various changes without departing from the scope of the invention.
References herein to computers generally means the computer functional assembly that may include but is not limited to combinations of processing circuitry, central processing unit (CPU), graphics processing unit (GPU), tensor processing unit (TPU), application specific integrated circuit (ASIC), field programmable gate array (FPGA), hardware accelerators, data buses, network connection interfaces, computer readable media (CRM), volatile and non-volatile memory, auxiliary storage, input/output components such as screen, keyboard, mouse and other connected peripherals, software operating systems, general purpose software, and specialized function software.
In addition, references to communication networks and interfaces generally means public or private networks, the Internet, intranets, wired or wireless networks, local area networks (LAN), wide area networks (WAN), satellite, cable, mobile communications networks and hotspots. The communications networks may utilize some form of communication protocols such as internet protocol (IP), transmission control protocol (TCP), or user datagram protocol (UDP). The communications network includes devices such as routers, gateway, wireless access points (WAP), base stations, repeaters, switches and firewalls.
In the first embodiment, referring to
Referring to
The digital image acquired from the camera is stored in CRM as a numerical array of pixel color intensity values. For example, an image acquired by the camera may have a width of 640 pixels and a height of 480 pixels and be recorded in a Red, Green, Blue (RGB) or YCbCr color space to have array dimensions of 640×480×3. A method for producing accurate image classification with computer vision has been the use of a convolution algorithm whereby the original pixels values are convolved with a numerical array whose width and height are smaller than the image width and height, known in image processing and computer vision literature as a filter, kernel, convolution matrix, mask, cell or window. The resulting output from the convolution operation are commonly known as the filtered image, convolved image, convolved image descriptor, convolved feature map or activation map.
Referring to
In a Haar feature object detector (Viola et. al. 2001), the image is typically first converted to grayscale. In this method, several different kernels are convolved with the input image. The kernels contain 2 regions described by 2 or more rectangles. Referring to
In a histogram of oriented gradients (HOG) object detector (Dalai et. al. 2005) and McConnel (U.S. Pat. No. 4,567,610), the kernel defines a window of pixel values called a cell that is convolved with a horizontal and vertical gradient filter to produce a set of gradient directions and magnitudes for each pixel. These gradient values for pixels within a cell are used to create a histogram,
In a convolutional neural network (CNN) classifier, the kernel is typically 3×3×3 or 5×5×3 elements in shape for a color depth 3 image. Referring to
The CNN kernel values are learned during training of the CNN classifier. A loss function is generated based on comparison of the predicted classifications with ground truth classifications of the training images. Gradient descent by back propagation of error defined by the loss function is used to modify the kernel values in an optimization process (Nielsen 2015, Rumelhart et. al. 1986). Current state of the art CNN based classifiers contain an architecture of multiple convolution operation layers typically combined with a softmax output layer to produce classification scores expressed as a probability. The architectures also include node weight regularization, normalization, pooling, dropout layers, different optimizers and other mathematical manipulations to improve computational efficiency and classification performance.
An extension of the CNN classifier is the region-based convolutional neural network (RCNN) whereby regions of interest (ROI) are generated as cropped subset portions of the original image by random assignment or by a region proposal network (RPN). Each subset portion is analyzed by the classifier algorithm, allowing for multiple instances of animals and their positions to be detected within the original image.
Several non-limiting examples of convolved image feature based classifiers have been described. Each classifier uses different methods, but all produce classification confidence scores that may be used to identify and differentiate between animals. Each classifier has different advantages and disadvantages when considering speed of training, prediction accuracy after training on limited image sets, ability to detect multiple object instances, image variation from camera sensor wavelength sensitivities and lighting conditions, prediction analysis loop times, computer memory consumption, and parallel processing efficiency. Another variable is that low cost computers are continually improving in processing power. As a result, embodiments may use different algorithms for different applications and evolutions of trap design.
After a convolution feature based classifier has been trained on a set of animal training images, it can be used for prediction to identify animals located within a newly acquired digital image. The classifier will produce a set of confidence scores for all animals that were included in the training image set. In a successfully trained classifier, the confidence score for an animal contained in the new input image will be high and animals not contained in the input image will have a low relative confidence score. Thus, for application in pest, non-pest detection, it is necessary to acquire a set of training images of the pest and non-pest animals and running the training procedure prior to deployment. However, differences in lighting, within species variation of the pest and non-pest animals, presence of new animals unknown to the classifier, or other factors may cause the animal trap pre-trained classifier to not meet sufficient classification confidence score differentiation between pest and non-pest animals to prevent misclassification errors and erroneous trap action. When the trap classifier attempts to classify an animal, but the classification probability does not meet a predefined threshold, the computer software or human operator may optionally instruct the trap to take no action except to acquire images of the animal for use in further training.
In a second embodiment, the wire cage trap assembly has a trap door with spring loaded pivot brace and pivot brace latch as in the first embodiment. A first computer is a 4-core CPU workstation with a 1664 processing core GPU and software to enable general purpose GPU computing that is located remotely from the wire cage trap assembly. The first computer is connected to the LAN with an ethernet cable. The second computer located in a weatherproof box attached to the wire cage trap assembly is a low-cost credit card sized micro-computer with a 4-core CPU and a general-purpose input output (GPIO) bus connected to 2 relays and an analog input. The second computer and GPIO interface controls the first electronic relay to switch the power source to the high voltage transformer, a second electronic relay to switch power to an electromechanical latch holding the trap door open, and to an analog output infrared proximity sensor to measure animal position within the trap. The second computer has a wireless connection to the LAN by way of a WAP.
In the second embodiment, the classifier training is accomplished by placing a digital camera in proximity to the baited trap to acquire training images. The trap doors are locked open, allowing animals to enter and exit unhindered. The camera contains a conventional complementary metal-oxide-semiconductor (CMOS) sensor with visible light and infrared wavelength detection. The camera also has infrared illumination that is autonomously turned on by the camera in low visible light conditions. The camera utilizes IP communication by an ethernet cable connected to the second computer. The second computer acquires a stream of digital images from the camera and applies a gaussian mixture model (GMM) adaptive motion detector. When motion is detected, digital images from the camera are saved as files on the second computer that are later transferred to the first computer over the network. This process is sustained, re-baiting the trap as needed, until a training image set of the animals is acquired. Preferably, this training image set includes at least 100 different images capturing different perspectives of each animal. Optionally, the training image set may be further expanded by converting some color images to greyscale to simulate infrared illumination images. In addition, an image augmentation procedure may be used to create random shifts, shears, rotations and flips to the images to create more image variations for each animal. A human operator defines the ground truth ROIs and animal classifications for each image used in the training set using an object tagging software tool on the first computer. In addition to the training images containing animals (positives), a set of non-animal images (negatives) are included in the training set for background reference. Ten percent of the positive training images are randomly selected and removed from the training set to serve as a test image set.
The second embodiment classifier model structure contains region proposal, convolution and classification algorithms based on Krizhevsky (2012) and Girshick (2015) known as a Fast-RCNN classifier. Model training is performed on the first computer GPU. Using transfer learning (Garcia-Gasulla et. al. 2017, Yosinski et. al. 2014), a pre-trained CNN is used in the model to reduce the computation time required to train on the set of animal images. The Fast-RCNN model is trained on the image set for approximately 8 hours or until the confidence score accuracy on the test image set is greater than predetermined threshold values. A non-limiting example Fast-RCNN training threshold expressed as a probability is 75% for each animal class. When training is completed, the trained Fast-RCNN classifier model is transferred to the second computer CRM.
The second embodiment metal cage assembly trap door locks are removed and the trap is baited to prepare for normal operation. Referring to
If the action decision algorithm 705 selects deterrent action, the second computer switches a GPIO digital output and connected power relay to turn on the power to the high voltage transformer 721. The metal cage is energized by the high voltage transformer to approximately 2000 volts and delivers a deterrent electric shock to the animal if it contacts the metal cage 722. The high voltage transformer remains powered until the software timer expires 723, then the second computer switches the GPIO digital output and connected power relay to turn the high voltage transformer off 724. The system returns to standby 725, 701.
If the action decision algorithm 705 selects kill or detain action for a pest animal, the second computer monitors the analog input data acquired through the GPIO bus from the proximity sensor 711. When the sensor signal indicates the animal is centrally positioned within the trap, the second computer switches a GPIO digital output and connected power relay to actuate the door latch thus releasing the trap doors 712. A message is sent to the first computer or another network connected device that an animal has been trapped 713. The metal cage trap assembly detains the animal until a human operator arrives 714.
If multiple pest and non-pest animals are detected simultaneously, or if a sensitive non-pest animal is detected, the action decision algorithm 705 may select no action, the timer is allowed to expire and the system returns to standby 731, 701.
The classification model may be pre-trained on images from other sources than the camera to minimize or negate the need for on-site training. Classification models for a particular pest application may be selected from a library of specialized models that are pre-trained on animals or within species variation anticipated for the specific application. The pre-trained models may be downloaded to the trap by over-the-air network software updates from a remote location. The computer or microprocessor on the trap may act as a server and client to message a remote user or server with images from the camera, current state of the trap and history of the animal detections and trap actions, or any related operational or system status information. The trap computer may be network connected to a platform as a service (Paas) or internet of things (IoT) information system for messaging, data downloads and over-the-air software updates.
In other embodiments, one or multiple cameras may be deployed to image the trap, inside the trap and the area in proximity to the trap. An actuator may be used to open and close one or more trap doors as instructed by the computer and may include application customizable actions such as closing the trap doors during daylight hours to better target a nocturnal animal, or responding to non-pest animals by closing the trap doors to prevent them from accessing the bait, or hold the doors in a closed position until a pest animal is detected. The trap may have one or more deterrent actions in place of or in addition to electric shock such as emitting pressure waves in the audible or ultrasonic range or electromagnetic radiation. The trap may have kill actions such as electrocution, pressure waves, percussion, or electromagnetic radiation.
In still other embodiments, the classification algorithm used in combination with the convolution algorithm may be different machine learning methods known within the body of knowledge such as boosted or randomized classification trees, and logistic regression. Variants of CNN and RCNN based algorithms such as described in U.S. patent application Ser. No. 15/001,417, U.S. patent application Ser. No. 15/379,277, and publications He 2017, Sabour 2017, He 2015 and Szegedy 2015 may be used. Open source code libraries implementing some of these approaches are available on the Internet.
An embodiment may have an auto-dispensing bait system or use light, chemical attractant or pheromone in place of the bait. An embodiment may not use a bait, but instead be placed or transported along a common travel route so that animals pass through the trap. An example common travel route may be a migration route such as a fish ladder. An embodiment may be configured not to have an enclosure or doors. For example, the trap may be a pad with embedded electrodes with bait placed in the center to draw the pest across the electrodes and the embodiment selects between a deterrent shock and an electrocution shock or other deterrent or kill action depending on the animal classification.
An embodiment may direct insects past a camera then divert them into release or kill/detention passages through the trap depending on the classification. An embodiment may be used for aquatic animals entering the trap at least partially submersed in a stream, river, lake, or ocean.
Embodiments may be applied to environmental management, biology or ecology studies where the animals are not categorized as pest, non-pest, but as different species or within species variants that are categorized as target and non-target.
Although the description of specific embodiments has been included, these should not be construed as limitations on the scope, but rather as an exemplification of possible embodiments. The selective action trap is able to identify animals by general appearance and thus has many advantages:
This application claims priority to U.S. provisional application Ser. No. 62/594,121 filed on Dec. 4, 2017, which is incorporated by reference in its entirety.
| Number | Date | Country | |
|---|---|---|---|
| 62594121 | Dec 2017 | US |
