Patent Grant
11849705
Honeybee populations have been in decline since the 1980s, and the term “Colony Collapse Disorder,” was coined in 2007 to describe the troubling state of the bees. Beekeepers in the past could rely on 10-15% hive losses (i.e., bee mortality rates) each year, which was perceived as a sustainable loss percentage. Today, beekeepers are experiencing an average hive loss rate of 30% each year, with hive loss rates reaching nearly 50% in some regions, which has resulted in a fairly significant decrease in honey production over the last couple of decades. Diverse beekeeping beliefs (i.e., decision-making regarding hive management) have contributed to the hive loss rates, typically with little to no actual data being used to support hive management practices.
Additionally, beekeepers generally do not keep an accurate accounting of the information associated with their apiaries (e.g., the condition of the hives, bee counts, environment conditions, losses, etc.). Further, current research on honeybees relies on self-selection citizen science, and is limited to the qualitative information supplied by beekeepers, which, as noted, can be largely inaccurate. In this field, there is a general lack of quantitative data on the ecology and production efficiency of apiaries and the bees/hives contained therein. Therefore, there is a need for improved technologies for monitoring and analyzing data of beehives in an apiary.
In one embodiment, a communications and control system and method for monitoring information about a plurality of beehives is provided. In one aspect of an embodiment of such a system, information or data is measured or otherwise observed, such as using suitable sensors, regarding the bees, and/or the beehives, and/or the environment around the beehives, or other relevant data or information. Such information measured may include one or more of temperature(s) at different points on or in the hive, gas (such as oxygen, carbon monoxide, and/or carbon dioxide gases) or other material or toxin levels, infrared or other wavelength information, weight and/or mass, activity level, decibel or other noise level measures, humidity levels, pressure, activity such as may be measure by movement or another visual measure, GPS or similar data regarding physical location, altitude, orientation or relative orientation, and other information or data as would occur to one of ordinary skill in the art regarding the beehive, bees, their activity, health, stress or other beehive effects.
In one embodiment, the sensors sense and transmit or store the data or information. Information may be stored at the beehive for later use, processing, or collection. The data or information may be collected through a variety of methods using a variety of systems and components well known in the art, such as automated or manual collection, pushing the information using a transmitter located at the beehive. Such data or information may be processed at the beehive or may be processed using devices or systems as disclosed in this disclosure or as would be known to one of ordinary skill in the art. The information may be processed at one or more of at the beehive, at another beehive that is in the same apiary or is separate from the beehive, and using other devices or systems remote from the beehives.
Other information and data can also be manually gathered or input, accessed, observed or collected relating to environmental conditions at or near each beehive and/or group of beehives. Such information may include the information described above, and/or other information such as wind speed and/or direction, precipitation type and amount, crop information (such as crop type and related data, plant location, plant density, crop yield information whether aggregate or by location in an orchard, field or plot), pesticide, fertilizer, and/or irrigation type, quantity and rate information, planting and harvest information, and other agricultural information. Such information and other data may be collected or accessed through other systems (such as farm implements and separate agricultural monitoring systems on the farm or orchard, or from other available databases or measurement/monitoring systems).
In one embodiment, the system and method may be configured to perform calculations and analyses. Calculations and analyses may be manual, automated, and may be performed using manually developed functions, predetermined or stablished functions, or machine learning. Such calculations and analyses may include for example recommended density or relative location of beehives placed for pollination in an orchard or in a field or plot, effectiveness or net yield change in a given crop, pesticide and/or fertilizer usage as a function of hive health, activity or other beehive information,
In one embodiment, the system and method may be configured to generate and/or communicate reports. Such reports may include graphs, charts, lists, and other displays known to those of ordinary skill in the art. Such reports may include data, analyzed data, results of calculations and other derived information from the data and information gathered or accessed as described herein.
In one embodiment, the system and method may be configured to generate and communicate suggestions, recommendations, or instructions such as physical movement of one or more beehives to a new and/or particular location in an apiary or group of beehives placed for pollination or relative to a portion of crops, replacement of a beehive, collection of honey in a beehive, replacement of a queen, inspection, repair, or maintenance of hive structural or sensor or other components as disclosed herein, and other actions to be taken whether automatically or manually by a famer, worker or beekeeper, and whether directly or remotely. These suggestions may be based on observed data and information gathered or may be based on analyses or calculations of such data.
In one embodiment, the system and method may be configured to generate and communicate actions. Such actions can be communicated by or within parts of the system, including those disclosed herein or as would be understood by one of ordinary skill in the art, and may illustratively include such actions that adjust a physical characteristic of a beehive such as changing beehive orientation, activating or positioning windscreens, activating or positioning sunscreens, opening ventilation equipment or doors, activating smoke, water, food, nutrients, medication, or other devices for providing a supply to the bees in or around the beehive.
In one aspect, a method for monitoring and analyzing data of beehives in an apiary includes receiving, by a remote data analysis computing device, sensor data from one of a plurality of sensors associated with the apiary; analyzing, by the remote data analysis computing device, the received data; determining, by the remote data analysis computing device, whether to perform an action as a function of a result of the analysis; and performing, by the remote data analysis computing device and in response to a determination the action is to be performed, the action.
In some embodiments, analyzing the sensor data comprises to analyze the data using one or more machine learning algorithms. In other embodiments, analyzing the sensor data comprises to analyze the data using one or more regression analysis methods. In still other embodiments, the method further comprising aggregating, by the remote data analysis computing device, the received data with previously received sensor data from at least a portion of the plurality of sensors, wherein analyzing the received data includes analyzing the aggregated data.
In some embodiments, the method further comprising comparing, by the remote data analysis computing device, the result of the analysis to one or more threshold values, wherein determining whether to perform the action as a function of the result of the analysis comprises determining whether to perform the action as a function of the comparison. In some embodiments, performing the action in response to a determination the action is to be performed comprises performing the action in response to a determination the result of the analysis meets or violates one or more threshold values. In other embodiments, performing the action comprises transmitting a message to a corresponding data monitoring application, wherein the message includes information related to the action that is usable to display to a user of the corresponding data monitoring application. In still other embodiments, performing the action comprises transmitting a command to an actuator of the apiary, wherein the command is usable by the actuator to adjust a physical characteristic of a beehive by positioning of a mechanism attached to the actuator. In yet other embodiments, performing the action comprises transmitting an email to an email address, wherein the email includes information related to the action that is usable to indicate to notify a user of the action. In still yet other embodiments, performing the action comprises transmitting a text message to a phone number, wherein the text message includes information related to the action that is usable to notify a user of the action.
The embodiments and other features, advantages and disclosures contained herein, and the manner of attaining them, will become apparent and the present disclosure will be better understood by reference to the following description of various exemplary embodiments of the present disclosure taken in conjunction with the accompanying drawings, wherein:
For the purposes of promoting an understanding of the principles of the present disclosure, reference will now be made to the embodiments illustrated in the drawings, and specific language will be used to describe the same. It will nevertheless be understood that no limitation of the scope of this disclosure is thereby intended.
As illustratively shown, each of the sensors 104 are communicatively coupled to one or more computing devices 120 either directly or over a network 118. The illustrative computing devices 120 include a gateway computing device 112, a remote data analysis computing device 124 (e.g., a cloud computing device), a client computing device 128, and a remote data provider computing device 132. It should be appreciated that additional and/or alternative computing devices 120 may be included in other embodiments.
The sensors 104 are configured to collect data of the environment in which they are deployed (e.g., internal or external to the beehives 102) and transmit (e.g., via messages, network packets, etc.) the collected data to the remote data analysis computing device 124. To do so, the sensors 104 are configured to transmit the collected data to the gateway computing device 112 over a wireless, cell, satellite, or other suitable network 114. In turn, and the gateway computing device 112 is configured to forward the collected data received from the sensors 102 to the remote data analysis computing device 124 via the network 118.
As will be described in further detail below, the remote data analysis computing device 124 is configured to monitor the data collected by and received from the sensors 104 to determine whether a condition exists such that an action should be taken (e.g., harvesting the produced honey from one or more of the beehives 102, attaching one or more supers to one or more of the beehives 102, moving/placing/repairing/replacing one or more of the beehives 102, adjusting a feeding schedule, etc.). In an illustrative example, the remote data analysis computing device 124 may be configured to determine locational placement of one or more of the beehives 102 within the apiary 116 in an effort to maximize honey production efficiency and/or reduce bee mortality rates.
The beehives 102 may be embodied as any type of enclosed structure in which a colony of honey bees is intended to live within. The beehives 102 may be configured as any type of artificial beehive 102, typically based on regional locale, such as, but not limited to, a top-bar hive, Wane (European style) hive, Langstroth (stacked) hive, etc. It should be appreciated that the beehives 102 as described herein are intended to be used such that honey produced from the bees living therein may be extracted therefrom. It should be further appreciated that, in some embodiments, the apiary 116, and more particularly the beehives 102 contained therein, may be intended to serve one or more additional and/or alternative purposes, such as the pollination of nearby crops, to provide safe-housing to mitigate the effects of colony collapse disorder, etc.
Each of the sensors 104 may be embodied as any combination of firmware, software, hardware, and/or circuitry capable of performing the functions described herein, including detecting a present condition and/or events/changes in the environment in which the sensors 104 are deployed. The illustrative sensors 104 include beehive sensors 106 and environment sensors 108. It should be appreciated that for the purposes of the description provided herein, the beehive sensors 106 include any type of sensor 104 usable to detect a state or condition of the beehive 102. In other words, beehive sensors 106 include any types of sensors 104 capable of measuring data (i.e., beehive data) related to a respective beehive 102 to which it is attached or in proximate location thereof. Examples of such beehive data may include a weight of the beehive 102, a temperature inside the beehive 102, an activity level inside the beehive 102, etc.
It should be further appreciated that for the purposes of the description provided herein, the environment sensors 108 include any type of sensor 104 usable to detect a state or condition of an environment condition the apiary 116. In other words, the environment sensors 108 include any types of sensors 104 usable for measuring data (i.e., environment data) of the apiary 116 in which the beehives 102 reside, or the immediately surrounding area of the apiary 116. Examples of such environment data may include weather conditions (e.g., temperature, wind, humidity, sunlight, shade, UV levels, etc.) of the apiary at a given point in time and/or over a particular window of time, a time of day (e.g., dawn, dusk, etc.), detected motion/activity (e.g., of beekeepers, bees, animals, etc.), etc.
In some embodiments, the system 100 may additionally include one or more actuators 110. The actuators 110 may be embodied as any combination of firmware, software, hardware, and/or circuitry capable of performing the functions described herein, such as moving or controlling a mechanism or component. As illustratively shown, one or more of the actuators 110 may be located proximate to one of the beehives 102. In other words, in some embodiments, one or more of the actuators 110 may be affixed to a mechanism or component internal to or otherwise attached or operatively coupled to one of the beehives 102. Additionally or alternatively, as also illustratively shown, one or more of the actuators 110 may be affixed to a mechanism or component located external to the beehives 102, in some embodiments. In an illustrative example, one of the actuators 110 may be configured to open a valve to release a variable amount of smoke from a smoker to which the respective actuator 110 has been affixed.
The computing devices 120 may be embodied as any combination of firmware, software, hardware, and/or circuitry capable of performing the functions described herein. Referring now to
The CPU 200 may be embodied as any type of hardware or combination of circuitry capable of processing data. Accordingly, the CPU 200 may include a single processing core (e.g., in a single-core processor architecture) or more than one processing core (e.g., in a multi-core processor architecture). Irrespective of the number of processing cores, the CPU 200 is capable of reading and executing program instructions. In some embodiments, the CPU 200 may include a cache memory (not shown). In such embodiments, the cache memory may be integrated directly with the CPU 200 or placed on a separate chip with a separate interconnect to the CPU 200. It should be appreciated that, in some embodiments, pipeline logic may be used to perform software and/or hardware operations (e.g., network traffic processing operations), rather than commands issued to/from the CPU 200.
The I/O controller 202, or I/O interface, may be embodied as any type of computer hardware or combination of circuitry capable of interfacing between input/output devices and the computing device 120. Illustratively, the I/O controller 202 is configured to receive input/output requests from the CPU 200, and send control signals to the respective input/output devices, thereby managing the data flow to/from the computing device 120.
The memory 204 may be embodied as any type of computer hardware or combination of circuitry capable of holding data and instructions for processing. Such memory 204 may be referred to as main or primary memory. It should be appreciated that, in some embodiments, one or more components may have direct access to memory, such that certain data may be stored via direct memory access (DMA) independently of the CPU 200.
The network communication circuitry 206 may be embodied as any type of computer hardware or combination of circuitry capable of managing network interfacing communications (e.g., messages, datagrams, packets, etc.) via wireless and/or wired communication modes. Accordingly, in some embodiments, the network communication circuitry 206 may include a network interface controller (NIC) capable of being configured to connect the computing device 120 to a computer network (e.g., the network 118).
The one or more I/O peripherals 208 may be embodied as any auxiliary device configured to connect to and communicate with the computing device 120. For example, the I/O peripherals 208 may include, but are not limited to, a touchpad, a keypad, a keyboard, a microphone, one or more buttons, a display (e.g., a liquid crystal diode (LED) display), a touchscreen, a speaker, one or more illuminating components (e.g., light emitting diode(s) (LEDs)), etc. Accordingly, it should be appreciated that some I/O peripherals 208 are capable of facilitating only input or output functions, while others are capable of performing both functions (i.e., input and output).
The data storage device 210 may be embodied as any type of computer hardware capable of the non-volatile storage of data (e.g., semiconductor storage media, magnetic storage media, optical storage media, etc.). Such data storage devices 210 are commonly referred to as auxiliary or secondary storage, and are typically used to store a large amount of data relative to the memory 204 described above.
Referring back to
In an alternative embodiment, the wireless network 114 may be configured as a mesh network, such as a wireless sensor network, in which each of the sensors 104 include repeater/routing capabilities. Accordingly, in the mesh network configuration, unlike the star network configuration, each of the sensors 104 are connected to one or more of the other sensors 104 of the wireless network 114. As such, information from one of the sensors 104 can be relayed onto another of the one or more sensors 104 in an effort to transmit collected data to the gateway computing device 112.
The network 118 may be implemented as any type of wired and/or wireless network, such as a local area network (LAN), a wide area network (WAN), a global network (e.g., the Internet), etc. Accordingly, one or more network communication technologies (e.g., Ethernet, cellular, satellite, etc.) and associated protocols may be used. The network 118 may include one or more communicatively coupled network computing devices (not shown) for facilitating the flow and processing of network communication traffic via a series of interconnects. Such network computing devices may include, but are not limited to, one or more virtual and/or physical access points, routers, switches, servers, network hubs, compute devices, storage devices, etc.
It should be appreciated that one or more of such network computing devices may be configured to couple one or more of the computing devices 120 (e.g., the gateway computing device 112, the remote data analysis computing device 124, the client computing device 128, and the remote data provider computing device 132 of the system 100 of
The remote data analysis device 124 may be embodied as any type of computing device 120 capable of monitoring, aggregating, and storing received data (e.g., data collected and transmitted by the sensors 104 of the apiary 116), as well as analyzing the data to determine whether an action should be taken as a result of the analysis. To do so, the illustrative remote data analysis device 124 includes or is otherwise communicatively coupled to an apiary data analysis engine 126 (see, e.g., the illustrative apiary data analysis engine 126 of
The remote data analysis device 124 may be embodied as any type of computing device with sufficient storage and compute capabilities to perform the functions described herein. It should be appreciated that while the remote data analysis device 124 is illustratively shown as a single computing device 120, the remote data analysis device 124 may include more than one computing device 120, in some embodiments, such as in distributed or parallel computing embodiments. For example, in some embodiments, the remote data analysis device 124 may be embodied as one or more servers (e.g., cloud servers, stand-alone, rack-mounted, etc.) and/or combination of compute blades and data storage devices (e.g., of a storage area network (SAN)).
It should be appreciated that the remote data analysis device 124 may be communicatively coupled to more than one apiary (e.g., via a respective gateway computing device 112 of each apiary to which the remote data analysis device 124 is communicatively coupled) at any given point in time. In other words, the remote data analysis device 124 may be configured to monitor, aggregate, store, and analyze the data corresponding to more than one apiary 116 at a time. Accordingly, in such embodiments, the remote data analysis device 124 may be configured to leverage data from one or more apiaries 116 to analyze another apiary 116.
The client computing device 128 may be embodied as any type of computing device 120 capable of displaying data received from the remote data analysis device 124, receiving input data from a user of the client computing device 128, and transmitting the received input data to the remote data analysis device 124. To do so, the illustrative client computing device 128 includes a data monitoring application 130 (see, e.g., the illustrative data monitoring application 130 of
In some embodiments, the client computing device 128 may be embodied as a mobile computing device that uses mobile-specific hardware and software (e.g., an operating system) for operating, executing, and providing services and applications to support a mobile architecture and portability. Illustrative examples of such mobile computing devices may include, but are not limited to, smartphones, wearables (e.g., smartwatches, smart glasses, etc.), tablets, laptops, notebooks, etc. Alternatively, in other embodiments, the client computing device 128 may be a stationary computing device, such as a desktop computer, an all-in-one computer, etc. It should be appreciated that, in some embodiments, the system 100 may include more than one client computing device 128. In other words, more than one client computing device 128 may be communicatively coupled to the remote data analysis device 124 at a given point in time.
The remote data provider computing device 132 may be embodied as any type of computing device capable of performing the functions described herein, including, but not limited to, aggregating data into an organized database and transmitting the aggregated data, or a result of an analysis performed thereon, to the remote data analysis device 124. For example, the remote data provider computing device 132 may be configured to collect and/or aggregate weather related data and transmit present and projected future weather conditions. Accordingly, the remote data provider computing device 132 may be embodied as a server, a compute device, a storage device, or a combination thereof. It should be appreciated that, similar to the remote data analysis device 124, the remote data provider computing device 132 may include more than one computing device 120, in some embodiments, such as in distributed or parallel computing embodiments.
Referring now to
The interface display database 302 is configured to store interface display information received from the remote data analysis computing device 124. Such interface display information may be usable to display or render/display a user interface of the data monitoring application to a display of the client computing device 128. The application settings database 304 is configured to store settings information of the data monitoring application 130. The settings information may include any settings usable by the data monitoring application 130 to determine which data types are being monitored, define one or more thresholds, set a weight and/or priority level to certain types of data, and/or set one or more actions associated with the collected data and corresponding thresholds. For example, the settings may include user-definable thresholds and/or parameters that may be usable to dynamically adjust one or more thresholds.
The illustrative environment 300 additionally includes an illustrative embodiment of the data monitoring application 130. The illustrative data monitoring application 130 includes a dashboard interface manager 306 and a data analysis engine interface manager 308. It should be appreciated that the interfaces described herein for managing the user interactions (e.g., displaying the GUI and receiving user inputs) are graphical user interfaces (GUIs) which allow the user to interact with the application referred to herein through GUI elements, graphical images, and visual indicators. Accordingly, the dashboard interface manager 306 is configured to render/display a GUI of the data monitoring application 130 to a display of the client computing device 128 such as, for example, the user interfaces 600, 700 of
In some embodiments, the dashboard interface manager 306 may be additionally configured to generate audible tones (i.e., initiate playback of a sound file), such as may be associated with a notification message, or some other indication of an event associated with the data monitoring application 130. The dashboard interface manager 306 is further configured to receive inputs (e.g., notes, comments, selections, etc.) from a user of the client computing device 128, such as may be received via a touchscreen display of the client computing device 128, or other peripheral device (e.g., stylus, mouse, keyboard, keypad, microphone, etc.) connected to the client computing device 128.
The data analysis engine interface manager 308 is configured to interface with the remote data analysis computing device 124. To do so, the data analysis engine interface manager 308 is configured to establish a communication channel with the remote data analysis computing device 124, which is usable to transmit and receive messages (e.g., network traffic, packets, messages, etc.) therebetween.
Accordingly, the data analysis engine interface manager 308 is configured to generate messages for transmission to the remote data analysis computing device 124, such as may be generated based on inputs received from a user by the dashboard interface manager 306. As such, the data analysis engine interface manager 308 can provide information (e.g., changed settings, dashboard display requests, initiated actions, etc.) that is usable by the remote data analysis computing device 124, and more particularly by the apiary data analysis engine 126, to update a setting associated with the data being monitored, adjust the analysis being performed, etc. Additionally, in some embodiments, the information may be usable by the remote data analysis computing device 124 to initiate an operation to be performed, or otherwise set a time/schedule for an operation to be performed, such as may be performed via one of the actuators 100 of the apiary 116. Additionally, in some embodiments, the occurrence of a condition (such as the actual or forecasted temperature drops below a threshold) may be usable by the remote data analysis computing device 124 to initiate an operation to be performed, or otherwise set a time/schedule for an operation to be performed, such as may be performed via one of the actuators 100 of the apiary 116. The data analysis engine interface manager 308 is additionally configured to process messages received from the remote data analysis computing device 124. For example, the data analysis engine interface manager 308 is configured interpret a message received from the remote data analysis computing device 124 to determine what information is to be displayed by the data monitoring application 130 to a display of the client computing device 128 (e.g., in a GUI dashboard of the data monitoring application 130) and/or other output device coupled to the client computing device 128. It should be appreciated that the received information may include data (e.g., in a payload of a network packet) that includes information related to an object that is usable to download and/or render one or more GUI elements, icons, graphics, text, etc., for output by the client computing device 128.
Referring now to
It should be appreciated that, in some embodiments, the data stored in, or otherwise represented by, each of the hive information database 402, the environment information database 404, and the settings database 406 may not be mutually exclusive relative to each other. In some embodiments, one or more of the hive information database 402, the environment information database 404, and the settings database 406 may include data from beehives 102 of multiple apiaries 116. In such embodiments, the data from one apiary 116 may be siloed from and/or combined with the data from other apiaries 116.
The illustrative environment 400 additionally includes an illustrative embodiment of the apiary data analysis engine 126. The illustrative data analysis engine 126 includes a sensor monitor 408, a data aggregator 410, a data analyzer 412, an action manager 414, and an application interface manager 416. It should be appreciated that, in some embodiments, one or more of the components of the illustrative data analysis engine 126 may be combined to result in fewer components (e.g., the data analyzer 412 may perform the function of the sensor monitor 408 and/or the data aggregator 410). Additionally, it should be further appreciated that, in some embodiments, one or more of the components of the illustrative data analysis engine 126 may be divided into a subset of components.
The sensor monitor 408, which may be embodied as any type of firmware, hardware, software, circuitry, or combination thereof, is configured to receive the data collected by the sensors 104 and store the received sensor data in the appropriate fields of the respective database (e.g., the hive information database 402 and/or the environment information database 404). It should be appreciated that the types of beehive data collected from the beehive sensors 106 may be predicated on the types of beehive sensors 106 in use in a particular beehive 102. Similarly, it should be appreciated that the types of environment data collected from the environment sensors 108 may be predicated on the types of environment sensors 108 in use in a particular apiary 116.
As described previously, the beehive data may include any data related to at least a portion of a beehive 102, such as a weight of the beehive 102, a temperature inside the beehive 102, an activity level inside the beehive 102, pheromone levels, etc. As also described previously, the environment data of the apiary 116 may include weather conditions (e.g., temperature, wind, humidity, sunlight, shade, UV levels, etc.), a time of day (e.g., dawn, dusk, etc.), detected motion/activity, etc. Such data can be used to make decisions regarding and issue action instructions to adjust a physical characteristic of the beehive 102 such as to close doors, change louvre positions, rotate the beehive, and the like.
In an illustrative example, one or more of the bees may be individually monitored, or tracked, using a micro-sized transponders/transceivers (e.g., a micro NFC/RFID transponder, a Bluetooth® Low Energy (BLE) micro module, or some other type of low power wireless communication technology module) that may be used for wireless tracking and/or identification purposes. In such embodiments, the micro-sized transponder may be configured to interface with one or more sensors 104 and/or readers associated with the micro-sized transponder (e.g., an RFID reader) that may be placed within the apiary 116, or more particularly within or otherwise associated with a particular beehive 102. For example, in such embodiments, ingress/egress pattern data may be captured for analysis.
Additionally or alternatively, in some embodiments, one or more GPS tracking devices may be employed. For example, in such embodiments, the queen bee of a particular beehive 102 may have a GPS tracking device affixed thereto such that movement, flight speed, flight pattern, and other properties can be tracked and monitored. Accordingly, in such embodiments, the sensor monitor 408 may be configured to interface with or otherwise be configured to interpret the tracking information received from such tracking technologies for analysis.
The data aggregator 410, which may be embodied as any type of firmware, hardware, software, circuitry, or combination thereof, is configured to aggregate, refine, and enhance the received data. To do so, the data aggregator 410 may be configured to gather data from a number of data sources other than the sensors 104. In other words, while the majority of the data described as being collected herein has been described in relation to the data received from the sensors 104, it should be appreciated that additional and/or alternative data elements may be received from a source other than the sensors 104, such that the externally received data may be usable to refine and/or enhance the sensor 104 received data.
For example, in some embodiments, the data aggregator 410 may be further configured to aggregate data received from a user (i.e., by way of the data monitoring application 130), such as may be received via the application interface manager 416, as well as any externally received and/or user input data. Accordingly, it should be appreciated that the hive information database 402 and/or environment information database 404 may include additional information that was not collected by the sensors 104, such as may have been acquired, gathered, or input by another source (e.g., a user of the data monitoring application 130 of the client computing device 128) or received from an external source (e.g., via an API of a data aggregator).
In an illustrative example, a user (e.g., a beekeeper, an apiary owner, an administrator, etc.) may input information (e.g., via the data monitoring application 130) usable by the data analysis engine 126 to refine and/or enhance the data of the hive information database 402 and/or environment information database 404 (i.e., the data that has been received from the beehive sensors 106 and/or the environment sensors 108). The user input data may include data related the bees in the apiary 116 or a particular one or more of the beehives 102.
For example, the user input data may include genetic information, breeding information, traits, wing span and thickness, coloration, etc. In another example, the user input data may be related to a particular hive, such as egg-laying levels, propolis levels, weight/size of individual bees in population (e.g., population size, number of drones, proportion of new/winter bees, proportion of workers to drones, etc.), honey production levels, parasite/pest types/levels (e.g., Varroa mites, small hive beetles, wax moths, etc.), travel routes/speed of the queen and/or other bee types, etc. In still another example, the user input data may be related to the feeding of the bees in the apiary 116 or a particular one or more of the beehives 102, such as a time of day of the feeding, a type/amount of food distributed, a type/amount of vitamin supplements added to the food, etc.
The user input data may additionally include data related a beekeeper prior to, during, or subsequent to interaction(s) with one or more beehives 102 of the apiary 116. Accordingly, it should be appreciated that such information may be input prior to, during, or subsequent to interaction(s), depending on the type of input. For example, the interaction data may include dietary information of the beekeeper, a mood of the beekeeper, a present health condition of the beekeeper, etc., prior to an interaction. Additionally and/or alternatively, the interaction data may include a duration of the interaction (e.g., with a particular beehive 102, in the apiary 116, etc.), a description of beekeeping attire worn (e.g., veils/masks, clothing, etc.), type of equipment used (e.g., smoker brand, size, fuel type, etc.) a mood of the beekeeper, noise(s) made, whether the beekeeper was stung (e.g., at a particular beehive 102, in the apiary 116, etc.), etc., during an interaction.
In another illustrative example, the data aggregator 410 may be configured to interface with an externally managed data source (e.g., an external data aggregator) to refine and/or enhance the data of the hive information database 402 and/or environment information database 404. Such externally acquired data may include weather prediction data (e.g., from a weather data aggregator), animal/insect migration patterns, air quality levels, etc. Additionally, the data aggregator 410 may be configured to collect external environmental data of one or more areas neighboring the apiary 116. Such neighboring environmental data may include, but is not limited to, a number and/or types of plants (e.g., food sources, trees, medicinal plants, etc.), other pollinators/apiaries, a number and/or types of animals/insects/humans, land type/usage, pesticide usage, water pollution levels, etc. It should be further appreciated that some input data may be discernable from other input data via an externally managed data source. For example, an address of the apiary 116 may be usable to determine an elevation, latitude and longitude, etc., of the apiary 116. GPS or other geo-location information and data, altitude, and orientation and other positioning information such as measured through accelerometers and other devices known to those of ordinary skill in the art.
While the data described herein is described as being collected from a particular source (e.g., beehive sensors 106, environment sensors 108, user input, 3rd party data aggregators, etc.), it should be appreciated that the data may be collected from an alternative data source, which may or may not be described herein. For example, in some embodiments, at least a portion of the data described herein as being collected via user input may be collected by one or more sensors 104 in other embodiments.
The data analyzer 412, which may be embodied as any type of firmware, hardware, software, circuitry, or combination thereof, is configured to analyze the received data, as well as the aggregated data (e.g., including user input data), such that correlations may be determined therefrom which are usable to identify one or more actions that may be taken as a function of the correlations and present condition information. To do so, the data analyzer 412 may be configured to use one or more machine learning algorithms, regression analysis methods, and/or the like to perform the analysis. The data analyzer 412 may be further configured to transmit the results of the analysis to the action manager 414 and/or the application interface manager 416, each of which are described in further detail below.
For example, the data analyzer 412 may be configured to compare at least a portion of a result of the analysis to a threshold (e.g., a statistics threshold) to determine whether the result violates (e.g., exceeds, is less than, is outside of a range, etc.) the threshold or that the results of the analysis otherwise indicate that an action should be taken (e.g., based on historical data). In some embodiments, one or more of the thresholds may be statically defined by a user (e.g., via the data monitoring application 130), the settings of which may be stored in the settings database 406. However, the data analyzer 412 may be configured to determine or otherwise adjust one or more other thresholds in a dynamic fashion, as may be determined based on previously performed analyses and the results related thereto.
Other data either measured or observed may include age of the hive, length of beehive deployment, number of moves of the hive, other movement of the beehive, beehive mechanical structure information. Further, information can be gathered and calculations and other functions performed regarding crop yield and yield changes over time, and data regarding hive density, placement and any relationship with crop yield, calculations can be made and recommendations or instructions communicated regarding adjustment of physical characteristics of the beehives and/or the apiary such as replacement of particular beehives, adding new beehives, removal of beehives, repositioning of beehives, and the like.
In another example, the data analyzer 412 may be configured to perform an image analysis on digital images captured (e.g., color, black and white, infrared, etc.) by one of the sensors 104. In an illustrative example, the data analysis engine 126 may be configured to analyze an image to determine an actual or estimated number of objects (e.g., eggs, types of bees, pollen, etc.) inside a particular beehive 102 based on one or more digital images received from a beehive sensor 106 embodied as an image sensor. It should be appreciated that the data analyzer 412 is configured to collect multiple data points for performing the analysis and/or updating a confidence level associated with a result of a particular analysis. In an illustrative example, the data analyzer 412 may be configured to analyze the movement of a queen bee such that liveness can be monitored. In such embodiments in which the data analyzer 412 determines the queen has not move for a particular duration of time such that it may be inferred that the queen bee's health has been compromised, the data analyzer 412 may be configured to analyze received digital images to validate a lack of movement, such as may be attributable to illness or death, rather than the transponder having fallen off the queen.
The action manager 414, which may be embodied as any type of firmware, hardware, software, circuitry, or combination thereof, is configured to determine which action is to be commenced or otherwise performed by the apiary data analysis engine 126, such as may be initiated upon a result of the analysis performed by the data analyzer 412 resulting in a determination that one or more thresholds have been violated. Such actions may include transmitting a message to a corresponding data monitoring application 130 and/or a user (e.g., via an email, text message, etc.), transmitting a command to one or more actuators 110 of the respective apiary 116, adjusting a time associated with a resource of the respective apiary 116 (e.g., an automated feeder schedule), adding a corresponding entry into a calendar (e.g., Google Calendar™, Outlook®, Thunderbird®, etc.) linked to the corresponding data monitoring application 130, etc. It should be appreciated that the action manager 414 may be configured to make the determination of which action to perform based on one or more settings corresponding to a user account associated with the apiary 116 being monitored. In some embodiments, the action settings and/or user account settings may be stored in the settings database 406.
The application interface manager 416, which may be embodied as any type of firmware, hardware, software, circuitry, or combination thereof, is configured to interface with the corresponding data monitoring applications 130. To do so, the application interface manager 416 is configured to establish a communication channel with the data analysis engine interface manager 308, which is usable to transmit and receive messages (e.g., network traffic, packets, messages, etc.) therebetween.
Additionally, the application interface manager 416 is configured to authenticate a user of the data monitoring application 130. To do so, the application interface manager 416 is configured to receive credentials from the user of the data monitoring application 130 and verify whether the user is an authorized user, such as may be performed based on a comparison between the received credentials and valid credentials of the user, which may be stored in the settings database 406. Further, the application interface manager 416 is configured to manage the access to data and/or the controls which are associated with the authenticated account.
In an illustrative example of the apiary data analysis engine 126 in use, a sensor 106 of a beehive 102 may be a weight sensor that is configured to detect a weight of at least a portion of the beehive 102. In such an embodiment, the weight sensor may transmit a weight value that is received by the sensor monitor 408, aggregated with historically received weight values, and analyzed by the data analyzer 412. The result of the analysis by the data analyzer 412 may result in a determination that the received weight value is greater than a weight threshold value, or is otherwise consistent with a weight threshold value that is indicative of a favorable time to extract honey from that beehive 102. Alternatively or additionally, weight loss for a sufficient period of time may indicate a deteriorating beehive health, which may result in a determination that the beehive needs to be checked by a beekeeper, or swapped with another beehive, moved, or some other action taken to improve the result.
Accordingly, as a function of the determination that the received weight value is greater than a weight threshold value and depending on the settings associated with an account corresponding to that beehive 102, the action manager 414 may be configured to perform one or more actions. Additionally, also dependent on the settings associated with an account corresponding to that beehive 102, the application interface manager may be configured to generate one or more messages for transmission to the respective data monitoring application that is usable to display one or more of a visual representation of the monitored data, the aggregated data, a result of the analysis of the monitored data and the aggregated data, and the action(s) to be taken, if applicable.
Referring now to
In block 504, the apiary data analysis engine 126 aggregates the received sensor data with previously received sensor data. In block 506, the apiary data analysis engine 126 analyzes the received sensor data and the aggregated sensor data (e.g., using statistical models). To do so, in some embodiments, in block 508, the apiary data analysis engine 126 is configured to analyze the data using one or more machine learning algorithms. Additionally or alternatively, in some embodiments, in block 510, the apiary data analysis engine 126 is configured to analyze the data using one or more regression analysis methods.
In block 512, the apiary data analysis engine 126 compares the results of the analysis to one or more thresholds. As described previously, the thresholds may be statically defined by a user (e.g., via the data monitoring application 130) or dynamically adjusted based on historical data (e.g., as may be determined based historical analysis performed and results related thereto). In block 514, the apiary data analysis engine 126 determines whether any thresholds have been violated or whether the results are otherwise indicative that an action should be taken. If the apiary data analysis engine 126 determines that an action should be taken in block 514, the method 500 advances to block 516.
In block 516, the apiary data analysis engine 126 performs one or more actions as a function of the comparison based on one or more settings of the apiary data analysis engine 126, as may be set by a user of a corresponding account via the data monitoring application 130. In some embodiments, in block 518, the apiary data analysis engine 126 may generate and transmit a message to the corresponding data monitoring application 130 (e.g., one or more instances of the data monitoring application 130 corresponding to the applicable account associated with the sensor data received in block 502. For example, in some embodiments, in block 520, the apiary data analysis engine 126 may be configured to display a notification in an interface of the data monitoring application 130. In another example, in some embodiments, in block 522, the apiary data analysis engine 126 may be configured to add an event to a calendar of the data monitoring application 130. It should be appreciated that, in some embodiments, the apiary data analysis engine 126 may be configured to add the calendar event to an external calendar application (e.g., Google Calendar™, Outlook®, Thunderbird®, etc.).
In some embodiments, in block 524, the apiary data analysis engine 126 may additionally or alternatively generate and transmit a command to a corresponding actuator (e.g., one of the actuators 110 of
It should be appreciated that, in some embodiments, sensor data received from one or more other accounts, apiaries 116, beehives 102, etc., may be used to trigger an action. For example, a present state of a monitored beehive 102 (i.e., as determined based on the most recently received sensor data associated with that beehive 102) may indicate that an optimal time to extract honey from the monitored beehive 102 may be in two days; however, data of a similar beehive 102 in another apiary 116 associated with another user's account may indicate that now is an optimal time to extract the honey from the monitored beehive 102, such as may be based on weather patterns that affected the similar beehive 102 previously or are presently affecting the similar beehive 102 and are anticipated (e.g., based on the analysis performed in block 506) to have a similar effect on the monitored beehive 102.
It should be further appreciated that some information to be displayed (e.g., monitored sensor data, results of data analysis, etc.) and/or certain actions to be performed which are described herein may be queued for a particular user or account to which the actions correspond until a future point in time. For example, if that particular user or account associated therewith is not presently logged into the data monitoring application 130, such information may be queued until that user logs into the data monitoring application 130.
Example data and/or types of measurements received by the remote data analysis computing device 124, and outcomes for the beehive(s) 102 and/or the user (e.g., the beekeeper) associated with the data and/or types of measurements are listed below in Table 1
As demonstrated above, the application of analytics to data collected from beehives can help growers ensure effective pollination. Use of the following embodiments described below decreases costs for growers by monitoring the number, health and distribution of pollinating bees. For a small amount per hive inspected, these embodiments allow growers and beekeepers to take infrared (IR) imagery from a smartphone attachment (or other IR imaging device) to automatically calculate the number of frames of bees inside the hives. As used herein, colony strength, size, and population are all synonymous. The colony refers to the cluster of bees, while the hive refers to the physical wooden structure as well as the bees inside.
Observing and recording the number of frames of bees in colonies throughout the year gives a much clearer picture of a colony's overall health by providing a seasonal history of its strength and size based on fluctuations in adult bee populations. In the past, a simple estimate of a hive's adult bee population required the beekeeper to crack the lid and smoke the top box gently. After the lid is removed and the top brood box is located, the beekeeper counts the number of frames covered with bees. In some cases, this can be accomplished without having to remove any of the frames by simply looking between each frame and counting the frames that are entirely or mostly covered with adult bees. In other situations, the frames must be removed to get an accurate assessment. This procedure is followed for each brood box, and the results are added together to get an estimate of the adult population of a hive expressed as frames of bees. This prior art methodology clearly creates an unwanted disturbance to the hive.
Pollination revenue ($655 million) makes up over half of the total beekeeping market, where over 90 crops need honeybees for proper pollination. Currently, growers in need of pollination will rent beehives from beekeepers, and the beekeepers seasonally move their hives to the grower's location for pollination. Almond pollination generates the highest demand for pollination, requiring ¾ths of the nation's beehives. For almond growers, their main concern is renting enough strong hives to pollinate their orchard, since 100% of an almond crop is reliant on honeybee pollination. If the growers do not rent enough hives above a minimum strength, they will produce lower yields that season.
Growers pay for hives based on colony strength, measured by the population of bees inside the hive, since this is directly related to the amount of pollination that a hive can perform. If a grower or a beekeeper thinks the price is wrong or is worried about hive strength, they can request a hive inspection. If an inspection reveals that some of the hives were weak or dead, the grower will contact a beekeeper and rent more hives.
In these inspections, hives are pried opened and the frames covered with bees are tallied to determine the frame count. Prices for hive rentals are then determined based on the average number of frames across all hives sampled. Inspections are done by hand at a rate of 7 hives per hour and are typically performed on a limited sample of 10-15% of hives rented. Because manual inspections rely on human judgement based on a small sample, they are inaccurate measurements of colony strength. Manual inspections are also stressful to the bees—they will not forage for 2-3 days after an inspection—resulting in lost pollination time for the grower.
The digital hive grading tool embodiments disclosed herein increase the accuracy of hive grading and can work without opening hives. Since manual inspections rely on human judgement, disputes often arise on the quality of the inspector. Using the embodiments disclosed herein, the hive grading data is undisputed as it is based in an unbiased methodology that grades identically across all hives. The embodiments disclosed herein also help beekeepers. The pre-pollination season hive inspections performed to verify that hives meet the minimum strength required by pollination contracts is a very labor-intensive task.
The hive grading embodiments disclosed herein also allow collection of colony size information for use in honeybee research (such as nutrition, pesticide impacts, etc), where this tool can be a more objective and affordable method of data collection. An often-cited problem in data collection is pesticide usage and its harmful effect on honeybee populations. Data collected from the embodiments disclosed herein can be used as a benchmark to test the impact of pesticides on honeybee populations. Currently in pesticide permitting, hive colony size endpoints are used to measure the impact of pesticides on these nontarget organisms. The IR analysis capabilities of the embodiments disclosed herein may also be applied to new verticals, including manufacturing, medical, agriculture, and energy sectors, to name just a few non-limiting examples.
The hive grading embodiments disclosed herein cut down the labor time required to get the hives ready for pollination by 93%. Easy-to-collect strength data also allows the beekeeper to increase transparency and reinforce growers' confidence in their decision to maintain their business relationship by demonstrating the value their bees provide. Beekeepers can also use the product as a point of comparison to other hives on the market, which they can use to justify better prices during negotiation.
In some embodiments, the digital hive grading tool uses infrared (IR) images of the hive. In order to establish the algorithm to use for the digital hive grading tool, data was collected by taking an infrared image of the hive and pairing the image with a manual inspection frame count. Images were collected from a variety of commercial beekeepers, which allows us to get a vast variety of data from beehives. The IR images were captured first, then hives were inspected and assigned frame counts. This order was important because opening the hive could affect the IR heat signature. IR image data and frame counts were collected consecutively—typically within minutes of each other—to ensure the two data points represented the colony at the same point in time. Data was collected from various beekeepers from California, Michigan and Florida. In total 3,197 IR images paired with frame counts were collected, and 6,489 IR images of hives were captured. In some embodiments, the IR images may be captured with a FLIR One Pro thermal camera that functions as an attachment to a smartphone, such as an IOS device. In some embodiments, the manual frame count data and other information noted by the inspector may be stored as metadata associated with each image.
Environmental sensors were installed in a research apiary to record local temperature and humidity values at the site. Environmental sensors were also installed inside the hives to record hive temperature and humidity, and weight sensors were installed to capture hive weight. Hive entrance traffic counters were also installed. Collected data was paired with open source data, such as weather or solar data to better capture ambient conditions affecting the temperature of the hive. Weather and solar data are triangulated from multiple sources (Table 2). Weather and solar data are relevant because ambient weather factors impact heat captured in the IR images. The environmental and open source data are used to validate the physics model by comparing the thermodynamics in the physics model to the actual temperature flow inside the hive.
During use of the FLIR One Pro IR image sensor in some embodiments, the camera was turned on in temperatures between 55-90° F. and it was ensured that the object being measured was above 32° F. Distance tests were also performed, where images were taken at 3, 5, 10 and 15 feet away from the hive. The closer the IR image sensor was to the hive, the greater resolution the thermal image contained. If an image is taken too close, however, the hive segmentation process (discussed below) is impaired. In some embodiments, images that were captured 3-6 feet from the front of the hive performed best.
As schematically illustrated in
The first step in the analysis chain is to process the images into a useful form for modeling. The irradiance data is extracted from the metadata of the image file created by the IR image sensor. Software is then used to extract and format the metadata from the image file. The software also calculates the temperature values from radiance data contained in the image file.
The next step is to build a segmentation model that can identify the beehive within the visual image. The IR image sensor used has two cameras, one infrared and one visual, that capture images simultaneously. The present inventors started with Mask_RCNN, a deep learning model commonly used for image recognition that detects objects in an image while simultaneously generating a high-quality segmentation mask for each instance. The model attempts to find a box in the image, which is well-suited for beehives. The segmentation model was improved and trained to meet the needs of the analysis. The model was trained using 1992 beehive images to develop its accuracy in locating a beehive within an image. This involved using the images sourced from Google Images and GIMP to define and store data on where the hive was in the image.
While training the model, it was determined that the hive lids and entrances skewed temperature data, so the segmentation model was improved to exclude lids and entrances. The model was optimized for hive detection so it didn't overfit the data. Because the segmentation model only needed to classify whether the image was a hive or not, which involves simpler operations, all of the algorithm layers in the RCNN model were not trained; the main layer of analysis was enough for accurate hive detection.
The segmentation model categorizes pixels as “hive” or “not hive”. This results in a shape cropped out of the visual image that is then applied to the thermal image to pull the relevant temperature values and exclude pixel values depicting background objects. To do this, a script was built to align the position and size of the visual image with the thermal image. This model accurately classifies each pixel 93.6% of the time.
The first step of modeling was to look at the temperature statistics (mean, median, range, standard deviation) distributions from the hive images and find a correlation with manually collected colony size data. The relationship between these two datasets was analyzed over the course of a day, and it was determined that a stronger relationship exists closer to sunrise.
A variety of different regression and classification models were tested to see if a relationship could be detected. In the early models, classification performed better, but as more data were collected, regression began performing better. For classification, categories were created to correspond with different levels of colony strength. This technique results in a loss of information because frames of bees are closer to a continuous variable than a category. The updated regression model can more closely output a representation of each hive's strength. The improvement over time of the regression model was due to incorporating feature engineering improvements (discussed below) and additional data.
As part of the feature engineering, the model included weather data (Table 3). Ambient factors were incorporated into the model to account for external heat sources affecting the hive's infrared heat signature.
The parameters of the models were then optimized, updating items such as the learning rate and the regularization factor, which allow the model to train more optimally to minimize error. The regression model used was a boosted model, a process which combines several weak models into one strong model. The model's performance was then evaluated. The model was first trained on a training set of 80% of the IR image and frame count data, then validated on a test set of the remaining data withheld from the model training. This helped determine how the model performs on unseen data.
To validate the data, a 1-dimensional physics-based model was created which incorporates the thermodynamics from the center of the hive out to the center of the hive wall as read from the infrared cameras. The simulated model comprised a heat flow model, and mathematical solvers were used for the differential equations. The simulated model was tested on real world data. The model was first tested using a simulated hive to get early validation of performance. In the simulated hive, Christmas lights were used as the heat source inside a beehive to test the model on the hive equipment before adding the bee complexity. The physics model matches the actual thermodynamics observed in the hive. To increase accuracy, the model takes into account the materials inside the hive.
To verify the 1-dimensional model, a simulated apiary is established using Christmas lights as the heat source in place of bees. In the experiments, IR pictures are taken periodically and the temperatures at important locations in the system are recorded. Temperature data was collected every five minutes via a probe, at various distances from the heat source. The experimental results are shown in
In some embodiments, the entire software chain is not automated—only the most critical components. For example, Images may need to be manually uploaded from the IR image sensor to Dropbox, where they may then be manually pulled into AWS. In some embodiments, although there is a model for segmenting images automatically, a manual segmentation flow may be used for images that do not properly auto segment. In some embodiments, this process includes a human check to ensure the segmentation model performed well.
In some embodiments, images uploaded in the S3 bucket create notifications (SNS) that are sent to the SQS queue. This is a distributed system (n number of hosts) processing the images in tandem. A load balancer calculates the number of images and/or amount of traffic in the SQS queue, then determines how many hosts are necessary in order to process the data. These images are then processed through the segmentation model, IR metadata extraction, and prediction models. Prediction results are then sent to a server, or serverless storage solution, which also stores all metadata, user, site, hive information, picture metadata, beekeeper and grower information. The results are securely processed and sent to the users. According to an embodiment, the results are sent to the users directly. The results may also be posted to a secure portal where the user may review the results.
Example 1: A system as described herein was tested with a commercial beekeeper, during which 120 hives were sampled to determine whether the image capture process worked and how the model performed. Actual frame counts collected manually averaged 9.02 frames across all 120 hives, and the model estimated 8.92 frames.
The system disclosed herein may be packaged in an easy-to-use mobile application so that personnel associated with the hives can perform the data collection themselves.
While the invention has been illustrated and described in detail in the drawings and foregoing description, the same is to be considered as illustrative and not restrictive in character, it being understood that only certain embodiments have been shown and described and that all changes and modifications that come within the spirit of the invention are desired to be protected.
This application is a continuation of U.S. patent application Ser. No. 16/780,663, which was filed on Feb. 3, 2020 and entitled SYSTEMS AND METHODS FOR MEASURING BEEHIVE STRENGTH, which claims the benefit of U.S. Provisional Application No. 62/800,276, filed Feb. 1, 2019, the contents of which are incorporated by reference herein in their entirety.
| Number | Name | Date | Kind |
|---|---|---|---|
| 20100198023 | Yanai | Aug 2010 | A1 |
| 20150366169 | Jiang | Dec 2015 | A1 |
| 20170079249 | Chapa | Mar 2017 | A1 |
| 20200167835 | Michel | May 2020 | A1 |
| Number | Date | Country |
|---|---|---|
| 105145410 | Dec 2015 | CN |
| 107549052 | Jan 2018 | CN |
| 207135954 | Mar 2018 | CN |
| 109145032 | Jan 2019 | CN |
| 111272211 | Jun 2020 | CN |
| WO-2015185899 | Dec 2015 | WO |
| WO-2018165051 | Sep 2018 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20220245937 A1 | Aug 2022 | US |
| Number | Date | Country | |
|---|---|---|---|
| 62800276 | Feb 2019 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16780663 | Feb 2020 | US |
| Child | 17689915 | US |
