Patent Application
20100114489
Unmanned aerial vehicles (UAVs) are being used in a variety of situations where the risk to the pilot of a manned vehicle is high, such as in military operations. UAVs often operate in swarms in which a large number of agents perform a simple set of tasks to accomplish a larger goal. Swarms are well suited to perform missions having a search stage where multiple entities must be located over a wide geographic area. The simplified tasks performed by each agent in the swarm allow a greater number of agents to be controlled and monitored by fewer human operators. Pheromone maps are a common coordination mechanism between agents that emulate insect foraging behavior. Insects mark their environment with pheromones to leave messages for other insects. A digital pheromone map overlays a digital grid onto a geographic area. Similar to insects, agents leave digital information for other agents in each cell of the digital grid. The digital pheromone map acts as an action selection input for the next movement for each agent.
Digital pheromone map systems rely on each agent having a relatively accurate copy of the map and may utilize either a central map that all agents connect to or a distributed map maintained by each agent. Centralized map systems are susceptible to network outages, such as those caused by the agent moving out of range or active jamming of the map signal. However, distributed map systems require a sophisticated synchronization strategy to ensure that each agent has an up-to-date map to conduct an effective search.
Existing swarming search strategies conduct searches based on moving agents using a pheromone gradient from areas of high pheromone concentration (i.e., a high number of searched cells) to areas of lower pheromone concentration (i.e., a low number of searched cells). Such an approach is subject to local minima in which an agent is in an area of low pheromone concentration surrounded by areas of high pheromone concentration. This causes the agent to unnecessarily search the local minima while ignoring faraway unsearched areas. Typical swarm search strategies also do not optimize the search coverage of each agent while maximizing the coverage of the network created by the distributed agents. A poorly distributed network of agents makes the search system more susceptible to agent attrition, such as when an agent malfunctions or is destroyed by an enemy. Finally, updates between distributed agents are often inefficient because they require unnecessary synchronization between agents that do not have relevant digital pheromone map differences.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
Aspects of the invention are directed to methods and systems for performing an operation (e.g., an exhaustive search) related to a geographic area using one or more agents. In particular aspects, the system first creates a digital map (e.g., a pheromone map) of the geographic region divided into cells that contain information about the progress of the operation over the geographic region. For example, a cell may contain an indication of whether a particular section of the geographic area has been searched by an agent. Next, the system calculates a distance gradient of the digital map indicating the distance from each cell to the next unoperated cell (e.g., an unsearched cell). The system determines the next move for each agent based on the distance gradient. For example, a search system may move an agent toward areas of the map that have not yet been searched by following decreasing gradient values.
In some aspects, each agent maintains an instance of the digital map and communicates with other agents to synchronize instances of the map. An agent (i.e., the sending agent) may send a key to other agents that contains a summary of the state of the sending agent's instance of the map that other agents can use to determine if they need to synchronize with the sending agent. If so, the other agents can request a more detailed update from the sending agent. The agents synchronize the received update information with their individual instances of the digital map.
In further aspects each agent's path is chosen to increase the communication range of a network between the agents and to reduce crowding among agents. Each agent may operate as a node in a wireless network and the farther apart agents are while staying within range of at least one other agent, the broader the coverage of the network will be. Likewise, the less crowded agents are the less likely an agent will cover an area that another agent has already covered or collide with other agents.
Methods and systems for performing operations related to a geographic area (e.g., search operations) are provided. Certain aspects and examples of the methods and systems are described generally below, and additional examples are further described in the context of the included figures. Many of these aspects are described in the context of search systems that conduct a search over a geographic area, but extend to more general systems that perform other operations as well.
In a particular embodiment, the system first creates a digital pheromone map of the geographic region divided into cells that contain information about the progress of a search or other operation related to the geographic region. For example, a cell may contain an indication of whether a particular section of the geographic area has been searched by an agent. Next, the system calculates a distance gradient of the digital pheromone map. The distance gradient indicates the distance from each cell of the digital pheromone map to the nearest unsearched cells (or unoperated cells in the case of an operation other than a search), such as by calculating the Euclidian distance between the cells. For example, a gradient value of zero for a cell may indicate that the section of the geographic area represented by the cell has not yet been searched, while a gradient value of two may indicate that the nearest unsearched cell is two cells away. At any point in time an agent can move in one of eight directions within the digital pheromone map (e.g., left, right, forward, backward, or diagonally one cell). The system determines the next move for each search agent based on the distance gradient. For example, a search system may move an agent toward areas of the map that have not yet been searched by following decreasing gradient values.
In some embodiments, the search system produces an exhaustive search of the geographic area. Each agent can perform a simple, reactive behavior based on the map gradient, and the combined result of multiple agents can produce an exhaustive, coordinated search. In general, each agent is either marking an area, on its way to an unmarked area, or spreading out network coverage around other marking agents. The distance gradient need not be subject to local minima like prior swarming search strategies because as areas of the map are searched, an absolute minima forms around unsearched areas. Therefore, following the distance gradient leads to the next closest unsearched area. This method leads to exhaustive coverage of the search area in a finite search time.
In other embodiments, search agents follow four basic behaviors: contour following, avoidance, gradient descent, and random movement. Contour following is selected when an agent is located at a boundary cell (e.g., a searched cell located next to at least one unsearched cell) when another agent is not nearby. The agent can move perpendicular to the distance gradient in a “lawnmower” pattern either spiraling from an outer boundary toward the center or from the center of an area toward the outer boundary. Avoidance behavior is selected when another search agent is within a predefined range. Avoidance behavior can distribute agents in the swarm, thus increasing coverage of the network and reducing crowding of agents. In a particular embodiment, each pair of adjacent agents can select a course perpendicular to their mutual collision course. This avoidance behavior divides pools of connected unsearched cells that are bounded by searched cells. In some embodiments, if there is more than one agent within the avoidance range, the search system avoids only the closest one. Gradient descent is selected when an agent is located at a searched cell that is not a boundary cell. Gradient descent facilitates moving the agent to the next nearest unsearched cell by drawing agents toward pools of unsearched cells. Finally, random movement is selected when none of the other behaviors produce a move for an agent. For example, when an agent is repelled by avoiding another agent against the boundary of the digital pheromone map, a random move may be selected.
In some aspects, contour following is guided by defining four-connected pools. A four-connected pool is a set of cells whose distance to the next unsearched cell is one cell. Although an agent can move diagonally, such a move could cause the agent to move diagonally from one unsearched cell inside a pool to another unsearched cell outside the pool in areas where the pool contains a diagonal boundary. By defining a pool based on four-connected cells and guiding the contour following behavior using these pools, the search system can prevent agents from leaving a pool with unsearched cells by an otherwise valid move. Staying within the pool prevents agent crowding and facilitates the completion of the search.
In some cases, the search system defines a minimum distance between agents when using gradient descent and adjusts the distance based on current conditions. This distance creates a repulsive force between agents that minimizes crowding and avoids collisions. However, sometimes the repulsive force prevents agents using gradient descent to get to the same pool of unsearched cells from entering the pool. Therefore, in some embodiments, if a pool is not getting smaller, the search system reduces the repulsive force by reducing the minimum distance between the agents until one of the agents is able to enter the pool and search the cells.
In some instances, the search system selects a size for cells of the digital pheromone map based on the footprint of a sensor carried by or otherwise associated with the search agent. For example, the search agent may be fitted with an infrared sensor that is able to detect heat in the environment within a 100-foot by 100-foot area. The search system may select a cell size equal to the sensor footprint or slightly smaller than the sensor footprint to allow for some overlap as agents pass over the section of the geographic area represented by each cell.
Aspects of the search system may mark searched cells of the digital pheromone map with an identifier of the search agent that searched the cell and the time that the cell was searched. This information can be used to detect areas that are stale and need to be searched again (e.g., for time-dependent searches), or can be used to detect areas that may need to be searched again when a particular agent is determined to be unreliable (e.g., such as when a faulty sensor is detected).
In some embodiments, the search agents form an ad hoc network. Each agent has an independent copy of the digital pheromone map and executes its own search strategy. When in range, search agents synchronize their copies of the digital pheromone map to determine areas searched by other agents. For example, two agents may start out at opposite sides of an island to be searched, and may be out of communication range with each other. As the agents independently search the island, they will eventually come into communication range and exchange data about the cells that each agent has searched. The agents then determine the next cell to search based on the remaining cells not searched by either agent. As agents come into communication range and synchronize map information, they spread the coverage of the ad hoc network formed by all the agents by propagating updates outward. The propagation of updates eventually connects separate subsets of the swarm.
In some cases, search agents exchange summaries of their copies of the digital pheromone map before performing a full synchronization with each agent. For example, the search system may assign a serial number to each agent that is incremented each time the agent makes an update to the digital pheromone map. Each agent can then exchange a list of agents and serial numbers that indicate what information that agent has received from each of the other agents. For example, if agents A and B exchange information that indicates that both have the same serial number from agent C, then it is not necessary for A and B to exchange information received from C because each is equally up to date with respect to C. The list of agents and serial numbers forms a key that summarizes the state of the instance of the digital pheromone map held by a particular agent. This method allows each agent to optimize the agents that it synchronizes with and to reduce the information requested from other agents during synchronization. In other embodiments, other pluggable, incremental algorithms for updates may also be used.
In some embodiments, the search system optimizes the agents that each agent synchronizes with. For example, an agent may choose to only synchronize with nearby agents, such as by applying a Delaunay filter, because nearby agents are most likely to have relevant information to the requesting agent's next move when there are still unsearched cells in close proximity. A Delaunay filter is a triangulation technique that removes neighboring agents that are not nearest neighbors from the list of agents to synchronize with. This allows each agent to synchronize with the smallest number of agents and results in a network topology between the agents that is an approximate minimal spanning tree. As another example, an agent may sort the summaries received from agents by the number of differences with the agent's own copy of the digital pheromone map. The agent can then choose to synchronize first with those agents having the most differences. This method may eliminate the need to synchronize with many of the other agents that have information that is redundant with the information received from the first agent.
In some aspects, the search system does not update the distance gradient of the entire digital pheromone map following a received update. Common optimization techniques may be used by a particular agent to reduce the area of the map that needs to be updated. For example, a wavefront propagation method based on Dijsktra's algorithm can be used to compute updated distances to unsearched nearby cells following an update while ignoring faraway cells.
In some cases, the search system instructs agents to operate in a post-search mode. Searching may only be the first phase in a sequence. For example, once a target is found, agents may switch to a target-destroying mode. In post-search modes the agents may no longer use the digital pheromone map, but may instead switch to control by a human operator or other control schemes applicable to the new mode.
The computing device on which the system is implemented may include a central processing unit, memory, input devices (e.g., keyboard and pointing devices), output devices (e.g., display devices), and storage devices (e.g., disk drives). The memory and storage devices are computer-readable media that may contain instructions that implement the system. In addition, the data structures and message structures may be stored or transmitted via a data transmission medium, such as a signal on a communication link. Various communication links may be used, such as the Internet, a local area network, a wide area network, a point-to-point dial-up connection, a cell phone network, an ad hoc network, and so on. The computing device may be connected to various peripheral devices such as avionics, sensors, weapons, and so on.
Embodiments of the system may be implemented in various operating environments that include personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, embedded processor-based systems, programmable consumer electronics, digital cameras, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and so on. The computer systems may be cell phones, personal digital assistants, smart phones, personal computers, programmable consumer electronics, digital cameras, unmanned aerial vehicles, and so on.
The system may be described in the general context of computer-executable instructions, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, and so on that perform particular tasks or implement particular abstract data types. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.
From the foregoing, it will be appreciated that specific embodiments of the search system have been described herein for purposes of illustration, but that various modifications may be made without deviating from the spirit and scope of the invention. For example, although search agents have been described in the context of UAVs, the techniques described may also be used in other contexts such as space exploration or undersea searches. Similar techniques may also be used to assist a human pilots in manned vehicles. Likewise, although searching has been described, similar techniques may be useful for tasks other than searching, such as crop dusting where a substance is distributed over a geographic area. Searches may also be performed on an ongoing basis, such as to track weather patterns where conditions of the environment change over time. Accordingly, the invention is not limited except as by the appended claims.
The present application claims priority to U.S. Provisional Application No. 60/686,362 (Attorney Docket No. 03004-8203US) filed on Jun. 1, 2005 and entitled “EXHAUSTIVE SWARMING SEARCH STRATEGY USING DISTRIBUTED PHEROMONE MAPS,” which is hereby incorporated by reference.
| Number | Date | Country | |
|---|---|---|---|
| 60686362 | Jun 2005 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 11440626 | May 2006 | US |
| Child | 12554788 | US |
