Patent Application
20100134285
Physical security systems that employ automated surveillance and/or intrusion detection, to secure a perimeter or volumetric area of a site, are intended to operate autonomously until an event occurs which indicates a possible security breach. At this point, a human operator is automatically alerted.
For any physical security system it is desirable to have low occurrences of false alarms and low occurrences of failures to detect an intruder. Measures taken to reduce one almost inherently increase the other. Failures to detect can be reduced by layering of sensors, that is, using multiple types of sensors and deploying them with overlapping detection areas. Simply “OR-ing” the detection outputs of these sensors then decreases detection failures but increases occurrences of false alarms. Of course, simply “AND-ing” the detection outputs of sensors with overlapping detection areas has the opposite effect.
While low in cost, the commonly used contact sensors (e.g., electric eyes), proximity sensors (e.g., passive infrared (PIR)), and video differential sensors, are all prone to high false alarm and nuisance alarm rates due to a variety of environmental and natural triggers. Such triggers include, for example, movements of small or large animals, wind blown debris, moving vegetation, temperature gradients, moving clouds, rain, snow, and moving water.
Many monitored sites are unattended and remotely located. Conventional security doctrine for military installations requires dispatching a team of military police to a site to directly assess the situation whenever a sensor is activated. False and nuisance alarms impose a significant cost in manpower and material. Sensitivities to smaller targets or environmental conditions can be reduced for existing sensors but only at the expense of reducing their ability to detect actual intrusions.
Accordingly, there is a need for improved security systems and methods that overcome the above deficiencies.
The present invention is related to a method of sensor data fusion in a physical security system of a monitored area. The method comprises receiving data inputs from one or more sensors in the physical security system, with the monitored area being overlaid with a grid defining a plurality of locations. The method further comprises selecting one or more potential paths of one or more potential intruders through the monitored area using an iterative process, which takes into consideration a sequence of sensor inputs and assumed limitations on the mobility of the one or more potential intruders. A confidence metric is then produced for each selected potential path.
Features of the present invention will become apparent to those skilled in the art from the following description with reference to the drawings. Understanding that the drawings depict only typical embodiments of the invention and are not therefore to be considered limiting in scope, the invention will be described with additional specificity and detail through the use of the accompanying drawings, in which:
The present invention is directed to a method of sensor data fusion for physical security systems having automated surveillance and/or intrusion detection. The present method utilizes an algorithm for performing sensor data fusion in the physical security application to reduce the occurrence of false and nuisance alarms. The method intelligently considers the indications of a number of sensors as they occur over time, and uses the combined and processed inputs as the basis for determining the likelihood that an actual security breach has happened.
The subject invention is applicable to various physical security areas, allowing the use of any number and variety of detection devices, and supporting continuous autonomous operation. In addition, the present approach includes numerous techniques for reducing processing load in order to improve the feasibility of implementation.
In general, the algorithm of the invention selects a set of possible paths of one or more intruders through a monitored area that have the highest likelihood, taking into consideration a sequence of sensor observations and the assumed limitations on the mobility of a potential intruder. The algorithm produces a confidence metric for each selected path that can be used both for a subjective decision regarding an actual intrusion and to provide its relative significance versus other paths.
For example, if the security system is concerned with human intruders on foot, a bird flying through the area would not cause an alarm because its velocity would be well outside the limits of a human on foot. As another example, passive infrared sensors can be activated by wind motivated vegetation. However, these activations are unlikely to correspond to a sequence produced by a human moving deliberately through the area. The algorithm would therefore assign a low metric to paths of a target that could cause this observed sequence of activations. The assumed mobility limitation can include direction of travel and lower velocity limits (e.g., slowly moving cars versus cars moving at highway speeds).
The subject invention provides a way to consider the indications of a variety of sensor types including first generation sensors (e.g., direct contact sensors such as switches, magnetically activated relays, electric eyes, etc.), second generation sensors (e.g., remote sensing such as passive infrared, and microwave proximity such as buried cable and fence motion sensors), and third generation sensors that have target location estimating capabilities (e.g., radar, LIDAR (light detection and ranging), video analytics, and seismic sensor arrays). The indications are considered in a systematic and consistent manner that is transparent to the type of sensor. This allows the algorithm to be used with any suite of sensors and to be easily modified to include new types of sensors as they become available or as the suite of sensors in a particular installation is modified.
The algorithm of the invention overlays the area of interest with a grid and defines the “state” of the system to be the grid location of a potential target (intruder). The system has a finite set of discrete states where each state corresponds to a specific grid location. The system state can be expanded to include target velocity and even acceleration provided that these are quantized so that the complete state remains finite. A “possible” state of the system (since the actual state is unknown) at an iteration (point in time) is represented as a value from this set. Using the term “iteration” is more accurate than using the term “time” because there is no assumption regarding time relationships other than that time increases monotonically with each iteration.
The term “state[i]” used herein refers to one of a finite set of discrete non-overlapping and contiguous areas. In this sense, any location (set of x-y coordinates) has an associated state value which is the state value assigned to the distinct area of the grid containing that location. No matter where a target is actually located in the monitored area, it has exactly one associated state which is the state assigned to the area in which it is located.
The algorithm defines a particular “path” of a potential target at any iteration of the algorithm as a sequence of time-stamped states. The algorithm considers all possible paths, that is, all possible sequences of state values. If the number of discrete states is M, the number of possible paths of length I is MI. For a reasonable number of states (e.g., 1024 for a 32×32 grid) the number of paths to consider exceeds the limits of practicality with just a few observation intervals. The present algorithm overcomes this by applying the Viterbi algorithm, which is described in Viterbi, “Error bounds for convolutional codes and an asymptotically optimum decoding algorithm”, IEEE Transactions on Information Theory 13(2):260-269 (April 1967), the disclosure of which is incorporated herein by reference. This reduces the total number of potential paths to retain for evaluation to a constant value of M*N regardless of the number of iterations. Here N is the number of potential paths in the set to be yielded by the algorithm as having the highest likelihood.
All that is needed to incorporate a particular instance and type of sensor into the algorithm is a function that provides a metric of the likelihood that the particular sensor indication would occur given that there is a target at some given location (i.e., some state location value). The independent variable for each of these functions is the grid location (state location value) of a potential target. For sensors that provide a binary output, as is typical of first and second generation sensors, this consists of two functions. One function yields the likelihood metric for the sensor indication being inactive given that there is a target at the location. The other gives the likelihood metric for the sensor's active indication when a target is at the location. For third generation sensors that provide an indication of suspected target location, a single function is needed that provides the likelihood metric for the sensor reporting a particular target position (or simply azimuth or range) given that the target is actually at the particular location given by the independent variable. These functions can be readily obtained for each particular type of sensor through its specifications and/or direct evaluation of its performance.
The method of the invention can be practiced by implementing in software all or a subset of the algorithm that is described hereafter. This software can be executed on any general purpose processor or set of processors having sufficient computing power and access in approximate real time to the data produced by the sensors. For example, low-cost custom hardware accelerators designed for high-speed video graphics calculations can provide the processing capability required to run the present algorithm in real time.
Further details of various aspects of the method of the invention are described hereafter with respect to the drawings and in the following sections.
The present fusion algorithm provides the operator with an automatically derived confidence level indication as to the presence or absence of one or more intruders. For example, the system may report something like:
Possible intrusion detected with confidence level of 8.
Possible intrusion detected with a confidence level of 6.
No other intrusions detected with confidence levels above 2.
The reported confidence level is only a value on a relative scale and is not related to a formal statistical definition. All that is needed is a metric whose relative value reflects how well the observed sequence of sensor activity corresponds to the presence or absence of an intruder. This is intended to assist the operator in deciding on the appropriate action in view of the system-wide status at the time. The present approach is a notable improvement over other algorithms which are limited to reporting only binary results from the fusion process. The present algorithm directly incorporates object location information provided by third generation sensors. The algorithm also makes use of the fact that a real intruder is a physical object with mobility limitations that restrict what can be considered to be reasonable sequences of observations. In other words, the algorithm attempts to make use of a more extensive history of detections by various sensors in order to assess the degree to which it is “believed” that this sequence has been caused by a particular type of intruder, such as a human on foot, a helicopter as opposed to a fixed wing airplane, etc.
The terms “probability” and “likelihood” are used in the following discussion to give an intuitive view of the reasoning behind the sensor fusion algorithm. In general, the values for these terms provide various metrics indicating a relative degree of confidence that some event has taken place in view of evidence provided by sensors and as processed by the fusion algorithm. These terms are used more or less interchangeably with the term “metric” in the discussion.
In the present approach, the area of interest is overlaid with a grid and the “state” of the system is defined to be the grid location of a target (intruder). Therefore, the system has a finite set of M discrete states:
Si i=1 . . . M,
where each state corresponds to a specific grid location.
A “possible” state of the system at an iteration (“point in time”) is considered to be one value of Si. Using the term “iteration” is more accurate than using the term “time” because there is no assumption regarding time relationships other than that time increases monotonically with each iteration. A particular “path” of a target is defined as the current iteration of the algorithm at some particular sequence of G states ending in the state.
The status of sensor k is represented as ek and the full set of detection status of K sensors as reported at the current iteration is represented as:
{e}={e1,e2, . . . ,eK}
It is necessary to determine a metric for a specific path that expresses a level of “belief” that the path was followed by a target given the complete history of sensor status observed over the most recent G iterations. This is expressed as the probability that a target has been at the sequence of locations defining a path given the history of sensor indications that have been observed over time. The assumption is that the target was at each of these locations on the path at the time a corresponding iteration of the algorithm was executed. It is not essential for this metric to be a true probability value. Instead it need only provide an indication of a relative level of confidence that this sequence occurred, given the sequence of sensor indications (“observations”) that has occurred over time.
In the basic approach, all possible sequences of state transitions of some length are considered and the metric for each one is determined. The paths with the highest metrics that also exceed some thresholds can then be presented to the operator along with the relative level of confidence in each.
A fortunate side effect of considering all possible paths and only attempting to assess their relative likelihood, rather than attempting to select “the” path from the outset, is that the algorithm is able to recognize the presence of multiple intruders (assuming they have multiple paths.)
In order to continuously provide a security assessment, it is necessary to update all path metrics each time there is a change in sensor status. Intuitively, it should be possible to avoid completely re-evaluating a metric for each possible path at each iteration and instead to determine the new metric iteratively. That is to say that the current path metric is ideally a function of the current set of sensor indications and the path metric for the “prefix” path. The prefix path is the path that includes all locations (states) except the location that is being added to this prefix as of the current iteration. This defines a path metric function that is based on a combination of:
the current set of sensor indications;
the newly appended state itself (representing a physical location); and
the metric that was previously determined for the first “prefix path”, i.e., the path minus the most recently added state.
An expression for the foregoing path metric function is the product of:
the probability that the target is at the newly added state (location) on the path given the set of sensor indications at that iteration; and
the probability of a transition being made by an expected target between from the most recent state on the prefix path and the newly added state during the time interval that has elapsed between the two iterations of the algorithm. Since the state is a two dimensional value, direction of travel between these two states may play a role in this as well as the distance and time interval.
From a practical standpoint, directly determining the probability that an intruder is at a location given the current set of sensor indications (assuming a mix of sensors of multiple generations) is all but impossible. On the other hand, if sensors do not interact with each other, determining values for the probability that a particular set of sensor indications would be observed given that the intruder is at a location is something that is at least approachable. According to Bayes equation:
To apply this to the fusion process, the elements of Bayes equation are assigned as follows:
P(A|B)=P(Si|{e}) This is the conditional probability that the target is at a state given the current set of sensor indications.
P(B|A)=P({e}|Si) This is the conditional probability of observing the current set of sensor indications given that the target is at a particular state.
P(A)=P(Si) This is the unconditional probability of the target being at the state.
P(B)=P({e}) This is the unconditional probability of the occurrence of the current set of sensor indications.
Substituting into Bayes equation gives:
Provided that the following two requirements can be met, this relation is an attractive candidate for the basis of determining a path metric:
P(Si)=P(Sj|{e})·P(Sj→Si)
where the two terms on the right are:
Therefore, the unconditional probability of the target being at the state is the product of the path metric for the prefix path and the probability of the transition from the most recent state in the prefix path to the current state. Intuitively this makes sense, saying simply that the probability of the target being at the current location is the product of the probability that the target was actually at the previous state at the start of the interval given the history of observations and transitions up to that time and the probability that a transition could take place from the previous state in the intervening time.
The complete expression for the path metric then becomes:
This effectively implements a recursion all the way back to the oldest state maintained by the path. This recursion effectively “rolls up” both the impacts of sensor detections and the likelihood of movement by the intruder. The general computational task then becomes the following:
The qualifier “possible” is used because it is reasonable to classify some state transitions as “impossible” based on the time interval and distance. An “impossible” state transition removes all paths terminating at a previous state as possible prefixes for paths that include a current state. Although less likely, it can also be decided that some observed sensor status is impossible if a target is at a given location. In this case, the location is not considered as a next state for all existing paths.
At each iteration, for each possible next state for each possible existing prefix path, the computational task is to evaluate the three terms on the right of equation (1).
The term P({e}|Si) is the probability that the observed collective sensor status would exist given the target state. As sensors are used that do not interact with each other, it is assumed that the sensor events have independent probability relationships. Therefore, the probability is simply the product of the probabilities for each of the individual sensor indications conditioned on the target being at a given location:
where P(ek|Si) is simply the probability of sensor k having the status that is currently observed, given that the system state is Si, i.e., the target is at the corresponding grid location.
The following sections describe the approaches used to determine the probability of the observations, meaning the information provided by the sensors, given a specific state, i.e., that the intruder is located in a specific grid cell. Different approaches are required based on the type of sensor.
The expression in the denominator of equation (1), P({e}), is a “normalizing” factor that effectively increases the path metric if the probability of the complete set of sensor indications is low, which is intuitively the desired result. A suitable value for this term can be calculated as part of the process of evaluating P({e}|Si) for each possible location (which is done at each iteration.) This value is simply the average of all these values.
If the individual cells of the grid are not all the same size, it would be better to compute a weighted average where each term is weighted by the relative size of the grid cell for the associated location. The result can be further improved by including in the summation a term to represent the probability of observing the collective sensor status given that there is no target in the grid.
A. Evaluating Binary Sensors
For a single binary sensor k (such as PIR, buried and fence), the probability of the sensor being active if the state (target location) is (xh,yh) is represented as:
Pb(xh,yh) (3)
Since (3) applies to just a single point in the horizontal plane, integrating over the dimensions of one cell and dividing by the area gives the average probability of the sensor being active given that the target is in a given cell.
where (Xi,Yi) are the coordinates of the center of the cell for state Sj and D is the length of the side of a cell. If binary sensor k is active:
P(ek|Si)=Pk(i) (5)
If binary sensor k is not active:
P(ek|Si)=1−Pk(i) (6)
The active case (5) appears to present a problem. If the location Si is not within the detection area of sensor k, it may not be reasonable to assign a low probability to the sensor being active because it could be responding to a real target at some other location. Likewise, if the location Si is within the detection area of sensor k, it may not be reasonable to assign a high probability to the sensor being active for the same reason. The inactive case (6) does not have the same problem. It is reasonable to assign a high probability if the location is not within the detection area and a low probability if the location is within the detection area. However, in reality, the algorithm has no notion of where an intruder may or may not be. With regard to binary sensors, the overriding objectives are:
Provided the probability values are never zero or one, the logic given in (5) and (6) appears to be able to achieve this objective. Since the path metric is computed purely as a product, injecting a probability value of zero effectively “kills” the path completely, eliminating any future chance of having its metric rise to compete with the metrics of other paths.
B. Evaluating Video Analytics
For detection information generated by video analytics, computing P(ek|Si) is a little more involved. Video analytics processing provides a periodic “object report” in which the estimated location (in pixels) in the video image plane is given for each tracked object. It can be assumed that there is some error in the reported location of the object so that there is a probability distribution for reported locations relative to the actual location of the object:
Pv(xe,ye|xv,yv) (7)
where (xe,ye) is the reported object location and (xv,yv) is the actual location, both expressed as pixel indices in the video image plane.
To incorporate the object reports into the fusion process, an expression is needed for the probability of a given reported location based on the object's actual location, both of these being in the state plane (typically earth's surface). To do this the image plane probability distribution is projected onto the state plane:
P
h(xe,ye|xh,yh)=Pv(xe,ye|Tx(xh,yh), Ty(xh,yh)) (8)
where (xh,yh) is the actual target location in the earth's surface plane and Tx(x,y) and Ty(x,y) transform horizontal plane coordinates into image plane coordinates. These transforms are geometric relations based on the camera location in three dimensions relative to the state plane and the camera orientation and field of view, which are assumed to be constants.
Since (8) applies to just a single point in the horizontal plane, in order to get the average probability for all locations in a state grid cell, it is necessary to integrate over the dimensions of one cell and divide by the area:
where (Xi,Yi) are the coordinates of the center of the cell for state Si and D is the length of the side of a cell. It should be noted that this does not require the camera's field of view to include the entire area covered by the location state grid. For cells whose state Si lies well outside the field of view, equation (9) should evaluate to zero, meaning video analytics will not report this object's location as being somewhere within its field of view.
For video analytics there is an important case that is not addressed by equation (9). For binary sensors, it is assumed that the detection status is always known. However, with video analytics it is necessary to deal with the case in which no object report is received. This can happen whether or not there is an intruder. For example, the intruder may be standing still or partially or fully hidden. One can argue that the absence of a video analytics object report should somehow result in a reduction in confidence levels meaning a reduction in path metrics. However, the reduction would have to be applied across the board to all currently possible states within the camera's field of view (since there is no basis for doing otherwise.) If the primary objective is to determine relative confidence levels for a set of paths, the simplest way to deal with the case of no report is to ignore it. This means that the video analytics object reports should be considered as part of the set {e} when the reports are received, and not part of the set otherwise.
Each video analytics report of a detected object may beneficially include a “track” identifier to indicate if video analytics believes the object is in fact the same one whose location was previously reported with the same identifier. This can be included in the fusion process using the following logic: If a path includes a state that previously fell within the probability zone of a video analytics object location, the reported track number is saved with that path. In evaluating the path metric, the transition probability is lowered if the saved track identifier does not match the identifier for the recently reported object for which the next state is being evaluated.
C. Evaluating Seismic Sensors
Analyzing the correlations of signals received by multiple point seismic sensors at the edge of an area can provide location information for a target inside the area. This can also provide location information for targets outside this area. However, it appears that limiting the detection to the direction of the target's location relative to the sensor set may provide more reliable results in this case. If use of seismic sensors is limited to analyzing the signals received by individual sensors, it may still be possible to provide an estimation of the range of the target from the sensor (based on signal amplitude). Otherwise, the fusion process would treat a seismic sensor as another type of binary sensor.
For each of the first three cases above (point location, direction and range), uncertainty in the analysis results is expressed as a complex two-dimensional probability distribution that is offset within the state grid. In the case of direction information, the distribution is also rotated based on the detection results.
It is assumed that seismic sensors are located at the edges of the protected area such that it is not necessary to include direction-only information in the fusion process. For the case of point locations, the problem is similar to that described for video analytics with the exception that the transform from ground to video planes is not needed. A relation is needed for the probability of a location being reported given some actual location:
Ps(xe,ye|xh,yh)
where (xe,ye) is the reported object location and (xh,yh) is the actual location.
Again, in order to get the average probability for a state grid cell, it is necessary to integrate over the dimensions of one cell and divide by the area.
where (Xi,Yi) are the coordinates of the center of the cell for state Si and D is the length of the side of a cell.
For the case of range-only information provided by single sensors, the probability distribution has only a single dimension, i.e., range, but, assuming the speed of sound through the earth is the same in all directions, it has angular symmetry:
Ps(de|dh)
where de is the reported object distance from the sensor and dh is the actual distance of the object from the sensor. The following is then used to obtain the average probability over an entire cell.
where (Xk,Yk) is the location of seismic sensor k.
The second term in the numerator in equation (2), the transition probability, P(Sj→Si), indicates the likelihood that an intruder could have moved from a previous location (Sj) to the current location (Si) during the time that elapsed between iterations. Both locations could be the same, which is entirely possible for a stationary or slowly moving object. Both target velocity and acceleration can be estimated based on previous states in the path:
where Ti is the clock time at the current iteration, etc., so that:
The actual function for FΔ may have to be derived empirically and then experimentally verified. It may end up being fairly involved. For example, it may be useful to reduce the probability for acceleration direction and magnitude as velocity increases. Given the real world limits on human mobility, it is reasonable for FΔ to evaluate to zero in some cases. The function may also take into account the direction of motion relative to the current position, taking advantage of the likely case of intruders having specific location objectives.
Since video analytics is not intended to detect a stationary object, in the case where no detection is reported by video analytics, it is possible to modify FΔ so that it falls off much more rapidly with velocity. In other words, there would be a strong preference for all states to transition only to themselves. Similar logic could be used when detections of the same object (same “track ID”) are reported in sequence by changing FΔ to favor velocities implied by the distance between the reported positions and the time interval between reports.
As indicated previously, no assumptions have been made regarding time intervals, etc. A question remains then as to what are the circumstances that initiate an iteration. In general, path metrics should be evaluated whenever new information is available from one or more sensors. For a binary sensor, this happens whenever there is a change in the detection status. For a video analytics source, it is receipt of a report of the objects detected in a video frame. For a seismic sensor, it could be receipt of a report that is sent periodically whether or not there has been a change in the results of its acoustic signal analysis. A report that a seismic sensor is not detecting any activity may be information that the fusion process should consider. Ideally, an iteration should be done for each of these events individually.
At the other extreme is the case where the site is completely secure and none of the sensors are reporting new information or one or more binary sensors are reporting the same state continuously. In this case, in order to gradually age out the older sensor inputs, iterations should continue based on some maximum time interval. This is a “polling” initiation as opposed to one that is event driven.
In general, a combination of event-driven and polling initiation of an evaluation should be used so as to bound the maximum time between iterations and to provide immediate assessment when there is sensor activity.
Real-world physical security systems must operate continuously and detect an intrusion that may occur at any point in time. In this environment, there can be no “time=0” at which metric evaluation for every possible path begins. Obviously, the metric evaluation time span for a path approximating that of a real intruder should be closely aligned with the time during which the intrusion occurs. In order to accommodate this dynamic environment, a given path must have beginning and ending times. Outside of this interval, the path's history of locations is not of interest because it is unlikely that they match those of a real intruder. One technique of tracking a changing environment that has been applied successfully is to simply ignore information whose age exceeds some limit. This has been referred to as “aging out” the data.
The path metrics should reflect only relatively recent sensor events. This can be done by removing the contributions to the path metric of the locations on the path for which the time that has elapsed since those locations were added exceeds some threshold. Since all paths are updated at each iteration, this has the effect of truncating all paths to some length G (the number of locations in the set that constitutes the path) which can be done simply by selectively removing the oldest number of locations from each path at each iteration based on the time that has elapsed since that location was added. With this approach, when there is low or no sensor activity, the time intervals between nodes will be at the maximum value as determined by the polling interval. Paths will be truncated to shorter lengths (lower values of G) when limiting the maximum “age” of the oldest nodes on each path. This is appropriate because there will be little or no change in the metrics between iterations. Conversely, when there is a lot of sensor activity, there will be a larger number of locations on each path since the time intervals between adding these is shorter (because evaluation is event driven by sensor activity.) Again this is appropriate in that it allows the algorithm to retain and thereby consider more information at this time.
As described so far, the algorithm gives no consideration to the ordering of the observed and derived values of P({e}|Si), P({e}) and P(Sj→Si) versus time. Intuitively, it may be desirable for more recent events to have more weight in determining the metric. Again, since we are concerned only with relative values of the path metrics, this can be done fairly easily. If all paths have the same length, a single weighting factor can be maintained for all paths. This factor is increased by some amount at each iteration. Because the factors in equations (1) and (2) are combined via multiplication, the weighting must be done by raising the factors that depend on the current information to a power. This changes equation (4) to the following:
where fW(i) is the weighting factor that decreases for the older locations on the path.
The cost of using this weighting is that equation (2) cannot be computed iteratively using the metric for the prefix path because this metric changes at each iteration due to age-weighting of the locations on the path.
It is useful to consider what indications the algorithm can be expected to give when the monitored area is secure and no sensors are active. When a binary sensor is off, the path metric is decreased for the locations that are inside the sensor's detection area. The logic is that the probability of an off indication is low if a target is in one of these locations. If all sensors are off, locations that are in the detection area of two sensors will have their metrics lowered more than those in the area of only one sensor. The overall result will be that the areas with the highest path metrics will be those for which the sensor coverage is weakest. The algorithm is saying simply that, given the current sensor states, if there are any intruders, they are most likely to be in the areas that have the worst coverage. If scaling and thresholds are set correctly, the highest path metrics will be reported with confidence levels that are low enough that the site can be considered to be secure. If, in this quiet state, a single sensor becomes active, the metrics for the paths through the locations that are in this sensor's detection area will be increased relative to the others.
If the number of discrete states is M, the number of possible paths after I iterations is MI. For a reasonable number of states (1024 for a 32×32 grid) the number of paths to track goes beyond the limits of practicality with just a few iterations. By using the Viterbi algorithm, and making one key assumption, it is not necessary to keep track of all possible paths. In fact, to track a single target, it is only necessary to maintain one path for each of the discrete states, no matter how many iterations there are.
The key assumption is that the change to the path metric as a result of a particular state transition depends only on the starting and ending states of that transition. Consider the new set of paths formed by the set of paths entering a state and one path exiting this state. The extension of each of the entering paths by the exiting path will result in a change to each path's metric. If we can assume that this change depends only on the starting and ending states of the new path segment, then the metrics for all of these paths will be changed by the same amount. Therefore, since the change in the metric due to the exiting path is the same for all paths in this set, within this exiting set of paths, the path with the highest metric must be the one that includes the entering path having the highest metric. This is true for any path exiting the state. At each iteration, each state that is reachable can be reached by up to M paths. It is then necessary to determine the new metric for each of these paths. However, once this is done, only the path having the highest metric needs to be remembered. All of the remaining paths can be disregarded. Therefore, no matter how many iterations are made, to track a single target, a maximum number of M paths needs to be retained for consideration at the next iteration.
Tracking Multiple Targets with the Viterbi Algorithm
In order to track a maximum of R targets, the Viterbi algorithm must be modified so that each state retains R paths. At each iteration, there will be M·R possible paths that reach any given state of which the paths having the R highest metrics are retained. The present algorithm uses this approach to keep track of the R paths terminating in each state having the highest metric. In general, a maximum value of R must be set at design time but lower values can be used in operation depending on the circumstances.
As stated previously, path metrics can be kept more “current” by removing locations on the path based on the time elapsed since these were added and/or weighting metrics based on elapsed time. In either case, the metric of the affected path is reduced. As discussed above, when using the Viterbi algorithm, a limited number of paths (R) is maintained for each state. Reducing the metric of paths whose “strongest” nodes occurred prior to the current aging threshold (or farther back in time) effectively improves the competitive position of all candidate paths now being considered to be one of the R paths to be retained for that state.
Thus, reducing the strength of existing paths based on elapsed time enables the emergence of new paths having stronger current metrics and this in turn leads to the demise of older paths since only the strongest set of paths will be retained for each state. True paths, i.e., those that closely correspond to an intruder's motion, should be able to retain their competitive stance as the boost given to metrics by more recent sensor activations compensates for the loss of the contributions of the older locations. False paths, however, will not have the benefit of this “replenishment” and consequently should be reliably deleted.
It may appear that there is a need to provide an initial value of the path metric to start the recursion in equation (2). However, at the time a path is selected for inclusion in the set of R paths that are retained by a state, this path is already one of the R full length paths held by each of the other M states. So the path is already initialized. The only time this is not true is for the first G-l iterations. What is needed then, is to establish this complete set of M*R G-length paths at initialization. This can be easily done while the monitored area is free of all intruders so that all sensors remain inactive.
An initial value of G is set based on the path node age limit and the polling rate. A complete set of R paths each including G locations are established for each of the M states. The metrics are all set to their lowest values (i.e., unity). After this, the algorithm is run with periodic iterations initiated via polling for several times the maximum path length. This achieves the desired steady state condition for no intruders in which all of the path metrics are based on the sensor coverage for their associated locations as described previously.
Considering Acceleration when Using the Viterbi Algorithm
The approach to calculating the transition probability P(Sj→Si) as discussed previously violates the key assumption that enables use of the Viterbi algorithm. This is because two states prior to the new one are needed in order to calculate the acceleration vector
In order to use acceleration in determining transition probability, the velocity vector
must be included as part of the state and, since the state must be discrete, the velocity vector needs to be quantized (in two dimensions.) Even with fairly coarse quantization (e.g., 4-8 levels), this would inflate the already substantial state space to a size that may be impractical. To avoid this and still use the Viterbi algorithm to keep the number of tracked paths to a manageable value, acceleration is not used. The transition probability function (12) is redefined as:
Intuitively, acceleration may not be especially useful for tracking a human on foot and attempting to use acceleration may provide a way for an intruder to defeat the fusion algorithm by deliberately moving erratically.
It can be assumed that there is some reasonable maximum velocity for a human on foot. If the evaluation intervals are sufficiently short, for any given next state the number of possible previous states is limited by the distance that would have to be covered during the evaluation interval. This can significantly reduce the number of state transitions to evaluate at each iteration. However, it does not reduce the number of paths that have to be maintained. Since it is assumed that the true location cannot be known for certain, all states are always potential next states. The assumptions about mobility locations can only limit the number of paths that can potentially lead to a given state.
A world-class sprinter may cover up to 10 meters in one second. If the evaluation interval is one second, the protected area is a square 50 meters on a side, and the state grid is 32×32, the maximum number of previous states is about
This is an 87% reduction versus the 1024 previous states per current state for which the metric would have to be evaluated otherwise.
The following explains the approach employed to calculate the path metrics that are used to determine which paths are retained as prefix paths for future paths and which provide an indication of the likelihood that an alarm condition exists as of the current iteration. Also, discussed is how the path metric is calculated within the sequence as embodied in the flow diagrams of
The complete path metric is calculated according to the following:
For the sake of efficiency, to find the criterion for selection of the set of new paths that are to be retained for a particular state, only the portion of this metric which is unique to a new path is initially evaluated for each possible new path. This is:
NewPath.Metric=NewPathEvaluationMetric=StateTransitionMetric*PrefixPathMetric
After the set of retained new paths entering a State has been determined, the metrics for the retained paths only are corrected for the current set of sensor indications as they apply to this State.
NewPathSet[r].Metric=NewPathSet[r].Metric*TargetIsAtStateMetric[i]
After the retained paths for all States have been determined, their partially computed metrics are “normalized” for the current set of sensor indications to produce the complete path metric as follows:
NewPathSets[i][r].Metric=NewPathSets[i]][r].Metric/CurrentSensorIndicationsMetric
In order to support time-weighting of sensor indications and past transitions so that the older these get the less impact they have on the path metric, the StateEntryMetric is maintained for each PathNode. This is the portion of the path metric that is independent of the prefix path and reflects only the conditions at that iteration that pertain to a transition into that State location.
NewPathSets[i][r].Nodes[1].StateEntryMetric=TargetIsAtStateMetric[i]*StateTransition Metric/CurrentSensorIndicationsMetric
The full path metric without time weighting can be calculated as the product of the StateEntryMetric value for each PathNode on a Path.
The following table summarizes the steps carried out by the fusion algorithm of the invention according to the flow diagrams of
Instructions for carrying out the various process tasks, calculations, and generation of signals and other data used in the operation of the methods of the invention can be implemented in software, firmware, or other computer readable instructions. These instructions are typically stored on any appropriate computer readable media used for storage of computer readable instructions or data structures. Such computer readable media can be any available media that can be accessed by a general purpose or special purpose computer or processor, or any programmable logic device.
Suitable computer readable media may comprise, for example, non-volatile memory devices including semiconductor memory devices such as EPROM, EEPROM, or flash memory devices; magnetic disks such as internal hard disks or removable disks; magneto-optical disks; CDs, DVDs, or other optical storage disks; nonvolatile ROM, RAM, and other like media; or any other media that can be used to carry or store desired program code in the form of computer executable instructions or data structures. Any of the foregoing may be supplemented by, or incorporated in, specially-designed application-specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). When information is transferred or provided over a network or another communications connection (either hardwired, wireless, or a combination of hardwired or wireless) to a computer, the computer properly views the connection as a computer readable medium. Thus, any such connection is properly termed a computer readable medium. Combinations of the above are also included within the scope of computer readable media.
The methods of the invention can be implemented by computer executable instructions, such as program modules, which are executed by a processor. Generally, program modules include routines, programs, objects, data components, data structures, algorithms, and the like, which perform particular tasks or implement particular abstract data types. Computer executable instructions, associated data structures, and program modules represent examples of program code means for executing steps of the methods disclosed herein. The particular sequence of such executable instructions or associated data structures represents examples of corresponding acts for implementing the functions described in such steps.
The present invention may be embodied in other specific forms without departing from its essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is therefore indicated by the appended claims rather than by the foregoing description. All changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope.
