Patent Grant
8102260
The invention relates to methods, systems and devices for detecting threatening objects passing through a security screening system.
The goal of detecting and locating threatening objects or items such as weapons has increased in importance as society becomes more violent. In response to this goal, security screening systems have become more prevalent and are being used in facilities and places where the need for screening was previously not considered necessary. To increase safety while keeping public inconvenience at a minimum, the focus of the security screening industry is to increase the accuracy of distinguishing between threatening and non-threatening objects while maintaining a high throughput.
Exemplary security screening systems (also referred to as “system(s)”) are configured to rely on passive magnetic sensors or magnetometers to detect threatening objects. Such configurations of security screening systems depend on the unvarying and uniformity of the Earth's magnetic field to operate effectively. That is, passive magnetic sensors (also referred to as “sensor(s)”) define a sensing region that extends into a portal passageway of the systems for detecting disturbances or variances in the uniformity of the magnetic field of the Earth. The variances in the magnetic field are called gradients. Exemplary weapons and/or threatening objects are routinely formed from ferrous or ferromagnetic material (iron). As ferrous or ferromagnetic material passes through a portal passageway, the Earth's magnetic field is disturbed or varied and is registered by the passive sensors. That is, the sensors detect this change or variance in the Earth's magnetic field as a gradient and output a response that is configured as a voltage signal. The security screening system interprets the gradient (voltage signal) as the detection of a ferrous object. In this manner, the security screening system indicates the presence of a potential weapon(s) within the portal passageway of the system.
However, the Earth's magnetic field varies slowly, and randomly, over a period of time that interrupts the operation of security screening systems based on passive sensor configurations. For example, the periodic rising and setting of the Sun causes diurnal variations to the Earth's magnetic field. Additionally, unpredictable solar flares and magnetic storms produced by the Sun randomly impact and vary the uniformity of the Earth's magnetic field. These influences are referred to as “far-field disturbances.” Furthermore, “local disturbances” can influence and vary the uniformity of the Earth's magnetic field. Exemplary local disturbances include man-made objects such as wheelchairs and cars, and even larger ferromagnetic objects such as airport subways.
Security screening systems are designed to compensate for these far-field and local disturbances. However, baseline responses produced by the sensors of the systems tend to wander over a period of time as result of these far-field and local disturbances. Additionally, electronic noise and instability inherent in the sensors combine with the far-field and local disturbances to compound the detrimental effects on operational capabilities of security screening systems.
Accordingly, there is a need to provide data analysis methods and detection/location methods for security screening systems to compensate for far-field disturbances, local disturbances, electronic noise, and instability inherent in the sensors. Moreover, there is a need to improve the signal-to-noise ratio of the magnetic sensors with data analysis methods and detection/location methods that compensate for DC drift and single-point response spikes, which are induced or outputted by magnetic sensors of security screening systems.
Some aspects of the invention provide methods for detecting threatening objects. One exemplary detecting method comprises the step of classifying unique features of magnetic data as representing a threatening object. Another step comprises acquiring magnetic data. Still another step comprises determining if the acquired magnetic data comprises a unique feature.
Another aspect of the invention comprises an exemplary security screening system. The system includes a portal structure defining a passageway. The system further includes an array of magnetic sensors arranged in the portal structure and configured to output magnetic data. The system includes a camera positioned to photograph the passageway. The system includes a processor coupled to each magnetic sensor.
Still another aspect of the invention includes a method for classifying magnetic signature data as representing specific objects. An exemplary classifying method comprises the step of simulating security screening scenarios by passing objects through a security screening system. Another step includes collecting magnetic signature data that is representative of the objects. Still another step comprises extracting features from the magnetic signature data that distinguish respective objects. Still further, another step comprises performing a pre-classification optimization method on the features.
Preferred embodiments of the invention are described below with reference to the following accompanying drawings.
This disclosure of the invention is submitted in furtherance of the constitutional purposes of the U.S. Patent Laws “to promote the progress of science and useful arts” (Article 1, Section 8).
Referring to
Still referring to
Still referring to
Still referring to
Still referring to
Moreover, in some embodiments, system 100 can optionally include one or more trigger devices 117 that signal when a person or object is approaching the entrance and leaving the exit of passageway 108 of portal frame 106. Activating trigger device 117 prompts system 100 to initiate a screening or measurement event and obtain magnetic data of the person or object passing through system 100. Alternatively, system 100 can be prompted by other methods and means. For example, a person operating system 100 can manually initiate a screening or measurement event and obtain magnetic data.
Referring to
It should be understood that as the ferrous object passes within the scanning region of the first magnetic sensor (and sensing or screening region of system 100). The first magnetic sensor senses, measures, outputs and/or registers the gradient or change in the orientation of the Earth's magnetic field. The sensed gradient is outputted as a magnetic signal or response, collectively over the period of time termed magnetic data, and illustrated as response curve 156 of
For example, still referring to
For example, referring to
Referring to
Data analysis methods according to various exemplary embodiments of the invention are described, which negate or null the DC components or offsets caused by the large and small environmental influences on the ambient magnetic field. Additionally, data analysis methods according to various exemplary embodiments of the invention are described to detect and locate ferrous objects passing within the screening region of the security screening system 100. These exemplary data analysis methods comprise detection and location methods that increase the operational capabilities and selectivity of security screening systems.
An exemplary data analysis method according to one of various embodiments of the invention is appropriately termed the “feature extraction method.” The feature extraction method is performed on the magnetic data received from the security screening system 100 wherein each magnetic sensor (also referred to as “sensor”) detects or senses a gradient, individually. The feature extraction method processes the magnetic data or raw magnetic data (output signals or responses of raw gradient data) from each sensor. In exemplary various embodiments of the feature extraction method, three separate and distinct values are reached: 1) a summary gradient value for each sensor; 2) a total power value of the gradient signal detected by each sensor; and 3) a dimensionless ratio of time value configured as the first instant in the time period window that each sensor detects an object over or relative the entire time period window.
Referring to
Still referring to
Still referring to
Still referring to
Referring to
Still referring to
Still referring to
Still referring to
Still referring to
Still referring to
Referring to
Still referring to
Still referring to
Still referring to
Still referring to
The above exemplary various embodiments of the feature extraction methods are completed and provide individual magnetic sensor data that is summarized using the “features” data computed above. Various other embodiments of data analysis methods are now described that verify detection and provide location information for a ferrous object within portal passageway 108 of system 100. These additional data analysis methods can be characterized as the “composite portal analysis and object location methods” (hereinafter, also referred to as the “object location methods”). The object location method is directed to determining the location of a ferrous object within a passageway wherein the location includes a vertical aspect relative the ground level and a horizontal aspect relative a lateral distance from at least one sensor or sensor array (alternatively stated, relative a lateral distance from one column of magnetic sensors).
To illustrate various exemplary embodiments of the object location methods, the computations to be described were based on output responses from sensors in a security screening system, such as system 100, measuring or sensing a ferrous object positioned in a portal passageway (for example, portal passageway 108) at the following location: 1) a ferrous object (hereinafter, also referred to as an object) placed in a front shirt pocket of a person passing through portal passageway 108 of system 100 (
Referring to
Still referring to
Still referring to
Still referring to
Still referring to
Referring to
Still referring to the graphical representation 530 of
The above computation finishes the initial vertical position determination of the ferrous object according to the first exemplary step 501 of the object location method 500. A horizontal aspect or position of the ferrous object can now be determined. After determining this horizontal aspect of the ferrous object, a data analysis method is presented which computes a final vertical position of the ferrous object.
It should be understood that horizontal position is defined as a horizontal distance between a ferrous object and a magnetic sensor or column of either one of the pairs of arrays 132 and 134 of system 100. For example, returning to
To determine the horizontal aspect of the ferrous object, begin with the peak signal power values (also referred to as “integrated signal power peaks”) computed and interpretations realized in respective sub-step 508 and sub-step 510 from the first exemplary step 501 of the object location method 500 (
The 1/r2 model mentioned above is represented by the following equation:
where:
This equation will estimate the behavior of the near-field disturbance Io (signal power value) and its intensity as a function of horizontal distance from the ferrous object. The premise is that the integrated signal power I (signal power value I) of the magnetic field at the magnetic sensor is proportional to the inverse of the distance squared from the ferrous object. The horizontal aspect is determined by noting the measured or calculated integrated signal power (signal power value) at both sides of the portal structure for the integrated signal power peak(s) of interest and solving for the integrated signal power (signal power value) at the ferrous object using gradient (magnetic) data from both sides of the portal structure (in
Accordingly, determining the initial horizontal position aspect of the object location method 500 comprises rearranging the
equation for both columns of sensors (response curves 538 and 536 of
The quadratic equation uses the left side or column of the portal structure as a reference point (or zero point) with horizontal distance “r” increasing as a distance from the left side increases (and alternatively as distance to the right side of the portal structure decreases). It should be understood that the right side or column of the portal structure could have been used as the reference point wherein horizontal distance “r” would be represented as a negative (−) value (negative in sign). Selecting the left side or column of the portal structure as the reference point will result in a more conventional coordinate system. Horizontal distance “r” is a variable that spans the entire width of the passageway of the portal structure.
Accordingly, solving the quadratic equation provides the horizontal distance “r” of the ferrous object relative a sensor in the left side or left column of the portal structure. Accordingly, the ferrous object was detected as existing in the portal passageway, and an initial vertical position and a horizontal position of the ferrous object within that portal passageway has been determined.
Relying on the 1/r2 model just described, another embodiment of an exemplary data analysis method is described for adjusting the initial vertical position of the ferrous object, that is, a final vertical position. The initial vertical position of the ferrous object was determined as having the same vertical position as a vertical position of one of the sensors. That is, no determination of the vertical location or position of the ferrous object between respective, vertically spaced sensors. Accordingly, vertical adjustments are made using the 1/r2 model and comparing the measured magnetic disturbances between respective vertically spaced sensors next to or surrounding an identified peak signal power value (integrated signal power value). Between the two sensors, the one sensor outputting the larger integrated signal power value proximate the peak integrated signal power value (in gradients) will influence the determination of the location of the ferrous object in that direction (up or down) toward the one sensor.
For example, referring to
0=(Pl−Pu)*x2−2*ss*Pl*x+(Pl−Pu)*L2+ss2*Pl, where:
Still referring to
Regarding the above-described exemplary data analysis methods using integrated signal power methodologies, such methods may produce anomalies for some structural designs or configurations of ferrous objects. That is, two or more ferrous objects may be allegedly detected or indicated when only one ferrous object exists in the portal passageway 108. For example, two or more integrated signal power peaks (peak signal power values) called “ghost alarms” may be present in the integrated signal power curves for a single ferrous object. Exemplary structural designs that produce ghost alarms characteristically have one dimension that is significantly thin and longer relative any other dimension of the ferrous object. This configuration of a ferrous object (also referred to as “ghost object”) tends to produce separate and distinct magnetic field poles, a positive pole and negative pole. These separate and distinct poles are detected by the array of sensors, which influences the shape of the integrated signal power curves relied upon for implementing the embodiments of the object location method 500.
For example, as the magnetic field changes from one pole to the other, the shape of the response curve dips or has a null region (local minimum value) leaving two local maximum values (or two integrated signal power peaks) in the response curve. That is, an ideal response curve for a single ferrous object will have a single integrated signal power peak with a steadily increasing and decreasing shape (laterally extending bell curve) as illustrated in
For a first exemplary embodiment 581 of the ghost alarm reduction 580, consider
Still referring to
Still referring to
The rationale or logic for consolidating the two integrated signal power peaks is based on the following assumptions: a) that the peak values were generated by a single, long and slender object; and b) the single, long and slender object was oriented at an angle with respect to the vertical axis of the portal passageway 108. In this orientation of the single, long and slender ferrous object, one of the magnetic poles produced by the ferrous object was “cast” to (or was detected by) an elevationally different sensor (lower or higher) which was located in the opposite column (opposite side) of the portal structure 106. In the integrated signal power curve, the consolidation will provide the single integrated signal power peak centrally between the two original integrated signal power peaks, in both the vertical aspect and the horizontal aspect. It should be understood that, generally, the greater move or repositioning will occur in the vertical aspect of the curve, that is, along the vertical axis of the curve because the two original integrated signal power peaks were nearly equal along the horizontal axis (i.e., had substantially equal horizontal positions). Accordingly, not much repositioning is needed along the horizontal axis, or in the horizontal aspect of the response curve.
Moreover, it should be understood that because the two integrated signal power peaks were determined in step 584 not to be outputted from the two opposite arrays or columns of sensors, conclude that the only other orientation is that the two integrated signal power peaks are outputted from the same column and array of sensors, and go to step 586.
Still referring to
The rationale or logic for combining these two integrated signal power peaks outputted from the same array of sensors is because the features of the long ferrous object provide the positive and negative magnetic poles that are clearly resolvable by the sensors. As the response curve registers (or outputs) the transition of one magnetic pole to the other, as stated previously, the response curve goes through a null region that appears to the sensors to be void of ferrous material or an object. It should be understood that this logic assumes that the sensors are not capable of resolving or discerning signatures or outputs from two large ferrous objects that are closer than the distance between two vertically spaced sensors.
The ghost alarm reduction method 580 consolidates the ghost alarms whether they occurred as signals from a single column of portal structure 106 or from opposite columns of system 100. Another exemplary method for addressing ghost alarms and locating ferrous object positions is based on the analyses and methods disclosed in U.S. Pat. No. 6,150,810, which were based on maximum signal methods. These maximum signal methods can be used to supplement the integrated signal power data analysis disclosed in the present application. To summarize, the maximum signal methods reduce the magnetic data acquired from each sensor during the magnetic data acquisition period into a single maximum gradient value. Comparing the graphical representation (plot) of gradient values using the maximum signal analysis with the graphical representation (plot) of gradient values using the integrated signal power analysis demonstrates how the maximum signal analysis resolves ghost alarms.
Consider outputted magnetic data from the same ferrous object, for example, a small gun, having one dimension that is significantly longer than the other dimensions. The gun is positioned approximately 44 inches above ground level 118 on the right side of portal passageway 108 (right of center line 120 of
For example, referring to
The response or signature curve 658 (
Other exemplary methods for analyzing raw magnetic data according to various embodiments of the invention are now described and are collectively termed “pattern classification methods.” The “pattern classification methods” use aspects of, and values determined from, the previously described data analysis methods. Accordingly, background for the previously described data analysis methods (and the previously described data analysis methods themselves) is reiterated and summarized in a different perspective to facilitate understanding of the various inventive embodiments of the “pattern classification methods.”
Generally stated, an exemplary embodiment of a pattern classification method extracts unique features from raw magnetic response data or signals (magnetic data, magnetic signal or magnetic response). These features are used to discriminate between threatening objects and non-threatening objects. The features are processed by various classification methods to automatically identify the class of object being detected or sensed during a measurement event. The automatic identification of the class of object being detected results in an intelligent security screening system to greatly reduce false alarms that result from benign objects, such as shoe shanks.
It should be understood that for successful classification, magnetic signals or data must contain information characteristic of the object being sensed or detected. Moreover, methods have to be available to extract the characteristic information from the magnetic signals or data. The characteristic information must be unique, reproducible and readily processed by the pattern classification methods. Exemplary embodiments of the pattern classification methods according to various embodiments of the invention use quantitative anomaly detectors and physics-based discrimination schemes to distinguish between threatening and non-threatening objects.
The inventive pattern classification methods resolve the typical analysis and processing challenges that occur using passive magnetic sensing applications. Exemplary typical analysis and processing challenges include: (1) a relative small sampling period which is a function of the speed of passage through the portal by a ferrous object (i.e., the speed of a moving person); (2) the raw magnetic data or response, which is relatively a narrowband signal, near DC; and (3) the raw magnetic data or response, which is a transient signal. These typical challenges arise because conventional magnetic sensing applications are poorly suited to process magnetic signals or data that change suddenly and/or unpredictably, which is characteristic of the unpredictability of magnetic fields that produce the magnetic signals or data. However, as stated above, it is recognized that the instantaneous magnetic field variations often carry the specific information characteristics (or magnetic data) indicative of specific classes of ferrous objects that can be used as “fingerprints” to discriminate between the specific classes of ferrous objects.
Various embodiments of the “pattern classification methods” will use aspects of the following previously discussed data analysis methods:
According to various embodiments of the pattern classification methods, the following values computed by the previously discussed data analysis are provided to the pattern classification methods:
In addition to these previously discussed data analysis methods, the following new data analysis methods are disclosed to further assist the classification process performed by the inventive exemplary pattern classification methods:
In other exemplary embodiments of the invention, a fusion of methods is performed to optimize performance of the passive magnetic sensors in an exemplary security screening system. In one embodiment of the invention, the following exemplary analysis methods are fused: i) time domain feature extraction; ii) wavelet analysis; iii) matched filter detection; and iv) model-based frequency analysis. The values derived or computed from these exemplary analysis methods are further processed, for example using fuzzy logic, to increase the probability of accurate classification using various embodiments of the inventive pattern classification methods.
One of the various exemplary pattern classification methods begins by gathering raw magnetic data, for example, from security screening system 100. Raw magnetic data is obtained from magnetic sensors and represented as magnetic field gradients relative time. The magnetic data is acquired at user-selectable sample rates, for example, ranging from DC to 100 KHz. Sensor-level digital signal processor (DSP) firmware extracts features and magnetic patterns from the raw magnetic data. Moreover, the sensor-level magnetic data and features thereof can be remotely interrogated through any of various communication protocols, for example, TCP-TP data transfer, RS-485 and USB. Additionally, the extracted features and magnetic patterns can be further post-processed by a standard desktop or laptop computer using custom software, for example, National Instruments' LABVIEW™. The extracted features and magnetic patterns from each sensor of security screening system 100 are analyzed as groups to provide additional information, such as symmetry and complex dipoles from large ferrous objects.
For the following described exemplary methods of the invention, it should be understood that “features” or “feature extraction” is defined as “repeatable characteristics in the raw magnetic data that are consistent for the same group or class of detectable ferrous objects.” An example of a “group” or class includes: guns, knives, cell phones, bras having structures with wire, and steel shank shoes. According to various embodiments of the invention, the features are available in the time domain, the frequency domain, and the two frequencies combined, that is, the time/frequency domain. Other pertinent features of the magnetic data conducive to classification of the ferrous objects are obtained, which include magnitude and location of the magnetic response.
The previously described data analysis methods analyzed the features using a combination of empirical and physics-based models to pinpoint ferrous objects and the relative “magnetic” sizes. That is, the dominant ferrous object (represented as outputting the magnetic signal response having the greatest magnitude or peak) is located generally vertically within the portal passageway (portal) by first associating the vertical location of the magnetic sensor that outputs the magnetic signal response. The same dominant ferrous object is located horizontally by solving the quadratic equation (0=(Pr−Pl)r2−2wPrr+Prw2) for horizontal distance “r,” which represents horizontal distance from the ferrous object to the magnetic sensor outputting the magnetic response. Solving the quadratic equation uses the 1/r2 formulation to model the magnetic strength of the dominant object as a function of distance from the magnetic sensor. If more than one ferrous object is identified as possibly being a dominant ferrous object, then the effects of the larger disturbances (larger outputted magnetic signals) are distinguished from each identified object using the 1/r2 model. That is, all the detected objects that are potential dominant ferrous objects are located horizontally and vertically. Minor adjustments to the vertical position are done based on the relative strengths of the magnetic signals outputted from adjacent sensors.
A summary of these methods follows:
It should be understood that the raw magnetic data is representative of the Earth's magnetic field, which can be visualized as laterally spaced, generally parallel, and uniform lines of force called “magnetic field lines.” The density or magnetic flux of the magnetic field lines determines the strength of the magnetic field. When the Earth's uniform magnetic field is disturbed by the passage of a magnetic conductive material such as a ferrous object, the magnetic field lines are concentrated or channeled, which induces a dipole moment having a north/south pole.
Additionally, it must be understood that ferrous objects comprised of ferromagnetic materials have a positive and negative magnetic pole. As discussed previously, for long and slender ferrous objects, the magnetic poles are separated sufficiently wherein both poles are detected by magnetic sensors during the measurement event. The detection of both poles is the classic magnetic dipole response or signature. However, for small and compact ferrous objects, the magnetic poles are close together wherein generally only one of the two magnetic poles is detected, that is, the stronger of the two magnetic poles is detected or sensed which results in the magnetic monopole response or signature.
It should be remembered that the magnetic dipole response is graphically represented with a crossover or inflection point of the magnetic responses wherein the magnetic poles switch polarity. It should be remembered that vertical locations of objects strongly correlate to the crossover or inflection point of the magnetic responses. Furthermore, it should be remembered that the magnetic signal peaks of the magnetic dipole decrease significantly with increasing distance from the magnetic sensors, and that decrease is represented as a rate of 1/r2. Additionally, the width of the magnetic response widens with increasing distance from the magnetic sensors.
It should be understood that the composite or alloy of a ferrous object (e.g., a handgun) affects the amplitude of the magnetic dipole response. In fact, different types of handguns can be distinguished from one another based on the differing composites and differing amounts of ferromagnetic material contained in each handgun. The more massive the ferromagnetic material in the handgun, the greater the amplitude and width of the magnetic response representing the handgun. It should be remembered that a width of the magnetic signal is a function of the number of adjacent sensors that sense or detect the same item or ferrous object during the measurement event.
It should be further remembered that determining the position of the ferrous object within the portal passageway requires the analytical analysis of relationships between the various parameters of the magnetic signals, for example: peak amplitude, peak location and inflection points in the magnetic response. The position of the object is further validated by comparing responses of adjacent and opposite sensor pairs.
As previously discussed, the amplitude of the measured magnetic signal decreases with increasing distance from the magnetic sensor. There are two physical mechanisms that produce signals: magnetization of the object and distortions in the Earth's magnetic field due to the ferrous mass of the object. Since both mechanisms can be present at varying magnitudes, a single theoretical signal decay function is incapable of predicting the amplitude of the magnetic signal for a given magnetic target (ferrous object) at a given location. Similarly, because a magnetic target may provide a simple and/or small dipole, or a larger more complex magnetic structure (e.g., quadrupole), then a viable theoretical model for all real-world target objects would have to be very complex.
Accordingly, the inventors of this application implemented techniques to properly describe a magnetic signal decay function for the desired targets. However, before the signal decay function could be determined, several conditions had to be taken into consideration:
These conditions greatly affect the repeatability of any magnetic signal collected from locations very close to the magnetic sensors, which, logically, is important to understand for the pattern classification methods. In addition, for targets with small magnetic signal variations in the noise floor, the magnetic signals can be on the order of the signal amplitude of the ferrous object. These situations greatly affect the repeatability of any magnetic signal collected from locations very far from the magnetic sensors. These conditions for magnetic signal repeatability can generate significant difficulties when trying to accurately characterize the signal decay function of each ferrous object.
Accordingly, in one embodiment of the invention, a pattern classification method uses a fusion of classification analytical methods to improve the signal-to-noise performance of magnetic sensors and to extract unique spectral features. Specific classification analysis methods include: wavelets, matched filters and model-based frequency analysis.
One exemplary embodiment of a classification analysis method according to the invention is a wavelet method. The wavelet method provides the means to extract secondary or complex dipole moments. The exemplary wavelet method allows the simultaneous extraction of both low-frequency and high-frequency magnetic signals having different frequency resolutions. Additionally, the wavelet method preserves the timing information (time domain) of the magnetic signal that other data analysis methods fail to maintain. The wavelet method is dependent upon deriving a waveform transform that best matches the magnetic signal characteristics of the object being analyzed (for example, a gun). The wavelet method is not limited to a sinusoidal function. The function of the wavelet method provides a “best fit” of the wavelet to the pertinent portions of the magnetic signature waveform. The fundamental wavelet transform function is understood by those skilled in the art and defines the theoretical basis for deriving mother wavelets, which will be used for feature extraction from magnetic signals.
Wavelets derived from the wavelet method are well-suited for the analysis of predominately non-stationary magnetic signals that have sudden spikes or peak values and a transient existence. The wavelet method uses wavelets for feature extraction. That is, the numerical implementation of the wavelet transform is a filter bank designed for processing of magnetic signals that have a short duration (transient). The wavelet transform uses a correlation operation to compare real-time signals to an elementary function. The wavelet transform compares the magnetic response signal to a predefined set of short waveforms called the fundamental wavelet (or mother wavelet). The wavelets have different time durations, or scales, that mathematically represent impulse-like functions. This enables near real-time processing of impulse signals, such as magnetic signals representing complex dipole moments of a gun.
The wavelet transforms of the wavelet method indicate the frequency of the magnetic signal and indicate the timing of when the frequency occurs. That is, the wavelet method applies wavelets to characterize a magnetic signature simultaneously in both the time and frequency domains. Accordingly, the wavelets are used to:
During an exemplary measurement event by a security screening system, the magnetic sensors are modulated, which introduce detrimental noise artifacts in the sensor response signal. The wavelet transforms are used to improve the magnetic sensor signal-to-noise ratio. For example, in one embodiment of the wavelet method, the process includes: a) taking the wavelet transform of the magnetic baseline signal of the magnetic sensors by applying the wavelet function to smooth out or negate undesirable spikes and drifts in the signal; and b) inverting the wavelet transform to reconstruct the original magnetic signal minus the noise. Performing the wavelet transformation to decompose the magnetic signal provides a set of wavelet coefficients. These wavelet coefficients represent characteristics of the original magnetic signal. The wavelet coefficients having a magnitude below a chosen threshold value are set to zero. The threshold value for the wavelet coefficients will ideally represent the magnetic noise to be removed. After setting the wavelet coefficients that are below the threshold value to zero, an inverse wavelet transform is performed to provide the magnetic signal without the magnetic noise.
In another embodiment of the invention, the wavelet method is employed to selectively discard undesired components, such as far-field noise and sensor thermal drift trends, which may corrupt the original magnetic signal. A drift in the magnetic sensor's DC offset can mask other important magnetic signal features. The trend often appears as a strong DC component in the frequency spectrum. Typical detrending techniques use low-pass filters, which can also impact or alter desired signal features. However, wavelet-based detrending will preserve the important features of the original signal.
It has been demonstrated that various objects (guns, cell phones, etc.) generate a unique magnetic signature or response. The uniqueness is not readily apparent with analysis methods that use only one basis function (complex sinusoidal). The wavelet method will reduce to practice a series of “mother wavelets” that are tailored to match the magnetic signals of interest.
In real-time, the magnetic signature of a ferrous object is acquired, such as, during the period of time a person is walking through the portal passageway 106 of system 100 with a gun. Multiple waveform transforms of the magnetic signature are performed to match the response to a known threat, that is, a known magnetic waveform that represents a known ferrous object, either non-threatening or threatening. For example, the potential exists to derive a series of mother wavelet functions for magnetic waveforms, each approximating the magnetic response from different classes of ferrous objects including cell phones, PDAs, cameras, underwire bras and steel shank shoes. Each one of these classes would be considered non-threatening ferrous objects.
Another exemplary embodiment of a classification analysis method according to the invention comprises a matched filter method. The matched filter method provides the means to filter out typical “false alarm” noise responses by processing the measured spatial data and identifying magnetic dipolar responses. The results are compared to modeled magnetic data. The fundamental matched filter correlation function is understood by those skilled in the art and serves as the basis for deriving application-specific matched filters. A matched filter can be used in communications to “match” a particular transit magnetic waveform to achieve the maximum signal-to-noise ratio (SNR) and to emphasize certain signal bands where high-fidelity information is present while de-emphasizing regions that are more prone to noise corruption. In contrast, the matched filter method relies on a matched filter correlation function to process concealed weapon spatial data and identify magnetic dipolar responses.
The identification of magnetic dipolar responses is accomplished by comparing model magnetic data generated through a training process to real-time field magnetic data. The resulting (or measured real-time) field magnetic data includes parameters such as spatial location, dipole strength, and orientation. When an optimum match is found in the comparison, the above parameters are stored in a computer memory. The measured parameters are then compared to modeled parameters that are expected for indicating a weapon or threatening ferrous object. The inventors of this application established parameters and features of known weapons from testing under standardized conditions. This comparison analysis provides a means for filtering out superfluous magnetic responses acquired during a measurement event, such as the magnetic responses outputted as the result of metallic clutter and environmental conditions.
Another exemplary embodiment of a classification analysis method according to the invention is a model-based frequency analysis method (super-resolution). Magnetic responses for objects having minimal ferromagnetic material can include a time-dependent spectra that is mainly a DC component. To extract out other frequency components, additional preprocessing is required. Accordingly, it is proposed that some important features of the magnetic response are not evident in the time waveform of the magnetic signal. Analyses of magnetic sensor responses to concealed weapons show that the frequency response is near DC, with features clustered in bins as close as 0.02 Hz. Therefore, the resolution of this frequency response is difficult to resolve and acquire the important features of the magnetic responses for additional processing and/or analysis.
The model-based frequency analysis method is a model-based analysis technique to improve resolution, and includes using the following various methods: matrix pencil, covariance, Prony's, and principle component auto-regressive (PCAR). In one embodiment according to the invention, the frequency analysis method is based on the “matrix pencil” method. Using the matrix pencil method, the magnetic response will be modeled as a time series and approximated with a recursive difference equation. For example, published Spectral Analysis Algorithms are used incorporating the matrix pencil method and understood by those skilled in the art. That is, exemplary mathematical equations and derivatives of the matrix pencil method are understood by those skilled in the art. Using the matrix pencil method in this manner will quantify the resonance frequency and the primary frequencies where the power of the magnetic signal resides. Moreover, the matrix pencil method will be optimized to obtain super-resolution power spectra, even when only small magnetic data sets are available.
In other embodiments of the invention, methods were developed for analyzing and compensating for narrowband signals whose frequencies change slowly with time. Magnetic responses of this type include slowly varying background conditions from diurnal effects, solar storms, or drift in the sensor's DC offset. For example, raw magnetic sensor data is collected in the time domain. Different ferrous objects provide the unique waveforms that are useful for pattern classification. The raw magnetic data is analyzed to determine statistical relationships, such as standard deviation and mean, and made available for pattern classification.
Referring to
Fuzzy logic 910 is a problem-solving control system methodology and is implemented, for example, by software. Fuzzy logic 910 provides a simple way to arrive at a definite conclusion based upon sometimes vague, ambiguous, imprecise, noisy, or missing input information. Fuzzy logic 910 methodology is an approach to control problems by mimicking how a person would make decisions, only much faster. Fuzzy logic 910 incorporates a simple, rule-based, IF X AND Y THEN Z, approach to solving a control problem rather than attempting to model a system mathematically. The fuzzy logic 910 model is empirically based, relying on historical knowledge and experience. Because of the rule-based operation, any reasonable number of inputs can be processed by the methodology of fuzzy logic, for example, a range of one input to eight or more inputs can be handled. Fuzzy logic 910 can process nonlinear systems that would be difficult or impossible to model mathematically.
An exemplary fuzzy logic 910 explores relationships between multiple data inputs to reach empirical conclusions. The fuzzy logic 910 methodology is tailored to mimic human logic and experience acquired from operation of exemplary testbeds operated by the inventors. For example, in one embodiment, fuzzy logic 910 is used to assign different weights to features based on location of the ferrous object and magnitude of the magnetic response. Fuzzy logic 910 is also used to weigh the confidence level of the classification decision. For instance, if multiple ferrous objects are detected within a clustered region, the confidence level would be decreased.
Still referring to
Referring to
In one embodiment of the security screening system 930 according to the invention, a camera 938 is electrically coupled to circuitry and processors of security screening system 930. Camera 938 is secured to portal structure 932 on a swivel mechanism (not shown) according to one of various embodiments of the invention. In another embodiment of the security screening system 930, trigger device 117 is provided to indicate when an individual or person is approaching portal structure 932, passing through portal structure 932, and exiting portal structure 932. Accordingly, trigger device 117 provides the capability to initiate activation of camera 938 (and magnetic data processing). An exemplary trigger device 117 includes an infrared break beam sensor or photo-detector. The swivel provides the capability for camera 938 to take real-time snapshot images of an individual approaching security screening system 930, passing through security screening system 930, and exiting security screening system 930.
Still referring to
Still referring to
Referring to
According to one of various embodiments of the invention, inputs from the variously described methods are provided to neural networks 912 and 956 for further processing and described below:
1) Raw magnetic sensor data reduction method using signed minimum/maximum function: This method reduces raw magnetic sensor data to a single point per magnetic sensor. Each magnetic sensor in the portal passageway is continuously sampled at operator selectable data rates, for example, 1 kHz. The magnetic data is filtered, averaged, and baseline corrected. This data reduction method is implemented on the sensor-level digital signal processor.
2) Magnetic moment classification:
3) X, Y position of detected ferrous object: Vertical location of the object strongly correlates to the crossover or inflection point of the magnetic moment response. The derived vertical position of the inflection point is fed into the neural network 912. This method requires a complete portal spectral waveform and therefore is processed at the portal computer level.
4) Time domain analysis of the raw spectral waveform from each sensor: The number of peaks, individual and relative peak amplitudes, peak widths, and peak rise and fall times are calculated and fed to the neural network 912. The peaks of the magnetic moments decrease significantly with increasing distance from the sensors. Theoretical peak response should decrease at 1/r2. The width of the response widens with increasing distance from the sensors. The composite or alloy of handguns affects the amplitudes of the magnetic responses. Types of handguns can be discriminated based on the mass of ferromagnetic material contained in respective handguns. The more mass, the higher the amplitude of the response. Response width is a function of the number of adjacent sensors that detect or sense the same ferrous object. The method is implemented through a time domain analysis performed by the sensor-level digital signal processor.
5) Frequency domain analysis of the raw spectral waveform from each sensor: Fast Fourier Transform (FFT) functions are used to calculate the primary frequency components and power content of the magnetic spectra. The power spectrum of the magnetic sensor data is also calculated by squaring the magnitude of the Fast Fourier Transform of the signal. Both inputs are fed to the neural network 912. The method is implemented through a frequency domain analysis performed by the sensor-level digital signal processor.
6) Z-axis, Time-positioned data analysis: This function determines whether the target is located in front of a body (F) or at the back of a body (B). A two-state position flag (F, B) is passed into the neural network. This method is implemented through synchronizing data to a start and stop break beam and calculating the total elapsed time between start/stop pulses. The elapse time is used to calculate the walking speed of an individual passing through the portal passageway of, for example, security screening systems 100 and 930. The elapse window is divided into segments, representing a person entering the portal, the person directly within portal, and the person exiting the portal. The occurrence of the major magnetic peak versus time is correlated to when it occurred within the sampling (time) window allowing a best-fit assignment to the front or back. This information is very pertinent in the discrimination of chest region alarms, such as underwire bras. In one exemplary embodiment of the invention, this method is implemented through time domain analysis and synchronization with the infrared break beam.
7) Portal zone where alarm occurred:
8) Post-processing JTFA analysis of raw sensor data: A more advanced joint time-frequency analysis (JTFA) method is used to extract unique features from the raw magnetic data. It is noted that the standard FFT provides the average frequency content of the magnetic signal over the entire time that the magnetic signal is acquired. That method is more accurate for stationary magnetic signal analysis. For measuring frequency information that may be changing during acquisition, the joint time-frequency analysis is used. The JTFA method is used to calculate the instantaneous power spectrum and to extract the specific frequencies of the major peak. This information is also provided to the neural network 912. In one exemplary embodiment of the invention, this method is implemented using a National Instruments utility and executes under LABVIEW™ on the computer system supported on or proximate the portal structure.
9) Symmetry fitness calculations: Objects such as shoe shanks and underwire bras each have a symmetrical magnetic signature or fingerprint. Moreover, the opposite and adjacent magnetic sensors located in the portal structure will detect and output a similar magnetic response for each object. For one exemplary embodiment of the invention, adjacent magnetic sensors are compared to determine if magnetic signatures being detected are statistically equivalent. For another exemplary embodiment of the invention, opposite magnetic sensors are compared to determine if magnetic signatures being detected are statistically equivalent.
10) Gun coefficient analysis: It has been noted that complex dipole objects, such as guns, induce a unique magnetic moment with multiple inflection points. This property can be used as an indicator of potential threat items.
For one exemplary embodiment of the invention, a neural network is implemented for training data representative of ferrous objects, and therefore, a general discussion on neural network theory is warranted. The training data includes many sets of input variables and a corresponding output variable. In statistical terms, the inputs are called independent variables and the output variable is called the dependent variable (represented as classifications 914 in
The neural network begins by finding linear relationships between the inputs and the output. Weight values are assigned to the links between the input and output neurons. After those relationships are found, neurons are added to the hidden layer so that nonlinear relationships can be found. Input values in the first layer are multiplied by the weights and passed to the second (hidden) layer. Neurons in the hidden layer “fire” or produce outputs that are based upon the sum of weighted values passed to them. The hidden layer passes values to the output layer in the same fashion, and the output layer produces the desired results, for example, classifications 914 and 956 for
The network “learns” by adjusting the interconnection weights between layers. The answers the network is producing are repeatedly compared with the correct answers, and each time, the connecting weights are adjusted slightly in the direction of the correct answers. Additional hidden neurons are added as necessary to capture features in the data set.
Eventually, if the problem can be learned, a stable set of weights evolves and will produce good answers for all of the sample decisions or classifications. The real power of neural networks is evident when the trained network is able to produce good results for data which the neural network has never “seen” or handled previously.
For an exemplary embodiment of the invention, the following procedures are performed in developing a neural network database and classification function. Referring to
Exemplary step 972, simulate field deployed screening scenarios. Step 972 includes simulating real-world conditions in a laboratory environment. Various threatening and non-threatening items are introduced to an exemplary portal of an exemplary security screening system, such as system 100 described above. Step 972 includes testbeds and data capture rigs.
Exemplary step 974, collect magnetic signature data. Step 974 includes developing a library of magnetic spectra of signature data or features for threatening and non-threatening items, and saving the library of magnetic spectra in a memory file. An exemplary memory file is a comma-delimited text file. Step 974 includes statistically determining sample size.
Exemplary step 976, simplify the magnetic data and extract features from the magnetic data. Step 976 includes at least the following exemplary methods to extract features: the time domain method, the frequency domain method and the joint time/frequency method, all of which are described above. In one exemplary embodiment of the exemplary process flow 970 according to the invention, step 976 is optional, wherein process flow 970 moves from step 974 to step 978.
Exemplary step 978, perform pre-classification optimization. Step 978 includes at least the following exemplary methods to perform pre-classification optimization: the wavelet method, the matched filters method and the fuzzy logic method, all of which are described above.
Exemplary step 980, populate a classification database. Step 980 includes sensor data being associated with a solution(s).
Exemplary step 982, apply a training strategy. Step 982 includes configuring a Neural Network Training Strategy. In one of the various embodiments of configuring a Neural Network Training Strategy, such an embodiment includes selecting a Neural Network method such as probabilistic (PNN) or genetic. Step 982 further includes defining a maximum number of hidden neurons to avoid over-fitting the model. Step 982 still further includes configuring user-defined Fitness Coefficients by assigning weights to inputs based on relevance.
An exemplary sub-step of step 982 includes a Training Mode or training process. The training mode includes initiating software utilities to derive a classification function (Neural Network training mode). The training mode includes monitoring reports of percentage correct classifications versus increasing a number of iterations. The training mode further includes monitoring reports of significance of each input in predicting the output value. The training mode is stopped or discontinued when acceptable confidence limits are achieved.
Another exemplary sub-step of step 982 includes a Test Mode. The test mode includes validating the trained data by applying to an out-of-sample test database. The test mode includes reviewing probabilities of classification. The test mode further includes monitoring reports of an “Agreement Matrix,” that is, true positive, false positive, true negative and false negative. The test mode still further includes reviewing the ROC Curve.
The exemplary step 982 further includes saving the Neural Network database and classifications function (Network). Saving the Network includes saving the trained and validated Network to a data file with .net extension.
The exemplary step 982 still further includes integrating the saved Network file into software onto a computer, for example, a Portal Control Computer. Integrating includes downloading network file(s) to the computer into a pre-assigned directory. Integrating further includes incorporating LABVIEW™ software to implement network files wherein LABVIEW™ software has a DLL that fires the neural network and applies it to real-time data. Outputs of the neural network are produced that comprise probabilities representing whether input pattern data (magnetic signature data) belong to a specific category of an item, such as a threatening item (gun) or a non-threatening item (cell phone).
Exemplary step 984, apply trained network to new magnetic data. Step 984 includes performing system level validation tests. For example, a person will walk through the portal of system 100 with various “trained” items. Step 984 further includes verifying proper classification of the various “trained” items.
Exemplary step 986, report classification results to an operator. An exemplary report includes displaying classification results on an interface.
In compliance with the statute, the invention has been described in language more or less specific as to structural and methodical features. It is to be understood, however, that the invention is not limited to the specific features shown and described, since the means herein disclosed comprise preferred forms of putting the invention into effect. The invention is, therefore, claimed in any of its forms or modifications within the proper scope of the appended claims appropriately interpreted in accordance with the doctrine of equivalents.
This invention was made with Government support under Contract DE-AC07-05-ID14517 awarded by the U.S. Department of Energy. The Government has certain rights in the invention.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5801631 | Hayashi | Sep 1998 | A |
| 6150810 | Roybal | Nov 2000 | A |
| 6269324 | Rakijas et al. | Jul 2001 | B1 |
| 6469624 | Whan et al. | Oct 2002 | B1 |
| 7312752 | Smith et al. | Dec 2007 | B2 |
| 20030167139 | Swartz et al. | Sep 2003 | A1 |
| 20030174057 | Kim et al. | Sep 2003 | A1 |
| 20040021778 | Oldani et al. | Feb 2004 | A1 |
| 20040234132 | Jojic et al. | Nov 2004 | A1 |
| 20050182590 | Kotter et al. | Aug 2005 | A1 |
| 20050242817 | Hoult | Nov 2005 | A1 |
| 20060140340 | Kravis | Jun 2006 | A1 |
| 20070013372 | Murray et al. | Jan 2007 | A1 |
| 20080136631 | Fluck | Jun 2008 | A1 |
| 20090121888 | Reeves et al. | May 2009 | A1 |
| Number | Date | Country |
|---|---|---|
| WO 2006036752 | Jun 2006 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20080231443 A1 | Sep 2008 | US |
