Systems and methods for detecting threats and contraband in cargo

Description

FIELD

The present specification discloses systems for scanning objects and, more specifically, systems for inspection of contents of tankers, containers and vehicles for identifying threat objects and contraband.

BACKGROUND

Cargo containers need to be inspected at ports and other points of entry or transportation to assess the quantity of contents, quality of contents, and any possible anomalies that may be associated with the contents of these containers, including contraband such as explosives, narcotics, currency, chemical and nuclear weapons. In addition, cargo container inspection is useful for cargo manifest verification. A cargo manifest is a physical or electronic shipping document that accompanies the cargo and provides important descriptive information about the cargo, including bills of lading issued by the carrier or its representative(s), the shipment's cosigner and/or consignee, cargo description, amount, value, origin, and/or destination.

Current security systems are limited in their ability to detect contraband concealed in cargo. Standard and advanced X-ray systems also have difficulty detecting anomalies in break-bulk cargo. Computed Tomography (CT) based systems have been shown to be more suitable for the difficult task of detecting aviation-threat explosives in luggage and, more recently, in larger objects. However, the configuration of commonly employed CT systems prevents scaling the system up to long objects such as large cargo containers, large skids and tankers.

The problem is further compounded by the fact that as a result of the image modulation according to atomic numbers of various materials, it is common for X-ray imaging systems to produce images with dark areas. Although these dark areas might indicate anomalies or presence of threat materials, they yield little information about the exact nature of the anomaly or threat. Also, radiographs produced by conventional X-ray systems are often difficult to interpret because objects are superimposed. Therefore, a trained operator must study and interpret each image to render an opinion on whether or not a target of interest, a threat, is present. Operator fatigue and distraction can compromise detection performance, especially when a large number of such radiographs is to be interpreted, such as at high traffic transit points and ports. Even with automated systems, it becomes difficult to comply with the implied requirement to keep the number of false alarms low, when the system is operated at high throughputs.

This difficulty of inspection is magnified when inspecting larger and oftentimes, cluttered pallets, cargo containers, and cargo-carrying vehicles, such as tanker trucks, which are uniquely designed to carry fluid/liquids or dry bulk loads. Current X-ray inspection systems may not provide an accurate indication of the nature of anomalies present in tankers, which can, among other indications be evidence of theft and/or dilution of tanker contents. An X-ray image itself is insufficient to discern an amount, type, and quality of a commodity within a tanker. These parameters are typically needed to verify a manifest corresponding to the truck, or to ensure that there is no theft underway. There is thus added difficulty in inspection of tanker contents and verification of the manifest.

In addition to reporting the quantity and quality of tanker contents, there is a need to identify and report anomalies associated with the tanker contents. It should be noted that an identified anomaly may end up being contraband as well.

Therefore, there is a need to provide an automated detection system that further includes assistance tools to help operators improve their throughput by scrutinizing cargo images more efficiently, thereby increasing detection and analysis speed. Additionally, an accurate automated content detection and assessment system, such as those for tankers, is needed as a means of preventing theft of tanker contents and as a tool for quality control. Such a tanker content assessment can be a manifest verification tool, and may also have applications in cases when the manifest is not useful. Such applications may include: use at a checkpoint where all passing tankers are expected to be full or empty; to confirm that the tanker content is consistent with expected contents (for example, contents are gasoline and not water); and to check for inner-container alterations including possible alterations made to compartments, walls, and other inner portions of a container/tanker that may be an indication of nefarious intents.

There is also a need for automated detection systems that include assistance tools for detecting the presence specific contraband or threat items, such as currency, drugs, cigarettes and firearms during non-intrusive X-ray imaging of vehicles and cargo. There is also a need for algorithms for processing radiographic images that can be employed to characterize cargo and provide estimates of cargo configuration, such as cargo weight and cargo dimensions, for assisting the operators in detecting illegal items. There is further a need for detection algorithms that enable detection of differences between sequential X-ray inspection images of the same object or vehicle, so that changes in an object or vehicle may be tracked over time or distance traveled.

SUMMARY

The present specification discloses a method for verifying a type of a container, the method comprising: obtaining a scanned image of the container; and, applying at least one pixel-value line integral to the scanned image for verifying the type of container.

Optionally, obtaining the scanned image comprises obtaining an X-ray scanned image.

Optionally, applying the pixel-value line integral comprises obtaining pixel attenuation values in a vertical direction at a fixed position within the scanned image. Obtaining pixel attenuation values may further comprise indicating a shape of at least one edge of the container. Optionally, the method comprises indicating the shape of the at least one edge to be at least one of a rounded shape and a straight shape, wherein the indicating the rounded shape verifies the type of container to be a tanker and the straight shape verifies the type of container to be a cargo container.

The present specification also discloses a method for verifying contents of a tanker, the method comprising: generating an X-ray image using an X-ray scanner; using said X-ray image, detecting presence of contents within the tanker; using said X-ray image, determining a capacity of the tanker; calculating a degree of fullness of the tanker based on said contents; and analyzing the contents.

Optionally, detecting presence of contents comprises using a pixel attenuation value threshold to determine a presence of contents within the tanker. Optionally, the pixel attenuation value threshold is 2,500 for a low-energy channel of a 16-bit normalized dual-energy scanned image.

Optionally, determining the capacity of the tanker comprises determining a bottom edge and a top edge of the tanker. Optionally, the method comprises determining whether a lower edge of the contents of the tanker coincides with the bottom edge of the tanker. Optionally, the method further comprises using a pixel attenuation value threshold for determining the top edge of the tanker. Optionally, the pixel attenuation value threshold is 10,000 for a low-energy channel of a 16-bit normalized dual-energy scanned image.

Optionally, calculating a degree of fullness comprises calculating a quantity of the contents relative to the capacity of the tanker.

The method may comprises using X-ray attenuation characteristics for analyzing the contents.

The present specification also discloses a method for identifying contraband in cargo, the method comprising: generating a scanned image using a radiographic source; analyzing the scanned image to compute features of said cargo; comparing the computed features of said cargo using the features of known contraband stored in a database; calculating a probability of the cargo features matching contraband features; and determining that the cargo matches contraband when a value of said probability is more than a threshold value.

The contraband may comprise at least one of currency, drugs, cigarettes, and firearms.

The present specification also discloses a method of identifying firearms within radiographic images of a conveyance carrying the firearms, the method comprising: obtaining from a database a template image of a firearm; generating a radiographic image of the conveyance using a radiographic source; comparing a set of predefined features of the template image with the radiographic image to obtain image regions on the radiographic image comprising features matching with the predefined features; filtering the obtained image regions of the radiographic image to obtain a final set of image regions; classifying the final set of image regions for extracting image properties to determine one or more regions indicating presence of a firearm; and marking the one or more regions on the radiographic image to communicate an alarm.

Optionally, a template image of a firearm is a radiographic image of a firearm with minimal background and no clutter or is based on the type of conveyance in which such a firearm is transported.

The conveyance may be one of a cargo container, a vehicle, a trailer, a boat, a truck, a bus, and a truck driver cab.

The present specification also discloses a method for estimating the weight of a cargo, the method comprising: generating a scanned image of said cargo using a radiographic source; analyzing the scanned image to convert image attenuations to effective organic and steel thickness; determining an atomic density of said cargo; using said scanned image to generate material thickness values and atomic density values for various positions in the cargo; using the material thickness values and atomic density values to compute a weight at a given position in said cargo; and adding the computed weight of all positions to obtain total weight of said cargo.

The present specification also discloses a method for estimating the average pallet size and density in a cargo, the method comprising: generating a scanned image of said cargo using a radiographic source; analyzing the scanned image to generate organic and steel thickness profiles from image attenuation values; correcting the steel thickness profiles for background signal from the cargo; fitting the organic and steel thickness profiles using optimized pallets of cargo; and comparing the fitted profiles with known pallet data.

The present specification also discloses a method for automatically detecting anomalies in a vehicle, using the sequential images of the same vehicle, the method comprising: scanning the vehicle under inspection to generate a current scan image; identifying a most recent image of said vehicle from a database, said image being a historical scan image; determining a transformation that optimally maps the current scan image to the historical scan image; applying a normalization function to intensity profiles of the current scan image and the historical scan image; comparing intensity levels of said current scan image with intensity levels of said historical scan image on a pixel-by-pixel basis; and identifying pixels representative of anomalies in the current scan image.

The aforementioned and other embodiments of the present specification shall be described in greater depth in the drawings and detailed description provided below.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features and advantages of the present specification will be appreciated, as they become better understood by reference to the following detailed description when considered in connection with the accompanying drawings, wherein:

FIG. 1 is an illustration of a scanned image of a tanker, in accordance with an embodiment;

FIG. 2 is an exemplary flow chart illustrating a process of assessing contents of a tanker, in accordance an embodiment;

FIG. 3 illustrates an exemplary output for a scanned image of a tanker;

FIG. 4 illustrates an exemplary architecture of a distributed inspection network that uses non-intrusive X-ray scanning, according to one embodiment of the present specification;

FIG. 5 is a diagram presenting the overall system architecture of an exemplary imaging system as described in an embodiment of the present specification;

FIG. 6 illustrates an exemplary dashboard for a service post, as employed in one embodiment of the system described in the present specification;

FIG. 7 depicts an exemplary interface for presenting manifest information, as employed in one embodiment of the system described in the present specification;

FIG. 8 shows an exemplary user interface screen for a data center, as employed in one embodiment of the system described in the present specification;

FIG. 9 shows another exemplary user interface screen for a data center, as employed in one embodiment of the system described in the present specification;

FIG. 10 is flowchart illustrating one process for preparing a features database, according to one embodiment of the system described in the present specification;

FIG. 11 illustrates the use of the features database described with respect to FIG. 10 to determine if cargo under inspection matches manifest information;

FIG. 12 illustrates the process of using the features database described with respect to FIG. 10 to determine if cargo under inspection matches the manifest, when there is more than one type of cargo present in the shipment;

FIG. 13 illustrates how currently scanned images may be visually compared with images from the database of the present specification to determine if cargo matches the manifest;

FIG. 14 illustrates the segregation of cargo into various cargo types based upon scanned images;

FIG. 15 illustrates three different types of radiographic firearm images with each one positioned in multiple orientations in three-dimensional space;

FIG. 16A illustrates an exemplary template of an assault rifle, in accordance with an embodiment of the present specification;

FIG. 16B illustrates an exemplary template of a hand gun, in accordance with an embodiment of the present specification;

FIG. 17 is a flowchart illustrating the steps of an advanced image processing method, in accordance with an embodiment of the present specification;

FIG. 18A is an exemplary radiographic image of a cargo container comprising firearms located throughout various orientations;

FIG. 18B illustrates an exemplary firearm template comprising a plurality of Maximally Stable External Regions (MSER) features marked thereupon, in accordance with an embodiment of the present specification;

FIG. 18C illustrates the radiographic image of FIG. 18A comprising a plurality of MSER features marked thereon, in accordance with an embodiment of the present specification;

FIG. 18D illustrates an exemplary image obtained by comparing the MSER features marked in FIG. 18C with those marked in FIG. 18B, in accordance with an embodiment of the present specification;

FIG. 18E illustrates the radiographic image of FIG. 18A with markings representing Speeded Up Robust Features (SURF), in accordance with an embodiment of the present specification;

FIG. 18F illustrates a filtered version of the radiographic image shown in FIG. 18D after application of a filtering method, in accordance with an embodiment of the present specification;

FIG. 18G illustrates a radiographic image of a cargo container after application of a marker box, in accordance with an embodiment of the present specification;

FIG. 19 illustrates an exemplary mobile inspection system, to which the methods of the present specification may be applied;

FIG. 20 illustrates an exemplary mobile inspection system in deployed mode, which may use the methods of present specification for automatic detection of threats and contraband;

FIG. 21 is a flowchart illustrating a method of detecting currency, according to one embodiment of the system described in the present specification;

FIG. 22 is flowchart illustrating one process for preparing a features database, according to one embodiment of the system described in the present specification;

FIG. 23 illustrates the use of the features database described with respect to FIG. 22 to determine if cargo under inspection carries a specific contraband;

FIG. 24 shows an exemplary case of use of an empty container verification tool;

FIG. 25A shows an example of a uniform cargo;

FIG. 25B shows an example of a non-uniform cargo;

FIG. 26 shows an exemplary case of anomaly detection in periodic cargo;

FIG. 27 illustrates an exemplary image with pallet counting;

FIG. 28 illustrates how scanned images may be visually compared with images from a database to determine if cargo matches the manifest, according to one embodiment of the present specification;

FIG. 29 is a flowchart illustrating a method of automatic threat detection, in accordance with one embodiment;

FIG. 30A illustrates a historic image of a vehicle acquired by an inspection system;

FIG. 30B illustrates the current image of the vehicle of FIG. 30A;

FIG. 30C illustrates the difference between the historic image and current image of a vehicle;

FIG. 30D illustrates contraband in an acquired and processed image;

FIG. 30E illustrates an image with a contraband item marked for display;

FIG. 31 illustrates an exemplary screen of an operator interface, according to one embodiment of the present specification;

FIG. 32 illustrates how the current and historic vehicle images may be used for identifying changes in the vehicle structure over time;

FIG. 33 illustrates the use of vehicle images for identifying changes in cargo contents over a traveled distance;

FIG. 34 illustrates another example of the use of vehicle images for identifying changes in cargo contents over a traveled distance;

FIG. 35 is a flowchart illustrating an exemplary method of characterizing cargo by estimating cargo weight, according to one embodiment;

FIG. 36 is a flowchart illustrating an exemplary method of characterizing cargo by estimating pallets size and density, according to one embodiment;

FIG. 37 illustrates a scan image of a cargo container, according to one embodiment;

FIG. 38A is a graph illustrating a linear fit to steel thickness as a function of attenuation counts;

FIG. 38B is a graph illustrating the accuracy in predicting steel thickness as a function of attenuation counts;

FIG. 39A illustrates an exemplary plot of the attenuations for 3.5″ thick steel at different positions along the detector array;

FIG. 39B illustrates an exemplary plot of the attenuations for 7″ thick steel at different positions along the detector array;

FIG. 40 shows a scan image for an exemplary uniform cargo;

FIG. 41 illustrates a cargo attenuation (areal density) graph with background correction, in accordance with one embodiment;

FIG. 42 is a graph illustrating fits to the attenuation profile using a single pallet and two pallets of cargo, in accordance with one embodiment;

FIG. 43 is a table illustrating a summary of fits to attenuation profile of contents of an exemplary cargo;

FIG. 44 illustrates how location and geometry of pallets can be inferred from the attenuation profiles obtained from various detectors in a detector array;

FIG. 45 illustrates another exemplary container scan image using the present method, according to one embodiment;

FIG. 46 illustrates the inferred location and orientation of steel plates in a cargo container, corresponding to the attenuation profile of FIG. 45, in accordance with one embodiment;

FIG. 47 illustrates an exemplary dual-energy scan image of cigarettes in a cargo container; and

FIG. 48 is a table illustrating the elemental composition of selected materials, in accordance with one embodiment of the present specification.

DETAILED DESCRIPTION

The present specification is directed towards multiple embodiments. The following disclosure is provided in order to enable a person having ordinary skill in the art to practice the invention. Language used in this specification should not be interpreted as a general disavowal of any one specific embodiment or used to limit the claims beyond the meaning of the terms used therein. The general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the invention. Also, the terminology and phraseology used is for the purpose of describing exemplary embodiments and should not be considered limiting. Thus, the present invention is to be accorded the widest scope encompassing numerous alternatives, modifications and equivalents consistent with the principles and features disclosed. For purpose of clarity, details relating to technical material that is known in the technical fields related to the invention have not been described in detail so as not to unnecessarily obscure the present invention. In the description and claims of the application, each of the words “comprise” “include” and “have”, and forms thereof, are not necessarily limited to members in a list with which the words may be associated.

It should be noted herein that any feature or component described in association with a specific embodiment may be used and implemented with any other embodiment unless clearly indicated otherwise.

It should be noted that all of the processes and methods described herein are implemented by a computing architecture comprising at least one server executing a plurality of programmatic instructions which may be remotely or locally stored in non-transient memory. The programmatic instructions may be executed or controlled using graphical user interfaces that are displayed on client devices which are in wired or wireless data communication with the computing architecture.

In one embodiment, the present specification discloses a method for reporting the quantity and quality of tanker contents, and for identifying and reporting anomalies associated with the tanker contents. Anomalies could be an indication of nefarious intents bringing a need to check a cargo container that is expected to be empty or full, to confirm that the cargo content is consistent with its expected contents; and to check for inner-container alterations including possible alterations made to compartments, walls, and other inner portions of a container/tanker. An identified anomaly could end up being contraband. Anomalies can also be evidence of occurrences of theft and/or dilution of tanker contents, which are difficult to detect. In some embodiments, the method uses scanned images of tankers that are under inspection, obtained by non-intrusive X-ray imaging techniques. This allows the operator or inspector to quickly ascertain and verify the contents of the tanker that is currently being inspected.

In one embodiment, images generated during a scan by an X-ray based inspection system are analyzed for specific features and parameters characteristic of specific contraband or threat items, such as currency, firearms, cigarettes or drugs. Various operator assistance tools are employed for automatic identification of specific banned items in radiographic images. In one embodiment, the system described in the present specification automatically applies detection algorithms to the image and provides alerts to operator if it detects the presence of illegal items in the object being scanned.

In another embodiment, the present specification discloses an automated method of threat detection based on processing and comparison of the X-ray images of a vehicle over time. In one embodiment, the present system provides an automated means to detect differences between sequential X-ray inspections of the same vehicle.

In another embodiment, the present specification discloses a method for characterizing cargo by providing estimates of cargo configuration, such as cargo weight as a function of position or length, pallet size and density. The present specification discloses multiple embodiments. The following disclosure is provided in order to enable a person having ordinary skill in the art to practice the invention. Language used in this specification should not be interpreted as a general disavowal of any one specific embodiment or used to limit the claims beyond the meaning of the terms used therein. The general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the invention. Also, the terminology and phraseology used is for the purpose of describing exemplary embodiments and should not be considered limiting. Thus, the present specification is to be accorded the widest scope encompassing numerous alternatives, modifications and equivalents consistent with the principles and features disclosed. For purpose of clarity, details relating to technical material that is known in the technical fields related to the invention have not been described in detail so as not to unnecessarily obscure the present invention.

One of ordinary skill in the art would appreciate that the features described in the present application can operate on any computing platform including, but not limited to: a laptop or tablet computer; personal computer; personal data assistant; cell phone; server; embedded processor; DSP chip or specialized imaging device capable of executing programmatic instructions or code.

It should further be appreciated that the platform provides the functions described in the present application by executing a plurality of programmatic instructions, which are stored in one or more non-volatile memories, using one or more processors and presents and/or receives data through transceivers in data communication with one or more wired or wireless networks.

It should further be appreciated that each computing platform has wireless and wired receivers and transmitters capable of sending and transmitting data, at least one processor capable of processing programmatic instructions, memory capable of storing programmatic instructions, and software comprised of a plurality of programmatic instructions for performing the processes described herein. Additionally, the programmatic code can be compiled (either pre-compiled or compiled “just-in-time”) into a single application executing on a single computer, or distributed among several different computers operating locally or remotely to each other.

In embodiments, an application implements the method disclosed in the present specification. In embodiments, the application is integrated with an X-ray inspection/detection system deployed at checkpoints or service posts. In one embodiment, manifest data is imported via the application. In other embodiments, the application processes detected data independent of manifest data. In one embodiment, the application works within the framework of a distributed network, wherein a point of image detection is connected to one or more points of image analysis, whereby an operator can analyze the X-ray image of the cargo. When the X-ray image has been analyzed, a service post which performed the non-intrusive X-ray scan will be notified automatically by the application integrated with the X-ray system. This allows the service post operator to make a decision to either release the cargo or to hold the cargo for further inspection.

It should be noted herein that any feature or component described in association with a specific embodiment may be used and implemented with any other embodiment unless clearly indicated otherwise.

In embodiments, the system of present specification provides several automated tools to make the image-inspection process more efficient and effective. These tools provide assistance to image operators in analyzing the images to identify a tanker and to determine its quantity and type. A tanker is different from any other cargo container due to its shape and because it is designed to carry liquefied loads, dry bulk cargo, or gases. A tanker is typically, but not always, cylindrical in shape and lies horizontally. It can be transported by truck, together referred to as a tanker truck, or by rail. FIG. 1 illustrates an exemplary scanned image 100 of a tanker that is part of a tanker truck. The scanned images, like image 100, may be obtained by dual-energy transmission systems that may range from 1 MV to 9 MV. In some embodiments, transmission systems with multiple views may be used to obtain the scanned images. When compared with other scanned images (such as those subsequently illustrated in FIGS. 13 and 14) rounded edges 102 of the tanker are seen in contrast from straight edges of other containers. Many variants of tankers exist due to wide variety of commodities that can be transported. Tanker trucks tend to be large; they may be insulated or non-insulated; pressurized or non-pressurized; and designed for single or multiple loads (often by means of internal divisions in their tank).

Embodiments of the specification describe exemplary methods and systems for assessing the contents of a tanker or a tanker truck (hereinafter also referred to as ‘tanker’).

FIG. 2 is an exemplary flow chart illustrating a process of assessing contents of a tanker, in accordance with an embodiment. In embodiments, the described process is used by an algorithm, such as a software algorithm to assess a tanker's contents. The algorithm may be implemented at one or both of a point where a non-intrusive X-ray scan is performed and a point where image analysis is performed (refer FIG. 4). At 202, a scanned image is presented to the algorithm. The scanned image may be an X-ray scanned image. Tanker identification may be performed in a first sub-step by using object shape, size, aspect ratio, and patterns to eliminate cars, pickup trucks, vans, or any other type of vehicle (except cargo containers and tankers) from the process. The remaining image may be of a container or tanker. In a second sub-step, for a scanned image that survives the first sub-step, line integrals may be applied to identify the presence of a tanker by distinguishing it from cargo containers.

As discussed, a distinguishing feature of a tanker vs. a container is the rounded edge of a tanker in contrast to the straight edge of a container. Applying a pixel-value integration technique distinguishes tankers from containers. In an example, for an image, if f(x,y) represents the attenuation value of a pixel at location (x,y), then ∫f(x,y)dy provides the sum of attenuation values observed along the y-direction at position x. By generating these vertical line integrals across the entire image, rounded and straight edges can be identified to establish the presence of a tanker or cargo container, respectively. FIG. 3 illustrates an exemplary simplified algorithm input for a scanned image of a tanker 302 with rounded edges and of a container 304 with straight edges. Integrating the scanned image of the container 304 along its vertical edges results in sharp peaks 307 corresponding to the straight edges of the container. The shape is different from 306 obtained from integrating the ends of a scanned image of a tanker, resulting from the rounded-edges of tanker 302 For the decision-making process, if the maximum line-integral result, i.e., the maximum of the sums of attenuation values, exceeds 230,000, then it may be determined that the object is not a tanker. If the maximum integral result falls between 100,000 and 230,000, then the conveyance may be determined to be a tanker if other factors, such as and not limited to object height, vertical dimension, and horizontal distance between the edges, are consistent with tanker geometry. In an alternative embodiment, a binary image of the region of interest may be used in place of transmission values of the image.

Referring again to FIG. 2, at 204, the contents of the tanker identified at 202, are analyzed and detected. In embodiments, a watershed segmentation method is used to segment the input image into meaningful physical objects based on magnitude of image gradient. In embodiments, the largest object that meets pre-defined attenuation thresholds may represent the tanker content. In an embodiment, an attenuation threshold of an integral result of 2,500 for an object, measured from a 16-bit normalized dual-energy image generated from the low-energy channel, is used to determine the presence of content in the tanker. A top edge of the largest object identified (content) may represent the content level. If the largest object identified (content) does not meet or exceed the attenuation threshold, then the tanker may be considered to be empty.

Once 204 is completed and the tanker is detected with contents, at 206, capacity (or volume) of the tanker is detected. A bottom edge of the interior space within the tanker may coincide with a lower edge of content of the tanker. In embodiments, a top edge of the tanker may be determined by searching upward from a top surface of the tanker content. In embodiments, the top edge of the tanker is determined if the pixel value is within a pre-determined threshold level. In an embodiment, using the 16-bit normalized dual-energy image generated from the low-energy channel, a threshold transmission pixel value of 10,000 is used to determine the top edge of the tanker. If the pixel value is greater than the threshold, then the pixel is determined to be not part of the top edge of the tanker, and the algorithm may continue searching for the top edge.

At 208, after the top and bottom edges of the tanker are determined, a relative fullness of the tanker may be calculated by identifying location of a top surface of the tanker's content, with respect to the tanker's inner (top and bottom) edges.

Knowledge of the tanker's type, or any other information describing the tanker, may be available from its manifest. The knowledge may be used to map relative fullness of the tanker to its capacity (absolute volume). Alternatively, the tanker type may be identified by individually observing interior and exterior characteristics of the tanker, that are unique to a particular tanker type. Calibration data for various tanker types and various fullness levels may be collected to train the algorithm and validate accuracy of the calculated absolute-volume.

In some cases, surface-level turbulence may be present in the tanker's content during the tanker's scan and acquisition of the scanned image. Characterization of this turbulence may be required to estimate with acceptable precision, the relative or absolute volume of the content.

At 210, material of the tanker's content may be analyzed. In embodiments, tanker's image may be scanned by x-rays that are captured by a flat, 2D detector array after passing twice through the side walls of the tanker. As a result, measured attenuation of each radiated beam that passes through the tanker may vary with height of the tanker. The attenuation will be a function of its exit point from the tanker and the amount of content inside the tanker. The algorithm, in accordance with embodiments of the specification, will characterize the x-ray attenuation profile as a function of height, the content types of interest, the various tanker types, the amount of content present in the tanker, the percent of the tanker that is occupied, and any other information. Calibration data may be used to model the characterization process, and to validate accuracy of the characterization process. In embodiments, the calibration data is based on empirical data such as but not limited to scans of different liquid types (regular gasoline, premium gasoline, diesel fuel, water, or any other type of liquid, solid, or gas type that may be contained within a tanker) at various percentage fill levels (for example, in 5% increments). Characteristics of these images may be used to compare the scanned image and further refine the detection result.

Measured attenuation values for both high- and low-energy x-rays may be associated with different depths of content, independent from the tanker. Therefore, the measured attenuation values may be used to determine the attenuation properties of the content in isolation. Once the attenuation values of the content are determined, mu-ratio, mu-difference, and effective atomic number values can be generated. In an embodiment,

Mu-ratio=Log(HE)/Log(LE); and

Mu-difference=Log(HE)−Log(LE);

where, LE is the pixel value for the normalized low energy channel of the scanned image, HE is the pixel value for the normalized high energy channel of the scanned image, and Log is the natural logarithm.

This information may be used to differentiate content types, such as to differentiate water from gasoline, and one type of fuel grade from another. In embodiments, surface turbulence of tanker's content is shown to vary with content-type. In such cases, turbulence features in a scanned image may be used to identify a content-type.

In embodiments, tanker content assessment, described in context of FIGS. 1-3, is a part of a distributed inspection network, which may be deployed at ports, borders, and other types of check posts. FIG. 4 illustrates the architecture of an exemplary distributed inspection network that uses non-intrusive X-ray scanning disclosed in the various embodiments. The components of system architecture are described as follows:

Service Post and Regional Center

Referring to FIG. 4, service post 401 is the point where a non-intrusive X-ray scan is performed. It should be noted herein that one exemplary scanning and inspection system that may be employed with the systems and methods of the present invention includes, but is not limited to the Rapiscan Eagle Mobile inspection system. Any suitable system for inspecting cargo, cargo containers, and their contents may be employed. As such, U.S. patent application Ser. Nos. 12/780,910; 13/370,941; 13/548,873; 13/532,862; 13/168,440; 13/175,792; 13/433,270; 13/281,622; 13/108,039; 12/675,471; 12/993,831; 12/993,832; 12/993,834; 12/997,251; 12/919,482; 12/919,483; 12/919,484; 12/919,485; 12/919,486; 12/784,630; 12/784,465; 12/834,890; 13/009,765; 13/032,593; 13/368,178; and 13/368,202, all assigned to the assignee of the present invention represent various systems that may be employed with the present invention and are herein incorporated by reference in their entirety. In addition, U.S. Pat. Nos. 5,638,420; 6,542,580; 7,876,879; 7,949,101; 6,843,599; 7,483,510; 7,769,133; 7,991,113; 6,928,141; 7,517,149; 7,817,776; 7,322,745; 7,720,195; 7,995,705; 7,369,643; 7,519,148; 7,876,879; 7,876,880; 7,860,213; 7,526,064; 7,783,004; 7,963,695; 7,991,113; 8,059,781; 8,135,110, 8,170,177; 8,223,919; and 8,243,876 all assigned to the assignee of the present invention represent various screening systems that may be employed with the present invention are herein incorporated by reference in their entirety.

Service post 401 further comprises at least one, and preferably a set, of non-intrusive inspection (NII) servers 411 through which the service post interfaces with other components of the system. After scanning, the operator responsible for controlling or operating service post 401 can verify that the X-ray image produced by the non-intrusive X-ray scan is of sufficient quality to be effectively analyzed. In one embodiment, the image analysis is performed at a regional center 402. In one embodiment, if the image is incomplete, or is corrupted, black (from attenuating cargo) or is unacceptable in any manner, the service post operator may request a rescan. This can happen in cases where the time between the scan and analysis is close and the truck is still available. In embodiments, application disclosed in FIGS. 1 to 3 is implemented at either one or both of a service post 401 and a regional center 402.

Servers 411 at service post 401 comprise standard non-intrusive inspection software. When a vehicle is about to be scanned, the software at the service post queries a predictive or routing software application 403 to receive an instruction, routing information, or any other data to identify a target regional center for analysis. Regional center 402 comprises at least one server 421 and inspection monitors 422. As a new X-ray image is generated at service post 401, it is transmitted onward from service post 401 to server 421 located at regional center 402, pursuant to routing information received from software application 403, for analysis by an inspection operator located at that regional center and for subsequent storage. It should be appreciated that, in embodiments, regional center 402 and service posts 411 are geographically remote from each other.

In one embodiment, the image is allocated to a regional center and/or an operator within that regional center via predictive or routing software 403, but the work is only allocated after the image transmission is complete. In one embodiment, to streamline the data transmission activity, predictive software 403 allocates an image to regional center 402 before the image has been completely generated.

In one embodiment, in the event of the operator becoming unavailable, such as due to PC failure, log off, etc., another operator in local regional center 402 is selected automatically by predictive software 403.

Further, the system will fall back on an alternative regional center in the event of a transmission error. In one embodiment, images are buffered until a center comes back on line.

In one embodiment, each X-ray inspection image is associated with a GUID (Globally Unique Identifier), which is a unique ID across all systems. The GUID is used for associating each image with its particular manifest data. In one embodiment, identifying information, such as license plate, CCTV images etc. are also associated with the GUID at the time of scanning. In one embodiment, the GUID is a 128-bit number displayed in hexadecimal. This information may be transmitted to the inspection operators at the regional center, if required.

When the X-ray image and (optionally) manifest data have been analyzed, service post 401 which performed the non-intrusive X-ray scan is notified automatically by means of a data transmission from a software application referred to herein as CertScan 405. The CertScan application presents an interface to the operator at the service post 401, which shows the operator a rolling status of all non-intrusive X-ray scans performed at that service post, along with relevant data to allow service post 401 to either release the cargo or to hold it for further inspection. In one embodiment, the relevant data includes license plate number, work order number, and results of scan. CertScan 405 application system may also be responsible for importing the manifest data associated with the cargo or vehicle being scanned. In one embodiment, manifest data can come in one or more of several forms, such as but not limited to a) a hardcopy of the manifest; b) from a computer owned and connected to the customer database; or c) from a customer database accessed directly by CertScan 405. The format in which manifest data is supplied depends on the customer, and their local requirements and regulations.

Predictive Software 403

Predictive software 403 operates to optimally balance the load distribution of image analysis among multiple regional centers and operators. Predictive software 403 processes meta-data from multiple regional centers and service post connectors to analyze and predict the best distribution of images to operators. For example, predictive software 403 uses historical meta-data on inspection queue lengths, workload, contention time, and a randomization factor to varying degrees, to allocate work to regional centers and individual operators.

Logging and Validation

At various stages of the process, the system provides localized and centralized logging, auditing, and accounting for each action performed by an X-ray scanning operator and an X-ray image inspection analyst. Centralized logging may be provided at a data center 404. During all steps of the process, from scanning, through inspection to search, the system provides a journal of actions for each non-intrusive X-ray scan and X-ray image inspection analysis.

Inspection Performance and Metrics

In one embodiment, the system records several X-ray image inspection metrics, such as image coverage, tools used, mean time to inspect, and time pending, among other variables. These metrics can yield information for operators/image analysts such as what tools were used (for example, zoom, contrast, brightness, and other parameters), how long it took to analyze the image, and/or what part of the image was analyzed using tools. This information can then be applied to measure attentiveness and diligence of operators. For example, this information may be reviewed for each X-ray image inspection analyst, and is useful in training, review and performance evaluation. In one embodiment, inspection metrics may be measured quantitatively and be assigned minimum and maximum values, against which the operators' performance may be evaluated.

Besides helping to assess the proficiencies of the analysts, data logs also allow an assessment of inspection volumes at regional centers and the speed at which analyses are performed.

In one embodiment, the system provides for secondary X-ray image inspection, for a percentage of images, or if required, on targeted users. If required in certain cases, the X-ray image inspection process is repeated twice to cross-check results. The second X-ray image inspection can be assigned to either a purely random X-ray image scanning operator, or to nominated workstations for quality and training purposes, in various embodiments. The final X-ray image inspection result would not be sent to the service post until both inspections are complete. If either result is “suspicious”, the suspicious result would be recorded, and any disagreement would be flagged.

In one embodiment, training images may be inserted into the workflow to pass on suspicious images to operators as part of their standard workload. The system then carefully segregates the results from these images, without the X-ray scanning operator knowing the difference. This allows for discrete and impromptu training of operators.

If a suspicious finding is communicated back to the service post, the operators can choose to manually open and search the suspicious cargo. In one embodiment, the system allows the operators to record detailed comments about the manual search process, which can provide both useful information about the suspicious cargo and useful feedback to trainers.

CertScan Software Application 405

Still referring to FIG. 4, the primary goal of CertScan application 405 is to present manifest information clearly for the non-intrusive X-ray image analysis inspector to quickly ascertain the contents of the cargo container or light vehicle that is currently being inspected. Application 405 may be executed on an application server 451 and may interface with a master database 452. In one embodiment, the manifest information and related data that CertScan application 405 provides may be imported into master database 452 through any suitable means, such as EDI (Electronic Data Interchange), web services, or OCR scanning of manifest documentation. The manifest information that is provided by these sources may include, but is not limited to, the following data elements:

Container Number

Arrival Date

Shipping Line

Bill of Lading Number

Port of Origin

Exporter

Consignee

Container Manifest

Besides use in security inspections, additional related data captured in CertScan application database 452 may be used for internal statistical analysis, financial forecasting, and operational reporting. In one embodiment, application 405 generates various reports, including daily, weekly, and monthly data related to the expected arrival dates of cargo containers and light vehicles, as well as data regarding actual cargo containers and light vehicles scanned. In one embodiment, captured data further includes information such as the number of containers scanned at each site, average time taken to analyze a scan, scans without supporting data, number of scans with threats and without threats, etc. In one embodiment, this data is presented in real time on a user interface, referred to herein as ‘Dashboard’.

In one embodiment, the use of CertScan system 405 is extended to provide reporting through online customer portals or electronic data exchange. Additionally, CertScan 405 may also be extended to provide web services for supporting “cloud” type solutions. In one embodiment, web services include obtaining manifest data and publishing or transmitting results of the scan along with any anomalies noted. These additional features are all value-added services for the security scanning system. Thus, the reports provided by CertScan application 405 may be coupled with x-ray images (JPG) which are produced by the scanning software, to create a combined reporting package. These reports may be provided to customers for their own analysis and audit purposes.

FIG. 5 is a diagram presenting an overall system architecture of a CertScan application 500 (shown as 405 in FIG. 4), according to one embodiment of the present invention. The hardware for running CertScan application 500 may include an application server 501 and a master database 502. CertScan application 500 may provide manifest data to a regional center 503, which may be used by an operator in conjunction with a scanned X-ray image to analyze and determine the disposition of the cargo or light vehicles. In one embodiment, each regional center has a graphical user interface (GUI)—the CertScan Application Dashboard or CertScan Dashboard, which shows the analyst all non-intrusive X-ray scans ready for analysis. Using the CertScan Application Dashboard, the image analyst may select an X-ray Image for analysis. At the time of selection, CertScan Dashboard may display the cargo and/or light vehicle manifest data along with its X-ray image. Once adjudication is determined, the image analyst may record the result in a database associated with CertScan Application 500. The CertScan Dashboard at a service post 504 that performed the X-ray scan, may be then updated with the result. The result may allow the service post operator to take appropriate action of releasing or holding for further inspection, the cargo and/or light vehicles.

As mentioned earlier, scan images are packaged with metadata and sent from service post 504 to a data center 505 and regional center 503. The metadata may also be processed and loaded into CertScan master database 502. In one embodiment, the scan images and metadata are packaged together as a scan transaction file 506, with a ‘.stf’ extension, for easy communication between service post 504, regional center 503, data center 505, and CertScan application database 502. In one embodiment, metadata includes information such as time of scan, the operator ID, and whether a rescan is required. This data may help establish the time it takes to transmit images and to analyze a scan. This information may also enable work quality monitoring and statistical reporting.

In one embodiment, a CertScan primary application may be a web-based application that resides at data center 505. The CertScan Dashboard in the data center may display all non-intrusive X-ray scans being performed and information on all regional centers, as well as all status information. The data center may also serve as a storage location for all X-ray images.

In one embodiment, CertScan Application 500 is externally integrated with web services 507, which may be used to generate reports as described earlier. In one embodiment, CertScan application 500 is integrated with the inspection software to provide a comprehensive solution for efficient non-intrusive X-ray inspection.

Collection of Manifest Data

As described above, manifest data can come in one or more of several forms, such as but not limited to a) a hardcopy of the manifest; b) from a computer owned and connected to the customer database; or c) from a customer database accessed directly by CertScan Application 500. In one embodiment, CertScan Application 500 accepts cargo and light vehicle manifest data in multiple formats including, but not limited to:

Electronic Data Interchange

Formatted Data Files (Fixed Width or XML)

Transportation Management System Interfaces

2D Bar Code Reader

Manifest Documentation

Some methods, such as Electronic Data Interchange (EDI) of formatted data files may be preferred to facilitate faster import of data into CertScan master database 502 before the cargo arrives. When using EDI to acquire the cargo container and/or light vehicle data provided by the customer, data integration may be accomplished by importation of a formatted flat file. In embodiments, application 500 is designed to support other data exchange formats that are widely accepted by Freight Management Systems (FMS) standards, web services, or OCR scanning of manifest documentation. One of ordinary skill in the art would appreciate that the system may be configured to accept additional or other forms of manifest input.

In one embodiment, a lack of manifest information may be used to detect hidden compartments and contraband such as weapons, nuclear materials, among other contraband. More specifically, in one embodiment, incomplete or otherwise inadequate manifest information may be indicative of cargo that requires further inspection.

Thus, in one embodiment, the present specification includes systems and methods for automatically and rapidly detecting the presence of high-atomic-number (high-Z) materials such as nuclear materials; nuclear weapons; and, shielding materials that may be used to shield radiation emitted by such materials as well as by radiological dispersal devices, which can prevent them from being detected by radiation detectors. The present specification also includes the detection of other types of high-Z materials that may be smuggled in cargo due to their value, such as gold and platinum bullion, and works of art and antiquities containing high-Z materials.

The present specification therefore advantageously employs a threat detection algorithm that uses physical properties such as material density, mass absorption coefficient, and dimension to determine whether high-Z materials are present in the cargo.

The threat detection method and algorithm requires a much shorter analysis time and, thus, allows for higher system throughput compared to a conventional system, which requires an inspector manually reviewing the image or cargo for objects that are highly attenuating. For example, if multiple objects that are highly attenuating are identified, the inspector would need to make contrast enhancements with each object using a computer and input device, such as mouse. Each object has to be then evaluated for its total attenuation (or transmission) value by using the computer to select a region of interest within the object and making an estimate of the average attenuation (or transmission) value, which reflects the total attenuation (or transmission) along the X-ray path through the cargo. Before the net attenuation (or transmission) of the object can be estimated, the attenuation (or transmission) of the surrounding background material has to be analyzed. Then, to generate an average net attenuation (or transmission) of the object, the background must be subtracted from the total attenuation (or added to the transmission). Finally, the inspector must examine the shape and size of the object, and combine these estimates with the estimated net attenuation (or transmission) to reach a conclusion of whether the object represents a threat. This procedure would have to be repeated for each object and, therefore, if performed accurately, would be a very time-intensive procedure.

The threat detection process described in the present specification, in one embodiment, operates by first receiving, on a computing platform, a radiographic image of an object from an X-ray imaging system, which typically comprises a radiation source positioned opposite to, or away from, a detector array. At least a part of the area bounded by the radiation source and detector array is an inspection region, through which the cargo being inspected passes, or within which the cargo is positioned. In one embodiment, the screening system acquires the original image, which is then processed by the methods described herein. The X-ray imaging system is in electrical communication, either wired or wirelessly, with the computing platform. The threat detection algorithm then performs a first level analysis to generate a first “suspicious object” binary map by measuring a number of physical attributes. Each area on the initial binary map is used as a mask to electronically crop out part of the X-ray radiographic image for analysis, including its surrounding background attenuation (or transmission) and physical characteristics such as attenuation, size, and shape. Then, a decision is made of whether that area or portion could represent a high-Z object. This decision process results in a second binary map, which highlights those regions that represent potential high-Z threats.

In using the threat detection method and algorithm with the methods of the present specification the threat or no-threat decision time ranges from typically less than one second for cargo determined not to have any suspicious objects, to less than approximately 5 seconds for cargo having a plurality of objects or areas of interest. U.S. patent application Ser. No. 12/780,910, entitled “Systems and Methods for Automated, Rapid Detection of High Atomic Number Materials” is herein incorporated by reference in its entirety.

Dashboard for Real-Time Updates

As mentioned earlier, data is presented by CertScan application 500 in real time through a GUI referred to herein as a “Dashboard”. In embodiments, the CertScan Dashboard is executed on all or a combination of the three components of the system—the service post, the regional centers and the data center. In one embodiment, the CertScan Dashboard displays a rolling list of non-intrusive X-ray scans, with data elements that are appropriate for each of the three locations.

In one embodiment, the CertScan application controls the flow of all X-ray image manifest data to ensure all three components have the content and data necessary to carry out their operations.

Service Post Dashboard

FIG. 6 illustrates an exemplary GUI (Dashboard) for the service post that is provided by the CertScan Application. This GUI has the goal of providing the service post operator with the optimal information to assist in deciding if the cargo being scanned is to be released or held for further inspection. Referring to FIG. 6, the data displayed on the Service Post Dashboard may include the container ID number 601, scan start time 602 and scan end time 603, time of start 604 and time of completion 605 of analysis of image and data at the regional center, the status (result) 606, as conveyed by the regional center, and comments 607, if any, from the regional center analyst. In one embodiment, the status or result 606 is indicated visually and by means of color coding. Thus, for example, green 606a may indicate ‘ready to clear’, red 606b may indicate the need for manual or visual inspection, blue 606c may indicated ‘under analysis’, and yellow 606d may represent already ‘cleared’.

The CertScan Dashboard located at the service post need not display any information about which regional center performed the X-ray image analysis or the identity of the image analyst who performed the analysis.

Regional Center Dashboard

This CertScan Dashboard aims to provide the regional center image analyst with the information required to quickly and efficiently analyze the X-ray image for potential threats or contraband, and enables the analyst to record the results of the image inspections.

The image analyst uses the CertScan Dashboard to select an X-ray scan ready for analysis. The CertScan Dashboard located at the regional center may not display any information about which service post performed the non-intrusive X-ray scan or the identity of the service post operator who performed the X-ray scan.

In one embodiment, CertScan application interface for the image analyst is designed to be easy to use, and presents manifest information in a manner such that the analyst requires minimal time to evaluate the cargo container and/or light vehicle manifest data and record scan results.

The CertScan user interface at the regional center is integrated with the inspection software to retrieve the cargo container and/or light vehicle manifest information once the X-ray scan is complete. An exemplary interface presenting the manifest information to the image analysis inspector is shown in FIG. 7. Referring to FIG. 7, the interface screen provides manifest data such as shipper ID 701, container number 702, expected date of arrival of shipment 703, type (size) of container 704, and names of the exporter 705 and the consignee 706. The screen also includes a manifest table 707 which provides data such as description of item (contents), harmonized tariff schedule (HTS), item unit, and unit quantity.

The X-ray image analysis inspector can thus verify if information about the cargo container and light vehicle matches with the scanned images. The image analysis inspector can then record the inspection result in the interface screen, using the color coded result buttons 708. In most cases the result will be ‘Cleared’, which is represented by a green button in one embodiment. However, there may be instances where certain areas in the X-ray Image cannot be identified clearly or it may be identified that contents are potentially harmful. In these cases, two additional results may be recorded—‘Irregularity’ or ‘Possible Threat’, represented by yellow and red respectively, in one embodiment. In one embodiment, blue color is used to indicate ‘Rescan required’ in case the image is unreadable. This may happen, for example, due to an environmental condition which may affect the quality and clarity of the X-ray image. In this case the cargo and vehicle under inspection need to be scanned again.

Data Center Dashboard

The data center uses the CertScan Dashboard to select an X-ray scan at any point of its lifecycle. The CertScan Dashboard located at the data center displays comprehensive information about the service posts performing the non-intrusive X-ray scan and the regional center where analysis of the X-ray image is being performed.

The CertScan application user interface screens for the Data Center provide all the functionality of the regional center, in addition to other functions. FIG. 8 shows an exemplary user interface screen for the data center. Referring to FIG. 8, the interface allows the dater center personnel to search for past scan records 801 as well as un-scanned cargo 802 whose manifest data is loaded in the system. The operator may also search for specific details of a cargo by container number 803 or by arrival date range 804. The search yields records for the specific container, which include data such as container type 805, shipper name 806, vessel name 807, expected arrival date 808, scan date 809 and scan results 810.

FIG. 9 illustrates another exemplary screen for the data center that shows completed scans. Referring to FIG. 9, scan records may be filtered by shipper name 901, or other attributes, such as consignee name, exporter name, date of arrival, among other parameters. In one embodiment, the completed scan records include container number 902, shipper name 903, vessel name 904, voyage number 905, and expected arrival date 906.

One of ordinary skill in the art would appreciate that all the interface screens may be customized to meet the customer's needs, and data may be selected for display accordingly.

System Logging

In one embodiment, the CertScan application performs logging of all activities throughout the full non-intrusive X-ray scanning operation. The application log provides information and reports such as:

Timings related to the non-intrusive X-ray scan process

CertScan Application performance monitoring

CertScan Application system health

CertScan Application error traps

One of ordinary skill in the art would appreciate that CertScan application log data may be used for internal system monitoring as well as for reporting based on customer needs.

The applications of the present inventions may be extended to security inspection at ports, borders, aviation checkpoints as well as supply chain security. The system can import manifest data from a port, border or aviation data management system, as the case may be, and compare the obtained information with image of container. In one embodiment, the system of present invention automatically applies detection algorithms to the image and provides alerts to operator, if there are any mismatches with the manifest. This ‘Operator Assist’ function enables the security personnel to identify threats or other contraband more efficiently, and they can determine if de-vanning or opening the container is required. In one embodiment, multiple operators work in a matrix or networking environment and review the alarms generated automatically by the system. The operators then decide to clear or further investigate the alarms. The application of the system may be extended to supply chain security, where devices that are capable of sending messages through cell phones or satellite networks, may be attached to pallets and containers. These devices may be used to send alarms remotely to a central monitoring station, along with X-ray and video images if there is an alarm.

One of ordinary skill in the art would appreciate that although the process of an operator inspecting an image to verify that the cargo matches the manifest is much more efficient than manually opening the container, it still requires significant labor. The labor-intensive nature of the problem is even more evident in applications such as inspecting each railcar in a long train with hundreds of railcars and trying to identify thousands of cargo types. Often, it is difficult to identify the cargo from the numerous images in such cases.

To address this problem, in another embodiment, the present invention is directed towards the analysis of images generated by non-intrusive cargo inspection systems with the goal of improving the efficiency of the process to verify that cargo matches the manifest.

For the purpose of this specification, cargo manifest is defined as a manifest that lists all cargo codes carried on a specific shipment. Further, cargo codes may be standard, also known as harmonization codes, or may be provided by various local custom agencies and may be different depending on the jurisdiction.

In one embodiment, predetermined image features of inspected cargo with an associated cargo code are computed and compared with features associated with the same cargo code saved in a database. The comparison results in a probability that the inspected cargo matches the declared cargo in the manifest. If the probability is greater than a predetermined threshold, the cargo will be declared as matching the manifest. Otherwise, the cargo does not match the manifest. In another embodiment, the probability is presented to the operator and the operator makes the decision. These processes are illustrated by means of flowcharts in FIGS. 10, 11 and 12.

Referring to FIG. 10, the process of preparing a features database is shown. In a first step 1001, the system obtains the image of the container. The image is obtained through non-intrusive scanning at any of the service posts, as described above. It should be understood by those of ordinary skill in the art that the radiographic images could be generated by low, medium or high-energy X-rays, gamma rays, neutrons or other type of radiation. The images could also contain atomic-number information generated from any modality of dual-energy or dual-species inspection. The images could be generated by one or more views and could be reconstructed in three-dimensions, from the views.

After obtaining the image, the system obtains cargo code associated with the image, as shown in step 1002. Cargo codes are obtained from manifest data, as described above. Thereafter, features of the image are computed, in step 1003. Computed features and their standard deviation are then saved in the database along with the number of images used to compute the features, and are associated with that cargo code, as shown in step 1004.

The features include, but not limited to, attenuation, texture, atomic number, and/or cargo height. For tomographic and multi-view systems, density is also a useful feature. This also would include elemental composition or features derived from the composition for neutron-based interrogation. It should be understood by those of ordinary skill in the art that other features not listed here could be used to match the cargos.

In the next step 1005, the system checks if any entries for that cargo code are already stored in the database. If so, the system combines features from the containers with same cargo code. This is shown in step 1006. The combination of the feature values takes into account the number of images used to compute the feature value and is weighted accordingly. Also, the user is notified of outlier feature values (values that are outside the three standard deviations or other selected range) for acceptance before the combination takes place. Thereafter the combined set of features for that particular cargo code is saved in the database, as shown in step 1007. Thus, the features saved in the database per cargo code are computed from a combination of feature values from a large number of cargo images with same cargo code. The feature values are updated as additional cargo images are collected. Additional features can also be computed as their usability becomes available.

FIG. 11 illustrates a method for performing cargo-manifest verification for an individual cargo container. In the first step 1101, an image captured at a service post is associated with one or more cargo codes, depending on the contents of the shipment as defined in manifest data. Then, the features of the image are computed, in step 1102. Thereafter, the system obtains features for that cargo code stored in a database, and compares them to the computed features. This is shown in step 1103. The system then determines the probability ‘P’ that cargo matches manifest, in step 1104. Probability ‘P’ is then compared to a threshold value in step 1105. If ‘P’ is greater than the threshold value, it implies that cargo matches manifest information declared, as shown in step 1106. If ‘P’ is less than the threshold value, it indicates that the contents of the cargo are not the same as declared in the manifest, as shown in step 1107.

In one embodiment, the threshold value may be determined in accordance with the user's preferences. For example, if custom office is using the system and they want to detect most contraband even at the expense of higher false alarm rate, they may be able to set a high threshold value, such as 90%. Conversely, if the custom agency does not want to have a high false alarm rate, they can choose to set a low threshold value, such as 60%. Further, the customer may decide that some categories of goods are more important, such as those associated with higher duties, than others, and place different thresholds for different types of goods.

Further, before flagging cargo, a predetermined minimum set of images may be used to compute the features. The customer may decide that the features database is complete and more images do not need to be used. In this case, there is no need to add more images to the database. However, if the database did not use enough images, or the customer wants to improve the accuracy of detection, an authorized operator can request to add more images to the database. The operator should have a high confidence that the cargo matches the manifest, which is generally achieved with experience with the type of cargo coded in the manifest or a manual inspection and verification of the container contents.

When a shipment contains more than one type of cargo, the image is analyzed for different types of cargo and segregated. This process is illustrated in FIG. 12. Referring to FIG. 12, the system first associates the image of scanned cargo with manifest information in step 1201. The image is then analyzed to determine if there are multiple cargos, in step 1202. The system then segregates each cargo type, as shown in step 1203. The segregation of cargo types is discussed in greater detail with respect to FIG. 14. The features for each cargo type are then computed in step 1204 and compared in step 1205 with the feature values stored in the database for each cargo type listed in the manifest. A list of probabilities for each segregated cargo is then produced. Thus, ‘P_i’ is the probability that i^thcargo matches with the declared manifest. This is shown in step 1206.

Each ‘P_i’ is then compared to the threshold value, as shown in step 1207. One of ordinary skill in the art would appreciate that since there are more than one type of cargos, there may be more than one threshold value for comparison. The system checks if P_iis more than the threshold value for all “i” in step 1208. If P_iis more that the threshold value for all “i”, it is determined that the cargo matches the manifest, as shown in step 1209. Otherwise, if one or more segregated cargos do not match features for one of the cargo codes in the manifest, the cargo(s) will be assigned as not matching the manifest and all cargos that do not match the manifest are listed. This is shown in step 1210. Alternatively, the probabilities for each segregated cargo may be displayed to the operator for decision.

In one embodiment, an operator can separate the cargo visually and/or with the help of tools, such as a “rubber band” type of tool. In another embodiment, cargo may be automatically segmented and features of the different parts of the cargo may be computed, as shown in FIG. 14. Segmented regions with similar features are assumed to be same cargo. Thus, on the basis of features cargo in image 1400 of FIG. 14 may be segregated into Type 1 1401 and Type 2 1402.

In another embodiment, the operator inspects the image of a container with associated manifest. The operator then requests to retrieve from the image database a number of images of cargo with same cargo code. The operator compares the images visually and/or aided with various image manipulation tools to determine whether the cargo matches the manifest. If the manifest lists more than one cargo code, the operator would request images for each cargo code for comparison.

Another method to assist the operator for determining whether a cargo image matches the manifest is to retrieve a number of images from the image database that have the same cargo type. This is shown in FIG. 13, wherein the current image 1301 of the cargo can be visually compared by the operator with images 1302 of the same cargo type from database. Additional assistance is provided by displaying values of various cargo features of the current and previously imaged cargo. In the example, shown, and by way of example only, the current image 1301 is different from the database images 1302. Thus, the operator should make a decision that the cargo does not match the manifest, because the current image is different from those in the database.

In another embodiment, the present specification discloses a system and method for automatic identification of firearms in radiographic images.

Radiographic systems such as X-ray machines are generally used to detect weapons concealed in a cargo, containers, or vehicles. However, currently available radiographic scanning systems provide two-dimensional images of the objects which by themselves are not always sufficient to accurately identify to the operator the presence or absence of a firearm. In embodiments, a firearm includes but is not limited to a weapon such as handgun, pistol, revolver, rifle, shotgun, BB/pellet gun, and assault rifle. Generally, the X-ray system operators review the real time two-dimensional images collected at X-ray checkpoints and check each image for the presence of objectionable items such as a firearm. As an object, such as a firearm, can be positioned in an infinite number of orientations in a three dimensional space, two-dimensional images of such objects often lack the clarity needed to determine the exact shape of the object just by looking at such images.

The above complexity involving two-dimensional images is described with reference to FIG. 15 which shows three different types of firearm images, with each one positioned in multiple orientations in three-dimensional space. As shown in FIG. 15, the images 1501a, 1501b and 1501c represent the same firearm object whereby each image captures the firearm object in a different orientation. Similarly, images 1502a and 1502b represent a single firearm object whereby each image captures the object in a different orientation. Similarly, images 1503a, 1503b and 1503c represent another single firearm object whereby each image captures the object in a different orientation. As a single three-dimensional object may be placed in any of the infinite number of possible three-dimensional orientations in a cargo or vehicle, it is not possible to simply compare a two-dimensional image of such object with a predefined reference image and predict, with high probability, the presence or absence of a firearm. One can appreciate that while it is easy to observe the shape of a firearm in some of the presented images, it may be difficult to identify a firearm from the shapes of other images as well. This task is very subjective and depends on the operator.

In an embodiment, the present specification describes a method wherein the images generated by radiographic systems are further processed through advanced image processing techniques to automatically identify objects such as firearms. While, in an embodiment, the systems and methods disclosed in the present specification are used to support the inspection of land- and sea-transported cargo, cargo-carrying truck cabs, and light vehicles, however, in other embodiments, the method disclosed in this specification may be applied to any transmission x-ray based system inspecting other conveyances and other concealment methods, and is therefore not limited to the discussion herein. In embodiments, firearms such as handguns and rifles can be detected either as single objects or as multiple objects in a commercial package. In embodiments, the systems and methods of the present specification can be used to detect firearms in cargo containers, vehicles, trailers, boats, trucks, buses, and truck driver cabs. In other embodiments, the systems and methods of the present specification can be adapted for any other type of transport and conveyance methods such as luggage and even air passenger carry-on items such as cardboard boxes, without departing from the spirit and scope of the present specification.

In an embodiment, the present specification describes an advanced image processing method for detecting the presence of firearms from two-dimensional radiographic images that involves comparing such images to reference images, which in some embodiments, represent standardized templates of firearms. In an embodiment, the present specification describes a method for detecting the presence of firearms from two-dimensional radiographic images by using a combination of image processing techniques such as Maximally Stable External Regions (MSER) and Speeded Up Robust Features (SURF), resulting in a greatly enhanced probability of detecting firearms placed in any orientation within a three-dimensional space. The method of the present specification results in a lower processing time as well as a lower false alarm rate, and a greatly improved detection rate as compared to prior art radiographic-image based firearm detection methods.

In an embodiment, the present specification describes the creation of reference images, which in some embodiments are predefined templates that are used to identify suspicious regions in an image. A template consists of a radiographic image of a firearm with minimal background and no clutter. These templates are compared with structures within the image-under-inspection to identify firearm candidates. Because of the versatility of each template, its size can be modified and it can be tailored to the type of firearm of interest. In an embodiment, such templates are generated from radiographic image data during a development phase. Template candidates are selected during algorithm development. In an embodiment, the templates comprising image features that result in successful template-matching with radiographic images of firearms are eventually used for firearm detection purposes.

FIG. 16A illustrates an exemplary template of an assault rifle 1602, in accordance with an embodiment of the present specification. FIG. 16B illustrates an exemplary template of a hand gun 1604, in accordance with an embodiment of the present specification. A radiographic image obtained from a container or object under inspection may be compared with at least one of templates 1602, 1604 and, if an assault rifle or handgun is present in the object under inspection, a match may result based on the comparison. In embodiments, the template of a firearm is based on the type of conveyance in which such a firearm is transported. Each type of conveyance has a characteristic image background and typical clutter objects for consideration in development of templates. For example, a template used to inspect a vehicle will account for the passenger seats and the steering wheel, and will be optimized to inspect the vehicle floor for firearms. In embodiments, the number and sizes of applicable templates varies for various modes of conveyance such as a light vehicle, container or a truck driver cab. In an embodiment, a radiographic image may be matched against any one of the pre-defined templates for a corresponding mode of transport.

In an embodiment, the features that are used for carrying out a comparison between pre-defined templates and the radiographic image being inspected are based on Maximally Stable External Regions (MSER) techniques, known in the art, which are effective for blob detection and are insensitive to affine transformation. In this context, affine transformation refers to a change in the orientation or size of an imaged firearm due to the firearm's rotation or distance from the x-ray source, respectively. MSER techniques/algorithms are commonly used to find correspondences between image elements/features from two images with different viewpoints, and to extract a comprehensive number of corresponding features between the two images. These features capture the most difficult aspects of recognizing a firearm in a free form of 2D presentation, namely, at various orientations and for various magnifications. In an embodiment, the matched results generated through template matching then undergo a filtering process utilizing the Speeded Up Robust Features (SURF) technique, known in the art. Finally, other features are extracted to support a classification process based on the image region's clutter, background, material, and statistical properties.

FIG. 17 is a flowchart that describes the steps of the advanced image processing method, in accordance with an embodiment of the present specification. As shown in FIG. 17, at step 1702, a radiographic image is input to the image processing system for analysis. The radiographic image may be of a vehicle/cargo carrying goods in which firearms may or may not be present. In an embodiment, a radiographic system such as an X-ray inspection system is coupled to the image processing system such that the images captured by the radiographic system are automatically transmitted to the processing system for advanced image processing in accordance with the present specification. In an embodiment, the image processing system is a part of the radiographic system itself, and in another embodiment, the image processing system is located at a remote location accessible through a central server.

It should be noted herein that the systems of the present specification further comprise at least one processor to control the operation of the entire system and its components. It should further be appreciated that the at least one processor is capable of processing programmatic instructions, has a memory capable of storing programmatic instructions, and employs software comprised of a plurality of programmatic instructions for performing the processes described herein. In one embodiment, the at least one processor is a computing device capable of receiving, executing, and transmitting a plurality of programmatic instructions stored on a volatile or non-volatile computer readable medium.

The input image is analyzed in step 1702 to characterize the conveyance object in the image as either a cargo container or a vehicle, such as but not limited to, cars, trucks, buses, and vans. In embodiments, at transit points that involve the movement of a variety of transport means, the exact transport or conveyance system such as a cargo or a truck corresponding to any radiographic image is also determined through radiographic scanning. In an alternate embodiment, the transport system corresponding to the radiographic image may be manually fed into the image processing system along with the radiographic image to be inspected. In embodiments, in case a cargo unit is characterized in the image, an additional step of determining presence of a tractor unit containing a driver cab coupled with the cargo unit is also performed. Due to the distinctive features of vehicles, containers, and driver cabs, a unique image processing method may be applied for analysis of each.

At step 1704, the processing system compares the radiographic image with predefined templates to obtain possible matching region or points. In an embodiment, regions within the radiographic images are compared with the predefined templates using the MSER method, which serves as the primary feature-detector for firearms identification. The benefits of using the MSER method include its insensitivity to affine transformations, which preserves neighboring intensity topology; and multi-scale detection, which senses both small and large structures in the image. In an embodiment, a set of features are detected by applying the MSER method to both the template and the radiographic image-under-inspection, which can be a transmission image or its inverse (an attenuation image). The features may comprise a central location, orientation scale, and all pixel locations with a predefined region.

Hence, in an embodiment, at step 1704 in order to compare the radiographic image with a predefined template, using the MSER method, a predefined set of features are marked in both the template and the radiographic image, and then are compared together to obtain a feature set match. FIG. 18A illustrates an exemplary radiographic image of a cargo container 1802 comprising firearms 1804 imaged at various orientations. FIG. 18B illustrates an exemplary firearm template image comprising a plurality of features marked thereon, in accordance with an embodiment of the present specification. In an embodiment, the features 1806 marked on the firearm 1808 image template are determined using the MSER method. FIG. 18C illustrates the radiographic image of FIG. 18A comprising a plurality of features marked thereon, in accordance with an embodiment of the present specification. In an embodiment, the features 1810 marked on the cargo container image 1802 are determined using the same MSER method as used to mark features 1806 on the firearm image 1808 template in FIG. 18B. As can be seen in FIG. 18C a plurality of regions within the cargo container 1802 have features 1810 that are similar to features 1806 marked on the firearm image template 1808. Whether or not each of these regions in the cargo container image 1802 shown in FIG. 18C is a firearms candidate is determined by measuring the rate of change of certain image properties (P) over each incremental distance ΔL in the region. If the measured rate of change (ΔP) is within a certain range of a template value, that is, within the upper and lower thresholds, then the ΔP measurement is repeated over the next ΔL, and so on. If all measured image properties are within the respective thresholds for the template, then the region and the template are determined to be “matched”, and this region is stored for further processing. However, if an image-property value is outside a template threshold range, then the region and the template are determined to be “unmatched”, and the unmatched region is eliminated as a firearms candidate. In this “thresholds of minimum-intensity increments” process, the image properties, the respective thresholds, and the incremental ΔL value can be varied to optimize the accuracy of firearms-candidate identification.

FIG. 18D illustrates an exemplary image obtained by comparing the MSER features marked in FIG. 18C with those marked in FIG. 18B, in accordance with an embodiment of the present specification. In FIG. 18D each white dot 1820 represents an image region comprising marked MSER features that matches with a similarly marked region in the template image shown in FIG. 18B. The robustness of the MSER based image processing method of the present specification is made evident by FIG. 18D illustrating that a single firearm template (FIG. 18B) can be used to identify a variety of firearm types, orientations, and sizes. In embodiments, obtaining an exact match with a template is not necessary, because the aim is to detect firearms as a whole, not their components. In embodiments, sometimes, some regions in FIG. 18D may be identified falsely as firearms, due to reasons such as, the use of imperfect template regions, the match threshold distance being too large, or certain image regions possessing characteristics similar to a firearms image.

In embodiments, similar to the process described above for cargo containers, for the detection of firearms in vehicles and in driver cabs of tractor units, a set of templates is used to generate MSER features used in a template-matching step as detailed with reference to step 1704 of FIG. 17. In an embodiment of the present specification, for cargo containers containing large quantities of cargo, an attenuation radiographic image is used for analysis, rather than a transmission radiographic image, in order to obtain accurate results.

Referring back to FIG. 17, at step 1706, the results obtained at step 1704 are filtered to minimize any matching errors. In an embodiment, a filtering method such as Speeded Up Robust Features (SURF), commonly known in the art, is used to identify salient features in the radiographic image shown in FIG. 18A. The SURF method is a fast and robust algorithm for local, similarity invariant representation and comparison of images. Similar to many other local descriptor-based approaches, using the SURF method, interest points of a given image are defined as salient features from a scale-invariant representation. Such a multiple-scale analysis is provided by the convolution of the initial image with discrete kernels at several scales (box filters). The filtering method also comprises building orientation invariant descriptors, by using local gradient statistics (intensity and orientation). The main advantage of using the SURF method for filtering the results obtained at step 1704, lies in its fast computation of operators using box filters, thus enabling real-time applications such as tracking and object recognition.

In an embodiment, at step 1706, salient/SURF features are identified in the radiographic image shown in FIG. 18A. FIG. 18E illustrates the radiographic image 1802 of FIG. 18A with markings 1830 representing SURF features, in accordance with an embodiment of the present specification. Since the MSER and SURF techniques generate different features, requiring a positive result from both, i.e., applying a MSER “and” SURF logic to each image region obtained at step 1704, reduces the matching error without compromising detection (true matches). As can be observed in FIG. 18C, a plurality of MSER feature markings 1810 are present near the ceiling of cargo container 1802, while in the image illustrated in FIG. 18E a plurality of SURF markings 1830 are found in the floor region of cargo container 1802. By application of the filtering method both these regions (floor and ceiling) having plurality of markings are discarded as an “AND” logic operation is applied. The application of both the MSER and SURF image processing techniques greatly enhances the probability of accurate detection of firearms as compared to prior art methods using only one type of image processing technique.

FIG. 18F illustrates a filtered version of the radiographic image shown in FIG. 18D after application of the filtering method, in accordance with an embodiment of the present specification. In FIG. 18F, each white dot 1840 represents an image region containing a possible firearm. The filtering process eliminates a plurality of matching errors. Comparison of FIG. 18F with FIG. 18D shows a significant reduction in the matched image regions denoting possible firearms. Further, in an embodiment, some feature markings present along the roof line shown in FIG. 18F, which actually represent matching errors, can be filtered out by applying image processing, wherein the first step is to determine that these markings are located on the roof lines of the container. The roof lines can be identified by their geometric locations with respect to the overall container, and by the uniformity of transmission and attenuation values along the lines. Then, if the regions are isolated within the roof line, and do not extend with template-like properties beyond the roof line, they can be eliminated from consideration as a firearm.

In an embodiment, another filtering method known in the art as Deep Belief Method (DBM) that utilizes a deep neural network composed of inter-connected multiple layers of feature variables may be used at step 1706. DBM is a machine-learning method and may further reduce the template-matching errors.

In embodiments, various other filtering methods may be used in case the radiographic image being analyzed comprises vehicles and driver cabs. These conveyances tend to contain greater clutter density than cargo containers, resulting in more obstructions. In addition, the images typically include engines, steering wheels, seats, glove compartments, and vehicle frames, and identification of these structures and their spatial relationships help to reduce matching errors. For example, using attenuation-based segmentation techniques, the engine can be extracted out, and its surrounding area can be identified by combining the information of the vehicle frame and the engine location. With this information, some template-matching errors associated with the vehicle or tractor-unit structure can be eliminated. Moreover, using the engine-location information together with the segmented passenger-seat information, their relative position can be established to provide the passenger leg-room area. Since vehicle leg-room imagery is invariant, any matched objects within this region would be more highly scrutinized.

At step 1708, once the filtering process is completed, additional properties of the filtered image regions are extracted and used as features for classifications. In embodiments, properties such as effective density, attenuation and associated standard deviation, and geometric properties such as size, area, and aspect ratio, may be extracted. Although these properties are common to all kind of conveyances, in different embodiments, unique classifier routines are applied to each. After the classification performed at step 1708 a set of image regions comprising firearm possibilities are obtained.

At step 1710, for each image region obtained at step 1708, a marker box is produced and laid upon the image region in the radiographic image under inspection to communicate an alarm location. FIG. 18G illustrates a radiographic image of a cargo container after application of the marker box, in accordance with an embodiment of the present specification. As shown, marker box 1850 is placed on the radiographic image of a cargo container denoting a location comprising firearms.

In another embodiment, the present specification discloses a system for automatically detecting the presence of currency, when a cargo container or a light vehicle is being inspected using non-intrusive X-ray imaging techniques. This allows the operator or inspector to quickly ascertain and verify the contents of the cargo container or vehicle that is currently being inspected. In one embodiment, the system described in the present specification automatically applies detection algorithms to the image and provides alerts to operator if it detects the presence of currency. In one embodiment, images of the cargo generated during a scan by an X-ray based inspection system are analyzed for specific features and parameters characteristic of currency, such as size, shape, texture, density, atomic number, and the location within the container or vehicle. This functionality enables security personnel to identify contraband more efficiently, without requiring opening or manual search of every container being inspected.

In one embodiment, the system of present specification provides several automated tools to make the image-inspection process more efficient and effective. These tools provide assistance to image operators in analyzing the images for contraband, particularly currency and also help in verifying that the cargo being carried is the same as declared in the cargo's manifest.

One of ordinary skill in the art would appreciate that the aforementioned automated tools can operate on any computing platform associated with the X-ray scanning system including, but not limited to: a laptop or tablet computer; personal computer; personal data assistant; cell phone; server; embedded processor; DSP chip or specialized imaging device capable of executing programmatic instructions or code.

FIG. 21 is a flowchart illustrating a method for identifying the presence of currency in an image of a scanned cargo container or vehicle, according to one embodiment of the present specification. Referring to FIG. 21, the first step 2101 of the method involves segmentation of the input image. The input image is the transmission X-ray image and represents the average of the high energy (HE) channel and low energy (LE) channel, which provides density and material discrimination information (atomic number). In one embodiment, the present specification uses a watershed segmentation method to segment the input image into meaningful physical objects based on the gradient magnitude.

The next step 2102 involves a dual-energy feature extraction. As explained earlier, the X-ray scanning system used with the present specification employs a dual energy method of scanning, which is useful for material discrimination. In step 2102, the following features are extracted for each segmented object obtained from the input image in step 2101:

a) Average Intensity: (HE+LE)/2

b) Intensity Difference: HE−LE

c) Mu Ratio: ln(HE)/ln(LE)

d) Mu Diff: ln(HE)−ln(LE)

e) Zeff (effective atomic number) estimated from the standard lookup table (LUT)

f) Zeff estimated from a new lookup table (LUT) generated by Mu Ratio

Where HE=High Energy channel and LE=Low Energy Channel.

One of ordinary skill in the art would appreciate that the above characteristics of objects in the image help in classifying the objects as being organic, inorganic, metallic, radioactive, etc., and also help in identifying the particular material composition of each segmented object. Further, the above features are determined (using various functions of the high energy and low energy values) and those determined features are compared against a values from a database storing feature values which have been previously determined for actual currency. In one embodiment, X-ray data is collected for vehicles that contain currency, and features specific of currency are established through analysis. In operation, these features are measured for the object under scan and compared to the established currency specific features.

In step 2103, currency candidate detection is performed. In this step, potential currency or currency-related objects are identified by thresholding the dual-energy features extracted in step 2102. In one embodiment, thresholds are established through analysis of x-ray images that contain currency in realistic environments, and through analysis of potential false-positive candidates.

In step 2104, texture feature extraction is performed. In one embodiment, the following second-order histogram-based texture features are extracted for each potential currency object identified in the previous step:

a) Energy

b) Entropy

c) Inverse different moment

d) Inertia

e) Cluster shade

f) Cluster prominence

In the following step 2105, objects identified in the previous steps that could potentially be currency or currency-related verified in order to determine currency objects. In one embodiment, to minimize false-positive detections, the dual-energy features extracted in step 2102 and the texture features extracted in step 2104 for each currency candidate are used in combination as input to a support vector machine classifier to determine whether it is a currency object.

Once a currency object is identified by the process of the present specification, an alert is issued to the operator of the scanning system so that they may take appropriate action.

In one embodiment, features and characteristics that form the basis of detection of currency and are determined as described above, are saved in a database for future reference. This image data can be referred by the scan operators to determine if a cargo currently being scanned has the features characteristic of currency. In one embodiment, these processes of saving relevant image data to the database and comparing the data with current images are integrated with the automated process of currency detection and help to substantially reduce false positives. These processes are illustrated by means of flowcharts in FIGS. 22 and 23.

FIG. 22 is a flowchart showing a process of preparing a features database. In the first step 2201, the system obtains the image of the container. The image is obtained through non-intrusive scanning by using a suitable scanning system, as described above. In one embodiment, radiographic images are generated that contain atomic-number information generated from any modality of dual-energy or dual-species inspection. The images could be generated by one or more views and could be three dimensional reconstructed from the views.

Thereafter, features of the image are computed, in step 2202. The process of computing features is explained above with reference to steps 2102 through 2104 in FIG. 21. In the next step 2203, computed features and their standard deviations are then saved in the database along with the number of images used to compute the features, and are associated with the particular contraband, which in this case is currency. In one embodiment, specific codes are assigned to different types of contraband to help associate specific features with them.

In one embodiment, the computed features include, but are not limited to, attenuation, texture, atomic number, elemental composition and/or cargo height. For tomographic and multi-view systems, density is also a useful feature. It should be understood by those of ordinary skill in the art that other features not listed here could be used to match the cargo images.

In the next step 2204, the system checks if any entries for that contraband are already stored in the database. If so, the system combines features from the containers with same type of contraband. This is shown in step 2205. The combination of the feature values takes into account the number of images used to compute the feature value and is weighted accordingly. Also, the user is notified of outlier feature values (values that are outside the three standard deviations or other selected range) for acceptance before the combination takes place. Thereafter the combined set of features for that particular contraband is saved in the database, as shown in step 2206. Thus, the features saved in the database per contraband are computed from a combination of feature values from a large number of cargo images found with the same contraband. The feature values are updated as additional cargo images are collected. Additional features can also be used computed as their usability becomes available.

FIG. 23 illustrates a method for comparing images of an individual cargo container with the database of contraband features. In the first step 2301, an image is captured by the cargo scanning system. Then, the features of the image are computed, in step 2302. Thereafter, the system obtains features for a specific contraband, such as currency, stored in a database, and compares them to the computed features. This is shown in step 2303. The system then determines the probability ‘P’ that some contents of cargo match the contraband, in step 2304. Probability ‘P’ is then compared to a threshold value in step 2305. If ‘P’ is greater than the threshold value, it implies that some contents of the cargo match with the features of the specific contraband, as shown in step 2306. If ‘P’ is less than the threshold value, it indicates that the contents of the cargo are not the same as the specific contraband, as shown in step 2307.

In one embodiment, the threshold value may be determined in accordance with the user's preferences. For example, if custom office is using the system and they want to detect most contraband even at the expense of higher false alarm rate, they may be able to set a high threshold value, such as 90%. Conversely, if the custom agency does not want to have a high false alarm rate, they can choose to set a low threshold value, such as 60%. Further, the customer may decide that some categories of goods are more important, such as those associated with higher duties, than others and place different thresholds for different types of goods.

In one embodiment, the system provides additional tools that help in the detection of currency carried in cargo. In one embodiment, an operator assist tool helps to determine when a cargo does not match manifest, thereby indicating presence of contraband. This tool helps in detecting situations such as cargo density being too low or too high for declared cargo, presence of organic material in cargo when manifest cargo is metallic or vice-versa and cargo being too uniform or heterogeneous for declared cargo, for example when an image shows high-density areas which are not compatible with a declared cargo of clothing. In other embodiments, tools for operator assistance include but are not limited to, automatic verification that a container/trailer/railcar is empty; Detection of hidden compartments; and Pallet counting.

Empty Container Verification

The goal of this tool is to verify that a cargo container is empty, and if the container is empty, to eliminate the need for the operator to view the image. Thus, this feature helps in improving operational efficiency of the scanning system. In one embodiment, the definition of not empty is the detection of any object greater than a specified size that attenuates the beam by a specified value. For example, any object greater than 1 cm in any direction that attenuates the beam by more than 5%. Other specifications as minimum detectable mass are also possible.

It may be noted that either the container and/or the system could be slightly tilted, increasing the width at the edges. Also, the truck might be passing through the system at different angles. Also, sometimes images contain artifacts due to bad detectors or linac instabilities. The algorithm for empty container verification takes into account these variations in one embodiment, and further is able to analyze container of any size, detect multiple containers and verify each of them separately.

The empty container verification tool is also useful in detecting more difficult cases of contraband smuggling, where the image of container structure may mask contraband. Examples of this type of smuggling include currency placed in door frames, door panels, container frame, corner blocks or other cases.

FIG. 24 shows an exemplary case where the empty container verification tool detects objects 2401 placed on the floor of the container 2402. The algorithm uses parallax to detect objects on the floor without alarming on container structure, such as beams 2403 and other truck support hardware. In the same manner, the tool is able to detect objects placed under the floor as well.

Cargo Uniformity

One the methods to perform cargo-manifest verification and determine the presence of contraband such as currency, is to check whether a cargo that is supposed to be uniform, also shows a uniform image. In one embodiment, a cargo database specifies which cargos are supposed to be uniform and the cargo uniformity tool alarms if the image is inconsistent with the manifest. Examples of uniform 2529 and non-uniform 2519 cargos are shown in FIGS. 25A and 25B, respectively.

In one embodiment, the algorithm for cargo uniformity is configured to detect uniformity based on a scale parameter “granularity”. The average density can also be used to verify the cargo type.

Material Discrimination

In one embodiment, dual-energy images are used for cargo manifest verification by checking whether the cargo is mainly organic, mainly metallic or combination. The images are then compared with the material type declared in the cargo manifest.

Anomaly Detection

In one embodiment, anomalies are automatically detected when the cargo is homogeneous and minimum characteristics of the cargo are specified. The algorithm relies on this information to detect any contraband concealed in the homogeneous cargo.

In one embodiment, anomalies are further automatically detected in periodic cargo. An exemplary case of periodic cargo is illustrated in FIG. 26. Referring to FIG. 26, the detection algorithm does not produce an alarm with repetitive objects 2601, but generates an alarm with an object 2602 of different attenuation or shape.

Pallet Anomalies

In one embodiment, data from manifest and cargo database are used to automatically identify anomalies such as hidden currency inside large bags that are used to carry dry groceries. The operator assist algorithm in this case determines whether the pallet should be homogeneous, and generates an alarm if it is not.

In some cases, it is useful to count the number of pallets to determine whether the shipper declared them all. In one embodiment, an automated tool detects the total number of pallets and indicates the corresponding locations of detected pallets. In one embodiment, the tool is able to determine when some pallets are placed behind others. FIG. 27 illustrates an exemplary image with pallet counting. As can be seen from the figure, the algorithm is sufficiently sophisticated to determine when pallets are placed behind others.

In one embodiment, an operator assist tool identifies the cargo types in the container. This is useful in cases where the manifest indicates more than one cargo type. As an example, in the image of a container carrying three cargo types, each of the cargo types may be indicated with a different color. One of ordinary skill in the art would appreciate that any other means for indication may be employed to illustrate different types of cargo.

In one embodiment, various operator assist tools may employ pattern recognition algorithms to detect and identify the type of contraband, including currency.

Image Comparison

This assistance tool allows the operator to invoke a number of images of certain type of cargo previously acquired and compare them with the image under analysis. This helps the operator in determining whether a cargo image matches the manifest, and indicates the potential presence of contraband if the present image is very different from the images in the database. This is shown in FIG. 28, wherein the current image 2801 of the cargo can be visually compared by the operator with images 2802 of the same cargo type from the image database. Additional assistance is provided by displaying values of various cargo features of the current and previously imaged cargo. In the example, shown, and by way of example only, the current image 2801 is different from the database images 2802. Thus, the operator should make a decision that the cargo does not match the manifest, because the current image is different from those in the database.

In various embodiments, operator assist tools are also useful in cases where contraband is concealed in parts of the truck other than the container or trailer—for example, when currency is concealed in the spare tire of a truck.

In one embodiment, an operator assist tool allows the operator to request images and analysis results, including past seizures, from previous inspections of a shipper or vehicle with same ID. This historical data helps the operator or automated software to provide a more intelligent decision.

X-ray screening of vehicles is often utilized at checkpoint entrances to Government facilities to search for illicit cargo such as drugs, weapons, and explosives. In many instances, a vehicle will pass through the same checkpoint on a regular basis. While the cargo within this vehicle will change from visit to visit, the vehicle structure (chassis, fuel tank, etc.) is expected to remain unchanged. Automated detection of differences in vehicle structure through inspection of sequential x-ray images of the same vehicle can improve security and reduce inspection times.

FIG. 29 is a flowchart illustrating a method of automatic threat detection, in accordance with one embodiment. Referring to FIG. 29, in the first step 2901, the vehicle under inspection is scanned using a high-energy transmission x-ray system. In one embodiment, upon entering the system, the vehicle license plate number or other vehicle identifier is read by the system's optical character recognition (OCR) software, and this unique character set is stored with the just-acquired image. In the next step 2902, the most recent vehicle image in the database is identified. For this, the system software uses the vehicle identifier to query the system database for images associated with the same identifier, and uploads the most recently-acquired of these stored images. Next, in step 2903 the current vehicle image is registered with the historic vehicle image. One of ordinary skill in the art would appreciate that a variety of image registration methods may be applied to determine the transformation that best maps the current image to the historic image.

In the next step 2904, the intensity profiles of the registered images are normalized and scaled similarly. After normalizing, image comparison is performed in step 2905. In this step, the intensity levels of the normalized images are compared on a pixel-by-pixel basis. In one embodiment, an optimized change detection algorithm is employed, wherein if the intensity difference between two pixels is greater than a predefined threshold, it is classified as a potential-difference pixel. The potential difference pixels are then clustered into potential-difference objects based on their spatial location. Finally, in step 2906 true anomalies are identified. In this step, each potential difference object is evaluated by a set of features, such as its size, location, and texture. Based on established alarm-decision criteria, the system reports the location of those potential-difference objects to the operator, which are determined to be true anomalies in need of investigation.

Image Registration Techniques

Image registration is required to compare two images of the same vehicle that differ geometrically. Differences may be due to variances in the respective scan processes, such as non-constant vehicle speed during the scan and differing magnification or view angle due to vehicle position or orientation with respect to the radiation source.

The canonical steps in image registration are detection/matching, transform model estimation, and image re-sampling. It may be appreciated that a variety of frequency-, intensity- and/or feature-based methods may be used to find matching control points to align the historic and current images. For example, a corner-matching feature-based method finds the transformation between two images by matching corresponding corners in the images. A normalized cross-correlation based method is an intensity-based method that finds the transformation between two images by finding the maximum of their normalized cross-correlation value. A priori knowledge of the geometry and ensuing transformations is essential to proper control-point selection.

The method of present specification may use one or more transformation methods, such as but not limited to, affine transformation, piecewise affine transformation, and deformable transformations. Transform model estimations under consideration include a mean-square difference, normalized correlation, and mutual information-based metrics. Both rigid transformation and non-rigid transformation methods, and their combinations, may be employed to account for the image-dimension variation between the current and the historic image due to differences in the scan geometry.

Filtering Techniques

In one embodiment, the present method uses one or more image-smoothing or filtering techniques for reducing image noise prior to the application of the change detection algorithm. Techniques that may be used include Gaussian Blur, a type of image-smoothing filter; Median Filtering, a noise-reduction technique; and Bilateral Filtering, a noise-reduction and edge-preserving technique.

Change Detection Algorithm

In one embodiment, the present method compares the current vehicle image with the historic vehicle image to identify significant differences between the two. For this purpose it makes use of one or more of the following approaches:

i) Ratio-Based Method

The observed pixel value or intensity at a location is defined as I(x, y), and then the ratio of the pixel value or intensities at location #2 to the value at location #1 can be expressed as

R(x,y)=I₂(x,y)/I₁(x,y)

Where I₂(x, y) refers to the current image and I₁(x, y) refers to the reference image. The ratio of intensities is expected to be a constant if the vehicle did not undergo any change, and hence we can expect R(x, y) to remain constant. This suggests that we can use the variance of R(x, y) in a window around a pixel as an indication of a change between the images. If this variance is higher than an established threshold, that pixel can be labeled as a change pixel.

ii) Statistical Method

In this method, the observation D is modeled as a Gaussian random variable with zero mean and variance σ²under the null hypothesis H₀, that is, the hypothesis that there is no change between the images. D can be any difference measure. The parameter σ²can be estimated as the sample variance of a training dataset. Hence the conditional density function may be presented as:

$f (D ❘ H_{0}) = \frac{1}{\sqrt{2 π σ^{2}}} e^{\frac{D^{2}}{2 σ^{2}}}$

The null hypothesis can be accepted if f(D|H₀) is greater than a threshold.

iii) Pixel Difference Method

In some embodiments, change detection is achieved by applying a pixel-difference method. On a pixel-by-pixel basis, the intensity levels of the normalized images are compared. If the intensity difference between two pixels is greater than a predefined threshold, it is classified as a potential-difference pixel.

Grouping Potential-Difference Objects

In one embodiment, potential-difference pixels are grouped into potential-difference objects based on their spatial connectivity, using a connected component labeling algorithm. This algorithm scans an image and groups its pixels into components based on pixel spatial connectivity. All pixels in a connected component have similar intensity values and are in some way connected with one other. Once all groups have been determined, each object is labeled with a gray-level or a color according to the component to which it was assigned.

Anomaly Identification

Classification methods are utilized, in one embodiment, to determine if a potential-difference object represents a true anomaly. The relevant features of a potential-difference object are classified, and objects that satisfy pre-defined classification criteria are labeled as true anomalies. For the purpose of image background subtraction and anomaly identification, some of the techniques that may be used, for example, include—Global Threshold using Otsu's Method, Adaptive threshold method, Edge-based method and Watershed-based method.

Vehicle Identification Number Capture

In one embodiment, for inspection of cargo and vehicles inspection, the present system provides an optional identification number capture system that automatically captures a license plate number, cargo container identification number, or other vehicle identification number. In one embodiment, the identification number capture system comprises cameras positioned to capture the license plate number or other identification number and OCR software that processes the camera images and extracts the ID number. The captured identification numbers are added to the corresponding inspection record.

FIGS. 30A through 30E illustrates an example of application of the automatic detection system and method, based on image comparison. FIG. 30A illustrates an exemplary historic image of a vehicle 3000 acquired by an inspection system. FIG. 30B illustrates the current image of the same vehicle, shown by the way of example only, with a contraband item 3001. Here, the current image is transformed (registered) to the space of the historic image.

Next, a pixel difference method is applied, to reveal the differences between the current image and historic image. This difference image is shown in FIG. 30C. It may be noted that any non-zero differences in the image of FIG. 30C, other than at the anomaly location, are due to random noise.

Next, using a thresholding technique, pixel-difference values are then evaluated against an established threshold value. Only those values greater than the threshold value survive to define potential-difference objects. A classification technique is then applied to determine which potential-difference objects represent true anomalies. In the present example, the only potential-difference object to survive the classification step is shown in the image of FIG. 30D, where the contraband item 3010 is clearly visible in the image.

The true anomaly is then transformed back to the space of the current image, and identified to the operator, as shown in FIG. 30E, where the contraband item 3030 is marked and presented on the display.

In one embodiment, an interface is provided for storing to, and recalling images from a database, based on vehicle identification number (VIN), license plate number, or other unique identifier. In one embodiment, the interface further comprises various operator assist tools, as described in the previous sections, to help in automatic identification of contraband and threat items. In one embodiment, the interface further comprises cargo manifest information management system and interface, as described above.

FIG. 31 illustrates an exemplary screen of the operator interface of the present system for threat/contraband detection based on vehicle image comparison. Referring to FIG. 31, when a cargo vehicle, such as a truck, comes for inspection scan, two images 3110 and 3120 of the same truck, acquired on different dates, are found in the reference image database. Since the database is also linked to the vehicle/cargo manifest system, it provides information to the operator regarding origin and destination of the cargo, shipper and consignee, container identification number and the vehicle license plate number in tables 3112 and 3122 alongside the scan images.

FIG. 32 illustrates how the current and historic vehicle images may be used for identifying changes in the vehicle structure over time, in accordance with one embodiment. Referring to FIG. 32, when a vehicle 3201 is scanned, its license plate number 3202 is also recorded. The license plate number is used to retrieve the historical images and data for the same vehicle from the system database. Thereafter, the current image 3203 is compared against a historical image 3204. A change detection algorithm is applied to the two images, which identifies anomalies 3205. In the present example, the identified anomaly comprises an object hidden under the floor of the vehicle, which becomes apparent when the two images are compared using the change detection algorithm.

FIG. 33 illustrates how vehicle images may be used for identifying changes in the cargo contents over a traveled distance. Referring to FIG. 33, a cargo vehicle 3301 is scanned at a first checkpoint 3302 and its image 3303 is recorded. The cargo vehicle is again scanned at a second checkpoint 3304 and its image 3305 is recorded. The system then automatically compares the two images 3303 and 3305 and applies a change detection algorithm to them. As mentioned earlier, the two images are linked in the system database by means of common attributes of the vehicle, such as the license plate. After comparison, the change detection algorithm identifies the anomalies, if any. In the present example, an additional pallet 3306 is detected in the cargo, which is added in the distance between the first and second checkpoints. In this case, if the cargo manifest remains unchanged, as reported by the manifest information database, the operator may flag down the vehicle for further inspection.

FIG. 34 illustrates another example of how vehicle images may be used for identifying changes in the cargo contents over a traveled distance. Referring to FIG. 34, a cargo vehicle 3401 is scanned at a first checkpoint 3402 and its image 3403 is recorded. The cargo vehicle is again scanned at a second checkpoint 3404 and its image 3405 is recorded. The system then automatically compares the two images 3403 and 3405 and applies a change detection algorithm to them. As mentioned earlier, the two images are linked in the system database by means of common attributes of the vehicle, such as the license plate. After comparison, the change detection algorithm identifies the anomalies, if any. In the present example, it is detected that the vehicle has emptied its load in the distance between the first and second checkpoints, and in fact the container in the second image 3405 is not the same as the container in the first image 3403.

In one embodiment, the method to obtain the weight of cargo comprises of combining the measured attenuation profile of the scanned cargo with the known geometry of the scanner. In one embodiment, for computing the pallets size and density, a priori assumptions about how the cargo is arranged in the container are used. Specifically, it is assumed that the cargo is arranged in one or two palletized rectangular blocks of uniform-density material that rest on the floor of the cargo container. This simplified model can be used to fit the measured attenuation profile to provide estimates of the dimensions including length, height and width of the cargo blocks, their front-to-back location in the container, and their effective steel density. For dual-energy scans, the atomic number information is used to improve the cargo characterization.

FIG. 35 is a flowchart illustrating an exemplary method of characterizing cargo by estimating cargo weight, according to one embodiment. Referring to FIG. 35, in the first step 3501, the relationship between steel thickness of the cargo blocks and attenuation for the scanner is established. Next, in 3502 a correction is developed for the angular dependence of the attenuation on steel thickness, using a model for the Linac spectrum and the known filtration on the radiation source. Thereafter in step 3503, image attenuations are converted to effective organic and steel thickness.

Next, in 3504 the atomic density of cargo is determined, if available. The sum of material thickness x density over height is then computed in step 3505, to get weight at given (length) position. Finally, all positions are summed to get total weight, in step 3506.

FIG. 36 is a flowchart illustrating an exemplary method of characterizing cargo by estimating pallets size and density, according to one embodiment. Referring to FIG. 36, in the first step 3601, the relationship between steel thickness of the pallet blocks and attenuation for the scanner is established. Next, in 3602 a correction is developed for the angular dependence of the attenuation on steel thickness, using a model for the Linac spectrum and the known filtration on the radiation source. Thereafter in step 3603, image attenuations are converted to effective organic and steel thickness.

Next, in 3604 the steel thickness profiles are corrected for background from the container. The resulting profiles are then fit with optimized pallets of cargo, varying the height, depth, and effective steel density of the cargos, as shown in step 3605. The resulting profiles are then compared with the known pallet data in step 3606. Optionally, the analysis may be extended to more irregular cargos so that it can be determined whether the assumptions are reasonable.

The above steps of flowcharts of FIGS. 35 and 36 are explained in further detail below.

Effective Steel Thickness and Attenuation

FIGS. 37, 38A and 38B illustrate how data collected using a radiation source, which is a 9 MV Linac in this example, is used to determine the relation between measured attenuation and steel thickness along the beam centerline, in accordance with one embodiment of the present specification. Referring to FIG. 37, a scan image of a cargo container with the test fixture holding 14″ (35.56 cm) of steel plate near the centerline of the Linac beam is shown. The attenuation profile is measured along the region-of-interest (ROI) 3701, yielding an average attenuation of about 16000 counts for the steel plate. The measurement is repeated for a number of scans as a function of the known steel thickness, which also contains a contribution from the container walls of 0.49 cm. The attenuations of the steel plates plus the container walls are then fit with a straight line, which is shown in FIG. 38A. Referring to FIG. 38A, graph 3801 illustrates a linear fit to steel thickness as a function of attenuation counts.

A 4-parameter fit to obtain the steel thickness from attenuation at this location in the detector array can be computed as:

y(cm steel)=(x−x₀)/mu−((x−x₀)/c)^m (1)

where:

x₀=88.76 counts, attenuation offset observed for air

mu=449 counts/cm, linear attenuation coefficient for steel

c=17345 counts, attenuation where the value of mu drops

m=7.505, exponent that indicates how fast mu changes.

The above computation is depicted by the graph 3802 of FIG. 38B, which shows that the fit predicts the steel thickness to an accuracy of better than 0.5 cm.

Correction for Angular Dependence

FIGS. 39A and 39B illustrate exemplary plots of the attenuations for 3.5″ and 7″ thick steel respectively, at different positions along the detector array. Referring to FIG. 39A, plots 3901 and 3902 represent attenuations for 3.5″ thick steel for two different 6 MV cargo scanner systems. Similarly in FIG. 39B, plots 3910 and 3920 represent attenuations for 7″ thick steel for two different 6 MV cargo scanner systems. This example illustrates the angular dependence of steel attenuation that results from a decrease in the energy of the x-ray spectrum with increasing angle.

In another example, the attenuation for a 9 MV-linac was measured and found to be around 3% lower than the 6 MV-linac attenuation. This may be associated with differences in Linac target thickness and filtration, while the down-turn in attenuation at the ends of the array is likely associated with scatter from material beyond the ends of the detector array where the flux is not attenuated by steel.

Using the results from the simulations, Equation (1) can be modified to include the angular dependence for three of the filters:

y(cm steel)=(x−x₀)/mu(θ)−((x−x₀)/c(θ)^m(θ) (2)

where the source-detector angle, θ(deg), is measured relative to the central detector, 270, and the detectors are numbered from 0 to 543 starting from the top of the array.

Fit to Uniform Cargoes

FIG. 40 shows a scan image for an exemplary uniform cargo comprising water bottles, used to develop the fitting procedure with respect to equations (1) and (2). Referring to FIG. 40, cargo is indicated by the rectangle 4001, while rectangle 4002 indicates a background region of interest (ROI). In one embodiment, equation (2) is multiplied by the nominal density of steel, 7.8 g/cm³, to yield the formula for converting image attenuation counts to areal density (g/cm²). Applying the background correction yields the cargo attenuation (areal density) shown as 4101 in FIG. 41. Application of least-square fit to the background-corrected steel areal density then yields the fits shown in FIG. 42 for one pallet of cargo 4201, and for two pallets of cargo 4202. The results of fits to attenuation profile of water bottles are summarized in a table in FIG. 43. Referring to FIG. 43, it can be seen that the cargo dimensions and characteristics obtained for single pallet 4301 and double pallet 4302 using the present method, are close to the measured dimensions and characteristics 4303.

In one embodiment, the present method also takes into account the air gap between pallets while computing the attenuation values to determine cargo characteristics. FIG. 44 illustrates how location and geometry of pallets can be inferred from the attenuation profiles obtained from various detectors in the detector array. Referring to FIG. 44, the rectangle 4401 represents the single pallet fit, while the rectangles 4402 and 4403 represent the double pallet fit. The outside corners of the cargo container are represented by 4411, 4412, 4413 and 4414. Projected lines 4405 from source through the corners indicate the detector number where the corners appear in the image. Rectangle 4406 indicates the position, where the inside walls of the cargo container are estimated to be located.

Referring together to FIGS. 42 and 44, the primary features in the areal density plot of FIG. 42 can be understood as follows. Starting from the bottom of the detector package in FIG. 44, the corner of the pallet on the right 4403 is first encountered, which leads to an increasing areal density. At the gap between pallets, the curve levels off, indicated by arrow 4203 in FIG. 42. This is because in that region, the x-rays only penetrate the left and right sides of the second cargo pallet 4403 with a near constant path length. The rays then encounter the lower right hand corner of the left pallet 4402 and the attenuation increases up to the top plateau 4205. This plateau is where rays pass through the sides of both cargo pallets, so there is only a small change in attenuation with height on the detector array. The subsequent decrease in attenuation then follows a similar set of transitions as the x-rays cross the gap between cargos, indicated by second arrow 4204 in FIG. 42.

In one embodiment, several types of uniform cargos were tested and it was found that cargos that are periodic in attenuation along the scan direction yield reasonably stable fit densities with the present method, as long as the region of interest is several periods wide. Since any changes in background attenuation due to cargo container features will limit the accuracy of the derived densities, in one embodiment, the present method further includes a useful tool for operator-assisted analysis. The tool enables an operator to select adjustable ROIs to exclude areas from the main ROI that show obvious changes in background.

Scans of Non-Uniform Cargo

FIG. 45 illustrates another exemplary container scan using the present method. Referring to FIG. 45, a non-uniform cargo comprising a container of the steel plates is scanned. The resulting attenuation graph 4501 representing steel penetration shows a small but high-attenuation region 4502. In this case, the curves 4503 for single and 4504 for double pallet scans yield nearly identical results, because the gap in the double pallet fit goes to zero.

FIG. 46 illustrates the inferred location and orientation of steel plates in the cargo container, corresponding to the attenuation profile of FIG. 45. Referring to FIG. 46, the outer and inner sides of the container are represented by 4601 and 4602, respectively. The location and orientation of steel plates in the cargo container is represented 4603.

It may be noted that cargos that are grossly non-uniform and are not rectangular in shape may not yield accurate cargo distributions. For example, if the steel plates in a cargo are stacked in a tilted or haphazard manner, it violates the basic assumption of the fit model that assumes square corners (90 deg) for the steel cargo with the plates standing upright. In such cases, additional scan views may be required to constrain the model further to provide a more accurate fit.

In one embodiment, the extracted cargo parameters are used for different weight calculations, including determination of pallet-by-pallet weight and full cargo weight.

In one embodiment, cargo weight (W) is estimated using the following equation, which is applied pallet-by-pallet:

$W = \sum \frac{\ln (I / I_{0})}{- μ_{M}} * pixelWidth * pixelHeight,$

where the sum is taken over all pixels in the volume of interest, μ_Mis the mass attenuation coefficient, and I/I₀represents the normalized pixel intensity value;

pixelWidth=containerLength in cm/containerlength in pixels; and

pixelHeight is obtained from the geometric information of the scanner.

To estimate the palletized-cargo weight, μ_M, pixelWidth and pixelHeight must be determined.

To determine μ_M: Based on high-energy data (HE) and low-energy data (LE), the material type is estimated by using μ_ratio. From the material type, the corresponding μ_Mis obtained using a look up table.

In one embodiment, the systems and methods of the present specification may be used for detection of specific contraband, such as cigarettes. FIG. 47 illustrates an exemplary dual-energy scan image of cigarettes 4738 in a cargo container 4758. The system of present specification takes into account the specific features and characteristics of cigarettes such us density, texture, spacing of the gaps, etc. to automatically detect the presence of cigarettes.

In one embodiment, the present method relies on various operator assist tools described earlier for detection of cigarettes, such as empty container verification, anomaly detection, pattern recognition, material discrimination, matching the cargo with manifest (verification) and pallet characteristics.

In one embodiment, the present system uses empty container verification tool to detect difficult cases of drugs concealed in containers, such as in door frames, door panels, container frame, corner blocks, in the floor or other cases where the image of container structure may mask contraband.

One of ordinary skill in the art would appreciate that dual-energy x-ray technology, even at high energy, cannot distinguish different types of organic materials. Therefore, anomalies in the x-ray images may require time-consuming, labor-intensive opening of containers potentially resulting in high false-alarm rate or in low detection due to the increased clearing to avoid opening containers.

Drugs are commonly found to be concealed in homogeneous organic cargo, such as that comprising fruits and vegetables or grains. Contraband can also be concealed in industrial equipment, and sometimes equipment must be damaged to determine whether there is contraband. If contraband is not found, the shipper must be compensated for the damage. There are other cases where intelligence indicates a high probability that contraband is hidden in a container or truck. In these and other cases, it would be useful to resolve the alarm without opening the container. Therefore in one embodiment, the system of present specification employs methods for determining the elemental composition of cargo for detection of contraband such as drugs and narcotics.

One of the methods to determine the elemental composition of cargo is employing neutrons. In this case, a pulsed neutron generator inspects an area of cargo suspected of containing contraband. A neighboring region could also be inspected for comparison. The results are an elemental map as a function of depth. The elemental composition is then employed to determine the probability that the cargo contains a specific type of contraband. The elemental composition of selected materials is shown in the table illustrated in FIG. 48. The last two columns 4801 and 4802 show some of features employed to identify the different materials. For example, a high a carbon-to-oxygen ratio (C/O) indicate the presence of cocaine, heroin and other drugs. Thus for example, a cargo of fruits and vegetables would result in a C/O of approximately 0.1. However, in the area where cocaine is present, the ratio would be about 3. If the ratio does not change significantly as a function of area, there is no need to open the container or destroy cargo, thereby saving considerable time and cost.

In one embodiment, the present system employs an additional concept of operation, by inspecting areas of the cargo for cargo-manifest verification. The elemental composition of the scanned areas is compared to the elemental composition of the claimed manifest to provide a probability that the manifest is correct. In one embodiment, the system further employs tools for detection of pallet anomalies for drug detection. Such tools determine, for example, if a pallet should be homogenous or not, based on the manifest and material data from the database. An alarm is raised in case an anomaly is found.

In various embodiments, a combination of operator assist tools are used and help identify cases where contraband is concealed in parts of the truck other than the container or trailer—for example, when drugs are concealed in the spare tire of a truck.

FIG. 19 is an illustration of an exemplary X-ray scanning system to which any of the methods of the present specification may be applied. It should be noted herein that one exemplary scanning and inspection system that may be employed with the systems and methods of the present specification includes, but is not limited to the Rapiscan Eagle Mobile inspection system. Other systems that are appropriate include inspection systems using a gantry, inspection systems having a portal configuration, and scanning systems designed specifically for rail cars. It may be noted that any suitable system for inspecting cargo, cargo containers, and their contents may be employed for the purpose of application of the methods of present specification. As such, U.S. patent application Ser. Nos. 12/780,910; 13/370,941; 13/548,873; 13/532,862; 13/168,440; 13/175,792; 13/433,270; 13/281,622; 13/108,039; 12/675,471; 12/993,831; 12/993,832; 12/993,834; 12/997,251; 12/919,482; 12/919,483; 12/919,484; 12/919,485; 12/919,486; 12/784,630; 12/784,465; 12/834,890; 13/009,765; 13/032,593; 13/368,178; and 13/368,202, all assigned to the assignee of the present invention represent various systems that may be employed with the present invention and are herein incorporated by reference in their entirety. In addition, U.S. Pat. Nos. 5,638,420; 6,542,580; 7,876,879; 7,949,101; 6,843,599; 7,483,510; 7,769,133; 7,991,113; 6,928,141; 7,517,149; 7,817,776; 7,322,745; 7,720,195; 7,995,705; 7,369,643; 7,519,148; 7,876,879; 7,876,880; 7,860,213; 7,526,064; 7,783,004; 7,963,695; 7,991,113; 8,059,781; 8,135,110, 8,170,177; 8,223,919; and 8,243,876 all assigned to the assignee of the present invention represent various screening systems that may be employed with the present invention are herein incorporated by reference in their entirety.

Referring to FIG. 19, in one embodiment, inspection system 1900 comprises a mobile inspection system that can be deployed at any checkpoint for inspecting vehicles and cargo containers. FIG. 19 illustrates a side view of the mobile inspection system in a deployed configuration. The system comprises a vehicle 1901, such as a truck, with a flat-bed surface and equipment mounted on it. The equipment comprises an X-ray source 1902, a boom 1903, a modulator 1904, chiller 1905 and hydraulics assembly 1906. A radiation detector 1907 is also mounted on the bed of the truck 1901.

The front part of the vehicle 1901 comprises the operator cab 1908. In one embodiment, the operator cab 1908 has two operator stations, each equipped with large high-resolution displays for scanning operations.

In one embodiment, the X-ray source 1902 and the boom 1903 are part of a single fixture that can be deployed or stowed.

In one embodiment, the mobile inspection system 1900 uses a dual energy mode of scanning. In this technique, an X-ray source emits alternating pulses of low and high energies. The difference or ratio of the transmissions for the two energies yields information about the atomic number (Z) of the cargo material traversed. Thus, in one embodiment X-ray source 1902 switches between two operating voltages in an interlaced fashion. In another embodiment, system 1900 comprises a first X-ray source 1902 and a second X-ray source (not shown) that deliver images from two views, horizontal and vertical, respectively. This allows the system to produce two separate simultaneous images from approximately perpendicular orientations, thereby providing more comprehensive imagery, reducing the need for repositioning and rescanning and enabling rapid and accurate threat detection. In one embodiment, both sources are of same type and have the same operating voltage, which is in the range of 200 kV. In one embodiment, the inspection system uses from three to five X-ray sources, each operating at a single voltage.

In one embodiment, the system provides dual view scanning, with the two views being generated by two X-ray sources or by a single source alternating operation between high and low energies. In one embodiment, each view is supported by an L-shaped array of dual-energy detectors that separate the x-rays received into a first energy bin and a second energy bin, wherein, in an embodiment, the first energy bin is low energy and the second energy bin is high energy. In one embodiment, each array contains two rows of detectors, with one set collecting the low-energy x-rays and the other set collecting the high-energy x-rays. This dual-energy data is the foundation for generating the atomic number (Z) information, thereby providing material discrimination essential for effective detection algorithms.

In one embodiment, the operating voltage of the X-ray source employed may range from 1 kV to 1 MeV, and can be varied depending upon the system in which the present invention is implemented. Further, it may be appreciated that an x-ray source that switches among two or more operating voltages may be used as well. Further, while the present example describes a dual-view system, it may be appreciated that a single-view or multi-view system may be used as well.

FIG. 20 illustrates an exemplary mobile inspection system in deployed mode, which may employ any of the methods described in the present specification for processing scanned images and detecting threats and contraband. A person of ordinary skill in the art would appreciate that the system is well capable of scanning large trucks and containers, such as the one 2002 shown in the figure, as well as smaller vehicles. As a vehicle 2002 passes through the portal 2003 created by the boom, it is scanned using the radiation from X-ray source (not shown). X-rays transmitted through (and attenuated by) the vehicle are detected by suitable detectors (not shown). In one embodiment, the detectors are integrated within the portions of the boom 2004 that are opposite to the X-ray source and in the beam path. An image is generated from the detected radiation and it forms the basis of material discrimination. Further details of X-ray scanning systems are provided in U.S. patent application Ser. No. 13/577,170, which is herein incorporated by reference in its entirety. The above examples are merely illustrative of the many applications of the system of present invention. Although only a few embodiments of the present invention have been described herein, it should be understood that the present invention might be embodied in many other specific forms without departing from the spirit or scope of the invention. Therefore, the present examples and embodiments are to be considered as illustrative and not restrictive, and the invention may be modified within the scope of the appended claims.

Claims

1. A method for determining a change in contents of a vehicle over time, the method comprising: receiving a first set of data representative of a first scanned image of a vehicle, wherein the first set of data representative of the first scanned image is generated at a first scanning location;receiving a second set of data representative of a second scanned image of the vehicle, wherein the second set of data representative of the second scanned image is generated at a second scanning location and wherein the first scanning location and second scanning location are physically separated; anddetermining the change in the contents of the vehicle between the first location and the second location by comparing the first scanned image to the second scanned image.
2. The method of claim 1, further comprising linking an identity of the vehicle at the first scanning location with an identity of the vehicle at the second scanning location based, at least in part, on data representative of a license plate number of the vehicle.
3. The method of claim 2, further comprising receiving the data representative of the license plate number of the vehicle at the first scanning location and receiving the data representative of the license plate number of the vehicle at the second scanning location.
4. The method of claim 2, wherein the identity of the vehicle further comprises at least one of an origin of the vehicle, a destination of the vehicle, a shipper information, a consignee information, or a container identification number.
5. The method of claim 1, wherein comparing the first scanned image to the second scanned image comprises: acquiring a first intensity value of a first pixel in the first scanned image;acquiring a second intensity value of a corresponding pixel in the second scanned image;determining a function of the second pixel intensity value and the first pixel intensity value; andusing a variance of the function in a window around at least one of the first pixel or the corresponding pixel to compare the first scanned image and the second scanned image.
6. The method of claim 5, wherein the function is a ratio of the second pixel intensity value and the first pixel intensity value.
7. The method of claim 1, wherein comparing the first scanned image to the second scanned image comprises modeling a measure of the change as a Gaussian random variable and calculating density as a function of the change.
8. The method of claim 1, wherein comparing the first scanned image to the second scanned image comprises: comparing pixel intensity levels of the first scanned image with pixel intensity levels of the second scanned image, on a pixel-by-pixel basis, to yield a plurality of pixel-difference values; andevaluating the pixel-difference values against a threshold.
9. The method of claim 8, wherein pixel-difference values greater than the threshold value indicate said change in the contents.
10. The method of claim 1, wherein determining the change in the contents comprises determining a difference object constituting the change.
11. The method of claim 10, further comprising determining a classification of the difference object, wherein the classification indicates a type of the difference object.
12. A computer readable non-transitory medium comprising a plurality of executable programmatic instructions wherein, when said plurality of executable programmatic instructions are executed by a processor, a process for determining a change in contents of a vehicle is executed, said plurality of executable programmatic instructions comprising: programmatic instructions, stored in said computer readable non-transitory medium, for receiving a first set of data representative of a first scanned image of a vehicle, wherein the first set of data representative of the first scanned image is generated at a first scanning location;programmatic instructions, stored in said computer readable non-transitory medium, for receiving a second set of data representative of a second scanned image of the vehicle, wherein the second set of data representative of the second scanned image is generated at a second scanning location and wherein the first scanning location and second scanning location are physically separated; andprogrammatic instructions, stored in said computer readable non-transitory medium, for determining the change in the contents of the vehicle between the first location and the second location by comparing the first scanned image to the second scanned image.
13. The computer readable non-transitory medium of claim 12, further comprising programmatic instructions for linking an identity of the vehicle at the first scanning location with an identity of the vehicle at the second scanning location based, at least in part, on a license plate number of the vehicle.
14. The computer readable non-transitory medium of claim 13, further comprising programmatic instructions, stored in said computer readable non-transitory medium, for receiving the license plate number of the vehicle at the first scanning location and receiving the license plate number of the vehicle at the second scanning location.
15. The computer readable non-transitory medium of claim 13, wherein the identity of the vehicle further comprises at least one of an origin of the vehicle, a destination of the vehicle, a shipper information, a consignee information, or a container identification number.
16. The computer readable non-transitory medium of claim 12, wherein the programmatic instructions for comparing the first scanned image to the second scanned image comprises: programmatic instructions, stored in said computer readable non-transitory medium, for acquiring a first pixel intensity value of a first pixel in the first scanned image;programmatic instructions, stored in said computer readable non-transitory medium, for acquiring a second pixel intensity value of a corresponding pixel in the second scanned image;programmatic instructions, stored in said computer readable non-transitory medium, for determining a function of the second pixel intensity value and the first pixel intensity value; andprogrammatic instructions, stored in said computer readable non-transitory medium, for using a variance of the function in a window around at least one of the pixel or the corresponding pixel to compare the first scanned image to the second scanned image.
17. The computer readable non-transitory medium of claim 12, wherein the programmatic instructions for comparing the first scanned image to the second scanned image comprises instructions for modeling a measure of the change as a Gaussian random variable and calculating density as a function of the change.
18. The computer readable non-transitory medium of claim 12, wherein the programmatic instructions for comparing the first scanned image to the second scanned image comprises: programmatic instructions, stored in said computer readable non-transitory medium, for comparing intensity levels of the first set of data representative of the first scanned image with intensity levels of the second set of data representative of the second scanned image, on a pixel-by-pixel basis, to generate pixel-difference values; andprogrammatic instructions, stored in said computer readable non-transitory medium, for evaluating the pixel-difference values against one or more threshold values.
19. The computer readable non-transitory medium of claim 18, wherein the programmatic instructions are further adapted to determine the change in contents if the pixel-difference values are greater than one or more threshold values.
20. The computer readable non-transitory medium of claim 12, wherein the programmatic instructions for comparing the first scanned image to the second scanned image comprises programmatic instructions, stored in said computer readable non-transitory medium, for determining a difference object constituting the change.
21. The computer readable non-transitory medium of claim 12 further comprising programmatic instructions, stored in said computer readable non-transitory medium, for determining a classification of the difference object, wherein the classification indicates a type of the difference object.

CROSS-REFERENCE

The present application is a continuation application of U.S. patent application Ser. No. 16/376,971, entitled “Systems and Methods for Detecting Threats and Contraband in Cargo” and filed on Apr. 5, 2019, which is a continuation application of U.S. patent application Ser. No. 15/431,011, of the same title, filed on Feb. 13, 2017, and issued as U.S. Pat. No. 10,302,807 on May 28, 2019, which, in turn, relies on U.S. Patent Provisional Application No. 62/298,383, entitled “Tanker Content Assessment” and filed on Feb. 22, 2016, for priority. In addition, U.S. patent application Ser. No. 15/431,011 relates to U.S. patent application Ser. No. 14/739,329, entitled “X-Ray Inspection System That Integrates Manifest Data With Imaging/Detection Processing”, filed on Jun. 15, 2015, and issued as U.S. Pat. No. 9,632,206 on Apr. 25, 2017, which is a continuation of U.S. patent application Ser. No. 13/606,442, of the same title, filed on Sep. 7, 2012, and issued as U.S. Pat. No. 9,111,331 on Aug. 18, 2015, which, in turn, relies on U.S. Patent Provisional Application No. 61/532,093, filed on Sep. 7, 2011, and entitled “X-Ray Inspection System with Integration of Manifest Data with Imaging/Detection Algorithms”. The above-referenced applications are incorporated herein by reference in their entirety.

US Referenced Citations (1091)

Number	Name	Date	Kind
2006	Read	Mar 1841	A
2636619	Alexander	Apr 1953	A
3124679	Tittman et al.	Mar 1964	A
3240971	Morgan	Mar 1966	A
3275831	Martin	Sep 1966	A
3374355	Parratt	Mar 1968	A
3439166	Chope	Apr 1969	A
3670164	Hardy et al.	Jun 1972	A
3780291	Stein	Dec 1973	A
3784827	Calhoun	Jan 1974	A
3790785	Paolini et al.	Feb 1974	A
3808444	Schneeberger et al.	Apr 1974	A
3832545	Bartko	Aug 1974	A
3835324	Weigle	Sep 1974	A
3837502	Hornagold	Sep 1974	A
3872287	Koeman	Mar 1975	A
3904923	Schwartz	Sep 1975	A
3924064	Nomura	Dec 1975	A
3980889	Haas	Sep 1976	A
3997787	Fearon et al.	Dec 1976	A
4020346	Dennis	Apr 1977	A
4047036	Smith	Sep 1977	A
4064440	Roder	Dec 1977	A
4158773	Novak	Jun 1979	A
4164138	Burkhart	Aug 1979	A
4173010	Hoffmann	Oct 1979	A
4217641	Naparstek	Aug 1980	A
4229654	Arya	Oct 1980	A
4239969	Galetta	Dec 1980	A
4247774	Brooks	Jan 1981	A
4251726	Alvarez	Feb 1981	A
4255659	Kaufman	Mar 1981	A
4338626	Lemelson	Jul 1982	A
4366382	Kotowski	Dec 1982	A
4379348	Haas	Apr 1983	A
4379481	Juner	Apr 1983	A
4383327	Kruger	May 1983	A
4418575	Hundt	Dec 1983	A
4430568	Yoshida	Feb 1984	A
4470303	Odonnell	Sep 1984	A
4480899	Sprague	Nov 1984	A
4482958	Nakayama	Nov 1984	A
4509075	Simms	Apr 1985	A
4535246	Shani	Aug 1985	A
4539648	Schatzki	Sep 1985	A
4558220	Evans	Dec 1985	A
4566113	Doenges	Jan 1986	A
4573198	Anderson	Feb 1986	A
4580219	Pelc	Apr 1986	A
4590558	Glover	May 1986	A
4598202	Koechner	Jul 1986	A
4599740	Cable	Jul 1986	A
4612666	King	Sep 1986	A
4637056	Sherman	Jan 1987	A
4651297	Schlunt	Mar 1987	A
4653109	Lemelson	Mar 1987	A
4658408	Amor	Apr 1987	A
4697594	Mayo, Jr.	Oct 1987	A
4709333	Crawford	Nov 1987	A
4722096	Dietrich	Jan 1988	A
4724543	Klevecz	Feb 1988	A
4725733	Horman et al.	Feb 1988	A
4736399	Okazaki	Apr 1988	A
4736401	Donges	Apr 1988	A
4737650	West	Apr 1988	A
4755680	Logan	Jul 1988	A
4756015	Doenges	Jul 1988	A
4759047	Donges	Jul 1988	A
4775895	Traupe	Oct 1988	A
4783794	Dietrich	Nov 1988	A
4788704	Donges	Nov 1988	A
4793261	Schwaemmle	Dec 1988	A
4795253	Sandridge	Jan 1989	A
4817123	Sones	Mar 1989	A
4819188	Matsubara	Apr 1989	A
4832447	Javidi	May 1989	A
4837733	Shiraishi	Jun 1989	A
4838644	Ochoa	Jun 1989	A
4841554	Doenges	Jun 1989	A
4849912	Leberl	Jul 1989	A
4862358	Kimura	Aug 1989	A
4869574	Hartman	Sep 1989	A
4870670	Geus	Sep 1989	A
4873708	Cusano	Oct 1989	A
4884289	Glockmann	Nov 1989	A
4887899	Hung	Dec 1989	A
4893015	Kubierschky	Jan 1990	A
4916722	Ema	Apr 1990	A
4933961	Rushbrooke	Jun 1990	A
4941162	Vartsky	Jul 1990	A
4955060	Katsuki et al.	Sep 1990	A
4957250	Hararat-Tehrani	Sep 1990	A
4973846	Lanza	Nov 1990	A
4989229	Negrelli	Jan 1991	A
5003616	Orita	Mar 1991	A
5012917	Gilbert	May 1991	A
5014293	Boyd	May 1991	A
5018178	Katsumata	May 1991	A
5020111	Weber	May 1991	A
5022062	Annis	Jun 1991	A
5034812	Rawlings	Jul 1991	A
5041728	Spacher	Aug 1991	A
5041993	Rawlings	Aug 1991	A
5056130	Engel	Oct 1991	A
5060249	Eisen	Oct 1991	A
5063602	Peppers	Nov 1991	A
5065418	Bermbach	Nov 1991	A
5070519	Stein	Dec 1991	A
5073782	Huguenin	Dec 1991	A
5079698	Grenier	Jan 1992	A
5091924	Bermbach	Feb 1992	A
5098640	Gozani	Mar 1992	A
5107351	Leib	Apr 1992	A
5109276	Nudelman	Apr 1992	A
5109691	Corrigan	May 1992	A
5125015	Shimoni	Jun 1992	A
5132811	Iwaki	Jul 1992	A
5132842	Yeh	Jul 1992	A
5132998	Tsutsui	Jul 1992	A
5138167	Barnes	Aug 1992	A
5150229	Takesue	Sep 1992	A
5151588	Kiri	Sep 1992	A
5162652	Cohen	Nov 1992	A
5175756	Pongratz	Dec 1992	A
5179581	Annis	Jan 1993	A
5181234	Smith	Jan 1993	A
5182764	Peschmann	Jan 1993	A
5185778	Magram	Feb 1993	A
5195629	Gottstein	Mar 1993	A
5197088	Vincent	Mar 1993	A
5198669	Namiki	Mar 1993	A
5200626	Schultz	Apr 1993	A
5202932	Cambier	Apr 1993	A
5216541	Takesue	Jun 1993	A
5237598	Albert	Aug 1993	A
5239595	Takemura	Aug 1993	A
5243664	Tuy	Sep 1993	A
5247561	Kotowski	Sep 1993	A
5257085	Ulich	Oct 1993	A
5257322	Matsuoka	Oct 1993	A
5259012	Baker	Nov 1993	A
5268967	Jang	Dec 1993	A
5283641	Lemelson	Feb 1994	A
5297222	Mori	Mar 1994	A
5298756	McCollum	Mar 1994	A
5299116	Owens	Mar 1994	A
5308986	Walker	May 1994	A
5309244	Katagiri et al.	May 1994	A
5309523	Iwaki	May 1994	A
5311359	Lucas	May 1994	A
5319544	Schmerer	Jun 1994	A
5319547	Krug	Jun 1994	A
5323004	Ettinger	Jun 1994	A
5323472	Falk	Jun 1994	A
5327286	Sampsell	Jul 1994	A
5339350	Thelosen	Aug 1994	A
5345081	Rogers	Sep 1994	A
5345173	Bito	Sep 1994	A
5363940	Fahrion	Nov 1994	A
5365560	Tam	Nov 1994	A
5365564	Yashida	Nov 1994	A
5367552	Peschmann	Nov 1994	A
5371542	Pauli	Dec 1994	A
5375156	Kuo-Petravic	Dec 1994	A
5376796	Chan	Dec 1994	A
5379334	Zimmer	Jan 1995	A
5379336	Kramer	Jan 1995	A
5388684	Peck	Feb 1995	A
5418380	Simon	May 1995	A
5420788	Vissers	May 1995	A
5425113	Ito	Jun 1995	A
5428657	Papanicolopoulos	Jun 1995	A
5430787	Norton	Jul 1995	A
5434415	Terada	Jul 1995	A
5464013	Lemelson	Nov 1995	A
5465284	Karellas	Nov 1995	A
5481584	Tang	Jan 1996	A
5481622	Gerhardt	Jan 1996	A
5483569	Annis	Jan 1996	A
5485312	Horner	Jan 1996	A
5490193	Kuroda	Feb 1996	A
5490218	Krug	Feb 1996	A
5493444	Khoury	Feb 1996	A
5493517	Frazier	Feb 1996	A
5493596	Annis	Feb 1996	A
5503424	Agopian	Apr 1996	A
5506880	Scardino	Apr 1996	A
5519225	Mohr	May 1996	A
5524133	Neale	Jun 1996	A
5528702	Mitsuoka	Jun 1996	A
5528703	Lee	Jun 1996	A
5541856	Hammermeister	Jul 1996	A
5546189	Svetkoff	Aug 1996	A
5568256	Korner	Oct 1996	A
5580471	Fukumoto	Dec 1996	A
5589162	Muraoka	Dec 1996	A
5591967	Moake	Jan 1997	A
5592561	Moore	Jan 1997	A
5595767	Cinquin	Jan 1997	A
5600303	Husseiny	Feb 1997	A
5600485	Iwaki	Feb 1997	A
5600700	Krug	Feb 1997	A
5604634	Khoury	Feb 1997	A
5606167	Miller	Feb 1997	A
5619596	Iwaki	Apr 1997	A
5625192	Oda	Apr 1997	A
5625717	Hashimoto	Apr 1997	A
5629669	Asano	May 1997	A
5638420	Armistead	Jun 1997	A
5642393	Krug	Jun 1997	A
5642394	Rothschild	Jun 1997	A
5647018	Benjamin	Jul 1997	A
5664574	Chance	Sep 1997	A
5668846	Fox	Sep 1997	A
5679956	Johnston	Oct 1997	A
5680525	Sakai	Oct 1997	A
5684565	Oshida	Nov 1997	A
5692028	Geus	Nov 1997	A
5692029	Husseiny	Nov 1997	A
5692446	Becker	Dec 1997	A
5698854	Gupta	Dec 1997	A
5699400	Lee	Dec 1997	A
5703921	Fujita	Dec 1997	A
5706816	Mochizuki	Jan 1998	A
5726449	Yoshiike	Mar 1998	A
5739539	Wang	Apr 1998	A
5740221	Norman	Apr 1998	A
5745542	Gordon	Apr 1998	A
5748305	Shimono	May 1998	A
5748697	Tam	May 1998	A
5754617	Itoh	May 1998	A
5754621	Suzuki	May 1998	A
5756875	Parker	May 1998	A
5757981	Kawakubo	May 1998	A
5761334	Nakajima	Jun 1998	A
5764683	Swift	Jun 1998	A
5764719	Noettling	Jun 1998	A
5768334	Maitrejean	Jun 1998	A
5777742	Marron	Jul 1998	A
5778046	Molloi	Jul 1998	A
5779641	Hatfield	Jul 1998	A
5784429	Arai	Jul 1998	A
5786597	Lingren	Jul 1998	A
5787145	Geus	Jul 1998	A
5794788	Massen	Aug 1998	A
5796802	Gordon	Aug 1998	A
5796868	Dutta-Choudhury	Aug 1998	A
5799100	Clarke	Aug 1998	A
5800355	Hasegawa	Sep 1998	A
5802133	Kawai	Sep 1998	A
5805660	Perion	Sep 1998	A
5809171	Neff	Sep 1998	A
5815198	Vachtsevanos	Sep 1998	A
5815264	Reed	Sep 1998	A
5828722	Ploetz	Oct 1998	A
5828774	Wang	Oct 1998	A
5834153	Hasegawa	Nov 1998	A
5835558	Maschke	Nov 1998	A
5835561	Moorman	Nov 1998	A
5838758	Krug	Nov 1998	A
5838759	Armistead	Nov 1998	A
5841828	Gordon	Nov 1998	A
5841907	Javidi	Nov 1998	A
5842578	Cordeiro	Dec 1998	A
5850465	Shimura	Dec 1998	A
5862198	Samarasekera	Jan 1999	A
5862258	Taylor	Jan 1999	A
5864598	Hsieh	Jan 1999	A
5866907	Drukier	Feb 1999	A
5870449	Lee	Feb 1999	A
5877849	Ramer	Mar 1999	A
5881123	Tam	Mar 1999	A
5893095	Jain	Apr 1999	A
5894345	Takamoto	Apr 1999	A
5895073	Moore	Apr 1999	A
5901196	Sauer	May 1999	A
5901198	Crawford	May 1999	A
5903623	Swift	May 1999	A
5909285	Beaty	Jun 1999	A
5909477	Crawford	Jun 1999	A
5909478	Polichar	Jun 1999	A
5910765	Slemon	Jun 1999	A
5910973	Grodzins	Jun 1999	A
5911139	Jain	Jun 1999	A
5917190	Yodh	Jun 1999	A
5926568	Chaney	Jul 1999	A
5933471	Kalvin	Aug 1999	A
5936249	Eisen	Aug 1999	A
5940468	Huang	Aug 1999	A
5943388	Tuemer	Aug 1999	A
5951474	Matsunaga	Sep 1999	A
5953452	Boone	Sep 1999	A
5958336	Duarte	Sep 1999	A
5960104	Conners	Sep 1999	A
5974111	Krug	Oct 1999	A
5978440	Kang	Nov 1999	A
5981949	Leahy	Nov 1999	A
5987095	Chapman	Nov 1999	A
5994706	Allen	Nov 1999	A
6005916	Johnson	Dec 1999	A
6008496	Winefordner	Dec 1999	A
6009142	Sauer	Dec 1999	A
6011266	Bell	Jan 2000	A
6011620	Sites	Jan 2000	A
6014628	Kovarik, Jr.	Jan 2000	A
6018561	Tam	Jan 2000	A
6018562	Willson	Jan 2000	A
6031890	Bermbach	Feb 2000	A
6035014	Hiraoglu	Mar 2000	A
6043870	Chen	Mar 2000	A
6049381	Reintjes	Apr 2000	A
6056671	Marmer	May 2000	A
6057761	Yukl	May 2000	A
6057909	Yahav	May 2000	A
6058158	Eiler	May 2000	A
6058159	Conway	May 2000	A
6060677	Ulrichsen	May 2000	A
6070583	Perelman	Jun 2000	A
6075591	Vokhmin	Jun 2000	A
6075880	Kollhof	Jun 2000	A
6076400	Bechwati	Jun 2000	A
6078638	Sauer	Jun 2000	A
6080994	Carrott	Jun 2000	A
6081580	Grodzins	Jun 2000	A
6084939	Tamura	Jul 2000	A
6088423	Krug	Jul 2000	A
6094472	Smith	Jul 2000	A
6097427	Dey	Aug 2000	A
6097483	Komatsu	Aug 2000	A
6118850	Mayo	Sep 2000	A
6149300	Greenway	Nov 2000	A
6151381	Grodzins	Nov 2000	A
6153873	Wolf	Nov 2000	A
6155179	Aust	Dec 2000	A
6157730	Roever	Dec 2000	A
6163403	Carrott	Dec 2000	A
6163591	Benjamin	Dec 2000	A
6175417	Do	Jan 2001	B1
6175613	Boutenko	Jan 2001	B1
6188747	Geus	Feb 2001	B1
6195413	Geus	Feb 2001	B1
6195444	Simanovsky	Feb 2001	B1
6198795	Naumann	Mar 2001	B1
6205195	Lanza	Mar 2001	B1
6205243	Migdal	Mar 2001	B1
6216540	Nelson	Apr 2001	B1
6218943	Ellenbogen	Apr 2001	B1
6220099	Marti	Apr 2001	B1
6222902	Lin	Apr 2001	B1
6229872	Amos	May 2001	B1
6233303	Tam	May 2001	B1
6236704	Navab	May 2001	B1
6236708	Lin	May 2001	B1
6246778	Moore	Jun 2001	B1
6249341	Basiji	Jun 2001	B1
6249567	Rothschild	Jun 2001	B1
6252929	Swift	Jun 2001	B1
6255654	Verbinski	Jul 2001	B1
6256370	Yavuz	Jul 2001	B1
6256404	Gordon	Jul 2001	B1
6263044	Joosten	Jul 2001	B1
6263231	Reitter	Jul 2001	B1
6266393	Ein-Gal	Jul 2001	B1
6271510	Boxen	Aug 2001	B1
6272204	Amtower	Aug 2001	B1
6272230	Hiraoglu	Aug 2001	B1
6272233	Takeo	Aug 2001	B1
6278760	Ogawa	Aug 2001	B1
6282258	Stein	Aug 2001	B1
6285030	Williams	Sep 2001	B1
6288974	Nelson	Sep 2001	B1
6289235	Webber	Sep 2001	B1
6292260	Lin	Sep 2001	B1
6292530	Yavus	Sep 2001	B1
6292533	Swift	Sep 2001	B1
6301327	Martens	Oct 2001	B1
6317509	Simanovsky	Nov 2001	B1
6324243	Edic	Nov 2001	B1
6324245	Tam	Nov 2001	B1
6345113	Crawford	Feb 2002	B1
6347132	Annis	Feb 2002	B1
6353673	Shnitser	Mar 2002	B1
6366638	Hsieh	Apr 2002	B1
6370222	Cornick	Apr 2002	B1
6373916	Inoue	Apr 2002	B1
6373970	Dong	Apr 2002	B1
6373979	Wang	Apr 2002	B1
6380540	Maor	Apr 2002	B1
6381297	Hsieh	Apr 2002	B1
6388788	Harris	May 2002	B1
6403960	Wellnitz	Jun 2002	B1
6404841	Pforr	Jun 2002	B1
6407390	Rozsa	Jun 2002	B1
6408042	Hsieh	Jun 2002	B1
6415012	Taguchi	Jul 2002	B1
6418184	Wang	Jul 2002	B1
6418189	Schafer	Jul 2002	B1
6418194	McPherson	Jul 2002	B1
6424692	Suzuki	Jul 2002	B1
6430255	Fenkart	Aug 2002	B2
6438577	Owens	Aug 2002	B1
6442288	Haerer	Aug 2002	B1
6445765	Frank	Sep 2002	B1
6448545	Chen	Sep 2002	B1
6453003	Springer	Sep 2002	B1
6459755	Li	Oct 2002	B1
6459761	Grodzins	Oct 2002	B1
6459764	Chalmers	Oct 2002	B1
6463181	Duarte	Oct 2002	B2
6473489	Bani-Hashemi	Oct 2002	B2
6477221	Ning	Nov 2002	B1
6479826	Klann	Nov 2002	B1
6480285	Hill	Nov 2002	B1
6480564	Kim	Nov 2002	B1
6483894	Hartick	Nov 2002	B2
6487307	Hennessey	Nov 2002	B1
6502984	Ogura	Jan 2003	B2
6507025	Verbinski	Jan 2003	B1
6507278	Brunetti	Jan 2003	B1
6515285	Marshall	Feb 2003	B1
6525331	Ngoi	Feb 2003	B1
6526120	Gray	Feb 2003	B1
6532276	Hartick	Mar 2003	B1
6542574	Grodzins	Apr 2003	B2
6542578	Ries	Apr 2003	B2
6542579	Takasawa	Apr 2003	B1
6542580	Carver	Apr 2003	B1
6542628	Muller	Apr 2003	B1
6545281	McGregor	Apr 2003	B1
6549683	Bergeron	Apr 2003	B1
6552346	Verbinski	Apr 2003	B2
6552809	Bergeron	Apr 2003	B1
6559769	Anthony	May 2003	B2
6570177	Struckhoff	May 2003	B1
6570708	Bergeron	May 2003	B1
6570951	Hsieh	May 2003	B1
6570956	Rhee	May 2003	B1
6574296	Stierstorfer	Jun 2003	B2
6574297	Tam	Jun 2003	B2
6580777	Ueki	Jun 2003	B1
6580778	Meder	Jun 2003	B2
6583895	Kuwahara	Jun 2003	B1
6584170	Aust	Jun 2003	B2
6586193	Yguerabide	Jul 2003	B2
6587575	Windham	Jul 2003	B1
6587595	Henkel	Jul 2003	B1
6597760	Beneke	Jul 2003	B2
6603536	Hasson	Aug 2003	B1
6608921	Inoue	Aug 2003	B1
6611575	Alyassin	Aug 2003	B1
6614872	Bueno	Sep 2003	B2
6618466	Ning	Sep 2003	B1
6621887	Albagli	Sep 2003	B2
6621888	Grodzins	Sep 2003	B2
6621925	Ohmori	Sep 2003	B1
6628745	Annis	Sep 2003	B1
6628982	Thomas	Sep 2003	B1
6628983	Gagnon	Sep 2003	B1
6636581	Sorenson	Oct 2003	B2
6637266	Froom	Oct 2003	B1
6644853	Kantor	Nov 2003	B1
6654443	Hoffman	Nov 2003	B1
6661867	Mario	Dec 2003	B2
6663280	Doenges	Dec 2003	B2
6665373	Kotowski	Dec 2003	B1
6671508	Mitsuoka	Dec 2003	B1
6702459	Barnes	Mar 2004	B2
6707879	McClelland	Mar 2004	B2
6713773	Lyons	Mar 2004	B1
6714623	Sako	Mar 2004	B2
6721387	Naidu	Apr 2004	B1
6721391	McClelland	Apr 2004	B2
6724922	Vilsmeier	Apr 2004	B1
6727506	Mallette	Apr 2004	B2
6731819	Fukushima	May 2004	B1
6735274	Zahavi	May 2004	B1
6735279	Jacobs	May 2004	B1
6738450	Barford	May 2004	B1
6744909	Kostrzewski	Jun 2004	B1
6746864	McNeil	Jun 2004	B1
6751349	Matama	Jun 2004	B2
6754374	Miller	Jun 2004	B1
6763148	Sternberg	Jul 2004	B1
6768421	Alioto	Jul 2004	B1
6785357	Bernardi	Aug 2004	B2
6785410	Vining	Aug 2004	B2
6791089	Caffrey	Sep 2004	B1
H2110	Newman	Oct 2004	H
6801647	Arakawa	Oct 2004	B1
6803997	Stanek	Oct 2004	B2
6804412	Wilkinson	Oct 2004	B1
6807458	Quackenbush	Oct 2004	B2
6813395	Kinjo	Nov 2004	B1
6819790	Suzuki	Nov 2004	B2
6825854	Beneke	Nov 2004	B1
6829585	Grewal	Dec 2004	B1
6837422	Meder	Jan 2005	B1
6839403	Kotowski	Jan 2005	B1
6839406	Ries	Jan 2005	B2
6843599	Le	Jan 2005	B2
6845873	Chattey	Jan 2005	B1
6856272	Levitan	Feb 2005	B2
6865287	Beneke	Mar 2005	B1
6865509	Hsiung	Mar 2005	B1
6868138	Clinthorne	Mar 2005	B2
6873261	Anthony	Mar 2005	B2
6876322	Keller	Apr 2005	B2
6891470	Bohinc, Jr.	May 2005	B2
6895072	Schrock	May 2005	B2
6895338	Hsiung	May 2005	B2
6899540	Neiderman	May 2005	B1
6918541	Knowles	Jul 2005	B2
6920197	Kang	Jul 2005	B2
6922461	Kang	Jul 2005	B2
6924487	Bolozdynya	Aug 2005	B2
6928141	Carver	Aug 2005	B2
6936828	Saccomanno	Aug 2005	B2
6937692	Johnson	Aug 2005	B2
6938488	Diaz	Sep 2005	B2
6940943	Claus	Sep 2005	B2
6950492	Besson	Sep 2005	B2
6952163	Huey	Oct 2005	B2
6970531	Eberhard	Nov 2005	B2
6972693	Brown	Dec 2005	B2
6980681	Hsieh	Dec 2005	B1
6982643	Garfinkle	Jan 2006	B2
6990171	Toth	Jan 2006	B2
6998617	Demilio	Feb 2006	B2
7000827	Meder	Feb 2006	B2
7005982	Frank	Feb 2006	B1
7012256	Roos	Mar 2006	B1
7020241	Beneke	Mar 2006	B2
7026944	Alioto	Apr 2006	B2
7030755	Bohinc, Jr.	Apr 2006	B2
7043474	Mojsilovic	May 2006	B2
7045787	Verbinski	May 2006	B1
7046761	Ellenbogen	May 2006	B2
7046768	Gilevich	May 2006	B1
7050616	Hsieh	May 2006	B2
7062074	Beneke	Jun 2006	B1
7064336	Archer	Jun 2006	B2
7065175	Green	Jun 2006	B2
7068751	Toth	Jun 2006	B2
7072434	Tybinkowski	Jul 2006	B1
7092485	Kravis	Aug 2006	B2
7098461	Endo	Aug 2006	B2
7099004	Masten	Aug 2006	B2
7099432	Ichihara	Aug 2006	B2
7100165	Eldridge	Aug 2006	B2
7103137	Seppi	Sep 2006	B2
7105828	Unger	Sep 2006	B2
7115875	Worstell	Oct 2006	B1
7116235	Alioto	Oct 2006	B2
7116749	Besson	Oct 2006	B2
7130456	Hillmann	Oct 2006	B2
7136716	Hsiung	Nov 2006	B2
7139406	McClelland	Nov 2006	B2
7142109	Frank	Nov 2006	B1
7142633	Eberhard	Nov 2006	B2
7151447	Willms	Dec 2006	B1
7154650	Lettington	Dec 2006	B2
7164138	McGregor	Jan 2007	B2
7164750	Nabors	Jan 2007	B2
7183554	Gallagher	Feb 2007	B2
7183906	Zanovitch	Feb 2007	B2
7193515	Roberts	Mar 2007	B1
7203276	Arsenault	Apr 2007	B2
7204125	Fine	Apr 2007	B2
7207713	Lowman	Apr 2007	B2
7212113	Zanovitch	May 2007	B2
7212661	Samara	May 2007	B2
7215738	Muenchau	May 2007	B2
7233644	Bendahan	Jun 2007	B1
7233682	Levine	Jun 2007	B2
7244941	Roos	Jul 2007	B2
7257189	Modica	Aug 2007	B2
7269527	Frank	Sep 2007	B1
7322745	Agrawal	Jan 2008	B2
7324921	Sugahara	Jan 2008	B2
7356115	Ford	Apr 2008	B2
7356174	Leue	Apr 2008	B2
7366282	Peschmann	Apr 2008	B2
7369643	Kotowski	May 2008	B2
7379530	Hoff	May 2008	B2
7391028	Rubenstein	Jun 2008	B1
7397891	Johnson	Jul 2008	B2
7400701	Cason	Jul 2008	B1
7411198	Holland	Aug 2008	B1
7417440	Peschmann	Aug 2008	B2
7418077	Gray	Aug 2008	B2
7430479	Holslin	Sep 2008	B1
7453987	Richardson	Nov 2008	B1
7471764	Kaval	Dec 2008	B2
7483510	Carver	Jan 2009	B2
7486768	Allman	Feb 2009	B2
7517149	Agrawal	Apr 2009	B2
7519148	Kotowski	Apr 2009	B2
7525101	Grodzins	Apr 2009	B2
7526064	Akery	Apr 2009	B2
7529341	Schlomka	May 2009	B2
7545965	Suzuki	Jun 2009	B2
7550738	Devito	Jun 2009	B1
7579845	Peschmann	Aug 2009	B2
7592601	Frank	Sep 2009	B2
7660388	Gray	Feb 2010	B2
7720194	Connelly	May 2010	B2
7720195	Allman	May 2010	B2
7734066	Delia	Jun 2010	B2
7734102	Bergeron	Jun 2010	B2
7742568	Smith	Jun 2010	B2
7759649	Frank	Jul 2010	B2
7769132	Hurd	Aug 2010	B1
7769133	Carver	Aug 2010	B2
7783004	Kotowski	Aug 2010	B2
7783005	Kaval	Aug 2010	B2
7792248	Strecker	Sep 2010	B2
7813540	Kraft	Oct 2010	B1
7817776	Agrawal	Oct 2010	B2
7851766	Frank	Dec 2010	B2
7856081	Peschmann	Dec 2010	B2
7860213	Akery	Dec 2010	B2
7876879	Morton	Jan 2011	B2
7876880	Kotowski	Jan 2011	B2
7899232	Gudmundson	Mar 2011	B2
7915596	Clothier	Mar 2011	B2
7928400	Diawara	Apr 2011	B1
7963695	Kotowski	Jun 2011	B2
7973697	Reilly	Jul 2011	B2
7982191	Friedman	Jul 2011	B2
7991133	Mills	Aug 2011	B2
7991242	Perron	Aug 2011	B2
7995705	Allman	Aug 2011	B2
8054938	Kaval	Nov 2011	B2
8059781	Agrawal	Nov 2011	B2
8073099	Niu	Dec 2011	B2
8116428	Gudmundson	Feb 2012	B2
8135110	Morton	Mar 2012	B2
8138770	Peschmann	Mar 2012	B2
8170177	Akery	May 2012	B2
8173970	Inbar	May 2012	B2
8243167	Liang	Aug 2012	B2
8243876	Morton	Aug 2012	B2
8275091	Morton	Sep 2012	B2
8284895	Haff	Oct 2012	B1
8304740	Frank	Nov 2012	B1
8356937	Kotowski	Jan 2013	B2
8385501	Allman	Feb 2013	B2
8389942	Morton	Mar 2013	B2
8428217	Peschmann	Apr 2013	B2
8433036	Morton	Apr 2013	B2
8457275	Akery	Jun 2013	B2
8472583	Star-Lack	Jun 2013	B2
8483356	Bendahan	Jul 2013	B2
8491189	Kotowski	Jul 2013	B2
8503605	Morton	Aug 2013	B2
8579506	Morton	Nov 2013	B2
8644453	Morton	Feb 2014	B2
8668386	Morton	Mar 2014	B2
8674706	Peschmann	Mar 2014	B2
8687765	Kotowski	Apr 2014	B2
8735833	Morto	May 2014	B2
8750452	Kaval	Jun 2014	B2
8774357	Morton	Jul 2014	B2
8798232	Bendahan	Aug 2014	B2
8831176	Morto	Sep 2014	B2
8837670	Akery	Sep 2014	B2
8840303	Morton	Sep 2014	B2
8885938	Hong	Nov 2014	B2
8903128	Shet	Dec 2014	B2
8908831	Bendahan	Dec 2014	B2
8929509	Morton	Jan 2015	B2
8958526	Morton	Feb 2015	B2
8971485	Morton	Mar 2015	B2
8993970	Morton	Mar 2015	B2
9014425	Perron	Apr 2015	B2
9020095	Morton	Apr 2015	B2
9020096	Allman	Apr 2015	B2
9025731	Kotowski	May 2015	B2
9042511	Peschmann	May 2015	B2
9052403	Morton	Jun 2015	B2
9057679	Morton	Jun 2015	B2
9086497	Bendahan	Jul 2015	B2
9111331	Parikh	Aug 2015	B2
9121958	Morton	Sep 2015	B2
9158027	Morton	Oct 2015	B2
9218933	Langeveld	Dec 2015	B2
9223049	Kotowski	Dec 2015	B2
9223050	Kaval	Dec 2015	B2
9223052	Morton	Dec 2015	B2
9268058	Peschmann	Feb 2016	B2
9274065	Morton	Mar 2016	B2
9279901	Akery	Mar 2016	B2
9285498	Carver	Mar 2016	B2
9310322	Panesar	Apr 2016	B2
9310323	Bendahan	Apr 2016	B2
9316760	Bendahan	Apr 2016	B2
9329285	Gozani	May 2016	B2
9332624	Morton	May 2016	B2
10059569	Fang	Aug 2018	B2
10210418	Liu	Feb 2019	B2
20010016030	Nicolas	Aug 2001	A1
20010021013	Hecht	Sep 2001	A1
20010021244	Suzuki	Sep 2001	A1
20010028696	Yamada	Oct 2001	A1
20010033636	Hartick	Oct 2001	A1
20010038681	Stanton	Nov 2001	A1
20010038705	Rubbert	Nov 2001	A1
20010038707	Ohara	Nov 2001	A1
20010048734	Uppaluri	Dec 2001	A1
20010053197	Murayama	Dec 2001	A1
20020000916	Richards	Jan 2002	A1
20020001366	Tamura	Jan 2002	A1
20020015475	Matsumoto	Feb 2002	A1
20020016546	Cerofolini	Feb 2002	A1
20020017620	Oomori	Feb 2002	A1
20020018199	Blumenfeld	Feb 2002	A1
20020024016	Endo	Feb 2002	A1
20020027970	Chapman	Mar 2002	A1
20020028994	Kamiyama	Mar 2002	A1
20020031246	Kawano	Mar 2002	A1
20020037068	Oikawa	Mar 2002	A1
20020044691	Matsugu	Apr 2002	A1
20020049660	Obrador	Apr 2002	A1
20020054694	Vachtsevanos	May 2002	A1
20020067259	Fufidio	Jun 2002	A1
20020067793	Stierstorfer	Jun 2002	A1
20020085046	Furuta	Jul 2002	A1
20020088952	Rao	Jul 2002	A1
20020094062	Dolazza	Jul 2002	A1
20020094064	Zhou	Jul 2002	A1
20020094119	Sahadevan	Jul 2002	A1
20020098518	Levinson	Jul 2002	A1
20020106052	Menhardt	Aug 2002	A1
20020122528	Besson	Sep 2002	A1
20020124664	Call	Sep 2002	A1
20020126800	Matsumoto	Sep 2002	A1
20020127586	Mortensen	Sep 2002	A1
20020141625	Nelson	Oct 2002	A1
20020150200	Zonneveld	Oct 2002	A1
20020161534	Adler	Oct 2002	A1
20020168083	Garms	Nov 2002	A1
20020168657	Chen	Nov 2002	A1
20020172324	Ellengogen	Nov 2002	A1
20020172409	Saito	Nov 2002	A1
20020175291	Reeder	Nov 2002	A1
20020175921	Xu	Nov 2002	A1
20020176351	Masaki	Nov 2002	A1
20020176534	Meder	Nov 2002	A1
20020186862	McClelland	Dec 2002	A1
20020188197	Bishop	Dec 2002	A1
20020191209	Yasumaru	Dec 2002	A1
20020198731	Barnes	Dec 2002	A1
20030012420	Verwoerd	Jan 2003	A1
20030023469	Lee	Jan 2003	A1
20030023592	Modica	Jan 2003	A1
20030024315	Merkel	Feb 2003	A1
20030031289	Hsieh	Feb 2003	A1
20030031291	Yamamoto	Feb 2003	A1
20030036006	Feke	Feb 2003	A1
20030038945	Mahner	Feb 2003	A1
20030043964	Sorenson	Mar 2003	A1
20030068557	Kumashiro	Apr 2003	A1
20030072414	Sakaida	Apr 2003	A1
20030072418	Albagli	Apr 2003	A1
20030072484	Kokko	Apr 2003	A1
20030076924	Mario	Apr 2003	A1
20030081720	Swift	May 2003	A1
20030081859	Kasutani	May 2003	A1
20030082516	Straus	May 2003	A1
20030085163	Chan	May 2003	A1
20030085348	Megerle	May 2003	A1
20030085353	Almogy	May 2003	A1
20030091145	Mohr	May 2003	A1
20030095633	Vanwoezik	May 2003	A1
20030095692	Mundy	May 2003	A1
20030108150	Franke	Jun 2003	A1
20030128812	Appleby	Jul 2003	A1
20030138147	Ongkojoyo	Jul 2003	A1
20030144800	Davis	Jul 2003	A1
20030148393	Woodbury	Aug 2003	A1
20030149346	Arnone	Aug 2003	A1
20030165213	Maglich	Sep 2003	A1
20030179853	Amemiya	Sep 2003	A1
20030194121	Eberhard	Oct 2003	A1
20030201394	Peoples	Oct 2003	A1
20030205676	Nelson	Nov 2003	A1
20030206649	Moshe	Nov 2003	A1
20030210139	Brooks	Nov 2003	A1
20030215051	Suzuki	Nov 2003	A1
20030215054	Fenkart	Nov 2003	A1
20030215143	Zakrzewski	Nov 2003	A1
20030231788	Yukhin	Dec 2003	A1
20030231791	Torre-Bueno	Dec 2003	A1
20040012853	Garcia	Jan 2004	A1
20040013239	Gregerson	Jan 2004	A1
20040016271	Shah	Jan 2004	A1
20040017882	Misawa	Jan 2004	A1
20040017883	Takagi	Jan 2004	A1
20040017888	Seppi	Jan 2004	A1
20040017935	Avinash	Jan 2004	A1
20040022425	Avinash	Feb 2004	A1
20040027127	Mills	Feb 2004	A1
20040037462	Lewis	Feb 2004	A1
20040041082	Harmon	Mar 2004	A1
20040051030	Olszak	Mar 2004	A1
20040051265	Nadeau	Mar 2004	A1
20040062342	Cahill	Apr 2004	A1
20040062349	Schuster	Apr 2004	A1
20040062351	Yoshioka	Apr 2004	A1
20040066882	Eberhard	Apr 2004	A1
20040066884	Hermannclaus	Apr 2004	A1
20040066890	Dalmijn	Apr 2004	A1
20040075058	Blevis	Apr 2004	A1
20040080315	Beevor	Apr 2004	A1
20040082846	Johnson	Apr 2004	A1
20040083958	Saidman	May 2004	A1
20040086075	Hein	May 2004	A1
20040086078	Adams	May 2004	A1
20040086160	Zimmermann	May 2004	A1
20040087844	Yen	May 2004	A1
20040101097	Wakayama	May 2004	A1
20040101098	Bijjani	May 2004	A1
20040102700	Asafusa	May 2004	A1
20040109231	Haisch	Jun 2004	A1
20040119591	Peeters	Jun 2004	A1
20040120009	White	Jun 2004	A1
20040120454	Ellenbogen	Jun 2004	A1
20040120857	Smith	Jun 2004	A1
20040126895	Overbeck	Jul 2004	A1
20040134986	Studer	Jul 2004	A1
20040141056	Izumi	Jul 2004	A1
20040141584	Bernardi	Jul 2004	A1
20040142386	Rigler	Jul 2004	A1
20040148137	Zerwekh	Jul 2004	A1
20040160599	Hamamatsu	Aug 2004	A1
20040161073	Nokita	Aug 2004	A1
20040175041	Miller	Sep 2004	A1
20040176677	Hwu	Sep 2004	A1
20040179647	Zhao	Sep 2004	A1
20040202154	Aklepi	Oct 2004	A1
20040212492	Boesch	Oct 2004	A1
20040212499	Bohinc	Oct 2004	A1
20040213377	Endo	Oct 2004	A1
20040213600	Watanabe	Oct 2004	A1
20040218729	Xue	Nov 2004	A1
20040225222	Zeng	Nov 2004	A1
20040232054	Brown	Nov 2004	A1
20040236520	Williams	Nov 2004	A1
20040240612	Suzuki	Dec 2004	A1
20040247071	Dafni	Dec 2004	A1
20040247171	Hashimoto	Dec 2004	A1
20040251415	Verbinski	Dec 2004	A1
20040252024	Huey	Dec 2004	A1
20040252870	Reeves	Dec 2004	A1
20040253660	Gibbs	Dec 2004	A1
20040256565	Adams	Dec 2004	A1
20040258198	Carver	Dec 2004	A1
20040258202	Wernick	Dec 2004	A1
20040263379	Keller	Dec 2004	A1
20040264624	Tanaka	Dec 2004	A1
20040264648	Claus	Dec 2004	A1
20040265175	Witty	Dec 2004	A1
20050001728	Appelt	Jan 2005	A1
20050008119	McClelland	Jan 2005	A1
20050008203	Dixon	Jan 2005	A1
20050011849	Chattey	Jan 2005	A1
20050017181	Kearfott	Jan 2005	A1
20050018812	Wolfs	Jan 2005	A1
20050023477	Archer	Feb 2005	A1
20050023479	Grodzins	Feb 2005	A1
20050024199	Huey	Feb 2005	A1
20050025280	Schulte	Feb 2005	A1
20050025350	Engelbart	Feb 2005	A1
20050025377	Avinash	Feb 2005	A1
20050029460	Iwatschenko-Borho	Feb 2005	A1
20050031069	Kaucic	Feb 2005	A1
20050031076	McClelland	Feb 2005	A1
20050053307	Nose	Mar 2005	A1
20050057354	Jenkins	Mar 2005	A1
20050058242	Peschmann	Mar 2005	A1
20050058350	Dugan	Mar 2005	A1
20050061955	Endo	Mar 2005	A1
20050069085	Lewis	Mar 2005	A1
20050074088	Ichihara	Apr 2005	A1
20050085721	Fauver	Apr 2005	A1
20050094856	Warren	May 2005	A1
20050098728	Alfano	May 2005	A1
20050100135	Lowman	May 2005	A1
20050105665	Grodzins	May 2005	A1
20050105680	Nabors	May 2005	A1
20050110672	Cardiasmenos	May 2005	A1
20050111618	Sommer	May 2005	A1
20050113961	Sabol	May 2005	A1
20050117683	Mishin	Jun 2005	A1
20050117693	Miyano	Jun 2005	A1
20050117700	Peschmann	Jun 2005	A1
20050123093	Lawaczeck	Jun 2005	A1
20050123174	Gorsky	Jun 2005	A1
20050128069	Skatter	Jun 2005	A1
20050133708	Eberhard	Jun 2005	A1
20050135535	Wallace	Jun 2005	A1
20050135668	Polichar	Jun 2005	A1
20050147199	Dunham	Jul 2005	A1
20050153356	Okawa	Jul 2005	A1
20050156734	Zerwekh	Jul 2005	A1
20050157842	Agrawal	Jul 2005	A1
20050157844	Bernardi	Jul 2005	A1
20050163354	Ziegler	Jul 2005	A1
20050169421	Muenchau	Aug 2005	A1
20050173284	Ambrefe	Aug 2005	A1
20050189412	Hudnut	Sep 2005	A1
20050190882	McGuire	Sep 2005	A1
20050198226	Delia	Sep 2005	A1
20050205793	Bohinc	Sep 2005	A1
20050206514	Zanovitch	Sep 2005	A1
20050207655	Chopra	Sep 2005	A1
20050212913	Richter	Sep 2005	A1
20050216294	Labow	Sep 2005	A1
20050219523	Onuma	Oct 2005	A1
20050220247	Ruddy	Oct 2005	A1
20050220264	Homegger	Oct 2005	A1
20050224719	Polichar	Oct 2005	A1
20050226375	Eberhard	Oct 2005	A1
20050240858	Croft	Oct 2005	A1
20050248450	Zanovitch	Nov 2005	A1
20050249416	Leue	Nov 2005	A1
20050251397	Zanovitch	Nov 2005	A1
20050251398	Zanovitch	Nov 2005	A1
20050258372	McGregor	Nov 2005	A1
20050259868	Sones	Nov 2005	A1
20050265517	Gary	Dec 2005	A1
20050271184	Ovadia	Dec 2005	A1
20050275545	Alioto	Dec 2005	A1
20050275831	Silver	Dec 2005	A1
20050276443	Slamani	Dec 2005	A1
20050279936	Litman	Dec 2005	A1
20050283079	Steen	Dec 2005	A1
20060000911	Stekel	Jan 2006	A1
20060002504	Deman	Jan 2006	A1
20060008054	Ohara	Jan 2006	A1
20060009269	Hoskinson	Jan 2006	A1
20060013455	Watson	Jan 2006	A1
20060013464	Ramsay	Jan 2006	A1
20060017605	Lovberg	Jan 2006	A1
20060018434	Jacobs	Jan 2006	A1
20060018517	Chen	Jan 2006	A1
20060019409	Nelson	Jan 2006	A1
20060027751	Kurita	Feb 2006	A1
20060034503	Shimayama	Feb 2006	A1
20060036167	Shina	Feb 2006	A1
20060045235	Bruder	Mar 2006	A1
20060045323	Ateya	Mar 2006	A1
20060056584	Allman	Mar 2006	A1
20060064246	Medberry	Mar 2006	A1
20060065844	Zelakiewicz	Mar 2006	A1
20060072702	Chapman	Apr 2006	A1
20060083418	Watson	Apr 2006	A1
20060084872	Ichikawa	Apr 2006	A1
20060086794	Knowles	Apr 2006	A1
20060093088	Sowerby	May 2006	A1
20060097171	Balchunas	May 2006	A1
20060098773	Peschmann	May 2006	A1
20060098842	Levine	May 2006	A1
20060098866	Whitson	May 2006	A1
20060109949	Tkaczyk	May 2006	A1
20060114477	Cox	Jun 2006	A1
20060115044	Wu	Jun 2006	A1
20060115109	Whitson	Jun 2006	A1
20060116566	Bruijns	Jun 2006	A1
20060119837	Raguin	Jun 2006	A1
20060133650	Xie	Jun 2006	A1
20060133659	Hammond	Jun 2006	A1
20060138331	Guillebaud	Jun 2006	A1
20060140341	Carver	Jun 2006	A1
20060141615	Lu	Jun 2006	A1
20060142662	Vanbeek	Jun 2006	A1
20060142984	Weese	Jun 2006	A1
20060173268	Mullick	Aug 2006	A1
20060176062	Yang	Aug 2006	A1
20060182221	Bernhardt	Aug 2006	A1
20060203960	Schlomka	Sep 2006	A1
20060204080	Sones	Sep 2006	A1
20060215811	Modica	Sep 2006	A1
20060249685	Tanaka	Nov 2006	A1
20060255929	Zanovitch	Nov 2006	A1
20060257005	Bergeron	Nov 2006	A1
20060262902	Wattenburg	Nov 2006	A1
20060269135	Ramsay	Nov 2006	A1
20060273257	Roos	Dec 2006	A1
20060274916	Chan	Dec 2006	A1
20060282886	Gaug	Dec 2006	A1
20060284094	Inbar	Dec 2006	A1
20070001123	Andrews	Jan 2007	A1
20070003122	Hey	Jan 2007	A1
20070058037	Bergeron	Mar 2007	A1
20070083414	Krohn	Apr 2007	A1
20070085010	Letant	Apr 2007	A1
20070118399	Avinash	May 2007	A1
20070140423	Foland	Jun 2007	A1
20070147585	Eilbert	Jun 2007	A1
20070156281	Leung	Jul 2007	A1
20070165777	Anwar	Jul 2007	A1
20070168467	Hu	Jul 2007	A1
20070172129	Tortora	Jul 2007	A1
20070189454	Georgeson	Aug 2007	A1
20070194909	Garfield	Aug 2007	A1
20070195994	McClelland	Aug 2007	A1
20070200566	Clark	Aug 2007	A1
20070206719	Suryanarayanan	Sep 2007	A1
20070210255	Bjorkholm	Sep 2007	A1
20070210921	Volpi	Sep 2007	A1
20070228284	Polichar	Oct 2007	A1
20070237293	Singh	Oct 2007	A1
20070269005	Chalmers	Nov 2007	A1
20070280502	Paresi	Dec 2007	A1
20070290136	Ivan	Dec 2007	A1
20080023631	Majors	Jan 2008	A1
20080037707	Rothschild	Feb 2008	A1
20080048872	Frank	Feb 2008	A1
20080075230	Oreper	Mar 2008	A1
20080084963	Clayton	Apr 2008	A1
20080089477	Eshed	Apr 2008	A1
20080095310	Edwards	Apr 2008	A1
20080118021	Dutta	May 2008	A1
20080128624	Cooke	Jun 2008	A1
20080130839	Rogers	Jun 2008	A1
20080152082	Bouchard	Jun 2008	A1
20080159591	Ruedin	Jul 2008	A1
20080170670	Bhatt	Jul 2008	A1
20080198967	Connelly	Aug 2008	A1
20080198970	Kirshner	Aug 2008	A1
20080205594	Bjorkholm	Aug 2008	A1
20080230709	Tkaczyk	Sep 2008	A1
20080236275	Breed	Oct 2008	A1
20080253653	Gable	Oct 2008	A1
20080260097	Anwar	Oct 2008	A1
20080304622	Morton	Dec 2008	A1
20090014662	Suhami	Jan 2009	A1
20090034790	Song	Feb 2009	A1
20090034792	Kennison	Feb 2009	A1
20090067575	Seppi	Mar 2009	A1
20090086907	Smith	Apr 2009	A1
20090116617	Mastronardi	May 2009	A1
20090127459	Neustadter	May 2009	A1
20090168964	Safai	Jul 2009	A1
20090174554	Bergeron	Jul 2009	A1
20090175411	Gudmundson	Jul 2009	A1
20090236531	Frank	Sep 2009	A1
20090238336	Akery	Sep 2009	A1
20090245462	Agrawal	Oct 2009	A1
20090257555	Chalmers	Oct 2009	A1
20090283690	Bendahan	Nov 2009	A1
20090285353	Ellenbogen	Nov 2009	A1
20090316851	Oosaka	Dec 2009	A1
20090323894	Hu	Dec 2009	A1
20100020937	Hautmann	Jan 2010	A1
20100161504	Casey	Jun 2010	A1
20100177868	Smith	Jul 2010	A1
20100177873	Chen	Jul 2010	A1
20100295689	Armistead	Nov 2010	A1
20100303287	Morton	Dec 2010	A1
20110019797	Morton	Jan 2011	A1
20110019799	Shedlock	Jan 2011	A1
20110038453	Morton	Feb 2011	A1
20110060426	Morton	Mar 2011	A1
20110064192	Morton	Mar 2011	A1
20110075808	Rothschild	Mar 2011	A1
20110129063	Bendahan	Jun 2011	A1
20110172972	Gudmundson	Jul 2011	A1
20110204243	Bendahan	Aug 2011	A1
20110235777	Gozani	Sep 2011	A1
20110266643	Engelmann	Nov 2011	A1
20120093367	Gudmundson	Apr 2012	A1
20120099710	Kotowski	Apr 2012	A1
20120104276	Miller	May 2012	A1
20120105267	Delia	May 2012	A1
20120116720	Klann	May 2012	A1
20130001048	Panesar	Jan 2013	A1
20140185771	Morton	Jul 2014	A1
20140197321	Bendahan	Jul 2014	A1
20150036798	Morton	Feb 2015	A1
20150078519	Morton	Mar 2015	A1
20150219785	Tudor	Aug 2015	A1
20150254522	Artyomov	Sep 2015	A1
20150301220	Morton	Oct 2015	A1
20150325010	Bedford	Nov 2015	A1
20150355117	Morton	Dec 2015	A1
20150355369	Morton	Dec 2015	A1
20160025889	Morton	Jan 2016	A1
20160033674	Allman	Feb 2016	A1
20170017667	Chen	Jan 2017	A1

Foreign Referenced Citations (59)

Number	Date	Country
2574402	Jan 2006	CA
2319958	Mar 2007	CA
2307439	Jul 2008	CA
101303317	Nov 2008	CN
0077018	Apr 1983	EP
0455177	Nov 1991	EP
0459648	Dec 1991	EP
0577380	Jan 1994	EP
0919186	Jun 1999	EP
1413898	Apr 2004	EP
2270547	Jan 2011	EP
3764280	Jan 2021	EP
2158572	Nov 1985	GB
2255634	Nov 1992	GB
2110037	Jun 1993	GB
2277013	Oct 1994	GB
2368764	May 2002	GB
2409268	Jun 2005	GB
2424065	Sep 2006	GB
2438317	Nov 2007	GB
2525228	Oct 2015	GB
H0422897	Jan 1992	JP
2001233440	Aug 2001	JP
2003287507	Oct 2003	JP
2005257400	Sep 2005	JP
1992003837	Mar 1992	WO
9855851	Dec 1998	WO
0009428	Feb 2000	WO
2000049428	Aug 2000	WO
2002082290	Oct 2002	WO
2003069498	Aug 2003	WO
2004010127	Jan 2004	WO
2004010162	Jan 2004	WO
2003107113	May 2004	WO
2005086616	Sep 2005	WO
2005098400	Oct 2005	WO
2006036076	Apr 2006	WO
2006053279	May 2006	WO
2006078691	Jul 2006	WO
2006119603	Nov 2006	WO
2006119605	Nov 2006	WO
2006119605	Nov 2006	WO
2007035359	Mar 2007	WO
2007055720	May 2007	WO
2007068933	Jun 2007	WO
2007103216	Sep 2007	WO
2008017983	Feb 2008	WO
2009106803	Sep 2009	WO
2009143169	Nov 2009	WO
2010114518	Oct 2010	WO
2011069024	Jun 2011	WO
2011091070	Jul 2011	WO
2013036735	Mar 2013	WO
2013116549	Aug 2013	WO
2013119423	Aug 2013	WO
2014107675	Jul 2014	WO
2014121097	Aug 2014	WO
2014124152	Aug 2014	WO
2016011205	Jan 2016	WO

Non-Patent Literature Citations (40)

Entry
Horner et al., “Phase-Only Matched Filtering”, Applied Optics, vol. 23, No. 6, Mar. 15, 1994, pp. 812-816.
Mahalanobis, et al. “Minimum Average Correlation Energy Filters”, Applied Optics, vol. 26, No. 17, pp. 3633-3640, Sep. 1987.
Kumar et al. “Spatial frequency domain image processing for biometric recognition”, Biometrics ICIP Conference 2002.
Caulfield, et al. “Improved Discrimination in Optical Character Recognition”, Applied Optics, vol. 8, pp. 2354-2356, Nov. 1969.
Morin, et al. “Optical Character Recognition (OCR) in Uncontrolled Environments Using Optical Correlators”, Proc. SPIE Int. Soc. Opt. Eng. 3715, 346; 1999.
International Search Report for PCT/US2012/054110, dated Dec. 24, 2012.
International Search Report for PCT/US2017/017642, dated Jun. 29, 2017.
International Search Report for PCT/US10/58809; Rapiscan Systems Inc.; dated Apr. 19, 2011.
Mobile X-Ray Inspection Systems, Internet Citation, Feb. 12, 2007, pp. 1-2, URL:http://web.archive.org/web/20070212000928/http://www.bombdetecti- on.com/cat--details.php?catid=20.
Molchanov P A et al: ‘Nanosecond gated optical sensors for ocean optic applications’ Sensors Applications Symposium, 2006. Proceedings of the 2006 IEEE Houston, Texas,USA Feb. 7-9, 2006, Piscataway, NJ, USA,IEEE, Feb. 7, 2006 (Feb. 7, 2006) , pp. 147-150, XP010917671 ISBN: 978-0-7803-9580-0.
Smith C. R. et al: ‘Application of 450 kV computed tomography to engine blocks with steel liners’ Materials Evaluation vol. 65, No. 5, 2007, pp. 458-461, XP055108238.
CRS Report for Congress, Aviation Security Technologies and Procedures: Screening Passengers and Baggage, Oct. 26, 2001, pp. 1-12.
Viggo Butler and Robert W. Poole, Jr., Rethinking Checked-Baggage Screening, Reason Public Policy Institute, Policy Study 297, Jul. 2002.
McLay, Laura A., Jacobson, Sheldon H., and Kobza, John E., A multilevel passenger screening problem for aviation security, Naval Research Logistics (NRL), vol. 53, issue 3, pp. 183-197, 2006.
Sun Olapiriyakul and Sanchoy Das, Design and analysis of a two-stage security screening and inspection system, Journal of Air Transport Management, vol. 13, Issue 2, Mar. 2007, pp. 67-74.
Kelly Leone and Rongfang (Rachel) Liu, The key design parameters of checked baggage security screening systems in airports, Journal of Air Transport Management, vol. 11, Issue 2, Mar. 2005, pp. 69-78.
Written Opinion on Patentability for PCT/US11/21758; dated Jul. 7, 2011; Rapiscan Systems.
International Search Report and Written Opinion for PCT/US12/54110, dated Dec. 24, 2012.
International Search Report for PCT/US11/21758; dated Jul. 7, 2011, Rapiscan Systems Inc.
International Search Report for PCT/US13/24191, Rapiscan Systems Inc., dated Jun. 25, 2013.
Written Opinion of the International Searching Authority for PCT/US2014/015126, dated May 27, 2014.
International Search Report for PCT/US13/23676, dated Jun. 28, 2013.
International Search Report for PCT/US14/56652, dated Apr. 27, 2015.
International Preliminary Report on Patentability for PCT/US2014/014198, dated Aug. 4, 2015.
International Preliminary Report on Patentability for PCT/US11/21758, dated Jul. 7, 2011.
International Search Report for PCT/US14/14198, dated May 16, 2014.
International Search Report for PCT/US2014/015126, dated May 27, 2014.
International Search Report for PCT/US2015/040653, dated Dec. 16, 2015.
International Search Report for PCT/US2014/010370, dated May 13, 2014.
ClearView Workstation, L3 Security & Detection Systems, Jun. 9, 2011.
“ClearView Workstation Cargo Inspection Workstation,” L-3 Communications Security & Detection Systems, Jul. 27, 2011, 2 pages.
Nayel Al-Zube et al: “Concealed Weapon Detection Using X-Ray Images”, International Journal of Electrical, Electronics and Data Communication, vol. 3, No. 10, Oct. 2015 (Oct. 2015), pp. 27-30, XP055756474, ISSN: 2320-2084.
Tao Lei et al: “Combining SURF with MSER for image matching”, 2013 IEEE International Conference on Granular Computing (GRC), IEEE, Dec. 13, 2013 (Dec. 13, 2013), pp. 286-290, XP032567415, DOI: 10.1109/GRC.2013.6740423 [retrieved on Feb. 13, 2014].
Kwang-Baek Kim et al: “Recognition of Identifiers from Shipping Container Images Using Fuzzy Binarization and Enhanced Fuzzy RBF Network”, Soft Computing; A Fusion of Foundations, Methodologies and Applications, Springer, Berlin, DE, vol. 11, No. 3, Apr. 25, 2006 (Apr. 25, 2006), pp. 213-220, XP019443769, ISSN: 1433-7479, DOI: 10.1007/S00500-006-0062-X.
Xinhua Shi: “Chapter 2: Overview of X-ray Detection Technologies” In: “Improving Object Classification in X-ray Luggage Inspection”, Feb. 22, 2000 (Feb. 22, 2000), Virginia Tech, XP055757888, pp. 18-55, * pp. 37-44 *.
Kolla P: “Detecting Hidden Explosives”, Analytical Chemistry, American Chemical Society, vol. 67, No. 5, Mar. 1995 (Mar. 1995), pp. 184A-189A, XP002106781, ISSN: 0003-2700 * p. 186, col. 2 *.
Wamiq M Ahmed et al: “Historical comparison of vehicels using scanned x-ray images”, 2011 IEEE Workshop on Applications of Computer Vision (WACV), Jan. 5, 2011, pp. 288-293.
Zheng Yufeng at al: A vehicle threat detection system using correlation analysis and synthesized x-ray images, Proceedings of SPIE, vol. 8709, May 31, 2013 (May 31, 2013), pp. 87090V1-87090V10, XP055793713, Orlando, Florida ISSN: 0277-786X, DOI: 10.1117/12.2016646 ISBN: 978-1-5106-4394-9.
International Search Report for PCT/GB2009/000497, dated Jan. 22, 2010.
International Search Report for PCT/GB09/00575, dated Apr. 7, 2010.

Related Publications (1)

	Number	Date	Country
	20200355631 A1	Nov 2020	US

Provisional Applications (1)

	Number	Date	Country
	62298383	Feb 2016	US

Continuations (2)

	Number	Date	Country
Parent	16376971	Apr 2019	US
Child	16926453		US
Parent	15431011	Feb 2017	US
Child	16376971		US

Systems and methods for detecting threats and contraband in cargo

Information

Patent Number

Date Filed

Date Issued

Inventors

Original Assignees

Examiners

Agents

CPC

Field of Search

US

CPC

International Classifications

Disclaimer

Term Extension

Abstract