Millimeter wave signals are used for radar and telecommunications. They are also capable of being used to produce an image of a subject by directing millimeter-wave signals at the subject and detecting the reflected signal. Examples of such imaging systems are described in U.S. Pat. Nos. 5,455,590; 5,557,283; 5,859,609; 6,507,309; 6,703,964; 6,876,322; and 7,034,746; U.S. Patent Application Publication Numbers 2004/0090359, 2004/0140924, 2006/0066469, and 2007/0075889; and U.S. patent application Ser. No. 11/088,470 filed Mar. 24, 2005. The complete disclosures of the above patent references are herein incorporated by reference for all purposes.
Imaging systems, including those systems described above, provide relatively detailed images of a subject, including a person's body and any objects carried by the person, whether or not the objects are concealed. An operator can view images of a subject and visually identify objects included with the person. Other detection devices can also be used to obtain other information about a subject. For instance, the subject can pass through a metal detector to determine if the subject has a detectable metal object.
Additionally, or alternatively, the surveillance system may include imaging systems and/or other detection devices to obtain information about subject-associated items, such as divested objects, checked baggage, and carryon baggage. For example, imaging systems may be used to produce images of checked baggage. An operator can view images of checked baggage and visually identify objects included with that baggage. Other detection devices can also be used to obtain other information about subject-associated items. For example, trace detection devices may be used to determine if the subject-associated item has an explosive substance or an illegal drug.
When the surveillance system is part of a process of boarding a public transportation vehicle, or entering a public or protected facility, substantial delays may be realized. Further, the effectiveness of the surveillance depends on the accuracy and attentiveness of the operator or operators.
Some embodiments provide a surveillance system. In some embodiments, the surveillance system may include a first screening apparatus configured to produce first screening data representative of screening at least a portion of a subject, the subject including a person and one or more objects with the person; at least one controllable information source of subject-associated item information data related to one or more subject-associated items, wherein at least one of the subject-associated items includes checked baggage; and at least one controller configured to receive the first screening data and to control operation of the at least one controllable information source of subject-associated item information data based, at least in part, on the first screening data.
In some embodiments, the surveillance system may include a first information source of checked-baggage information data related to checked baggage associated with a subject; at least one controllable information source, other than the first information source, of at least one of (a) subject information data related to the subject and (b) subject-associated item information data related to one or more subject-associated items; and at least one controller configured to receive the checked-baggage information data and to control operation of the at least one controllable information source, other than the first information source, based, at least in part, on the checked-baggage information data.
In some embodiments, the surveillance system may include a first screening apparatus adapted to transmit toward and receive from a subject in a subject position, first electromagnetic radiation in a frequency range of about 100 MHz to about 2 THz, from positions spaced from the subject position, the subject including at least a portion of a person in a subject position and detectable objects carried by the person, the first screening apparatus producing from the received radiation, a first image signal representative of a first image of at least a portion of the subject; a second screening apparatus different than the first screening apparatus, and adapted to detect a given characteristic of an object potentially carried by at least one subject-associated item, and to produce a screening signal representative of the detection of the given characteristic; and a controller adapted to control operation of the first and second screening apparatus, and to produce, from the image signal, image data representative of the image of the at least a portion of the subject and screening data from the screening signal, to relate the image data to the screening data, and to produce from the related image data and screening data, relational information data about whether at least one of (a) the person and (b) the at least one subject-associated item is potentially carrying an object having the given characteristic.
Some embodiments provide a surveillance method. In some embodiments, the surveillance method may include screening at least a portion of a subject-associated item associated with a subject; generating screening data based, at least in part, on the screening of at least a portion of a subject-associated item; analyzing the screening data; obtaining, after analyzing the screening data, information data related to at least one of (a) the subject and (b) one or more subject-associated items other than the screened subject-associated item; and reanalyzing the screening data based, at least in part, on the information data, and in a manner that is at least partially different from the analyzing of the screening data.
In some embodiments, the surveillance method may include screening at least a portion of a subject, the subject including a person and one or more objects with the person; generating screening data based, at least in part, on the screening of at least a portion of a subject; analyzing the screening data; obtaining, after analyzing the screening data, information data related to one or more subject-associated items; and reanalyzing the screening data based, at least in part, on the information data, and in a manner that is at least partially different from the analyzing of the screening data.
In some embodiments, the surveillance method may include associating, with respective identifiers, a person and one or more carried items selected by the person to be carried by the person into a controlled-access area; screening the one or more carried items while physically separate from the person; and matching the identifier of the one or more screened carried items with the identifier of the person, and, if the identifiers match, giving possession of the screened carried items to the person prior to the person entering the controlled-access area.
Some embodiments provide a surveillance portal. The surveillance portal may include a first screening apparatus adapted to detect on a subject in a subject position a first characteristic of an object potentially carried by at least one of (a) the subject and (b) one or more subject-associated items, and to produce a first screening signal representative of the detection of the first characteristic; a second screening apparatus that is different from the first screening apparatus and is adapted to detect on the subject in the subject position a second characteristic of an object potentially carried by at least one of (a) the subject and (b) the one or more subject-associated items, and to produce a second screening signal indicating the detection of the second characteristic; and a controller adapted to control operation of the first and second screening apparatus, to produce first screening data indicating the detection of the first characteristic from the first screening signal, and second screening data indicating the detection of the second characteristic from the second screening signal, to relate the first screening data to the second screening data, and to produce from the related first and second screening data, relational information data about whether at least one of (a) the subject and (b) the one or more subject-associated items is potentially carrying an object having at least one of the first and second characteristics.
A subject 32 includes all that is presented in an interrogation station of a screening apparatus, whether human, animal, or inanimate object. For example, if a person 36 is in an interrogation station, the subject 32 includes the person as well as any objects 38 with an/or supported on the person, such as watches, keys, jewelry, pocket or other knives, coins, clothing accessories, guns, and/or any other objects.
An example of an imaging apparatus 28 is an active imaging apparatus adapted to transmit toward and receive from a subject in a subject position, millimeter-wave electromagnetic radiation. In some examples, the millimeter-wave radiation may be in the frequency range of 100 Megahertz (MHz) to 2 Terahertz (THz) range. Generally, the corresponding wavelengths range from several centimeters to a few micrometers. Such radiation may be either at a fixed frequency or over a range or set of frequencies using several modulation types, e.g. chirp, pseudorandom frequency hop, pulsed, frequency modulated continuous wave (FMCW), or continuous wave (CW).
Some embodiments of imaging apparatus may use a transmitting signal that incorporates frequencies in the range of 24 to 30 GHz, FMCW modulation, and having signal content that meets FCC unlicensed operation requirements and is outside of any restricted US Government frequency bands. Pulse lengths may range from 2-10 microseconds. Antennae beam widths may range from 20-120 degrees for broad beam implementation, or less than 30 degrees for narrow beam width applications, depending on the image formation signal processor requirements. Various system polarizations may be used. Examples include same polarization, cross polarization, elliptical polarization, right circular polarization, and/or left circular polarization.
Certain natural and synthetic fibers may be transparent or semi-transparent to radiation of such frequencies and wavelengths, permitting the detection and/or imaging of surfaces positioned beneath such materials. When the subject of interrogation is a clothed individual, image information about portions of a person's body covered by clothing can be obtained with system 20, as well as those portions that are not covered by clothing. Further, image information relative to objects carried by a person beneath clothing, regardless of how they are supported on the person, can be provided with system 20 for metal and non-metal object compositions, such as those used for weapons and contraband.
Controller 21 can be adapted to operate the sensor apparatus 26. In the example shown in which a sensor apparatus is an imaging apparatus 28, the controller is adapted to produce image data representative of an image of the subject from received radiation. Supplemental subject information sources 30 provide information about the subject 32 that is relatable to objects 38 potentially carried by the person 36. The source may be of a type that is different from imaging apparatus 28. The controller then is adapted to produce relational information relating the produced image signal and the subject information.
In some surveillance systems 20, a supplemental source 30 of subject information is a second sensor apparatus 26 adapted to detect a given characteristic of an object potentially carried by a person in the subject position. The controller 21 in such a system 20 is accordingly adapted to produce relational information about whether the person is carrying an object having the given characteristic. The second sensor apparatus may be adapted to detect one or more of a variety of subject information. The sensor apparatus may function sequentially or concurrently with other sensor apparatus. For example, the second sensor apparatus may detect one or more of a substance, such as a metal, explosive or other chemical substance; a feature identifying the person 36, such as a retinal image, facial image, fingerprint, volumetric or topographical representation of all or a portion of the body, other biometric feature, an identification or categorization badge or insignia, such as an insignia representing that the person belongs to a certain group, such as a security or police force, or radio frequency identification (RFID) device. Trace element detection can be based on ion mass spectrometry, quadrupole resonance, Fourier-transform infrared spectroscopy, vapor detection, chemical biological sensing, laser measurement, fluorescence, DNA analysis, and MEMS. Optionally, a supplemental source 30 may include context data stored in a memory. Context data relates to a person or potential objects carried by the person and/or subject-associated item(s) associated with the person, such as historical data relating to information previously detected or input about the particular person, any clearance the person might have to carry particular types of objects, such as guns or other security or law-enforcement devices, or other data that is considered to assist operators of surveillance system 20 in detecting and assessing the significance of objects carried by the surveilled person and/or subject-associated item(s) associated with the person.
Controller 21 can include a processor 40 in the form of any suitable component for processing the image data, such as digital circuitry, analog circuitry, or both. Processor 40 may be programmable, such as a computer or microprocessor, a dedicated hardwired state machine, a logic unit, or a combination of these. For a multiple processor form, distributed, pipelined, and/or parallel processing can be utilized as appropriate.
Processor 40 may include a memory 42. Memory 42 can be solid state, electromagnetic, optical, or a combination of these, and it can be volatile and/or nonvolatile. Further, it may be dedicated, such as a resident hard drive, a random access memory (RAM), or a removable, such as a removable memory device (R.M.D.). Presently typical removable memory devices include a floppy disc, tape cartridge, optical disc (CD-ROM or DVD), or other appropriate type. Memory 42 also may be a remote memory coupled to the processor by a wired or wireless communication link via another processor and/or network.
Controller 21 may be coupled to suitable input and/or output devices 44 adapted to receive data and transmit output data, such as a communication link to another system or other remote output device. Exemplary input devices may include a keyboard, mouse or other cursor-control device, a voice recognition input system, or other appropriate device or system. Output devices may include visual or audio alarms, printers, communications links, or displays, such as cathode ray tube (CRT), Liquid Crystal Display (LCD), plasma, organic light emitting diode (OLED), or other appropriate display type. Appropriate communication links may include local area networks (LANs), municipal area networks (MANs), or Wide Area Networks (WANs), whether private or public.
A general flow chart 50 illustrating exemplary operation of surveillance system 20 is shown in
Where one or more subject information sources 24 is a non-imaging source 58, the data from the non-imaging source is acquired at 60. A non-imaging source may be a sensor that detects general features of the subject, such as the general detection of a substance, a feature identifying the person 36, or context data stored in a memory relating to the subject. Image features are then identified at 56. The detection of the existence of a substance or an identification of the person or, a characteristic, class or categorization of the person, and other appropriate indicators or information are considered features of the subject.
Image features from the various subject information sources 24 are then correlated with each other at 62. For example, the identification of an object on the side of a person from an imaging apparatus may be correlated with the detection of metal in the middle zone of the person, a badge identifying the person, and context data previously stored in memory indicating that the person is a security guard and has high security clearance.
These correlated features may then be classified at 64 based on the combination of features. Classification of features is a logical process for determining the likelihood that a detected feature is a suspect object or a false alarm. For example, the detection of metal in the same zone as an object indicates that there is a high likelihood that the object is metal. Further, given that the person is identified as a security guard, it is highly likely that the object is a gun. Also, the person may be authorized to carry a gun in this position as part of her duties. The object would thus be given a high weight of being a suspect object, but given a low weight as a security risk, due to the status of the person as a security guard.
The processor may use a relational database to relate the various possible features relating to a subject that is surveilled. As a general example, the following truth table illustrates how two or three features, identified as features A, B, and C, may be related. Based on a determined relationship, an attribute, weight, or value, Vx, may be assigned to each combination. In this example, reference is made to a value, which term is intended to include attribute, weight, or other relative indicator. These values may be different or the same for different relationships, depending the application.
B
(B
C)
C)
C)
C)
C)
Any set of corresponding features can be assigned a corresponding relative indicator, such as weight, value or attribute. For instance, an object identified in both a visible image and a millimeter-wave-based image can have a low value, since the object is visible. On the other hand an object identified on a millimeter-wave-based image that is not visible, can be assigned a high value. Infrared sensors can also be used to confirm the existence of hidden objects, and can identify a hidden object that is not detected on a subject using millimeter-wave-based imaging. An area of a subject can thus be assigned a high value even though no image object is detected. For example, a sheet of plastic explosive taped to a person's body may appear “colder” than the rest of the person's body. Also, the combination of explosive and metal detectors, may be used to identify an imaged object as likely to be a plastic bomb with metal pieces. Such an object may then be given a very high value.
An object's (or subject area's) value may thus be based on the substances the object (or subject) includes, such as metal, plastic, dielectric, ceramic, tissue, fabric, paper, stone, plant matter. The structure of the object also can be the basis of assigning a value, such as dimensions, shape and edge characteristics. The chemical make-up of an object can also be the basis for assigning a value, including drugs, and explosives, such as PETN, TNT, plastic, C-4 and datasheet.
Once the image features are classified, then conclusions are generated at 66 about the combinations of image features.
A(B^C): detected anomaly is metal and is explosive.
A(B^
A(
A(
Ā(B^C): undetected anomaly is metal and is explosive.
Ā(B^
Ā(
Ā(
The various conclusions may then be output at 68, as appropriate, such as via a display, report or alarm condition. However, the steps discussed above may be performed in different sequences and in different combinations, not all steps being required for all embodiments of surveillance system 20.
There are various ways in which a subject information assembly can be configured. A surveillance system, in which sensor apparatus are mounted relative to a subject position and relative to the different sensor apparatus, can provide for different moving or non-moving configurations. For example, a given sensor apparatus may be adapted to move relative to the subject position but not move relative to one or more other sensor apparatus.
As used herein, moving a specified element M relative to another specified element N means the same thing as moving element N relative to element M. For example, element M could be fixed to a frame, and element M could be moved relative to element N by moving element N relative to the frame.
An arrow between two components shown in
There are numerous possible configurations, a portion of which are shown in the figure. For example, where all three sensor apparatus are adapted to move relative to the subject position, this can be provided by one, two or three (or more) motion apparatus. Where two or three motion apparatus are used, the second and subsequent motion apparatus may be mounted relative to one or more other motion apparatus, thereby potentially providing for the capability of moving concurrently in multiple directions, depending on how the respective movement apparatus are operated.
An example of a surveillance system 20 having a subject-information assembly, 22 configured as shown in
As illustrated in
Imaging apparatus 102 may include an antenna apparatus 110 including a primary multiple-element sensing array 112. Subject information assembly 22 may include a frame 114 on which array 112 may be supported. Array 112 may extend the full height of frame 114. Motor 108 causes platform 106, and subject 32 to rotate about axis R. As a result, array 112 circumscribes a generally circular pathway about axis R. The antenna array may be about 0.5 to about 2 meters from radius R.
Antenna array 112 may include a number of linearly arranged elements 116 only a few of which are schematically illustrated. Each element 116 may be dedicated to transmission or reception of radiation, and the elements may be arranged in two generally vertical columns, with one column dedicated to transmission, and the other to reception. The number and spacing of the elements corresponds to the wavelengths used and the resolution desired. A range of 200 to about 600 elements can span a vertical length of about two or two and one-half meters.
Subject information assembly 22 includes a second sensor apparatus 118. Optionally, additional sensor apparatus may be mounted to second sensor apparatus 118, such as a third sensor apparatus 120. The second sensor apparatus may be mounted by a prime mover 122, for vertical movement along frame 114. Thus, as platform 106 rotates about axis R, sensor apparatus 118 scans the zone or location of the subject corresponding with the vertical position of the sensor and the rotational position of the subject.
Various other configurations for portal 100 and antenna apparatus 110 may be used. For example, a two-dimensional transmit and receive array may be used, as well as an array that moves around a fixed subject platform, or an array that moves vertically and extends horizontally. Further, many variations of an antenna apparatus are possible. The antenna apparatus may include one or more antenna units, and each antenna unit may include one or more transmitting antennae and one or more receiving antennae. An antenna unit may include a plurality of antennae that may receive radiation in response to transmission by a single antenna. The antennae may be any appropriate type configured to transmit or receive electromagnetic radiation, such as a slot line, patch, endfire, waveguide, dipole, semiconductor, or laser. Antennae may both transmit and receive. The antennae units may have one or more individual antennae that transmit or receive like polarization or unlike polarized waveforms such as plane, elliptical, or circular polarization, and may have narrow or broad angular radiation beam patterns, depending on the application. Beam width may be relatively broad, i.e. 30 to 120 degrees for imaging applications that use holographic techniques, while narrow beam widths in the range of 0 to 30 degrees may be used for applications having a narrow field of view requirement.
Further, a single antenna may scan a subject by mechanically moving about the subject in a one- or two-dimensional path. A one- or two-dimensional array of antenna units may electronically and mechanically scan a subject. An imaging system may include one or a plurality of antenna apparatus. The antennae apparatus may be protected from the environment by suitable radome material, which may be part of the apparatus, or separate, depending on the mechanical motion that is required of the antennae apparatus or array. Examples of other array configurations are illustrated in copending patent application Ser. No. 10/728,456 filed Dec. 5, 2003, entitled “Millimeter-Wave Active Imaging System” incorporated herein by reference.
Antenna apparatus 48 may be configured to transmit and receive electromagnetic radiation selected from the range of about 1 Gigahertz (GHz) to about 2 THz, or from the range of about 100 MHz to about 15 GHz, depending on the impulse duration. A range of about 1 GHz to about 300 GHz may also be used, with a range of about 5 GHz to about 110 GHz particularly useful for imaging. A range of 24 GHz to 30 GHz is used in some antenna apparatus. The antenna apparatus produces an image signal 68 representative of the received radiation.
Various configurations of second and third sensor apparatus are also possible. For example, a plurality of sensor apparatus 118 may be mounted along antenna array 112 in fixed positions. Each sensor apparatus 118 or 120 may have a sensor unit 121 (such as a receiver) on one side of subject position 34 and a complementary unit (such as a transmitter) on an opposite side or other spaced position, as represented by sensor unit 123.
A controller 21 may control operation of subject information assembly 22. Controller 21 may include a transceiver 124 including a switching tree 126 configured to irradiate subject 32 with only one transmitting element 116 at a time, and simultaneously receive with one or more elements 116. Transceiver 124 may include logic to direct successive activation of each combination of transmit and receive antenna elements to provide a scan of a portion of a subject 32 along a vertical direction as platform 106 and the subject rotate.
An image signal 125 received from antenna apparatus 110 may be downshifted in frequency and converted into an appropriate format for processing. In one form, transceiver 124 may be of a bi-static heterodyne Frequency Modulated Continuous Wave (FM/CW) type like that described in U.S. Pat. No. 5,859,609. Other examples are described in U.S. Pat. Nos. 5,557,283 and 5,455,590. In other embodiments, a mixture of different transceiver and sensing element configurations with overlapping or non-overlapping frequency ranges may be utilized, and may include one or more of the impulse type, monostable homodyne type, bi-static heterodyne type, and/or other appropriate type.
Transceiver 124 may provide image data 127 corresponding to the image signals to one or more processors 40. Processor 40 can include any suitable component for processing the image data, as appropriate. Processor 40 is coupled to a memory 42 of an appropriate type and number. As has been mentioned, memory 42 may include a removable memory device 129, as well as other types of memory devices.
Controller 21 may be coupled to motor 108, prime mover 122, or other drive element used, to selectively control the rotation of platform 106 or movement of a sensor apparatus 26. Controller 21 may be housed in a monitor and control station 128 that also includes one or more input/output devices 44, such as operator input devices 130 and one or more display or other type of output devices 132.
Antenna apparatus 110 includes array elements 116 that may be mounted by a primary motion apparatus, not shown, to frame 140 for moving vertically. A second sensor apparatus 26 may be mounted relative to an intermediate frame 146 by a second motion apparatus, also not shown. In turn, intermediate frame 146 may be mounted relative to base frame 140 by a further motion assembly, also not shown. The subject information assembly 22 shown in
A simplified top view of a further variation of a subject information assembly 22 for a portal 100 and usable in a surveillance system 20, is shown in
Array 112′ is shown mounted on a frame 114 having opposite distal ends 114a and 114b. Second sensor apparatus 26″ also includes a first sensor unit 152 and a second sensor unit 154 mounted, respectively, on frame ends 114a and 114b. Sensor units 152 and 254 may be the same type of sensor, or different types of sensors, when a single sensor unit is sufficient. If sensor apparatus 26″ is of a type that requires a send or transmit unit and a spaced-apart complementary receive unit, then the two sensor units can be the two components required. Examples of such sensor apparatus include metal detectors, some imaging systems, and trace chemical detectors. With trace chemical detectors, one sensor unit may direct air toward the subject, such as by a fan, and a second sensor unit draws in the air, which air is then checked for the trace element or elements of interest.
By traveling vertically with the imaging array 112, the second sensor apparatus 26″ senses subject information at a level corresponding with the position of the imaging array. As a result, characteristics of the image in the position or zone of the image corresponding to which supplemental subject information are detected, can be correlated with the subject information to provide further information relating to the detected image of the subject. Other configurations for the second sensor apparatus can also be realized. For example, the sensor units may be mounted in fixed positions on a fixed portion of frame 114. Also, a plurality of sensor units can be distributed vertically along fixed portions of frame 114, and mounted in fixed positions independent of array 112′, to provide zonal subject information. Optionally, the sensor units may be mounted independent of imaging apparatus 28′, and may move or not, as was discussed with reference to subject information assembly 22 shown in
A motion apparatus 80 is adapted to move the first and second sensor apparatus 26 vertically. Accordingly, a subject 32 may be imaged on one side by moving the array vertically from top to bottom or from bottom to top. In order to image both sides of a subject, then, it is necessary to scan the opposite side of the subject. This can be accomplished in various ways, such as having the subject turn around. Optionally, a second array, opposite the array shown, could be provided. Also, the array shown could be rotated around the subject position.
Following is a description that illustrates a method for providing a blurred image of a subject for protection of privacy concerns of surveilled subjects, as well as for automatic identification of portions of an image suspected of including objects carried by the surveilled person. This description is presented largely in terms of display images, algorithms, and symbolic representations of operation of data bits within a computer memory. It may be preferred to implement a program or algorithm as various interconnected distinct software modules or features. This is not necessary, as software, firmware, and hardware may be configured many different ways, and may be aggregated into a single processor and program with unclear boundaries.
An algorithm is generally considered to be a self-consistent sequence of steps leading to a desired result. These steps require physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. When stored, they can be stored, transferred, combined, compared, and otherwise manipulated. When stored, they may be stored in any computer-readable medium. As a convention, these signals may be referred to as data, bits, values, elements, symbols, characters, images, terms, numbers, or the like. These and similar terms may be associated with appropriate physical quantities and are convenient labels applied to these quantities.
The present disclosure also relates to apparatus for performing these operations, such as has been described. This apparatus may be specially constructed for the required purposes or it may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer or other apparatus. In particular, various general-purpose machines may be used with programs in accordance with the teachings herein, or it may prove more convenient to construct more specialized apparatus to perform the required method steps. The required structure for a variety of these machines will appear from the description given below. Useful machines for performing the disclosed operations include general-purpose digital computers, microprocessors, or other similar devices, as has been described.
The programs described need not reside in a single memory, or even a single machine. Various portions, modules or features of it can reside in separate memories, or even separate machines. The separate machines may be connected directly, or through a network, such as a local access network (LAN), or a global or wide area network, such as what is known as the Internet. Similarly, the users need not be collocated with each other, but each only with a machine that houses a portion of the program.
Various techniques may be used for identifying objects that may be associated with a person being imaged. For example, once a first image of a subject has been obtained, objects are detected visually by an operator of the system or automatically. When manual detection is used, a primary or first image 160, such as the image of
Once a suspect area containing a possible object other than the person, is identified, the antenna apparatus may be used to provide a more detailed image of the suspect area. For instance, a secondary antenna array, not shown, may be used to re-image the suspect area in greater detail than was originally provided.
Optionally, imaging arrays may be used to re-image the suspect area. Increased resolution may be possible by increasing the firings per surface area of the subject, such as by reducing the rate of movement of the subject relative to the antenna array, or increasing the scanning rate for the antenna array. It may also be possible to produce the first, low-resolution image using a portion of antenna elements in the array, and to produce a second, higher-resolution image using all of the antenna elements corresponding to the selected region of the subject.
Accordingly, the image signal, whether it be a reworking of the portion of the image signal corresponding to the first image or a portion of the image signal corresponding to a rescan of the selected subject area, a secondary image may be produced that has a higher resolution than the primary image. This image may then be displayed for observation by an operator, to better determine whether an object is present, and if so, the nature of the object.
Characteristics of image data may also be automatically analyzed to determine whether there may be areas of an image that may include an object. Regions of a subject image that have characteristics that correspond to characteristics of images of known objects may be identified. By automatically identifying suspect areas of an image, a system operator's attention may be directed to such areas, the information may be used to activate alarms, and the information may be used to initiate a rescanning of the suspect areas or to otherwise increase the resolution of the image.
It has been found that objects carried by a person may have the appearance of a “wavy texture” in the vicinity of objects hidden on a person.
The waviness may be detected by convolving the image with a waviness detection kernel. An exemplary kernel is given by the following five-by-five matrix of values:
The “wavy” nature of the above set of numbers is illustrated graphically in
The kernel numbers above can be thought of as a small image window covering a corresponding five-by-five square of picture elements, which window is moved above (associated with successive corresponding portions of) the image. Each value in the kernel is multiplied by the corresponding image intensity value (right below it) and the final result of the operation is the sum of the results of the 25 multiplications. Generally, the closer the relative values of the intensities of the pixels overlaid by the window are to the relative values in the kernel, the higher the magnitude or absolute value of the resulting sum.
An illustration of this operation is depicted in
This sum provides an indication of the correlation of the 25 pixels associated with the reference pixel to the relative variations or texture represented by the kernel. The higher the absolute value of the sum, then the higher the correlation.
Once the waviness image is produced from the original subject image, the brightest regions of the waviness image may be identified by running a threshold analysis on the image. Small contrasting regions or holes may also be eliminated in order to make the resulting image more uniform. In this analysis, a new image 182 as shown in
The intent is to identify graphically those regions of the subject image where there are suspected objects. These regions can then be displayed or otherwise identified to a system operator, so that they can evaluate the images or the subject further. The above process may result in minor regions being identified for which it is unlikely that they correspond with an object on the person. The image may accordingly be cleaned up by dilating the highlighted spots and then deleting smaller regions not expected to be significant. Dilating is a process of expanding highlighted regions. One way of doing this is by passing another “window” of analysis or kernel over the image. For a given reference pixel, if there are more white pixels than a preset threshold, then the pixels in the entire window are made white. An exemplary result is shown as dilated image 184 in
To simplify the imaging of the suspected object areas, rectangles that bound the highlighted regions may be identified.
This image may again be dilated to further combine closely spaced areas. The image 188 that results from one such process is shown in
To facilitate the identification of objects on the subject image, the outlines of the suspect areas, derived from the image of
As has been mentioned, a surveillance system 20 can be configured to provide both an image of a subject, as well as supplemental subject information. The source of supplemental subject information can be in the form of a sensor, such as a metal detector. As described with reference to the subject-information assembly 22 illustrated in
Various configurations of a subject image or images 204 can be provided. In this example, eight frames 212 of images corresponding to views of the subject at 45-degree increments around the subject are shown. Each frame includes a subject image 214, which in this instance is a privacy image 216 of the subject, such as provided by the process described with reference to
Supplemental display 206 visually associates the results of operation of a second sensor apparatus 26. In this example, the second sensor apparatus may be a metal detector that determines the existence of metal in different vertical zones of the person's body. A display element 222 is aligned with each of the detected zones of the image 220. The display element in this case has a positive state, represented by a dark indication or distinctive color, when metal is detected in the corresponding zone of the subject. When no metal is detected for a particular zone, the display element has a negative state represented by a light indication or distinctive color. Other symbols, indicia or representations could be used to indicate the results of the second sensor apparatus relative to the subject image.
It is seen that for the selected primary image 220 shown in
When there is a match or correlation of detected information in a particular location or particular locations of a subject, one or more images may be further enlarged or cropped to focus attention on and provide additional detail of the suspect regions or zones where metal is detected. An additional display 230 that may be provided, is shown in
Although not shown in the figures, a surveillance system may also be adapted to provide images of regions of a subject for which no suspect regions are identified, but for which a supplemental sensor detects a substance or other characteristic. In the case where metal is detected in a zone for which no suspect regions are identified, an enlargement of the image of the affected zone provides improved resolution or detail, to show whether an external object is present in that zone.
Another example of a general surveillance system is shown in
Subject-related information assembly 304 may include a subject information assembly 306 and a subject-associated item information assembly 308. The subject information assembly may include one or more of the structures and/or functions of structures of subject information assembly 22 discussed above. For example, subject information assembly 306 may include a plurality of subject information sources 310, which may at least partially include the structures and/or functions of the one or a plurality of screening apparatus 26 and/or the one or a plurality of supplemental subject information sources 30. The subject information sources may provide information related to subject 32 in subject position 34, such as information on person 36 and objects 38 with and/or supported on the person. Subject information sources 310 may be controllable or controlled by controller 302.
Subject-associated item information assembly 308 may include a plurality of subject-associated item information sources 312. The subject-associated item information sources may provide information related to one or more subject-associated items 314, such as any objects 316 carried by those items. “Subject-associated items,” as used herein, refers to items associated with the subject but not part of the subject, such as divested objects, carryon baggage, and checked baggage. Subject-associated item information sources 312 may be controllable or controlled by controller 302.
The subject-associated item information sources may include at least one subject-associated item screening apparatus 318. Screening apparatus 318 may be configured to screen at least a portion or a plurality of sections of the one or more subject-associated items and/or produce screening data or a screening signal representative of screening the at least a portion of the one or more subject-associated items. In some embodiments, screening apparatus 318 may be adapted to detect a given characteristic (and/or object information) of an object potentially carried by one or more subject-associated items and to produce a screening signal representative of the detection of that characteristic (and/or screening data indicating the detection of the characteristic). In some embodiments, screening apparatus 318 may be adapted to produce object data indicative of whether the subject-associated item(s) carries at least one object that includes the object information or a screening signal representative of the detection of the given characteristic. The screening data may be received by controller 302 and/or other structures of surveillance system 300, as further discussed below.
In some embodiments, the subject-associated item screening apparatus may include at least one subject-associated item imaging apparatus 320, which may include any suitable structure adapted to image at least a portion of the subject-associated item(s) and/or to produce an image signal representative of an image of the at least a portion of the subject-associated item(s). For example, imaging apparatus 320 may be adapted to transmit, toward the one or more subject-associated items, X-ray electromagnetic radiation for use in X-ray transmission, diffraction, and/or backscattering imaging. In some examples, the X-ray radiation may be in a frequency range of about 30 Petahertz (PHz) to about 30,000 PHz, with corresponding wavelengths that may range from 10 to 0.01 nanometers. Examples of the use of X-ray electromagnetic radiation for screening subject-associated item(s) are further described in U.S. Pat. No. 5,600,303 to Husseiny et al., the complete disclosure of which is herein incorporated by reference for all purposes.
Imaging apparatus 320 may alternatively, or additionally, include other suitable imaging technologies such as computed tomography (CT), magnetic resonance imaging (MRI), smart X-ray, multi-view X-ray, laminography, and/or other technologies discussed above. Structures and features of some examples of imaging apparatus 320 are further described in U.S. Pat. No. 6,952,163 to Huey et al., the complete disclosure of which is herein incorporated by reference for all purposes. Although imaging apparatus 320 is discussed to incorporate or use particular technologies, the imaging apparatus may alternatively, or additionally, include one or more suitable technologies configured to image the contents of the subject-associated item(s).
Screening apparatus 318 may alternatively, or additionally, be adapted to detect, in the one or more subject-associated items, one or more of a metal substance, an explosive substance (such as Semtex, C-4, nitroglycerin, PETN, RDX, Detasheet, TNT, tetryl, ANFO, and black powder), a chemical substance (such as illegal drugs, including cocaine, heroin, and MDMA), and a feature identifying the person associated with the one or more subject-associated items (such as one or more tags attached to the item(s) and having a barcode, RFID, and/or image of the person). Screening apparatus 318 may include one or more suitable technologies. For example, in some embodiments where the subject-associated item screening apparatus is adapted to detect metal, explosive, and/or chemical substance(s), the screening apparatus may include X-ray transmission, diffraction, and/or backscatter, nuclear quadropole resonance (NQR), trace detection, and/or other suitable technologies. The use of X-ray diffraction, NQR, and trace detection in screening one or more subject-associated items are further discussed in U.S. Pat. No. 5,600,303 to Husseiny et al. and U.S. Pat. No. 6,952,163 to Huey et al. Additionally, or alternatively, in some embodiments where the subject-associated item screening apparatus is adapted to detect a feature identifying the person associated with the one or more subject-associated items, the screening apparatus may include barcode reader(s), RFID reader(s), video camera(s), and/or other suitable technologies.
Subject-associated item information sources 312 may additionally, or alternatively, include context data stored in a memory. Context data relates to potential objects carried by the one or more subject-associated items, such as historical data relating to information previously detected or input about the one or more subject-associated items and/or the person associated with those items, any clearance the person associated with the one or more subject-associated items might have to carry particular types of objects, and/or other data that may be considered to assist in detecting and assessing the significance of objects in the one or more subject-associated items. The context data may include results of previous screening(s) of a particular person and/or their subject-associated items within the particular facility or area in which the surveillance system is used in (which may be stored, for example, in memory associated with surveillance system 300) and/or from other facilities and areas that have screened the particular person and/or their subject-associated item(s). Additionally, or alternatively, context data may include information from various public sources, such as the FBI, National Crime Information Center (NCIC), State Department databases, IRS, Social Security Administration, state motor vehicle and corrections departments, credit bureaus, and/or bank records, and/or private sources, such as organizations that generate preferred passenger lists.
Although subject-associated item information sources 312 are shown to include screening apparatus 318 and context data, the information sources may additionally, or alternatively, include other types of information sources configured to provide information regarding potential objects 316 carried by the one or more subject-associated items.
Controller 302 may be adapted to operate or control operation of one or more subject information sources 310 and/or one or more subject-associated item information sources. For example, when one or more of those sources include an imaging apparatus and a screening apparatus, the controller may be adapted to produce, from an image signal of the imaging apparatus, image data representative of the image of at least a portion of a subject and/or subject-associated item(s) (and/or image data representative of detection of a characteristic of an object potentially carried by a subject and/or one or more subject-associated items). Additionally, or alternatively, the controller may be adapted to produce, from a screening signal of a screening apparatus, screening data representative of screening at least a portion of a subject and/or subject-associated item(s) (and/or screening data indicating detection of a characteristic of an object potentially carried by the subject and/or one or more subject associated items). Alternatively, as discussed above, the imaging apparatus may be configured to produce the image data and/or the screening apparatus may be configured to produce the screening data. Although controller 302 is described as being adapted to produce image data and screening data from an image signal and a screening signal, respectively, the controller may be adapted to produce data from any suitable signal or combination of signals.
Additionally, or alternatively, the controller may be adapted to relate a first screening data to a second screening data, and to produce, from the related first and second screening data, relational information data relating the first screening data and the second screening data. For example, the controller may be adapted to relate image data to other screening data, and to produce, from the related image data and screening data, relational information data relating the image data and the screening data. Although controller 302 is described as being adapted to relate image data to screening data, the controller may be adapted to relate any suitable combination of data and to produce relational information data relating the combination of data. For example, the controller may alternatively, or additionally, be adapted to relate context data to screening data, and to produce, from the related context data and screening data, relational information data relating the context data and the screening data.
Alternatively, or additionally, controller 302 may be adapted to receive data from one or more subject information sources and/or one or more subject-associated item information sources, and control one or more operating parameters of one or more other subject information sources and/or one or more other subject-associated information sources based, at least in part, on the received data. For example, the controller may be adapted to receive first screening data from a first screening apparatus and control a sensitivity level of one or more screening apparatus other than the first screening apparatus. Additionally, or alternatively, the controller may be adapted to select for screening one or more portions of the subject and/or the subject-associated item(s) (or one or more portions of at least one section of the subject and/or one or more subject-associated items) based, at least in part, on the first screening data.
For example, the controller may receive data from screening a subject-associated item indicating that negligible amounts of an explosive substance were contained in the subject-associated item. The controller may then increase sensitivity levels for the same explosive substance when screening the subject and/or other subject-associated items. Although controller 302 is discussed to be adapted to control sensitivity level(s) or portion(s) to be screened, the controller may be adapted to control any suitable operating parameter(s) of downstream information sources based, at least in part, on received data from one or more upstream information sources.
Alternatively, or additionally, controller 302 may be adapted to analyze screening data, receive information data related to the subject and/or one or more subject-associated items, and then reanalyze the screening data based, at least in part, on the information data, and in manner that is at least partially different from the first analysis of the screening data. The controller may, in analyzing the screening data, assign one or more of a relative weight, value, attribute, and relative indicator to an object potentially with a screened subject and/or in screened subject-associated item(s) based, at least in part, on the screening data. Controller 302 may, after obtaining the information data, reassign one or more of a relative weight, value, attribute, and relative indicator to the object potentially with the screened subject and/or in the screened subject-associated item(s) based, at least in part, on the obtained information data.
For example, the controller may analyze screening data from screening of a subject-associated item and conclude that a flammable substance found in the subject-associated item is a low threat. Controller 302 may then obtain information data regarding the person associated with the subject-associated item that identifies the person as a suspected terrorist. The controller may then reanalyze the screening data with, for example, a higher sensitivity level and conclude that the flammable substance is a higher threat based, at least in part, on the obtained information data.
Alternatively, or additionally, the controller may analyze first screening data from screening of a first subject-associated item and conclude that a first portion of a potentially dangerous device found in the subject-associated item is a low threat without a second portion. Controller 302 may then obtain second screening data from screening the subject and/or other subject-associated item(s) indicating the presence of the second portion of the device. The controller may then reanalyze the first screening data and conclude that the first portion of the device is a higher threat based, at least in part, on the presence of the second portion of the device.
Controller 302 may include at least some of the structures and/or functions of controller 21 discussed above. For example, controller 302 may include a processor 322, a memory 324, and input and/or output devices 326, which may include at least some of the structures described above for processor 40, memory 42, and input and/or output devices 44, respectively.
Subject information sources 310 and subject-associated item information sources 312 may be configured, distributed, and/or arranged in any suitable manner to obtain information on a subject and/or subject-associated items. For example,
The subject and the subject-associated items begin the process at 402. Checked baggage 328 may be physically separated from the subject at 404, may be screened by at least one checked-baggage information source 338 having a processor 340 at 406, and then may be moved to the cargo portion of the vehicle. The subject-associated items that are selected by the person to be moved to the cargo portion of the vehicle also may be referred to as “cargo-area items.”
Subject 32 may be physically separated from the subject-associated items at 410 and then one or more objects 38 with or supported on person 36 may be physically separated from the subject at 412. Those objects may be referred to as “divested objects” when physically separate from the subject. The subject may be screened at 414 by at least one subject information source 342 having a processor 344. The divested objects may be screened by at least one divested object information source 346 having a processor 348 at 416. The screened divested objects may be returned to the screened subject or picked up by the screened subject at 418.
Carryon baggage 330 may be physically separated from the subject at 420 and may be screened by at least one canyon-baggage information source 350 having a processor 352 at 422. The screened carryon baggage may be returned to the screened subject at 424. The screened subject and the screened carryon baggage may then move to the passenger portion of the vehicle at 426. The subject-associated items and the divested objects that are selected by the person to be carried by the person into the passenger portion of the vehicle also may be referred to as “passenger-area items” or “carried items.”
One or more of checked-baggage information source 338, subject information source 342, divested object information source 346, and carryon baggage information source 350 may be at least partially controlled by controller 302 having processor 322 and memory 324. Although those information sources are shown to each have its own processor, one or more of the information sources may not have a separate processor, but rather may share a processor, such as processor 322.
Additionally, or alternatively, at least one of the information sources may be used to screen two or more of the checked baggage, the subject, the divested objects, and the carryon baggage. For example, the at least one carryon baggage information source may be used to screen the carryon baggage and the divested objects. Alternatively, or additionally, the at least one subject information source may be used to screen the subject, the divested objects, and the carryon baggage.
Moreover, although surveillance system 300 is shown to have a particular configuration for obtaining information regarding a subject and/or subject-associated items that are boarding a vehicle with at least one cargo portion 334 and at least one passenger portion 336, the surveillance system may be configured in any suitable way as appropriate for the facility, the vehicle, threat level associated with the facility and/or the vehicle, and/or other suitable factors. An example of configuring the surveillance system would be to exclude one or more information sources shown in
Furthermore, although surveillance system 300 is shown to be used in a vehicle facility, the general surveillance system may be configured for use in any type of facility. For example, surveillance system 300 may be used for an entrance of any type of building or facility having one or more controlled-access areas 333. When the general surveillance system is used in those applications, the system may exclude the at least one checked-baggage information source and/or other suitable information sources.
A flow chart 450 illustrating an example of a method of operation of surveillance system 300, is shown in
One or more of the subject information sources and/or the subject-associated item information sources may include an imaging apparatus that produces an image or image data representative of an image of at least a portion of the subject and/or the subject-associated item(s). The imaging apparatus may include apparatus that detects information relatable to zones, sections, portions, and/or positions of the subject and/or the subject-associated item(s), such as surfaces or characteristics, as may be realized, such as using received radiation of appropriate form, such as acoustical waves, optical radiation, infrared radiation, millimeter-wave radiation or other radio-frequency radiation, Terahertz radiation, and X-ray or Petahertz radiation. In some embodiments, the imaging apparatus may detect image information and produce a detected signal. Those signals may then be acquired and processed to construct image data.
Additionally, or alternatively, one or more of the subject information sources and/or the subject-associated item information sources may include non-imaging sources. A non-imaging source may be a sensor or screening apparatus that is configured to screen at least a portion of the subject and/or the subject-associated item(s), such as detecting general features of the subject and/or subject-associated item(s), such as a metal detector. The detection of the existence of a substance with the person and/or the subject-associated item(s), an identification of the person, a characteristic, class, or categorization of the person and/or the subject-associated item(s), and/or other appropriate indicators or information may be considered features of the subject and/or the subject-associated items. Alternatively, or additionally, the non-imaging source may be context data stored in memory relating to the subject and/or the subject-associated item(s).
The processed data from the subject information source(s) and the subject-associated information source(s) may then be used to identify features at 460. Those features may then be correlated with each other at 462. For example, the identification of an explosive substance on the person from a screening apparatus may be correlated with the detection of a triggering device in the person's carryon baggage and/or checked baggage.
The correlated features may then be classified at 464 based on the combination of features. As discussed above, classification of features is a logical process for determining the likelihood that a detected feature is a suspect object or a false alarm, and the process may use a relational database to relate the various possible features relating to a subject and/or subject-associated item(s) that are surveilled. Once the features are classified, then conclusions may be generated at 466 about the combination of features, and then outputted at 468. The outputs may be via display, report, and/or alarm condition. Additionally, or alternatively, the outputs may be used to control downstream information sources, reanalyze other screening data, and/or store data into memory. However, the steps discussed above may be performed in different sequences and in different combinations, not all steps being required for all embodiments of surveillance system 300.
A flow chart 500 illustrating another example of a method of operation of surveillance system 300, is shown in
The processed data from the upstream information source(s) may then be used to identify features at 508. Those features may be classified at 510 based on the above described logical process and/or other suitable logical process(es) for determining the likelihood that a detected feature is a suspect object or a false alarm. The identification of features and classification of features may be referred to as “analysis of the data.” Once the features are classified, then conclusions may be generated at 512. Based, at least in part, on the generated conclusions, the controller may adjust the operation of one or more downstream information sources, such as adjusting a sensitivity level or adjusting which portion(s) of the subject and/or subject-associated item(s) to screen.
For example, a chemical substance may be detected in checked baggage screening apparatus but may be present in a negligible amount. The controller may increase the sensitivity level for the same chemical substance in the subject and/or carryon baggage screening apparatus based, at least in part, on detecting the chemical substance in the checked baggage. Additional negligible amounts in the subject and/or the carryon baggage, combined with the amount in the checked baggage, may cumulatively be non-negligible. However, the steps discussed above may be performed in different sequences and in different combinations, not all steps being required for all embodiments of surveillance system 300.
A flow chart 550 illustrating another example of a method of operation of surveillance system 300, is shown in
Screening data may then be generated based, at least in part, on screening at least a portion of the subject-associated item at 554. The screening data may include, at least in part, the object information detected. The generated screening data may be analyzed, for example by the controller, at 556. Analyzing the screening data may include assigning a relative weight, value, attribute, and/or relative indicator to the object potentially in the screened subject-associated item based, at least in part, on the screening data. For example, a relative threat level (such as low risk, medium risk, or high risk) may be assigned to the object potentially in the screened subject-associated item.
After analyzing the screening data, information data related to the subject and/or one or more subject-associated items other than the screened subject-associated item may be obtained at 558. Obtaining information data may include screening at least a portion of the subject and/or the subject-associated item(s) other than the screened subject-associated item. That screening may include imaging at least a portion of the subject and/or the subject-associated item(s) other than the screened subject-associated item. Alternatively, or additionally, the screening may include detecting, in the subject and/or the subject-associated item(s) other than the screened subject-associated item, one or more of a metal substance, an explosive substance, a chemical substance, and a feature identifying the person associated with the subject-associated item. Alternatively, or additionally, the information data may include context data relating to the person and/or one or more subject-associated items associated with the person, other than the screened subject-associated item.
After obtaining the information data, the screening data may be reanalyzed at 560. The reanalysis may be based, at least in part, on the obtained information data, and may be in a manner that is at least partially different from the analysis of the screening data. Reanalyzing the screening data may include assigning a relative weight, value, attribute, and/or relative indicator to the object potentially in the screened subject-associated item based, at least in part, on the obtained information data. The relative weight, value, attribute, and/or relative indicator assigned from the reanalysis may be the same, lower, or higher than the relative weight, value, attribute, and/or relative indicator assigned from the analysis. For example, a knife potentially in the screened subject-associated item that was initially assigned a high threat level may be reassigned a lower threat level based, at least in part, on obtained information data that the person sells knives and has the appropriate approval to carry the knife in the screened subject-associated item. However, the steps discussed above may be performed in different sequences and in different combinations, not all steps being required for all embodiments of surveillance system 300.
A flow chart 600 illustrating another example of a method of operation of surveillance system 300, is shown in
Screening data may then be generated based, at least in part, on screening at least a portion of the subject at 604. The screening data may include, at least in part, the object information detected. The generated screening data may be analyzed, for example by the controller, at 606. Analyzing the screening data may include assigning a relative weight, value, attribute, and/or relative indicator to the object potentially with the screened subject based, at least in part, on the screening data. For example, a relative threat level (such as low risk, medium risk, or high risk) may be assigned to the object with the screened subject.
After analyzing the screening data, information data related to the subject and/or one or more subject-associated items may be obtained at 608. Obtaining information data may include screening at least a portion of the subject-associated item(s). That screening may include imaging at least a portion of the subject-associated item(s). Alternatively, or additionally, the screening may include detecting, in the subject-associated item(s), one or more of a metal substance, an explosive substance, a chemical substance, and a feature identifying the person associated with the subject-associated item. Alternatively, or additionally, the information data may include context data relating to the person and/or one or more subject-associated items associated with the person, other than the screened subject-associated item.
After obtaining the information data, the screening data may be reanalyzed at 610. The reanalysis may be based, at least in part, on the obtained information data, and may be in a manner that is at least partially different from the analysis of the screening data. Reanalyzing the screening data may include assigning a relative weight, value, attribute, and/or relative indicator to the object potentially in the screened subject-associated item based, at least in part, on the obtained information data. The relative weight, value, attribute, and/or relative indicator assigned from the reanalysis may be the same, lower, or higher than the relative weight, value, attribute, and/or relative indicator assigned from the analysis. For example, a gun potentially in the screened subject that was initially assigned a high threat level may be reassigned a lower threat level based, at least in part, on obtained information data that identified the person as a police officer. However, the steps discussed above may be performed in different sequences and in different combinations, not all steps being required for all embodiments of surveillance system 300.
As shown in
For example, as shown in
Identifiers 615 may include any suitable identifier(s), such as one or more of an image of the subject, a barcode, a code contained in an RFID tag, a color, and an alphanumeric code. Where the tag is a RFID tag, the tag may be adapted to produce an identifying signal, such as when read by a RFID reader or RFID reading device. In some embodiments, the RFID tag may include a memory 621 with the identifier stored in the memory. Although particular identifiers 615 are discussed, any suitable identifier(s) may alternatively, or additionally, be used.
As discussed, tags 614 also may include at least one attachment assembly 622. The attachment assembly may include any suitable assembly configured to secure the tag to subject 32 and/or the subject-associated items 314, such as one or more of ropes, cords, lanyards, pins, chains, etc. In some embodiment of tags 614, the attachment assembly may be excluded. Although tags 614 are shown to be used to associate a person with their subject-associated item(s), any suitable structure(s) may alternatively, or additionally, be used, such as sticker(s), marker(s), etc.
One or more subject-related information sources 305 of surveillance system 300 may include at least one reader 624 configured to read identifiers 620 of tags 614. For example, where the identifier includes a barcode, then reader 624 may include a barcode reader or barcode reading device. Additionally, or alternatively, where the identifier includes an RFID device, then reader 624 may include an RFID reader. The barcode and/or RFID reader(s) may be connected and/or in communication with controller 302. The processor of the controller may associate the identifiers and/or perform other suitable functions. Additionally, the memory of the controller may store the associations performed by the processor.
Additionally, or alternatively, surveillance system 300 may include one or more containers 626 configured to contain subject-associated item(s), as shown in
A flow chart 650 illustrating an example of a method of operation of surveillance system 300 related to associating the person and the one or more subject-associated items, such as carried items, is shown in
Various embodiments of the method shown in flow chart 650 are possible, such as (a) carried item tag method, (b) carried item tag and subject tag method, (c) container method, (d) container and subject tag method. In the carried item tag method, one or more subject-associated item tags 616 having subject-associated item identifier(s) 617 may be attached (or be required to be attached) to the carried item(s) as part of associating at 652. As discussed above, the subject-associated item identifiers may be one or more of an image of the person, a barcode, an RFID device, a color, an alphanumeric code, and/or other suitable identifiers.
Where the subject-associated item identifier is an image of the person, associating at 652 also may include producing the at least one subject-associated item tag with an image of the person (such as from an image taken of the person at a surveillance portal) and associating that tag with the one or more carried items. Matching identifiers at 656 may then include comparing the image of the person on the at least one subject-associated item tag to a visual appearance of the person. Where the tag is a RFID tag having memory 621, associating at 652 also may include storing an identifier of the person, such as a name of the person and/or other suitable identifiers, in the memory. Matching identifiers at 656 may then include reading memory 621 with a reading device and comparing the read identifier of the person with a document that identifies the person, such as an identity card and/or boarding pass.
Where the subject-associated item identifier is at least one barcode, associating at 652 also may include reading the barcode with a reading device and associating an identifier of the person with the barcode. Matching identifiers at 656 may then include reading the barcode with the reading device and comparing the identifier of the person associated with the barcode with a document that identifies the person.
The carried item tag and subject tag method, similar to the carried item tag method, also includes attaching or requiring attachment of at least one subject-associated item tag 616 having at least one subject-associated item identifier 617 (such as one or more of a barcode, an RFID device, a color, and an alphanumeric code) as part of associating at 652. However, unlike the previously discussed method, associating at 652 also includes providing, to the person, at least one subject tag 618 having at least one subject identifier 620 (such as one or more of a barcode, an RFID device, a color, and an alphanumeric code), reading the subject-associated item identifier(s) and the subject identifier(s), and associating or relating those two types of identifiers. For example, where subject-associated item identifier 617 and subject identifier 620 include barcodes, associating at 652 may include reading the barcodes with a reading device and associating the barcodes with each other.
Matching at 656 may then include reading the subject-associated item identifier(s) and the subject identifier(s), and confirming that the subject-associated item identifier(s) is associated with the subject identifier(s). In the example above where barcodes are used for identifiers, matching at 656 may include reading the barcodes with a reading device and confirming that the barcodes are associated.
In the container method, associating at 652 may include placing or requiring placement of the carried item(s) in one or more containers 626 having at least one container identifier 630. As discussed above, container identifier 630 may include one or more of an image of the person, a barcode, an RFID device, a color, and an alphanumeric code. Where the container identifier is an image of the person, associating at 652 also may include placing the image of the person on the container. Matching identifiers at 656 may then include comparing the image of the person on the container to a visual appearance of the person.
Where the container identifier includes an RFID tag having memory 621, associating at 652 also may include storing an identifier of the person, such as a name of the person and/or other suitable identifiers, in the memory. Matching identifiers at 656 may then include reading memory 621 with a reading device and comparing the read identifier of the person with a document that identifies the person, such as an identity card and/or boarding pass. Where the container identifier is at least one barcode, associating at 652 also may include reading the barcode with a reading device and associating an identifier of the person with the barcode. Matching identifiers at 656 may then include reading the barcode with the reading device and comparing the identifier of the person associated with the barcode with a document that identifies the person.
In some embodiments where container 626 includes cover 632 and locking mechanism 634 configured to secure the cover, associating at 652 also may include securing the cover, such as when the carried items are in the container and the container identifier has been associated with an identifier of the person. Additionally, matching identifiers at 656 also may include opening the cover to allow removal of the carried item(s) from the container, such as when the container identifier and identifier of the person match.
The container and subject tag method, similar to the container method, may also include placing or requiring placement of the carried item(s) in one or more containers 626 having at least one container identifier 630 (such as one or more of a barcode, a RFID device, a color, and an alphanumeric code) as part of associating at 652. However, unlike the previously discussed method, associating at 652 may also include providing, to the person, at least one subject tag 618 having at least one subject identifier 620 (such as one or more of a barcode, an RFID device, a color, and an alphanumeric code), reading the container identifier(s) and the subject identifier(s), and associating or relating those two types of identifiers. For example, where container identifier 617 and subject identifier 620 include barcodes, associating at 652 may include reading the barcodes with a reading device and associating the barcodes with each other.
Matching at 656 may then include reading the container identifier(s) and the subject identifier(s), and confirming that the container identifier(s) is associated with the subject identifier(s). In the example above where barcodes are used for identifiers, matching at 656 may include reading the barcodes with a reading device and confirming that the barcodes are associated.
In some embodiments where container 626 includes cover 632 and a locking mechanism to secure the cover, associating at 652 also may include securing the cover, such as when the carried items are in the container and the container identifier has been associated with an identifier of the person. Additionally, matching identifiers at 656 also may include opening the cover to allow removal of the carried item(s) from the container, such as when the container identifier and identifier of the person match. Although particular methods are discussed, any suitable method(s) may be used to associate the subject and the one or more subject-associated items, such as carried items. For example, the at least one identifier of the subject-associated item tag(s) may be associated with at least one identifier of the person's ticket and/or boarding pass (such as a barcode of the subject-associated item and a barcode of the person's boarding pass).
An example of surveillance system 300 having a subject information assembly 306 mounted in a surveillance station or portal 700 is shown in
Although enclosure 702 is shown to have entrance 704 and exit 706, the enclosure may alternatively have a single opening that serves as the entrance and the exit. Additionally, although enclosure 702 is shown to completely enclose subject 32, the enclosure may less than completely enclose the subject. For example, the enclosure may exclude a top or ceiling and/or may exclude a bottom or floor.
The plurality of screening apparatus may be adapted to detect on subject 32 in subject position 34 one or more characteristics of at least one object potentially carried by the subject and/or the subject-associated item(s). For example, the plurality of screening apparatus may include a millimeter wave imaging apparatus 710, a nuclear detection apparatus 712, a biochemical detection apparatus 714, trace element detection apparatus 716, a proximity card reader 718, and a subject recognition apparatus 720.
Nuclear detection apparatus 712 may include any suitable structure adapted to detect the presence of one or more radioactive materials in the subject and/or the subject-associated item(s). Examples of nuclear detection apparatus are provided in U.S. Pat. Nos. 7,142,625 to Jones et al. and 5,371,362 to Mestais et al., the complete disclosures of which are herein incorporated by reference for all purposes. Biochemical detection apparatus 714 may include any suitable structure adapted to detect the presence of one or more biological and/or chemical substances, such as biological and chemical warfare agents. The biochemical detection apparatus may include immunoassay systems, polymerase chain reaction units, liquid-phase systems, gas-phase systems, flame spectrophotometry, infrared spectroscopy, and laser techniques. Other examples of biochemical detection and/or nuclear detection apparatus are provided in U.S. Pat. Nos. 7,088,230; 7,084,753; and 7,061,388 to Maurer et al., the complete disclosures of which are herein incorporated by reference for all purposes.
Trace element detection apparatus 716 may include any suitable structure adapted to detect the presence of one or more substances in the subject and/or the subject-associated item(s). The trace element detection may be based on one or more technologies discussed above. Proximity card reader 718 may include any suitable structure adapted to read and/or identify one or more items, such as a boarding pass, an identity card, etc. The proximity card reader may include one or more barcode readers, radio frequency identification (RFID) readers, and smart card readers. Although proximity card reader 714 is included with portal 700, the portal may alternatively, or additionally, include other types of card readers, such as magnetic stripe readers, etc.
Person recognition apparatus 720 may include any suitable structure configured to identify the person in the portal. For example, the person recognition apparatus may include any suitable technologies for face imaging, retinal imaging, fingerprint and/or other biometric identification of the person. Additionally, or alternatively, person recognition apparatus 720 may include a video camera, and/or other suitable camera.
Although portal 700 is shown to have specific types of screening apparatus, the portal may have any suitable type(s) of screening apparatus adapted to screen the subject in the subject position. Additionally, although portal 700 is shown to include six types of screening apparatus, the portal may have any suitable number of screening apparatus. Moreover, although portal 700 is shown to include the subject information assembly, the portal may additionally, or alternatively, include the subject-associated item information assembly.
While the inventions defined in the following claims have been particularly shown and described with reference to the foregoing embodiments, those skilled in the art will understand that many variations may be made therein without departing from the spirit and scope of the inventions. Other combinations and sub-combinations of features, functions, elements and/or properties may be claimed through amendment of the present claims or presentation of new claims in this or a related application. Such amended or new claims, whether they are directed to different combinations or directed to the same combinations, whether different, broader, narrower or equal in scope to the original claims, are also regarded as included within the subject matter of the present disclosure.
The foregoing embodiments are illustrative, and no single feature or element is essential to all possible combinations that may be claimed in this or later applications. Where the claims recite “a” or “a first” element or the equivalent thereof, such claims should be understood to include one or more such elements, neither requiring nor excluding two or more such elements. Further, cardinal indicators, such as first, second, and third, for identified elements are used to distinguish between the elements, and do not indicate a required or limited number of such elements, nor does it indicate a particular position or order of such elements unless otherwise specifically stated.
The methods and apparatus described in the present disclosure are applicable to security, monitoring and other industries in which surveillance and imaging systems are utilized.
This application is a continuation-in-part of U.S. patent application Ser. No. 11/679,106, filed Feb. 26, 2007and entitled “Multi-Source Surveillance System,” which is a continuation of U.S. patent application Ser. No. 10/825,530, filed Apr. 14, 2004 and entitled “Multi-Source Surveillance System,” which issued as U.S. Pat. No. 7,205,926. Additionally, this application claims priority under 35 U.S.C. §119(e) to U.S. Provisional Patent Application Ser. No. 60/795,434, filed Apr. 26, 2006 and entitled “Security Checkpoint;” U.S. Provisional Patent Application Ser. No. 60/801,961, filed May 19, 2006and entitled “Security Checkpoint;” U.S. Provisional Patent Application Ser. No. 60/810,598, filed Jun. 1, 2006 and entitled “Security Checkpoint;” and U.S. Provisional Patent Application Ser. No. 60/846,590, filed Sep. 21, 2006 and entitled “Security Checkpoint.” This application further is being filed concurrently with U.S. patent application Ser. No. 11/740,133 titled “Surveillance Systems and Methods with Subject -Related Screening,” filed Apr. 25, 2007, and U.S. patent application Ser. No. 11/740,155 titled “Surveillance of Subject-Associated Items with Identifiers,” filed Apr. 25, 2007. The complete disclosures of the above applications are herein incorporated by reference for all purposes.
Number | Name | Date | Kind |
---|---|---|---|
4866424 | Parks | Sep 1989 | A |
4901084 | Huguenin et al. | Feb 1990 | A |
4910523 | Huguenin et al. | Mar 1990 | A |
4940986 | Huguenin | Jul 1990 | A |
4964309 | Jenkins | Oct 1990 | A |
5039981 | Rodriguez | Aug 1991 | A |
5047783 | Hugenin | Sep 1991 | A |
5073782 | Huguenin et al. | Dec 1991 | A |
5078952 | Gozani et al. | Jan 1992 | A |
5115459 | Bertozzi | May 1992 | A |
5162652 | Cohen et al. | Nov 1992 | A |
5170169 | Stephan | Dec 1992 | A |
5181234 | Smith | Jan 1993 | A |
5183008 | Carrano | Feb 1993 | A |
5202692 | Huguenin et al. | Apr 1993 | A |
5212909 | Morin | May 1993 | A |
5227800 | Huguenin et al. | Jul 1993 | A |
5323004 | Ettinger et al. | Jun 1994 | A |
5367552 | Peschmann | Nov 1994 | A |
5397986 | Conway et al. | Mar 1995 | A |
5455590 | Collins et al. | Oct 1995 | A |
5465607 | Corrigan et al. | Nov 1995 | A |
5557283 | Sheen et al. | Sep 1996 | A |
5592083 | Magnuson et al. | Jan 1997 | A |
5600303 | Husseiny et al. | Feb 1997 | A |
5680103 | Tuner et al. | Oct 1997 | A |
5753832 | Bromberg et al. | May 1998 | A |
5760397 | Huguenin et al. | Jun 1998 | A |
5784430 | Sredniawski | Jul 1998 | A |
5818047 | Chaney et al. | Oct 1998 | A |
5845692 | Kellem et al. | Dec 1998 | A |
5859609 | Sheen et al. | Jan 1999 | A |
5930326 | Rothschild et al. | Jul 1999 | A |
5974111 | Krug et al. | Oct 1999 | A |
6040191 | Grow | Mar 2000 | A |
6057761 | Yuki | May 2000 | A |
6073499 | Settles | Jun 2000 | A |
6078047 | Mittleman et al. | Jun 2000 | A |
6088423 | Krug et al. | Jul 2000 | A |
6094472 | Smith | Jul 2000 | A |
6119096 | Mann et al. | Sep 2000 | A |
6137896 | Chang et al. | Oct 2000 | A |
6215851 | Meilunas et al. | Apr 2001 | B1 |
6218943 | Ellenbogen | Apr 2001 | B1 |
6334365 | Linker et al. | Jan 2002 | B1 |
6342696 | Chadwick | Jan 2002 | B1 |
6344818 | Markov | Feb 2002 | B1 |
6375697 | Davies | Apr 2002 | B2 |
6469624 | Whan et al. | Oct 2002 | B1 |
6507278 | Brunetti et al. | Jan 2003 | B1 |
6507309 | McMakin et al. | Jan 2003 | B2 |
6518915 | Schutz et al. | Feb 2003 | B2 |
6642513 | Jenkins et al. | Nov 2003 | B1 |
6665373 | Kotowski et al. | Dec 2003 | B1 |
6703964 | McMakin et al. | Mar 2004 | B2 |
6708572 | Jenkins et al. | Mar 2004 | B2 |
6721391 | McClelland et al. | Apr 2004 | B2 |
6791487 | Singh et al. | Sep 2004 | B1 |
6901163 | Pearce et al. | May 2005 | B1 |
6927691 | Yuki | Aug 2005 | B2 |
6937182 | Lovberg et al. | Aug 2005 | B2 |
6952163 | Huey et al. | Oct 2005 | B2 |
6965340 | Baharav et al. | Nov 2005 | B1 |
6970088 | Kovach | Nov 2005 | B2 |
6972714 | Baharav et al. | Dec 2005 | B1 |
6985122 | Cohen | Jan 2006 | B2 |
7030761 | Bridgelall et al. | Apr 2006 | B2 |
7139406 | McClelland et al. | Nov 2006 | B2 |
7183963 | Lee et al. | Feb 2007 | B2 |
7225979 | Silverbrook | Jun 2007 | B2 |
7280068 | Lee et al. | Oct 2007 | B2 |
7283085 | Lee et al. | Oct 2007 | B2 |
7298318 | Baharav et al. | Nov 2007 | B2 |
7317390 | Huey et al. | Jan 2008 | B2 |
7327304 | Baharav et al. | Feb 2008 | B2 |
7333055 | Baharav et al. | Feb 2008 | B2 |
7492855 | Hopkins et al. | Feb 2009 | B2 |
7830254 | Huey et al. | Nov 2010 | B2 |
8031903 | Paresi et al. | Oct 2011 | B2 |
20030056113 | Korosec | Mar 2003 | A1 |
20030070076 | Michael | Apr 2003 | A1 |
20030085163 | Chan et al. | May 2003 | A1 |
20030128150 | McMakin et al. | Jul 2003 | A1 |
20030171939 | Yagesh et al. | Sep 2003 | A1 |
20030225612 | DeSimone et al. | Dec 2003 | A1 |
20030231788 | Yukhin et al. | Dec 2003 | A1 |
20040017930 | Kim et al. | Jan 2004 | A1 |
20040041724 | Levitan et al. | Mar 2004 | A1 |
20040090359 | McMakin et al. | May 2004 | A1 |
20040140924 | Keller et al. | Jul 2004 | A1 |
20040239549 | Morgan et al. | Dec 2004 | A1 |
20040252807 | Skatter et al. | Dec 2004 | A1 |
20040263379 | Keller | Dec 2004 | A1 |
20050024199 | Huey et al. | Feb 2005 | A1 |
20050031069 | Kaucic et al. | Feb 2005 | A1 |
20050031076 | McClelland et al. | Feb 2005 | A1 |
20050057354 | Jenkins et al. | Mar 2005 | A1 |
20050073418 | Kelliher et al. | Apr 2005 | A1 |
20050110672 | Cardiasmenos et al. | May 2005 | A1 |
20050232459 | Rowe et al. | Oct 2005 | A1 |
20060012510 | Rowe | Jan 2006 | A1 |
20060022140 | Connelly et al. | Feb 2006 | A1 |
20060066469 | Foote et al. | Mar 2006 | A1 |
20060115109 | Whitson et al. | Jun 2006 | A1 |
20060274916 | Chan et al. | Dec 2006 | A1 |
20070195994 | McClelland et al. | Aug 2007 | A1 |
20080174401 | Reilly et al. | Jul 2008 | A1 |
Number | Date | Country |
---|---|---|
S61-206885 | Dec 1986 | JP |
S63-255683 | Oct 1988 | JP |
H9-197042 | Jul 1997 | JP |
H10-40371 | Feb 1998 | JP |
H10-148673 | Jun 1998 | JP |
H10-153655 | Jun 1998 | JP |
H10-282247 | Oct 1998 | JP |
H11-83996 | Mar 1999 | JP |
H11-133141 | May 1999 | JP |
2001501304 | Jan 2001 | JP |
2001235554 | Aug 2001 | JP |
2001521157 | Nov 2001 | JP |
2007502415 | Feb 2007 | JP |
2007532907 | Nov 2007 | JP |
2007536506 | Dec 2007 | JP |
9811453 | Mar 1998 | WO |
9921148 | Apr 1999 | WO |
2005017559 | Feb 2005 | WO |
2005101295 | Oct 2005 | WO |
2005101307 | Oct 2005 | WO |
Number | Date | Country | |
---|---|---|---|
20120105267 A1 | May 2012 | US |
Number | Date | Country | |
---|---|---|---|
60795434 | Apr 2006 | US | |
60801961 | May 2006 | US | |
60810598 | Jun 2006 | US | |
60846590 | Sep 2006 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 10825530 | Apr 2004 | US |
Child | 11679106 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 11679106 | Feb 2007 | US |
Child | 11740193 | US |