This invention relates to the use of digital imaging for security applications.
Security checks, such as those performed at border crossings and entry points to high-risk areas, e.g., bridges, tunnels, government owned buildings, airports, harbors, etc., often involve stopping a vehicle, e.g., an automobile, airplane, train, or boat, and inspecting it to determine whether it contains contraband or explosives. For areas of the vehicle not easily visible by the inspector, e.g., the underside of a vehicle, use may be made of a mirror attached to an extended rigid handle, e.g., a long rod or a staff, to enable the inspector to inspect such areas.
Unfortunately, such a mirror only provides a limited view of the total underside of the vehicle. Also disadvantageously, the inspector has no way to compare what is being seen to what is appropriate to be present for the particular vehicle being inspected. Further disadvantageously, the inspectors view may be limited by the illumination that is available underneath the vehicle.
We have recognized that the problems of the prior art of vehicle inspection may be avoided by automatically a) inspecting areas of a vehicle that should not be altered and b) determining whether such areas have been altered. This may be achieved, in accordance with an aspect of the invention, by an apparatus that i) captures an image of an area of a vehicle that should not be altered from a known configuration and ii) compares the image to a reference image of the area as it should appear if it is unaltered. If the captured image and the reference image match, the vehicle is considered unaltered, and hence safe to pass. If the captured image and the reference image do not match, the vehicle is identified as likely to have been altered, and hence a candidate for further inspection.
In one embodiment of the invention, an automatic vehicle inspection system captures an image of the underside of a vehicle at a checkpoint. The image may be captured using, for example, a scanner built into, or placed on, the floor over which the vehicle is driven. An identifier of the vehicle, e.g., the vehicle's license plate number and/or the vehicle identification number (VIN), is obtained and used to obtain a stored copy of a reference image of the underside of the vehicle from a database. The database may be located local to, or remote from, the vehicle inspection system.
The vehicle inspection system performs an image matching process to compare the captured image of the vehicle's underside to the reference image to determine whether the images match. If the captured image and the reference image match, the vehicle is considered unaltered, and hence safe to pass through the checkpoint. If the captured image and the reference image do not match, the vehicle is identified as likely to have been altered, and hence not safe to pass, but instead to be a candidate for further inspection. In the latter case, areas of difference between the captured image of the vehicle's underside and the reference image may be identified, e.g., highlighted, and an alert is produced so that an inspector may investigate such differences.
Advantageously, when using the automatic vehicle inspection system an inspection may be made of the total underside of a vehicle, rather than merely a portion thereof, as was done in the prior art. Furthermore, the same approach can be used for other areas of a vehicle that should remain unaltered after a reference image is taken, e.g., the engine compartment.
The image capture portion of the automatic vehicle inspection system typically can capture an adequate image with less light than is required by a human being to make a comparable inspection. Also, the automatic vehicle inspection system may be able to operate with wavelengths of light that are not visible to human beings. Thus, the automatic vehicle inspection system does not need an environment that has adequate lighting for visual perception by a human being. Further advantageously, the automatic vehicle inspection system does not depend upon a human operator to determine whether areas of the vehicle that should not be altered have been altered, and so it is less prone to errors than a human-based inspection system.
Image capture system 30 captures at least one image of at least one area of a vehicle that should remain unaltered, e.g., the underside of a vehicle. To that end, image capture system 30 is positioned so that it can capture an image of the area of interest of vehicle 20. In the embodiment of the invention shown in
Image capture system 30 may be a digital camera, scanner, machine vision system, or the like. Image capture system 30 may be such that it can capture an image with less light than is required by a human being to make an inspection. Alternatively, image capture system 30 may be able to operate with wavelengths of light that are not visible to human beings.
Image capture system 30 may have a built-in controller or processor 40 may control it. The output of image capture system 30 is supplied to processor 40.
Vehicle identification device 35 is a device capable of obtaining information that identifies the vehicle being inspected. The vehicle identifying information may be obtained, for example, a) from character information written on the vehicle; b) from coded information printed on the vehicle, e.g., bar code or other pattern information; c) from a radio frequency identification (RFID) type tag in the vehicle; d) from a port on the vehicle, such as may be connected to the vehicle's computer; or e) from the shape and details of the exterior of the vehicle.
As noted, in one embodiment of the invention shown in
Similar to image capture system 30, vehicle identification device 35 may be a digital camera, scanner, machine vision system, or the like. Also, similar to image capture system 30, vehicle identification device 35 may be able to capture an image with less light than is required by a human being to perform an adequate inspection, or it may be able to operate with wavelengths of light that are not visible to a human being.
An image captured by vehicle identification device 35 may be analyzed using well-known character recognition techniques to determine the characters contained in the area. The image area may be analyzed by vehicle identification device 35 itself, or in conjunction with, or wholly by processor 40. The resulting character string is stored by processor 40.
In another embodiment of the invention, vehicle identification device 35 is a bar code scanner that scans a bar code pattern printed on the vehicle. The scanned pattern is then converted to a corresponding digital representation, e.g., the characters corresponding to the bar code.
In a further embodiment of the invention, vehicle identification device 35 is a RFID tag reader, which obtains an identifying code by use of an antenna that uses radio frequency waves. More specifically, vehicle identification device 35 may transmit a radio signal that activates transponder/tag 25 located in vehicle 20. When activated, transponder/tag 25 transmits back to the antenna of vehicle identification device 35 data which includes the identifying code. In such an embodiment of the invention, vehicle identification device 35 may be located at location 35.
In yet another embodiment of the invention, vehicle identification device 35 is capable of obtaining an image of a portion of a vehicle, e.g., part of the exterior of vehicle 20. Using image matching techniques the image area may be analyzed by vehicle identification device 35 itself, or in conjunction with, or wholly by processor 40, to determine the year, make, and model of vehicle 20 from the captured image.
Note that although two possible locations for vehicle identification device 35 have been shown, other locations are possible as well. Based on the foregoing, those of ordinary skill in the art will readily be able to place vehicle identification device 35 at an appropriate location to best capture the information of the type to be obtained by any particular embodiment of vehicle identification device 35.
Processor 40 may be any type of processor. Processor 40 can perform image matching techniques so as to determine whether areas of a captured image and a reference image match. Processor 40 should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (DSP) hardware, network processor, application specific integrated circuit (ASIC), field programmable gate array (FPGA), read only memory (ROM) for storing software, random access memory (RAM), and non volatile storage. The functions of processor 40 may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared.
In one embodiment of the invention, processor 40 may be a conventional computer that has software to perform image matching available to it. The image matching may be flexible enough to ignore minor differences between the captured image and the reference image, such as vendor names on replacement parts installed after delivery of a vehicle to its owner, and to yet consider such images to match. Also, processor 40 may be able to display, e.g., on monitor 43, the differences identified by the image matching.
Database 80 stores reference images of vehicles as they should appear if the vehicle is unaltered. The reference images are organized so that when information identifying a particular vehicle, or type of vehicle, is presented, the corresponding reference image may be retrieved. Thus, for example, database 80 may store a reference image of the underside of vehicle 20, and this stored reference image will be retrieved in response to presentation of an identifier of vehicle 20. The identifier may be any identifying information capturable by vehicle identification device 35. Database 80 may be stored local to processor 40, or remote therefrom.
The reference images used to populate database 80 may be supplied by a) a vehicle manufacturer, which could supply such an image immediately after manufacturing the vehicle; or b) a vehicle dealer, e.g., which could supply such an image before delivery of the vehicle to its buyer, so that the image may be taken after post manufacture vehicle customization. Also, special scanning stations may be provided by, for example, the government or security organizations, to capture, and supply for storage in database 80, images of vehicles already in use by their owners.
Those of ordinary skill in the art will readily be able to select processors, databases, vehicle identification devices and image capture systems appropriate for use in any particular implementation of an automatic vehicle inspection system.
Advantageously, when using the automatic vehicle inspection system 10 an inspection may be made of the total underside of vehicle 20, rather than merely a portion thereof, as was done in the prior art. Furthermore, the same approach can be used for other areas of a vehicle that should remain unaltered after a reference image is taken, e.g., the engine compartment. Also, advantageously, vehicle inspection system 10 can operate under lighting conditions not amenable to human inspections. Further advantageously, automatic vehicle inspection system 10 does not depend upon a human operator to determine whether areas of vehicle 20 that should not be altered have been altered, and so it is less prone to errors than a human-based inspection system.
In step 210 (
In step 220 (
In step 240 (
In optional step 280, areas of difference between the captured image and the reference image may be pointed out, e.g., by drawing a circle around the area, placing pointers on the image showing the area of difference, highlighting the area of difference, or using other conventional display techniques. The reference image and the captured image may be displayed side by side on a monitor for the convenience of a human operator. Further, the operator may be able to manipulate the image, such as magnifying and/or rotating it to better see the change that was made to the vehicle.
The process is exited in step 290.
Note that the signal indicating that the vehicle is safe to pass and/or the signal indicating that the vehicle is not safe to pass may be human perceivable.
The foregoing merely illustrates the principles of the invention. It will thus be appreciated that those skilled in the art will be able to devise various arrangements, which, although not explicitly described or shown herein, embody the principles of the invention, and are included within its spirit and scope.