Patent Grant
8295541
The present invention relates to image processing. In one particular form the invention relates to a method of determining whether an object, situated in a region of interest and viewed in a sequence of images is located in an expected position or has moved, been tampered with or otherwise altered.
In another form the present invention relates to a detection system, which in one example relates to a security system capable of monitoring whether a detector forming part of the security system has undergone tampering. It will be convenient to hereinafter describe this embodiment of the invention in relation to the use of a passive infra-red (PIR) detector in a security system. However, it should be appreciated that the present invention is not limited to the embodiments and applications that are described herein.
Video camera systems have long been used to monitor areas or regions of interest for the purposes of maintaining security and the like. One important application is the use of video camera systems to monitor sensitive areas in locations such as museums or art galleries which include valuable items that could be potentially removed by a member of the public. Typically such a system would include a number of video cameras which would be monitored by a security attendant. In this human based scenario, the attendant would be relied on to detect any changes in the areas being viewed by each of the individual cameras. Clearly, this approach has a number of significant disadvantages. Notwithstanding the expense of the labour involved, this approach is prone to human error as it relies on the ability of the attendant to detect that a change of significance has occurred within the area being viewed by the camera without being distracted by any other visual diversion.
With the advent of more sophisticated image processing algorithms, and the associated computer hardware to implement these algorithms in real time, a number of attempts have been made to automate this process. A naïve approach to this problem includes the direct comparison of either individual or groups of pixel intensities of subsequent sequential images or frames which make up a digital video signal. If the difference between a group of pixels over a number of sequential images is found to be over some threshold then an alarm is generated indicating that movement has occurred within the area being viewed by the camera. Clearly, this naïve approach when applied to a viewing area which naturally includes a subset of objects moving within it (e.g. patrons at a museum) and a number of stationary items (e.g. museum exhibits) fails as the movement of patrons will trigger the alarm.
One attempt to overcome this disadvantage is to apply background modelling techniques to the subsequent images or frames corresponding to the area being viewed by the camera. In this approach, portions of the image which do not change substantially from normal from image to image are determined to be part of the background. In the example of an art gallery or museum, the paintings or artefacts would form part of the “background” of an image as they are stationary in the subsequent images or frames of the digital video signal. If one of the “background” pixels corresponding to an artefact has an intensity which varies above a predetermined threshold then this pixel is in alarm condition. However, as would be appreciated by those skilled in the art, this approach is extremely sensitive to pixel intensity changes as would typically be caused by lighting changes resulting from shadows, time of day variation and other ambient light variation. Whilst some of these effects can be compensated by employing a more sophisticated background model, this also increases the overall complexity and tuning requirements of the surveillance system. Another disadvantage of the background modelling approach and other prior art detection systems is that they fail where there is a temporary total occlusion of an object of interest or in the case where there is permanent partial occlusion of the object.
In a related area of application, various detection or monitoring systems which may be arranged to provide security or detect and measure the behaviour of objects within a field of view or detection region of the system are well-known. Examples range from Doppler radar detectors used to measure or detect a characteristic such as the speed of vehicles and active beam detectors which measure or detect a characteristic such as the reflection of an incident beam off an object to devices such as passive infra-red (PIR) detectors which measure the characteristic of heat emanated by objects and are often used in security applications. A requirement of each of these devices is that they may be orientated to inspect a predetermined field of view which corresponds to the detection region of the device.
Clearly, the performance of these devices may be degraded or totally compromised if the actual field of view or detection region is different from that assumed during initial setup. In the example of a Doppler radar detector, the characteristic of speed calculated by the device will depend on the angle of travel of the moving vehicle with respect to the orientation of the detector and errors in setup may result in erroneous results.
In the example of a PIR detector, this device may typically be located and adjusted to view regions which are required to be kept secure such as an entranceway to a building or the like. If in fact the PIR detector is not pointing in the correct direction, a person moving along the viewed entranceway may not be detected, as they may not be within the field of view of the detector.
This illustrates a significant disadvantage with devices of this nature which have a detection region set by the orientation of the device. A person wishing to gain access to a building may during the day, when the PIR detector is inactive, change the detecting direction of the device so that it no longer points towards or views a given detection region. Accordingly, when the device becomes operative at night it may no longer be pointing in the correct direction thereby allowing an intruder to potentially gain access to the building. Similarly, a radar detector which has been positioned to detect the speed of vehicles moving in a given direction may provide incorrect results if it has been tampered with by changing its detecting direction.
Any discussion of documents, devices, acts or knowledge in this specification is included to explain the context of the invention. It should not be taken as an admission that any of the material forms a part of the prior art base or the common general knowledge in the relevant art in Australia or elsewhere on or before the priority date of the disclosure herein.
It is an object of the present invention to provide a method that enables detection of an object in a sequence of images which compensates for temporary total occlusion of the object.
It is a further object of the present invention to provide a method that enables detection of an object in a sequence of images which compensates for permanent partial occlusion of the object.
It is yet still a further object of the present objection to provide a method which can be implemented in real time on a digital video system or signal.
It is also an object of the present invention to provide a detection system capable of monitoring its operation and hence whether tampering or at least unauthorised alteration of the system has taken place.
In a first aspect the present invention accordingly provides a method of detecting change of an object state from an initial state, said object displayed in a plurality of sequential images, said method comprising:
Preferably, said measure is substantially illumination invariant.
Preferably, said substantially illumination invariant measure is derived from edge characteristics of said object.
Preferably, plurality of sequential images forms a digital video signal.
In a second aspect the present invention accordingly provides a method of detection comprising the steps of:
Preferably the set of predefined criteria comprises:
In a third aspect the present invention accordingly provides a method of detection comprising the steps of:
In a fourth aspect the present invention accordingly provides a method of detection comprising the steps of:
In a fifth aspect the present invention accordingly provides a method of detection comprising the steps of:
Preferably the method further comprises the steps of:
In a sixth aspect the present invention accordingly provides a computer program product comprising:
In a seventh aspect there is provided an apparatus for carrying out the method of any one of aspects one to five of the invention.
In an eighth aspect the present invention accordingly provides a device for detecting a characteristic of a detection region, said detection region associated with a detecting direction of said device, said device comprising:
Preferably, said tamper monitoring means generates a signal on a change of detecting direction of said device.
Preferably, said tamper monitoring means monitors said change in said detecting direction by image processing means.
Preferably, said image processing means comprises imaging means to view a viewing region related to said detecting direction, said image processing means operable to detect changes in said viewing region corresponding to a change in said detecting direction of said device.
Preferably, said imaging means also comprises said detection means.
Preferably, output generated by said detection means is stored.
In a ninth aspect the present invention accordingly provides a method for monitoring for the alteration or tampering of a detection device, said detection device operable to detect a characteristic of a detection region, said method comprising the steps:
Preferably, said determining step comprises:
Preferably, said detection device further comprises imaging means to perform said viewing of said viewing region and generate said plurality of sequential images.
Preferably, said detection device is dependent on said detecting direction.
In a tenth aspect the present invention accordingly provides a method for determining a contrast measure for an image; said method comprising the steps of determining a plurality of intensity measures associated with a plurality of regions of said image;
Preferably, said step of determining a contrast measure comprises determining a first frequency value corresponding to a maximum intensity range and calculating the difference between this value and a second frequency value corresponding to a minimum intensity range.
Preferably, said first and second frequency values are above a predetermined threshold.
In an eleventh aspect the present invention accordingly provides a method for compensating for contrast changes in an image change detection method, wherein said image change detection method is based upon a comparison of a current image with a reference image, said method comprising the steps of:
In an embodiment of the present invention there is provided an apparatus adapted to monitor for the alteration or tampering of a detection device; said apparatus comprising:
In another embodiment of the present invention there is provided an apparatus adapted to determine a contrast measure for an image; said apparatus comprising:
In yet another embodiment of the present invention there is provided an apparatus adapted to compensate for contrast changes in an image change detection method, said apparatus comprising:
In further embodiments the present invention also provides computer program products comprising:
Illustrative embodiment of the present invention will be discussed with reference to the accompanying drawings wherein:
In the following description, like reference characters designate like or corresponding parts throughout the several views of the drawings.
Further scope of applicability of the present invention will become apparent from the detailed description given hereinafter. However, it should be understood that the, detailed description and any specific examples, while indicating embodiments of the invention, are given by way of illustration only, since various changes and modifications within the spirit and scope of the invention will become apparent to those skilled in the art from this detailed description.
Referring now to
The sequence of images is first processed by edge detector module 110 which detects edges of the objects within the image by use of a Sobel filter that has been set with an appropriate threshold. Whilst in this embodiment a Sobel edge filtering function has been used, other edge detection functions such as a Canny filter may be used. As would be appreciated by those skilled in the art, any image processing function which is substantially illumination invariant and hence substantially insensitive to changes in intensity may also be employed. Some illustrative examples of other image processing techniques, that may be utilised either individually or in suitable combination include the use of colour information rather than intensity information, since this has less dependence on illumination intensity, the use of a “homomorphic” filtered image, which essentially removes illumination dependence from the scene or the use of a texture measure which will determine the visual texture of the scene in the vicinity of each pixel position.
Region masking module 120 allows an operator of the system to select a number of objects within the digital video signal which in turn corresponds to selecting these objects within each frame or image which make up the digital video signal. Typically this will involve selecting those pixels which represents the object including its boundary. In this embodiment, the region masking module 120 allows a user to select all pixels within an arbitrary closed freehand curve, this process being repeated for each set of pixels corresponding to an object. In this way a number of objects may be selected within a given viewing area. In the case of a museum or art gallery monitoring system, the objects selected would correspond to those valuables or artefacts for which an alarm is generated if movement or tampering of the artefact is detected.
For each selected object, region masking module 120 generates a mask 125 and respective masked edges 126 corresponding to a portion and hence a pixel subset of the image which corresponds to each object. In this embodiment masked edge information 126 is those pixels within the masked pixel subset which contain an edge as determined by the Sobel filter applied in the edge detector module 110.
To determine reference edge characteristics or modelled edges 131, to which the edge characteristics of subsequent images can be compared to, the reference edge modeller 130 performs a moving average on masked edge information 126. This involves computing the percentage of time each of the pixels contains an edge during a predetermined learning period. This percentage value is further thresholded, so that for example those pixels which correspond to those defined to have an edge for less than a predetermined percentage of time in the learning period will not form part of the reference edge characteristics or the modelled edges 131 which form an input to decision module 190. This allows an operator to tune the sensitivity of reference edge modeller 130 by varying the threshold value as required.
Clearly, as would be apparent to those skilled in the art, the updating of the reference edge characteristics or modelled edges 131 can be selected by an operator or alternatively these characteristics may be updated automatically according to other changes in the viewing area. The intent of updating the modelled edges 131 is to ensure that a reproducible model of the object being monitored is generated. An automatic process for updating the modelled edges 131 could involve a feedback mechanism to adjust the reference characteristics so that a figure of merit which is fed back to an updater is maintained. This figure of merit could be the number of pixels in the modelled edges 131 for a given object, or the percentage coverage of the object by edge pixels, or the uniformity of that coverage, or alternatively some combination of these factors. A different automatic process, that would not require feedback, could use a measure of the visual texture in the image to determine suitable threshold parameters for both the edge detector 110 and reference edge modeller 130.
The modelled edges 131 are inputted into the alarm decision processor 190 in the form of those pixels which contain an edge after processing for the particular masked portion of the overall image. AND gate 140 selects only those pixels 141 from the masked edges 126 of subsequent frames which correspond to the pixels of the modeled edges 131 as determined by reference edge modeller 130. In this manner, processing time is reduced as analysis is only performed on the subset of pixels known to contain edges in the modelled edge information 131. This information 141 is also inputted into alarm decision processor 190 along with original mask 121 information.
Referring now to
In the event that the comparison value rises above criterion C (i.e. output FALSE) for a predetermined number of frames or images as determined by NR then alarm counter 194 is reset. By varying NR, the system can be tuned to determine how much convincing it requires before an object is deemed to be visible again. This may prevent an ALARM 196 occurring, or reset ALARM 196 if it has already occurred. An extension of this is to latch ALARM 196 or record whenever it occurs so that all ALARM 196 events are noted.
Referring now to
The Hausdorff distance is defined for two finite point sets A={a1, . . , ap} and B={b1, . . . , bq}, as
H(A, B)=Max(h(A, B), h(B, A))
where
h(A, B)max min||a−b
aεA bεB
and ||.|| is some underlying norm on the points of A and B (e.g., the L2, or Euclidean norm).
The function h (A, B) is called the directed Hausdorff distance from A to B. It identifies the point aεA that is farthest from any point of B and measures the distance from a to its nearest neighbor in B (using the given norm ||.||), that is, h (A, B) in effect ranks each point of A based on its distance to the nearest point of B and then uses the largest ranked such point as the distance (the most mismatched point of A). Intuitively if h (A, B)=d, then each point of A must be within distance d of some point of B, and there also is some point of A that is exactly distance d from the nearest point of B (the most mismatched point).” The Hausdorff distance H (A, B) is then simply the maximum of the two directed Hausdorff distances h (A, B) and h (B, A).
By using the Hausdorff distance as a comparison measure, the edge characteristics of the reference image 271 are compared directly to those of the current image 251. The Hausdorff distance tests how well a model fits the image, as well as how well the image fits the model. Although these two tests seem identical, the following example highlights the importance of considering both aspects.
Consider the scenario where the valuable to be protected is a single, blank sheet of A4 paper. If the user selected a region slightly larger than the piece of paper, the edge model would consist of only four edges, ie the edges of the piece of paper. Now, if this “valuable” was replaced by piece of A4 paper but with a small picture on it, the current image edge map would consist of the four edges of the piece of paper, along with the edges of the picture on the paper. This scenario is similar to a thief stealing an artwork and replacing it with a replica—most of the original content is accounted for, but there are some differences. Now, the reverse partial Hausdorff distance (i.e., how well the model fits the image) would not return any difference, as all four edges in the model are accounted for by the edges of the replacement A4 paper (the AND-based matching method would not detect any differences either). However, the forward partial Hausdorff distance (i.e., how well the image fits the model) would detect that the picture edges were not present in the model.
This added ability means that to escape detection, a thief would have to replace the valuable with an exact replica, placed in exactly the same position and orientation.
By way of explanation, this example serves to define what is meant herein by detecting change of object state, whether that be detecting the actual movement of an object or, determining discrepancies between stored reference images and images of the object being captured under surveillance where, the object may have been tampered with or altered, for example, by way of replacing the object with a replica in an extended time interval between capturing the reference images of the original object and capturing images of the replica object.
Referring to
Forward distance calculation module 310 determines h20% (A, B), the K-th ranked value of the forward partial Hausdorff distance corresponding to 20% of the total number of pixels being compared. This value 311 is inputted to comparison module 330 and if it is greater than 0 a TRUE signal 332 is generated and alarm counter 360 will commence counting frames. This in effect tests whether more than 20% of the model is present in the image as by definition h20% (A, B) will be 0 if this is the case.
Reverse distance calculation module determines h65% (B, A), the K-th ranked value of the reverse partial Hausdorff distance corresponding to 65% of the total number of pixels being compared. This value 321 is inputted into comparison module 330 and if it is greater than 0 a TRUE signal 332 is generated and alarm counter 360 will commence counting frames. Similar to the forward partial distance calculation, this in effect tests whether more than 65% of the image is in the model as in this case h65% (B, A) will be by definition equal to 0.
Similar to the alarm generation section described in
These illustrative embodiments of the present invention provide a simple but extremely effective system for protecting valuables in a static scene. It has been shown to accurately detect the removal of protected items in scenes ranging from a sterile indoor environment to an outdoor scene on a windy day. Given the relatively small number of assumptions and the real-time operation achievable due to the simplicity of the algorithm the present invention may be applied successfully in a wide range of situations.
Referring now to
Whilst in this embodiment the present invention has been illustrated with regard to a PIR detector, as would be clear to those skilled in the art the invention can also be applied to those detection or monitoring systems which are initially aligned and orientated to measure a characteristic in a detection region.
As is shown figuratively, detection device 500′ whose orientation has been changed with respect to correctly aligned device 500 now views a substantially different detection region 600′. Accordingly, the new field of view ranging between left boundary 520′ and 530′ does not encompass region 540 which corresponds to area 610 of car park 600 not being viewed thereby resulting in this area being insecure. As would be clear to those skilled in the art, the field of views described herein extend in three dimensions having a length, width and depth.
Referring now to
As the orientation of CCD camera 515 is fixed with respect to the orientation of PIR detector 510, any changes in the orientation of PIR detector 510 may result in a different image being viewed by CCD camera 515. Monitoring of this image change results in an alarm signal being generated that indicates that monitoring device 500 has been tampered with.
Whilst in this illustrative embodiment CCD camera 515 is substantially co-aligned with PIR detector 510 to view a similar region this is only one convenient embodiment. Clearly, as long as the orientation of CCD camera 515 remains fixed in relation to that of PIR detector 510, then any tampering with the alignment of PIR detector 510 may be detected by CCD camera 515. Additionally, there may be multiple PIR detectors that are collocated with respect to a single CCD camera 515.
Change detection algorithms that are particularly suited to detecting changes in the region viewed by CCD camera 515 have already been described herein with reference to
For this application the change detection algorithms described previously with reference to
As this change detection algorithm is dependent in one embodiment on the detection of edges within the image a further low contrast detector may be included in the algorithm to ensure that the change detection algorithm operates in conditions where there is adequate image contrast.
In one embodiment, the low contrast detector determines a histogram of the whole image in terms of frequency of pixel intensities for a given intensity bin size or range. The difference between the maximum and minimum intensities for those bins which have a frequency of occurrence above some minimum threshold provides a contrast measure that is substantially insensitive with respect to point sources such as might occur with a generally low contrast region such as a car park at dusk which may have a number of lights operating.
When low contrast conditions are detected the alarm signal provided by the change detection algorithm is ignored or alternatively the change detection algorithm is bypassed. When contrast is restored the change detection algorithm resumes normal operation. As a reference image is retained by the change detection algorithm an alarm signal may be generated once contrast is restored if there has been any tampering with the alignment of device 500.
Other modifications to the change detection algorithm which may be incorporated comprise the ability to compensate for sudden changes in lighting which may occur when an area illuminated by a number of lights are turned off, resulting in the edge features of the image changing as the area is now only illuminated by background lighting. This may result in a false alarm condition being generated.
To overcome this issue, a number of reference images may be stored which correspond to different general lighting conditions. If a comparison between a first stored reference image results in an alarm condition then a further comparison is made with a subsequent reference image corresponding to different lighting conditions. If after this comparison, the alarm condition still exists, then a general alarm is flagged. Clearly, this use of a number of reference images which each corresponds to a change in the ambient conditions is equally applicable to those embodiments of the present invention which detect the change of an object state from an initial state as described with reference to
Clearly, this principle may be applied to incorporate any number of reference images and as this comparison may be made essentially instantaneously this does not add significantly to the real time performance of the change detection algorithm. The storing of these reference images would be incorporated into the setup of device 500.
Although in this embodiment of the present invention a CCD camera and associated change detection algorithm are employed to monitor the change of detecting direction of device 500 clearly other tamper monitoring means are contemplated to be within the scope of the invention. One example comprises a collimated detector incorporated with device 500 which detects emitted light from an alignment laser. If the laser is no longer detected this would imply that the detector is no longer in line with the laser and hence the orientation of device 500 has changed. Another example of a suitable monitoring device would be an Inertial Measurement Unit (IMU) fixedly located with respect to device 500 which would directly measure the geospatial orientation and provide an alarm signal corresponding to tampering when the orientation changes.
In another embodiment, the CCD camera may form both the detector which views the detection region and the tamper monitoring means which determines any changes in the viewing direction of the detector. Separate algorithms based on the image processing methods described herein or otherwise would then be employed to process the raw output image data from the CCD camera.
In this embodiment, a first “tamper monitoring” algorithm is tailored to detect those changes which correspond to a change of viewing direction of the detector, for example by concentrating on a fixed object of known orientation. A second separate algorithm would then be customised to determine if an object of interest is missing from the detection region. Alternatively, the CCD camera may simply record and store the images for later review by security personnel with an alarm only being generated when a change of the viewing direction of the detector has been determined by the “tamper monitoring” algorithm.
Throughout the description it will be understood that the following terms may be interpreted as follows:
While the present invention has been described in connection with specific embodiments thereof, it will be understood that it is capable of further modification(s). This application is intended to cover any variations, uses or adaptations of the invention following in general, the principles of the invention and comprising such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains and as may be applied to the essential features hereinbefore set forth.
As the present invention may be embodied in several forms without departing from the spirit of the essential characteristics of the invention, it should be understood that the above described embodiments are not to limit the present invention unless otherwise specified, but rather should be construed broadly within the spirit and scope of the invention as defined in the above disclosure. Various modifications and equivalent arrangements are intended to be included within the spirit and scope of the invention and the disclosure herein. Therefore, the specific embodiments are to be understood to be illustrative of the many ways in which the principles of the present invention may be practised.
Where stated in the above disclosure, means-plus-function clauses are intended to cover structures as performing the defined function and not only structural equivalents, but also equivalent structures. For example, although a nail and a screw may not be structural equivalents in that a nail employs a cylindrical surface to secure wooden parts together, whereas a screw employs a helical surface to secure wooden parts together, in the environment of fastening wooden parts, a nail and a screw are equivalent structures.
“Comprises/comprising” when used in this specification is taken to specify the presence of stated features, integers, steps or components but does not preclude the presence or addition of one or more other features, integers, steps, components or groups thereof.”
| Number | Date | Country | Kind |
|---|---|---|---|
| 2004903572 | Jun 2004 | AU | national |
| 2004904053 | Jul 2004 | AU | national |
| 2004906689 | Nov 2004 | AU | national |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/AU2005/000955 | 6/30/2005 | WO | 00 | 4/11/2007 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2006/002466 | 1/12/2006 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 4922093 | Fardeau et al. | May 1990 | A |
| 4991223 | Bradley | Feb 1991 | A |
| 6014183 | Hoang | Jan 2000 | A |
| 20020024446 | Grech-Cini | Feb 2002 | A1 |
| 20030227548 | Kawakami et al. | Dec 2003 | A1 |
| 20040189510 | Negro et al. | Sep 2004 | A1 |
| 20050031201 | Goh | Feb 2005 | A1 |
| 20060251328 | Boo | Nov 2006 | A1 |
| 20070054350 | Walker | Mar 2007 | A1 |
| 20090033745 | Yeredor et al. | Feb 2009 | A1 |
| 20090232412 | Hutchinson et al. | Sep 2009 | A1 |
| Number | Date | Country |
|---|---|---|
| 0 261 917 | Mar 1988 | EP |
| 2045 493 | Feb 1980 | GB |
| 2 150 724 | Jul 1985 | GB |
| 2 282 294 | Mar 1995 | GB |
| 2 308 260 | Jun 1997 | GB |
| 8-265740 | Oct 1996 | JE |
| WO 9923626 | May 1999 | WO |
| WO 0232129 | Apr 2002 | WO |
| WO 03001467 | Jan 2003 | WO |
| WO 2004079681 | Feb 2004 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20070230798 A1 | Oct 2007 | US |
