Patent Application
20090022371
This document relates to the field of movement detection and more particularly that of image sensors, such as CMOS imaging devices used in the visible or infrared range, wherein a movement detection method is used.
Movement detection involves detecting the movement of moving elements with respect to fixed elements in a field of captured images. These elements may be for example vehicles or even people. Such movement detection consists of isolating, among the signals supplied by an image sensor, those related to the moving elements, for example by detecting the significant variations on the mean or variance of a signal of a pixel or a group of pixels indicating a change in the nature of the element captured in this pixel or group of pixels, with respect to those related to the fixed elements of which the mean or variance remains for example substantially constant in time. For this purpose, detection methods or algorithms are used.
A first approach consists of using a “recursive average” algorithm for such movement detection. This algorithm is based on an estimated background calculation, which is to say of the fixed elements found in all of the captured images. Which is to say Xn is the background, or the mean, corresponding to an image n, Sn is the signal corresponding to the acquired image n and 1/N is a weighting coefficient, therefore:
A comparison is then made between a chosen threshold value Th and |Sn−Xn|. If the value obtained is positive, this means that a movement has been detected. Xn and Sn may be variables obtained from a signal supplied by a pixel or by considering several signals supplied by several pixels, for example a group of pixels located next to one another that form a macropixel, like a single signal, taking for example the mean of these signals.
Such an algorithm has especially as disadvantage a lack of robustness in the detection as the detection threshold Th is determined à priori, prior to the algorithm being used, and is global for all of the pixels of the matrix. This disadvantage results in low precision of the location of the movements in the captured images. The low pass type filtering carried out by this algorithm induces dephasing, and consequently a delay in the response with respect to the signal. This delay results in a drag effect which occurs downstream of the passage of a moving element.
A second approach consists of using a “sigma-delta” algorithm for the movement detection. This algorithm permits significant variations of the signal to be detected by calculating two variables that can be assimilated to the mean value and the variance of the signal.
Firstly, the sigma-delta mean M1n is calculated, with a constant incrementation and decrementation value, for example 1, of the signal Sn corresponding to the acquired image n. For this purpose, the signal Sn is sent as an input of the first means of calculating the sigma-delta mean 2. These means 2 first carry out an initialisation M10=S0. For the following images, which is to say for n>0, these means 2 compare M1n−1 and Sn. If M1n−1<Sn, then the value of M1n−1 is incremented such that M1n=M1n−1+1. If M1n−1>Sn, then the value of M1n−1 is decremented such that M1n=M1n−1. The value of the signal Δn=|M1n−Sn| is calculated by a subtractor 4 and absolute value calculation means 6. The calculation of N×Δn is then made by the multiplier 8, wherein N is a constant corresponding to the adaptive threshold of the algorithm whose value is chosen in function of the complexity of the scene. The calculation of a sigma-delta mean M2n of N.Δn is then made and sent as an input of second means of calculating the sigma-delta mean 10. Next, the initialisation M20=0 is carried out first. For the following images, which is to say for n>0, a comparison is made of M2n−1 and N.Δn. If M2n−1<N.Δn, then the value of M2n−1 is incremented such that M2n=M2n−1+1. If M2n−1>N.Δn, then the value of M2n−1 is decremented such that M2n=M2n−1−1. Finally, a comparison is made by a comparator 12 of the signal Δn and M2n. If M2n<Δn, this means that a movement has been detected.
As for the recursive average algorithm, the variables Sn, M1n, Δn and M2n may be obtained from a signal supplied by a pixel or by considering several signals supplied by several pixels, for example a group of pixels next to one another, like a single signal, taking for example the mean of these signals.
However, such an algorithm especially has the disadvantage of not filtering high frequency parasite movements which are considered as movements to be detected (for example, a movement of the leaves of a tree or snow falling). Furthermore, the constant N used must be determined à priori, which reduces the adaptability of the detection carried out by this algorithm.
Thus there is a need to propose a method of movement detection which permits the detection of high frequency parasite movements to be reduced or eliminated and which offers more efficient detection, for example in terms of precision of locating the movements, with respect to the methods of the prior art, and which reduces or eliminates the “drag” effect obtained by the methods of the prior art.
Also, there is a need to propose a method of movement detection which requires few calculation and memory hardware resources to be used, and that can be installed analogically in a very low consumption imaging device (with for example a mean consumption equal to approximately several hundred μW).
For this purpose, one embodiment proposes a method of movement detection comprising at least the following steps:
where k: non-null positive real number, and
n: natural whole number.
Using adaptive double filtering, by adaptive calculation of two means, one designed to estimate the background and the other to detect the variations in the captured images, band pass filtering may be generated to eliminate the high frequency parasite movements, for example snow falling, which constitute a noise and thus movements that are not to be detected. This pass band filtering also enables to delete statistical backgrounds. The calculation of the difference of the two means M1n and M2n enables to only consider the objects which have a speed of movement comprised between these two means.
The adaptability of the detection is also improved by eliminating certain constants that had to determined à priori in the methods of the prior art. The sensitivity of the detection is also adapted locally to the activity of the pixels, which is to say individually for each pixel.
The second mean M2n may be obtained from the following equation:
where M2−1=0, and
1/N2: non-null positive real number.
By choosing an appropriate value of 1/N2, the time constant of the calculation of this second mean may be chosen, which may be rapid to detect the variations recorded during the movement detection.
The first mean M1n may be obtained from the following equation:
where M1−1=0, and
1/N1: non-null positive real number.
Analogously to the choice of the value of 1/N2, the choice of the value of 1/N1 may determine the time constant of the calculation of this first mean, which may be slow to estimate the background of the captured images.
In one variant, the first mean M1n may be obtained at least by the following calculation steps:
M10=S0;
And for n>0:
M1n=M1n−1+c1 when M1n−1<Sn;
M1n=M1n−1−c1 when M1n−1>Sn;
where c1: non-null positive real number.
The value of the first mean M1n may be greater than a first non-null minimum threshold value SM1n.
The value of the second mean M2n may be greater than a second non-null minimum threshold value SM2n.
When M3n is the third mean of the signal Δn, M3n may be obtained from the following equation:
where M3−1=0, and
1/N3: non-null positive real number.
When M3n is the third mean of the signal k.Δn, M3n may be obtained from the following equation:
where M3−1=0, and
1/N3: non-null positive real number.
In one variant, when M3n is the third mean of the signal Δn, M3n may be obtained at least by the following calculation steps:
M30=Δ0;
And for n>0:
M3n=M3n−1+c3 when M3n−1<Δn;
M3n=M3n−1−c3 when M3n−1>Δn;
where c3: non-null positive real number.
When M3n is the third mean of the signal k.Δn, M3n may be obtained at least by the following calculation steps:
M30=k.Δ0;
And for n>0:
M3n=M3n−1+c3 when M3n−1<k.Δn;
M3n=M3n−1−c3 when M3n−1>k.Δn;
where c3: non-null positive real number.
The value of the third mean M3n may be greater than a third non-null minimum threshold value SM3n.
The values of SM1n and/or SM2n and/or SM3n may be comprised between approximately 1/250×the dynamic of the signal Sn and 1/25×the dynamic of the signal Sn, and for example equal to approximately 1/50×the dynamic of the signal Sn, the dynamic of the signal Sn corresponding to the possible maximum value of |Sn| (for example equal to 256 for a 8 bits coded signal).
The value of k may be between approximately 1.2 and 4.
The values of the weighting coefficients 1−N, as well as the values of the incrementation and decrementation coefficients c in the case of a sigma-delta type mean calculation, may be calculated in function of the speed of acquisition of the images, and may be easily determined. The time constant τ of a mean may be equal to the ratio: sampling period of the image capture/ln(1−1/N)−1, the sampling period corresponding to the period of the signal Sn period (noted period(Sn)).
The values of the weighting coefficients 1/N which may be used for the calculation of means M1n and/or M2n and/or M3n may be chosen such that the time constant τ of the first mean M1n is short, that is lower than about 0.5 second (for example equal to 380 ms), and that the time constants τ of means M2n and/or M3n are long, that is greater than about 1 second (for example equals to 1.18 s), at a sampling period equal to 25 Hz.
The values of the incrementation and decrementation coefficients c which may be used for the calculation of means M1n and/or M3n may be chosen and adapted during the method in order to satisfy the relation c<|dSn/dn|. In one variant, when the incrementation and decrementation coefficients have fixed values, for example equal to 1, it is possible to adapt a refresh period Tn of the method, that is the period of which the steps of the method are realized, in order to satisfy the relation c<|ΔSn/Tn|.
The values of 1/N1 and/or 1/N2 and/or 1/N3 may be chosen such that:
période(Sn)/ln(1−1/N1)−1<0.5 s;
période(Sn)/ln(1−1/N2)−1>1 s;
période(Sn)/ln(1−1/N3)−1>1 s;
and/or the values of c1 and/or c3 may verify the relation: c<|dSn/dn|.
The calculation of the third mean M3n may be realized when Δn has a non-null value.
Another embodiment also relates to a movement detection device comprising at least:
where k: non-null positive real number, and
n: natural whole number.
The means of calculating, or the calculator of, the second mean M2n may carry out at least the following operation:
where M2−1=0, and
1/N2: non-null positive real number.
The means of calculating, or the calculator of, the second mean M2n may comprise at least two switched capacities connected in parallel to one another, wherein a first of the two capacities comprises a capacity
and a second comprises a capacity
where 1/N2: non-null positive real number.
The means of calculating, or the calculator of, the first mean M1n may carry out at least the following operation:
where M1−1=0, and
1/N1: non-null positive real number.
The means of calculating, or the calculator of, the first mean M1n may comprise at least two switched capacities connected in parallel to one another, wherein a first of the two capacities comprises a capacity
and a second comprises a capacity
where 1/N1: non-null positive real number.
In one variant, the means of calculating, or the calculator of, the first mean M1n may comprise at least:
When M3n is the third mean of the signal Δn, the means of calculating, or the calculator of, the third mean M3n may carry out at least the following operation:
where M3−1=0, and
1/N3: non-null positive real number.
When M3n is the third mean of the signal
k.Δn, the means of calculating, or the calculator of, the third mean M3n may carry out at least the following operation:
where M3−1=0, and
1/N3: non-null positive real number.
The means of calculating, or the calculator of, the third mean M3n may comprise at least two switched capacities connected in parallel to one another, wherein a first of the two capacities comprises a capacity
and a second comprises a capacity
where 1/N3: non-null positive real number.
In one variant, when M3n is the third mean of the signal Δn, means of calculating, or the calculator of, the third mean M3n may comprise at least:
When M3n is the third mean of the signal k.Δn, means of calculating, or the calculator of, the third mean M3n may comprise at least:
The means of comparison may comprise at least one operational amplifier, or transconductance amplifier.
Finally, another embodiment also relates to an image capture device comprising at least one pixel matrix and one movement detection device as previously described.
This invention will be more clearly understood upon reading the following description of embodiments provided purely by way of illustration and in no way restrictively, in reference to the appended drawings in which:
Identical, similar or equivalent parts of the various figures described below have the same numerical references so as to facilitate the passage from one figure to another.
The different parts shown in the figures are not necessarily to a uniform scale, to make the figures easier to read.
The different possibilities (variants and embodiments) should be understood as not being mutually exclusive and may be combined with one another.
A double adaptive filtering movement detection device 100 and method, using a RAE algorithm (“Recursive average and Estimator”), according to one specific embodiment will now be described in relation to
A first step of this method is to estimate the background of the images in which a movement is to be detected, wherein the background is formed by the fixed elements in the captured images. For this purpose, firstly, using means 102, a first recursive average M1n is calculated wherein an important weighting coefficient 1/N1 of a signal Sn corresponding to the signal supplied by a pixel, or a group of pixels also called macropixel, of an n-th acquired image. Wherein:
In this calculation of the first mean M1n, the choice of the value of the weighting coefficient 1/N1 is important so that only the fixed elements in the acquired images are conserved. For an image capture device operating at 25 Hz, for example a weighting coefficient such as N1=24 is chosen. The value of the weighting coefficient 1/N1 is chosen in function of the amplitude and the frequency of the variations of the signal Sn.
Consequently, the value M1n calculated for a pixel or a macropixel corresponds to the luminous intensity emitted by the fixed element captured by this pixel or this macropixel, even if one or several moving elements temporarily pass before this pixel or this macropixel during the movement detection.
In one variant of this embodiment, the first mean M1n may not be a recursive average, but a sigma-delta mean, for example with a low level of incrementation and decrementation c1 (for example equal to 1), also permitting the estimation of the background to be calculated. This low level of incrementation and decrementation corresponds to an important time constant in a recursive average calculation. For this purpose, firstly the initialisation M10=S0 is carried out. For the following images, which is to say for n>0, M1n−1 and Sn are compared. If M1n−1<Sn, then the value of M1n−1 is incremented such that M1n=M1n−1+c1. If M1n−1>Sn, then the value of M1n−1 is decremented such that M1n=M1n−1−c1.
With respect to a recursive type average, a sigma-delta type mean value has the advantage of not directly depending on the amplitude of the variations of the signal for which the mean is calculated.
In parallel to the calculation of this first mean M1n, of the recursive or sigma-delta type, calculation means 104 calculate a second recursive average M2n, with a low weighting coefficient 1/N2, of the signal Sn. This provides:
By choosing the low weighting coefficient 1/N2, for example equal to 1/22, a filtering of the high frequency parasite elements (for example moving leaves of trees or falling rain or snow) is realized. The weighting coefficient 1/N2 is chosen in function of the amplitude and the frequency of the variations of the captured signal Sn.
Then the signal Δn=|M1n−M2n| is calculated using a subtractor 106 and absolute value calculation means 108.
The device 100 also comprises means 110 of calculating a third mean M3n, for example recursive with an important weighting coefficient 1/N3, of the signal Δn. This third recursive mean M3n is obtained using the following calculation:
The weighting coefficient 1/N3 is chosen in function of the amplitude and the frequency of the variations of the captured signal Sn. For example N3=26. The value of the operating frequency of the movement detection device also intervenes in the choice of the value of the weighting coefficient 1/N3.
In one variant, the third mean M3n may not be a recursive average, but a sigma-delta mean, for example with a constant incrementation and decrementation level c3 (for example equal to 1). For this purpose, firstly the initialisation M30=Δ0 is made. For the following images, which is to say for n>0, M3n−1 and Δn are compared. If M3n−1<Δn, then the value of M3n−1 is incremented such that M3n=M3n−1+c3. If M3n−1>Δn, then the value of M3n−1 is decremented such that M3n=M3n−1−c3.
Finally, to determine the presence or absence of movements in the captured image n, the device 100 comprises a comparator 114 which makes a comparison of the signal Δn and the product of the third mean M3n by an amplification constant k obtained at the output of a multiplier 112. The value of k is chosen in function of the operating frequency of the image capture device capturing the images processed. For example, k may have a value approximately between 1.2 and 4, or for example approximately between 1.5 and 2.5 for an image capture device operating at 25 Hz. If Δn>k.M3n, this means that a movement has been detected in the captured images.
In one variant, M3n may corresponds to the mean of the product of the signal Δn and the amplification constant k. In the case of a recursive third mean M3n, M3n may be obtained using the following calculation:
In the case of a sigma-delta third mean M3n, the initialisation M30=k.Δ0 is first made. For the following images, which is to say for n>0, M3n−1 and k.Δn are compared. If M3n−1<k.Δn, then the value of M3n−1 is incremented such that M3n=M3n−1+c3. If M3n−1>k.Δn, then the value of M3n−1 is decremented such that M3n=M3n−1−c3. Finally, when M3n is the mean of k.Δn, the comparator 114 makes a comparison of the signal Δn and the third mean M3n. If Δn>M3n, this means that a movement has been detected in the captured images.
On
This method is based on the generation of two adaptive means, which is to say a first recursive or sigma-delta average and a second recursive average, each with their own weighting coefficients 1/N1 (or c1 in the case of a first sigma-delta type mean) and 1/N2, respectively estimating the background and filtering the high frequency parasite movements. The thresholding of the variations detected is therefore adaptive as amplification is made with the third mean M3n of the absolute difference of the two previous first means M1n and M2n.
In general, the incrementation and decrementation values c1 and/or c3 and/or the weighting coefficients 1/N1, 1/N2 and 1/N3 used in the means calculations are adapted in function of the pixel resolution of the captured images, which is to say the number of levels of grey onto which the processed signal is encoded, as well as the operating frequency of the movement detection device 100.
In this
It is possible to impose a minimum threshold value SM3n below which this mean value M3n is not allowed to drop. For example, in this embodiment, the value of SM3n may be equal to about 1/50×the dynamic of the signal Sn.
One example of an image capture device 200, or optical imaging device, permitting a double adaptive filtering movement detection method previously described to be implemented is shown in
The image capture device 200 comprises pixel matrix 202 and a movement detection device 204. Each pixel of the matrix 202 is here formed by a photodiode and addressing and reading transistors. The double adaptive filtering movement detection method previously described may be applied to the signal Sn supplied by a pixel, by connecting a movement detection device similar to the device 204 to each column of pixels of the matrix 202. It is also possible that the movement detection method is applied to a signal Sn corresponding to the signals supplied by several pixels, for example the mean of these signals. Consequently, by considering the macropixels, it is possible to operate the image capture device 200 in low resolution zones, and only to detail these zones by using a double adaptive filtering movement detection method for each pixel of a macropixel when a movement is detected on this macropixel. It is therefore possible to reduce the number of movement detection devices 204 used in the image capture device 200 by only using a single movement detection device 204 per column of macropixels. A macropixel may for example be a square of 12×12 pixels, or any other value, for example 4×4 pixels as in
The movement detection devices 204 shown in
The operation of the movement detection device 204 will now be described in relation to the implementation of the movement detection method previously described.
In the case of a first mean M1n of the recursive type, the value of the signal Mn−1 stored in one of the memory registers 210 is supplied to the input of the multiplexer 216, by means of the address multiplexer 212b. When n=0, M−1=0. The first recursive mean M1n is then calculated. For this purpose, the signal Mn−1 is sent to the terminals of a first switched capacity 208 whose value C3 is equal to (N1−1)/N1. The value of the signal Mn−1 is stored at the terminals of this capacity.
The signal Sn supplied by the first macropixel of the matrix 202 is then supplied to the input of the multiplexer 216. The value of the signal Sn is stored at the terminals of a second switched capacity whose value C4 is equal to 1/N1.
By connecting in parallel the two switched capacities of values C3 and C4, the signal
is obtained at the terminals of these capacities.
It is also possible that the mean M1n is a sigma-delta mean. For this purpose, the initialisation M10=S0 is carried out by storing the value of the signal S0 in one of the memory registers 210. For the following images, M1n−1 and Sn are compared by applying one of the two signals to the positive input of the comparator 206 by means of the capacitor 222 in which this signal is stored, and by applying the other of the two signals to the negative input of the comparator 206. The result obtained at the output of the comparator 206 the permits one of the values +c1 or −c1 applied to the input of the multiplexer 216 (+c and −c in
Whether the mean M1n is of the recursive or sigma-delta type, its value is stored in one of the memory registers 210. In this embodiment, the movement detection device 204 comprises three memory registers 210 designed to store three different means M1n, M2n and M3n.
In a similar manner to the calculation of the first mean M1n when the latter is of the recursive type, the calculation of the second recursive mean M2n is carried out. This calculation is made by using another of the memory registers 210 and two other switched capacities of the set-up 208 with values C1 and C2, respectively of values (N2−1)/N2 and 1/N2.
Then the signal Δn=|M1n−M2n| is calculated. This operation may be carried out by means of the switched capacities 208.
In a similar manner to the calculation of the first mean M1n, the calculation of the third mean M3n is carried out. As for the first mean M1n, the third mean M3n may be of the sigma-delta type or the recursive type. This calculation is implemented by the same elements of the device 204 as those used to calculate the first mean M1n when M1n and M3n are of the same type.
A comparison is then made using the amplifier 206 of the signal Δn and the product of the third mean M3n by an amplification constant k whose value is obtained by the ratio between one of the capacities 220 and another of the switched capacities 208, or between the signal Δn and the third mean M3n when M3n corresponds to the third mean M3n of k.Δn. The value obtained at the output of the amplifier 206 is representative of a detection or non-detection of a movement on the macropixel considered. If Δn>k.M3n (or Δn>M3n when M3n corresponds to the third mean M3n of k.Δn), then it is considered that a movement has been detected on the macropixel considered.
The operation is then repeated for the following macropixels, line after line.
During the calculations of the third mean M3n, a minimum threshold value may be imposed, below which this mean is not allowed to descend. This minimum threshold value SM3n may be applied to the input of the multiplexer 216. When the value of this mean is below this threshold value, the value of M3n is then replaced by the threshold value SM3n.
When a movement is detected on the macropixel, it is possible, for an image n for which the movement detection has been carried out on macropixels, to store the values of the macropixels in the memory 218, then, on the macropixel(s) where movements are recorded, to implement the movement detection method previously described for each of the pixels of the macropixel. Consequently, the location of the movements detected may be defined precisely, without processing all of the pixels of the captured images.
It may be seen that the method may be implemented using few hardware calculation resources (an operational amplifier, several switched capacities with a clock frequency for the instructions of several tens of kHz and several multiplexers/demultiplexers) and memory resources (several analogue registers per pixel and a SRAM memory for example).
The device shown in
The signals obtained at the output of the movement detection devices may be used to display an image on which the background captured forms a black background onto which the moving elements detected are shown in white, for example as shown on the image on the right in
| Number | Date | Country | Kind |
|---|---|---|---|
| 07 56519 | Jul 2007 | FR | national |
