Patent Application
20090022227
This document relates to the field of movement detection and more particularly that of image sensors, such as CMOS imaging devices used in the visible or infrared range, wherein a movement detection method is used.
Movement detection involves detecting the movement of moving elements with respect to fixed elements in a field of captured images. These elements may be for example vehicles or even people. Such movement detection consists of isolating, among the signals supplied by an image sensor, those related to the moving elements, for example by detecting the significant variations on the mean or variance of a signal of a pixel or a group of pixels indicating a change in the nature of the element captured in this pixel or group of pixels, with respect to those related to the fixed elements of which the mean or variance remains for example substantially constant in time. For this purpose, detection methods or algorithms are used.
A first approach consists of using a “recursive average” algorithm for such movement detection. This algorithm is based on an estimated background calculation, which is to say of the fixed elements found in all of the captured images. This is to say Xn is the background, or the mean, corresponding to an image n, Sn is the signal corresponding to the acquired image n and 1/N is a weighting coefficient, therefore:
A comparison is then made between a chosen threshold value Th and |Sn−Xn|. If the value obtained is positive, this means that a movement has been detected. Xn and Sn may be variables obtained from a signal supplied by a pixel or by considering several signals supplied by several pixels, for example a group of pixels located next to one another that form a macropixel, like a single signal, taking for example the mean of these signals.
Such an algorithm has especially as disadvantage a lack of robustness in the detection as the detection threshold Th is determined à priori, prior to the algorithm being used, and is global for all of the pixels of the matrix. This disadvantage results in low precision of the location of the movements in the captured images. The low pass type filtering carried out by this algorithm induces dephasing, and consequently a delay in the response with respect to the signal. This delay results in a drag effect which occurs downstream of the passage of a moving element.
A second approach consists of using a “sigma-delta” algorithm for the movement detection. This algorithm permits significant variations of the signal to be detected by calculating two variables that can be assimilated to the mean value and the variance of the signal.
Firstly, the sigma-delta mean M1n is calculated, with a constant incrementation and decrementation value, for example 1, of the signal Sn corresponding to the acquired image n. For this purpose, the signal Sn is sent as an input of the first means of calculating the sigma-delta mean 2. These means 2 first carry out an initialisation M10=S0. For the following images, which is to say for n>0, these means 2 compare M1n-1 and Sn. If M1n-1<Sn, then the value of M1n-1 is incremented such that M1n=M1n-1+1. If M1n-1>Sn, then the value of M1n-1 is decremented such that M1n=M1n-1−1. The value of the signal Δn=|M1n−Sn| is calculated by a subtractor 4 and absolute value calculation means 6. The calculation of N×Δn is then made by the multiplier 8, wherein N is a constant corresponding to the adaptive threshold of the algorithm whose value is chosen in function of the complexity of the scene. The calculation of a sigma-delta mean M2n of N.Δn is then made and sent as an input of second means of calculating the sigma-delta mean 10. Next, the initialisation M20=0 is carried out first. For the following images, which is to say for n>0, a comparison is made of M2n-1 and N.Δn. If M2n-1<N.Δn, then the value of M2n-1 is incremented such that M2n=M2n-1+1. If M2n-1>N.Δn, then the value of M2n-1 is decremented such that M2n=M2n-1−1. Finally, a comparison is made by a comparator 12 of the signal Δn and M2n. If M2n<Δn, this means that a movement has been detected.
As for the recursive average algorithm, the variables Sn, M1n, Δn and M2n may be obtained from a signal supplied by a pixel or by considering several signals supplied by several pixels, for example a group of pixels next to one another, like a single signal, taking for example the mean of these signals.
However, such an algorithm especially has the disadvantage of not filtering high frequency parasite movements which are considered as movements to be detected (for example, a movement of the leaves of a tree or snow falling). Furthermore, the constant N used must be determined à priori, which reduces the adaptability of the detection carried out by this algorithm.
Thus there is a need to propose a method of movement detection which permits the detection of high frequency parasite movements to be reduced or eliminated and which offers more efficient detection, for example in terms of precision of locating the movements, with respect to the methods of the prior art, and which reduces or eliminates the “drag” effect obtained by the methods of the prior art.
Also, there is a need to propose a method of movement detection which requires few calculation and memory hardware resources to be used, and that can be installed analogically in a very low consumption imaging device (with for example a mean consumption equal to approximately several hundred μW).
For this purpose, one embodiment proposes a method of movement detection comprising at least the following steps:
is different from the sign of the value
or for which the sign of the value (Sa-1−Sa-2) is different from the sign of the value (Sa−Sa-1);
where n: natural whole number.
Consequently, a local adaptive movement detection threshold is used, which is to say a threshold specific to each pixel whose value is not constant but based on the activity of the signal from each of the pixels (amplitude, interference frequencies on the signal, . . . ) to determine the presence or absence of movement from significant variations of the signal.
The adaptability of the detection is also improved by eliminating certain constants that had to determined à priori in the methods of the prior art. The sensitivity of the detection is also adapted locally to the activity of the pixels, which is to say individually for each pixel.
When no movement is detected, the value of variations (signal En) is null or approximately null. The first mean M1n, the product En.M1n (signal γn) and the signal M2n are also approximately null. This method is thus well adapted to realize movement detection in a scene comprising few movements.
Moreover, this method is also adapted to realize movement detection in a scene comprising many movements. Whatever the magnitude of the signal Sn occurring during a movement (with for example the value of Sn varying from 10 to 100 or from 100 to 1000, and En thus varying from 0 to 90 or from 0 to 900), a movement detection is realized because the value of |Sn−M3n| is greater than the value of M2n during a movement. This movement detection method runs whatever the value of the analyzed signal Sn.
The first mean M1n may be obtained at least by the following calculation steps:
and for n>0:
where c1: non-null positive real number.
In one variant, the first mean M1n may be obtained at least from the following equation:
where M1-1=0, and
1/N1: non-null positive real number.
The value of the first mean M1n may be greater than a first non-null minimum threshold value SM1n.
The second mean M2n may be obtained at least by the following calculation steps:
and for n>0:
where c2: non-null positive real number.
In one variant, the second mean M2n may be obtained at least from the following equation:
where M2-1=0, and
1/N2: non-null positive real number.
The value of the second mean M2n may be greater than a second non-null minimum threshold value SM2n.
Moreover, the value of the second mean M2n may be less than a maximum threshold value, for example comprised between approximately 0.1×the dynamic of the signal Sn and 0.2×the dynamic of the signal Sn, the dynamic of the signal Sn corresponding to the possible maximum value of |Sn| (for example equal to 256 for a 8 bits coded signal).
The maximum or minimum threshold values may be imposed to these means during the detection method so that the means value does not exceed these threshold values.
The third mean M3n may be obtained at least by the following calculation steps:
and for n>0:
The value of the third mean M3n may be greater than a third non-null minimum threshold value SM3n.
The calculations of the first mean M1n, the signal γn and the second mean M2n may be realized when En has a non-null value.
The values of the weighting coefficients 1/N, as well as the values of the incrementation and decrementation coefficients c in the case of a sigma-delta type mean calculation, may be calculated in function of the speed of acquisition of the images, and may be easily determined. The time constant τ of a mean may be equal to the ratio: sampling period of the image capture/ln(1−1/N)−1, the sampling period corresponding to the period of the signal Sn period (noted period (Sn)).
The values of the weighting coefficients 1/N which may be used for the calculation of means M1n and/or M2n may be chosen such that the time constant(s) τ of these means are comprised between approximately 1 second and 10 seconds, or between approximately 1 second and 100 seconds, according to the referred application and/or the speed of the movements to detect.
The values of the incrementation and decrementation coefficients c which may be used for the calculation of means M1n and/or M2n may be chosen and adapted during the method in order to satisfy the relation c<|dSn/dn|. In one variant, when the incrementation and decrementation coefficients have fixed values, for example equal to 1, it is possible to adapt a refresh period Tn of the method, that is the period of which the steps of the method are realized, in order to satisfy the relation c<|ΔSn/Tn|.
The values of 1/N1 and/or 1/N2 may verify the relation: 1 s<période(Sn)/ln(1−1/N)−1<100 s;
and/or the values of c1 and/or c2 may verify the relation: c<|dSn/dn|.
The values of SM1n and/or SM2n and/or SM3n may be comprised between approximately 1/250×the dynamic of the signal Sn and 1/25×the dynamic of the signal Sn, and for example equal to approximately 1/50×the dynamic of the signal Sn.
Another embodiment relates to a movement detection device comprising at least:
is different from the sign of the value
or for which the sign of the value (Sa-1−Sa-2) is different from the sign of the value (Sa−Sa-1);
where n: natural whole number.
The means of calculating, or the calculator of, the first mean M1n may at least comprise:
In one variant, the means of calculating, or the calculator of, the first mean M1n may carry out at least the following operation:
where M1-1=0, and
1/N1: non-null positive real number.
The means of calculating, or the calculator of, the second mean M2n may at least comprise:
In one variant, the means of calculating, or the calculator of, the second mean M2n may carry out at least the following operation:
where M2-1=0, and
1/N2: non-null positive real number.
The means of calculating, or the calculator of, the third mean M3n may at least comprise:
The means of comparison, or the comparator, may at least comprise an operational amplifier, or transconductance amplifier.
Finally, another embodiment also relates to an image capture device comprising at least one pixel matrix and one movement detection device as previously described.
This invention will be more clearly understood upon reading the following description of embodiments provided purely by way of illustration and in no way restrictively, in reference to the appended drawings in which:
Identical, similar or equivalent parts of the various figures described below have the same numerical references so as to facilitate the passage from one figure to another.
The different parts shown in the figures are not necessarily to a uniform scale, to make the figures easier to read.
The different possibilities (variants and embodiments) should be understood as not being mutually exclusive and may be combined with one another.
An adaptive threshold sigma-delta algorithm movement detection device 100 and method, according to one specific embodiment will now be described in relation to
A first step of this method is to determine a threshold, which will subsequently be used to determine the presence or absence of movement in the captured images.
Firstly a calculation is made, using calculation means 102, of an envelope signal En of a signal Sn corresponding to the signal supplied by a pixel, or a group of pixels also called macropixel, of an n-th acquired image. The value of this envelope signal En is equal to the absolute value of the difference between Sn and the value of the final extreme value previously reached by the signal Sn. This final extreme value corresponds to the last value of the signal Sn in which the sign of the derivative
has changed.
is different from the sign of the value
or for which the sign of the value (Sa-1−Sa-2) is different from the sign of the value (Sa−Sa-1).
The envelope signal En obtained is indeed representative of the variations undergone by the output signal Sn of a pixel or a macropixel and thus enables the deletion of the direct component of the signal Sn.
Then the calculation is made, using calculation means 104, of a first sigma-delta mean M1n with a constant level c1 of incrementation and decrementation (for example 1 if the value of M1n is within a range of values of levels of grey) of the envelope signal En. The calculation of this first mean M1n permits the mean of the signal variations to be calculated, which is to say the mean deviation of dispersions, which can be expressed as the mean of differences between the signal value and his mean. For this purpose, the incrementation and decrementation value c1 chosen is low (in this case 1).
To calculate this first mean M1n, firstly the initialisation M10=E0 is carried out. For the following images, which is to say for n>0, M1n-1 and En are compared. If M1n-1<En, then the value of M1n-1 is incremented such that M1n=M1n-1+c1. If M1n-1>En, then the value of M1n-1 is decremented such that M1n=M1n-1−c1.
It is also possible that this first mean M1n is not a sigma-delta type mean, but a recursive type mean. In this case, the calculation means 104 calculate the first recursive mean M1n with a high weighting coefficient 1/N1, of the signal En. This first recursive mean M1n is obtained with the following calculation:
The value of the weighting coefficient 1/N1 is especially chosen in function of the amplitude and the frequency of the variations of the envelope signal En. For example N1=26 for image captured at around 25 Hz.
The signals M1n and En are then sent to the input of a multiplier 106 which calculates the signal γn=M1n×En.
Then the calculation is made, using calculation means 107, of a second sigma-delta mean M2n with a constant level c2 of incrementation and decrementation (for example equal to 1) of the signal γn.
To calculate this second mean M2n, firstly the initialisation M20=γ0 is carried out. For the following images, which is to say for n>0, M2n-1 and γn are compared. If M2n-1<γn, then the value of M2n-1 is incremented such that M2n=M2n-1+c2. If M2n-1>γn, then the value of M2n-1 is decremented such that M2n=M2n-1−c2.
It is also possible that this second mean M2n is not a sigma-delta type mean, but a recursive type mean. In this case, the calculation means 107 calculate the second recursive mean M2n with a high weighting coefficient 1/N2, of the signal γn. This second recursive mean M2n is obtained by the following calculation:
The value of the weighting coefficient 1/N2 is chosen in function of the amplitude and the frequency of the variations of the envelope signal γn. For example N2=26 for images captured at around 25 Hz.
Calculation means 108 then calculate a third sigma-delta mean M3n of the signal Sn. This third mean M3n is calculated with a non-constant level of incrementation and decrementation. This level of incrementation and decrementation corresponds to the value of the signal M2n. Firstly, the initialisation M30=S0 is carried out. For the following images, M3n-1 and Sn are compared. If M3n-1<Sn, then the value of M3n-1 is incremented such that M3n-1=M3n-1+M2n. If M3n-1>Sn, then the value of M3n-1 is decremented such that M3n=M3n-1−M2n.
It may therefore be seen that for the calculation of this third mean M3n, the threshold value used to increment and decrement is adaptive in function of variations of the signal Sn.
Finally, to determine the presence or absence of movements on the captured image n, a comparison is made between the signal Δn=|Sn−M3n|, obtained from a subtractor 110 and absolute value calculation means 112, and the signal M2n. If M2n<|Sn−M3n|, this means that a movement has been detected in the captured images.
This method is based on the calculation of a sigma-delta mean with constant incrementation and decrementation which varies in function of the variations in the captured images, filtering the high frequency parasite movements. The thresholding of the variations detected is therefore adaptive.
With respect to a recursive type mean, a sigma-delta type mean value has the advantage of not directly depending on the amplitude of the variations of the signal for which the mean is calculated.
In general, the incrementation and decrementation values c1 and c2 and/or the weighting coefficient values 1/N1 and 1/N2 used in the means calculations are adapted in function of the pixel resolution of the captured images, which is to say the number of levels of grey onto which the processed signal is encoded, as well as the operating frequency of the movement detection device 100.
In this
During the calculations of means M1n and M2n, it is possible to impose a minimum threshold value SMn below which the mean values are not allowed to drop. For example, in this embodiment, the values of SM1n and/or SM2n may be equal to about 1/50×the dynamic of the signal Sn.
One example of an image capture device 200, or optical imaging device, permitting the previously described movement detection method to be implemented is shown in
The image capture device 200 comprises pixel matrix 202 and a movement detection device 204. Each pixel of the matrix 202 is here formed by a photodiode and addressing and reading transistors. The movement detection method previously described may be applied to the signal Sn supplied by a pixel, by connecting a movement detection device similar to the device 204 to each column of pixels of the matrix 202. It is also possible that the movement detection method is applied to a signal Sn corresponding to the signals supplied by several pixels, for example the mean of these signals. Consequently, by considering the macropixels, it is possible to operate the image capture device 200 in low resolution zones, and only to detail these zones by using the movement detection method for each pixel of a macropixel when a movement is detected on this macropixel. It is therefore possible to reduce the number of movement detection devices 204 used in the image capture device 200 by only using a single movement detection device 204 per column of macropixels. A macropixel may for example be a square of 12×12 pixels, or any other value, for example 4×4 pixels as in
The movement detection devices 204 shown in
The operation of the movement detection device 200 will now be described in relation to the implementation of the movement detection method previously described.
Firstly, the envelope signal En of the signal Sn is calculated, For this purpose, the signal Sn supplied at the output of the pixel matrix 202 is sent to the calculation means 224 supplying in output the envelope signal En whose value is calculated according to the previously described method. The value of the envelope signal is then stored in one of the analogue memory registers 210.
Next, to calculate the first mean M1n, when it is of the sigma-delta type, initialisation M10=E0 is carried out by storing the value of the signal E0 in one of the memory registers 210. For the following images, M1n-1 and En are compared by applying one of the two signals to the positive input of the comparator 206 by means of the capacitor 222 in which this signal is stored, and by applying the other one of the two signals to the negative input of the comparator 206. The result obtained at the output of the comparator 206 then permits one of the values +c1 or −c1 applied to the input of the multiplexer 216 (+c or −c in
In one variant, the first mean M1n may be of the recursive type and be calculated from the following equation:
For this purpose, the signal Mn-1 (where M-1=0) is sent to the terminals of a first switched capacity 208 whose value is equal to (N1−1)/N1. The value of the signal Mn-1 is stored at the terminals of this switched capacity.
The signal Sn supplied by the first macropixel of the matrix 202 to the input of the multiplexer 216 is then read. The value of the signal Sn is stored at the terminals of a second switched capacity whose value is equal to 1/N1.
By connecting in parallel these two switched capacities, the signal
is indeed obtained at their terminals.
The value of the mean M1n is then stored in one of the memory registers 210.
Next, to obtain the signal γn, the product of En×M1n is calculated. For this purpose, the two values of En and M1n stored in the memory registers 210 are sent to the two inputs of the multiplication means 226, wherein the result of this multiplication may then be stored in one of the memory registers 210.
In a similar manner to the calculation of the first mean M1n of the envelope signal En, the calculation of the second mean M2n of the signal γn is carried out. As for the first mean M1n, the second mean M2n may be of the sigma-delta or recursive type. This calculation is implemented by the same elements of the device 204 as those used to calculate the first mean M1n when M1n and M2n are of the same type.
Next the calculation of the third sigma-delta mean M3n of the signal Sn is carried out with an incrementation and decrementation value of M2n. This calculation is implemented by the same elements of the device 204 as those used to calculate the first mean M1n when the latter is of the sigma-delta type.
Next the calculation of the signal Δn=|Sn−M3n| is carried out. This operation may be carried out using switched capacities 208.
A comparison is then made using the amplifier 206 of the signal Δn and the signal M2n. The value obtained at the output of the amplifier 206 permits the detection or non-detection of a movement on the macropixel considered. If Δn>M2n, then it is considered that a movement has been detected on the macropixel considered. This value obtained at the output of the amplifier 206 may be stored in the memory 218.
The operation is then repeated for the following macropixels, line after line.
During all of the calculations of the means M1n and M2n, a minimum threshold value may be imposed, below which the values of these means are not allowed to descend. This minimum threshold value SMn may be applied to the input of the multiplexer 216. When the value of one of these means drops below this threshold value, the value of this mean is then replaced by the threshold value. This threshold value may be different according to whether it is M1n or M2n that is calculated. In this case, several minimum non-null threshold values SM1n and SM2n may be applied to the input of the multiplexer 216. Moreover, a maximum threshold value may be imposed above which the value of the second mean M2n is not allowed to rise. When the value of M2n exceeds the value of this maximum threshold, the value of this mean M2n is replaced by the threshold value, for example applied to the input of the multiplexer 216.
When a movement is detected on the macropixel, it is possible, for an image n for which the movement detection has been carried out on macropixels, to store the values of the macropixels in the memory 218, then, on the macropixel(s) where movements are recorded, to implement the movement detection method previously described for each of the pixels of the macropixel. Consequently, the location of the movements detected may be defined precisely, without processing all of the pixels of the captured images.
It may be seen that the method may be implemented using few hardware calculation (an operational amplifier, several switched capacities with a clock frequency for the instructions of several tens of kHz and several multiplexers/demultiplexers) and memory resources (several analogue registers per pixel and a SRAM memory for example).
The device shown in
The signals obtained at the output of the movement detection devices may be used to display an image on which the background captured forms a black background onto which the moving elements detected are shown in white.
| Number | Date | Country | Kind |
|---|---|---|---|
| 07 56516 | Jul 2007 | FR | national |
