The present invention relates to an image change detection device and an image change detection method for determining whether or not an image has changed.
Non-Patent Literature 1 below discloses a singular spectrum transformation. The singular spectrum transformation is one of methods for determining whether or not an image in a change detection period has changed from an image in a period for comparison. In the singular spectrum transformation, left singular vectors of an image in a period for comparison among images generated on the basis of observation radio waves of a radar are calculated, and a subspace (hereinafter, referred to as a “first subspace”) whose bases are the left singular vectors is generated. In addition, in the singular spectrum transformation, left singular vectors of an image in a change detection period among images generated from the observation radio waves of a radar are calculated, and a subspace (hereinafter, referred to as a “second subspace”) whose bases are the left singular vectors is generated. In the singular spectrum transformation, the degree of change between the first subspace and the second subspace is calculated, and whether or not the image in the change detection period has changed from the image in the period for comparison is determined on the basis of the degree of change.
Non-Patent Literature 1: T. Ide and K. Inoue, “Knowledge discovery from heterogeneous dynamic systems using change-point correlations”, In Proceedings of 2005 SIAM International Conference on Data Mining (SDM 05), pages 571-575, 2005.
In a case of determining whether or not an image in each of a plurality of change detection periods has changed from an image in each of a plurality of periods for comparison, in the singular spectrum transformation disclosed in Non-Patent Literature 1, it is required to calculate left singular vectors of the images in the periods for comparison and generate first subspaces whose bases are the respective left singular vectors. The left singular vectors are obtained by performing singular value decomposition on an image. Since singular value decomposition generally has a high calculation cost, there is a problem that a change in an image cannot be detected in real time due to the calculation of the left singular vectors being a bottleneck.
The present invention has been devised to solve the above problem, and an object of the present invention is to obtain an image change detection device and an image change detection method with which, when an image in a certain change detection period has not changed from an image in a certain period for comparison, it is possible to determine whether or not an image in a change detection period next to the certain change detection period has changed from an image in a period for comparison next to the certain period for comparison without performing singular value decomposition on the image in the period for comparison next to the certain period for comparison.
An image change detection device according to the present invention includes processing circuitry to calculate, by a degree of change calculator implemented by the processing circuitry, a degree of change between a first subspace and a second subspace, when receiving the first subspace and the second subspace, the first subspace being a space whose bases are left singular vectors of an image in a certain period for comparison from among images generated on a basis of an observation radio wave of a radar, the second subspace being a space whose bases are left singular vectors of an image in a certain change detection period that is a period different from the certain period for comparison from among the images generated on a basis of the observation radio wave of the radar, to determine, by a determinator implemented by the processor, whether or not the image in the certain change detection period has changed from the image in the certain period for comparison on a basis of the degree of change calculated by the degree-of-change calculator, and to update, by a subspace updater implemented by the processing circuitry, the first subspace by rotating the first subspace so that the first subspace approaches the second subspace when the determinator determines that there has been no change. When the determinator determines that there has been no change, the degree-of-change calculator calculates another degree of change between the first subspace which has been updated by the subspace updater and the second subspace whose bases are left singular vectors of an image in another change detection period next to the certain change detection period, and the determinator determines that the image in the certain change detection period has changed from the image in the certain period for comparison when the degree of change calculated by the degree-of-change calculator is equal to or greater than a threshold, and determines that the image in the certain change detection period has not changed from the image in the certain period for comparison when the degree of change is less than the threshold.
According to the present invention, the image change detection device includes a subspace updating unit to update the first subspace by rotating the first subspace so that the first subspace approaches the second subspace when the determination unit determines that there has been no change. When the determination unit determines that there has been no change, the degree-of-change calculating unit calculates a degree of change between the first subspace which has been updated by the subspace updating unit and the second subspace whose bases are left singular vectors of an image in another change detection period next to the certain change detection period. Therefore, the image change detection device according to the present invention can determine, when an image in a certain change detection period has not changed from an image in a certain period for comparison, whether or not an image in a change detection period next to the certain change detection period has changed from an image in a period for comparison next to the certain period for comparison without performing singular value decomposition on the image in the period for comparison next to the certain period for comparison.
In order to describe the present invention in more detail, some modes for carrying out the present invention will be described with reference to the accompanying drawings.
In
The image generation unit 1 outputs, for example, a radio wave image in a period for comparison T1 among the generated radio wave images to a matrix-for-comparison generating unit 12 of a subspace generation unit 11 described later. Hereinafter, the radio wave image output from the image generation unit 1 to the matrix-for-comparison generating unit 12 is referred to as an image for comparison 2.
The image generation unit 1 outputs, for example, the radio wave image in each of the change detection periods T2, T4, . . . among the generated radio wave images to a later-described detection target matrix generating unit 15 of the subspace generation unit 11. Hereinafter, the radio wave image output from the image generation unit 1 to the detection target matrix generating unit 15 is referred to as a detection target image 3.
When the determination result output from a determination unit 19 described later indicates that the detection target image 3 in the change detection period T2 has changed from the image for comparison 2 in the period for comparison T1, the image generation unit 1 outputs the image for comparison 2 in a period for comparison T3 next to the period for comparison T1 to the matrix-for-comparison generating unit 12, and outputs the detection target image 3 in the change detection period T4 next to the change detection period T2 to the detection target matrix generating unit 15.
When the determination result output from the determination unit 19 indicates that the detection target image 3 in the change detection period T2 has not changed from the image for comparison 2 in the period for comparison T1, the image generation unit 1 outputs the detection target image 3 in the change detection period T4 to the detection target matrix generating unit 15 without outputting the image for comparison 2 in the period for comparison T3 to the matrix-for-comparison generating unit 12.
In the image change detection device 4 illustrated in
The image change detection device 4 includes the subspace generation unit 11, a degree-of-change calculating unit 18, the determination unit 19, and a subspace updating unit 20.
The image change detection device 4 compares the image for comparison 2 generated by the image generation unit 1 with the detection target image 3 generated by the image generation unit 1, and determines whether or not the detection target image 3 has changed from the image for comparison 2.
In addition, the image change detection device 4 compares the image for comparison 2 after updating by the subspace updating unit 20 with the detection target image 3 generated by the image generation unit 1, and determines whether or not the detection target image 3 has changed from the image for comparison 2 after the updating.
The subspace generation unit 11 includes the matrix-for-comparison generating unit 12, a first vector calculating unit 13, a first space generation processing unit 14, the detection target matrix generating unit 15, a second vector calculating unit 16, and a second space generation processing unit 17.
When receiving the image for comparison 2 in the period for comparison T1 from the image generation unit 1, the subspace generation unit 11 calculates left singular vectors LSV1 of the image for comparison 2 and generates a first subspace span(U1) whose bases are the left singular vectors LSV1.
The subspace generation unit 11 outputs the first subspace span(U1) to each of the degree-of-change calculating unit 18 and the subspace updating unit 20.
Further, when receiving the detection target images 3 in the change detection periods T2, T4, . . . from the image generation unit 1, the subspace generation unit 11 calculates left singular vectors LSV2, LSV4, . . . of the detection target images 3, and generates second subspaces span(U2), span(U4), . . . whose bases are the left singular vectors LSV2, LSV4, . . . , respectively.
The subspace generation unit 11 outputs the generated second subspaces span(U2), span(U4) . . . to each of the degree-of-change calculating unit 18 and the subspace updating unit 20.
When the determination unit 19 determines that there has been any change, the subspace generation unit 11 acquires the image for comparison 2 in the period for comparison T3 next to the period for comparison T1 from the image generation unit 1.
The subspace generation unit 11 calculates left singular vectors LSV3 of the image for comparison 2 in the period for comparison T3, generates a first subspace span(U3) whose bases are the left singular vectors LSV3, and outputs the first subspace span(U3) to the degree-of-change calculating unit 18.
When the determination unit 19 determines that there has been no change, the subspace generation unit 11 does not acquire the image for comparison 2 in the period for comparison T3 from the image generation unit 1. Therefore, the subspace generation unit 11 does not generate the first subspace span(U3) whose bases are the left singular vectors LSV3.
The periods for comparison T1 and T3 and the change detection periods T2, T4, . . . are different from each other. Among the periods for comparison T1 and T3 and the change detection periods T2 and T4, the period for comparison T1 is the most earliest period, the change detection period T2 is the second earliest period, the period for comparison T3 is the third earliest period, and the change detection period T4 is the latest period, for example.
The matrix-for-comparison generating unit 12 is implemented by, for example, a matrix-for-comparison generating circuit 31 illustrated in
The matrix-for-comparison generating unit 12 generates a matrix for comparison U1 indicating the image for comparison 2 in the period for comparison T1 output from the image generation unit 1, and outputs the matrix for comparison U1 to the first vector calculating unit 13.
When receiving the image for comparison 2 in the period for comparison T3 from the image generation unit 1, the matrix-for-comparison generating unit 12 generates a matrix for comparison U3 indicating the image for comparison 2 in the period for comparison T3, and outputs the matrix for comparison U3 to the first vector calculating unit 13.
The first vector calculating unit 13 is implemented by, for example, a first vector calculating circuit 32 shown in
The first vector calculating unit 13 calculates the left singular vectors LSV1 of the image for comparison 2 by performing singular value decomposition on the matrix for comparison U1 output from the matrix-for-comparison generating unit 12.
The first vector calculating unit 13 outputs the left singular vectors LSV1 to the first space generation processing unit 14.
When receiving the matrix for comparison U3 from the matrix-for-comparison generating unit 12, the first vector calculating unit 13 calculates the left singular vectors LSV3 of the image for comparison 2 by performing singular value decomposition on the matrix for comparison U3.
The first vector calculating unit 13 outputs the left singular vectors LSV3 to the first space generation processing unit 14.
The first space generation processing unit 14 is implemented by, for example, a first space generation processing circuit 33 shown in
When receiving, for example, the left singular vectors LSV1 from the first vector calculating unit 13, the first space generation processing unit 14 generates the first subspace span(U1) whose bases are the left singular vectors LSV1.
The first space generation processing unit 14 outputs the first subspace span(U1) to each of the degree-of-change calculating unit 18 and the subspace updating unit 20.
When receiving, for example, the left singular vectors LSV3 from the first vector calculating unit 13, the first space generation processing unit 14 generates the first subspace span(U3) whose bases are the left singular vectors LSV3.
The first space generation processing unit 14 outputs the first subspace span(U3) to each of the degree-of-change calculating unit 18 and the subspace updating unit 20.
The detection target matrix generating unit 15 is implemented by, for example, a detection target matrix generating circuit 34 illustrated in
The detection target matrix generating unit 15 acquires the detection target images 3 in the change detection periods T2, T4, . . . from the image generation unit 1.
The detection target matrix generating unit 15 generates matrices for detection U2, U4, . . . indicating the respective detection target images 3, and outputs the matrices for detection U2, U4, . . . to the second vector calculating unit 16.
The second vector calculating unit 16 is implemented by, for example, a second vector calculating circuit 35 shown in
The second vector calculating unit 16 calculates the left singular vectors LSV2, LSV4, . . . of the detection target images 3 by performing singular value decomposition on the matrices for detection U2, U4, . . . output from the detection target matrix generating unit 15.
The second vector calculating unit 16 outputs the left singular vectors LSV2, LSV4, . . . to the second space generation processing unit 17.
The second space generation processing unit 17 is implemented by, for example, a second space generation processing circuit 36 shown in
The second space generation processing unit 17 generates second subspaces span(U2), span(U4), . . . whose bases are the left singular vectors LSV2, LSV4, . . . , respectively, which are output from the second vector calculating unit 16.
The second space generation processing unit 17 outputs the second subspaces span(U2), span(U4), . . . to each of the degree-of-change calculating unit 18 and the subspace updating unit 20.
The degree-of-change calculating unit 18 is implemented by, for example, a degree-of-change calculating circuit 37 illustrated in
The degree-of-change calculating unit 18 acquires the first subspace span(U1) generated by the first space generation processing unit 14 of the subspace generation unit 11 and the second subspace span(U2) generated by the second space generation processing unit 17 of the subspace generation unit 11.
The degree-of-change calculating unit 18 calculates a degree of change c(t2) between the first subspace span(U1) and the second subspace span(U2).
The time t2 is included in the change detection period T2. The time t2 may be a start time of the change detection period T2 or an end time of the change detection period T2. The time t2 may be a time between the start time and the end time of the change detection period T2.
The degree-of-change calculating unit 18 outputs the degree of change c(t2) to the determination unit 19.
When the determination unit 19 determines that the detection target image 3 in the change detection period T2 has not changed from the image for comparison 2 in the period for comparison T1, the degree-of-change calculating unit 18 acquires a first subspace span(U3′) which is the first subspace span(U1) updated by the subspace updating unit 20 and a second subspace span(U4) generated by the second space generation processing unit 17.
The degree-of-change calculating unit 18 calculates a degree of change c(t4) between the first subspace span(U3′) and the second subspace span(U4).
Assuming that the first subspace span(U3) with respect to the period for comparison T3 is generated by the first space generation processing unit 14, the first subspace span(U3′) is similar to the first subspace span(U3).
The second subspace span(U4) generated by the second space generation processing unit 17 is a second subspace whose bases are the left singular vectors LSV4 of the detection target image 3 in the change detection period T4.
When the determination unit 19 determines that the detection target image 3 in the change detection period T2 has changed from the image for comparison 2 in the period for comparison T1, the degree-of-change calculating unit 18 acquires the first subspace span(U3) generated by the first space generation processing unit 14 and the second subspace span(U4) generated by the second space generation processing unit 17.
The degree-of-change calculating unit 18 calculates a degree of change c(t4) between the first subspace span(U3) and the second subspace span(U4).
The first subspace span(U3) generated by the subspace generation unit 11 is a subspace whose bases are the left singular vectors LSV3 of the image for comparison 2 in the period for comparison T3.
The determination unit 19 is implemented by, for example, a determination circuit 38 illustrated in
The determination unit 19 determines whether or not the detection target image 3 in the change detection period has changed from the image for comparison 2 in the period for comparison on the basis of the degree of change calculated by the degree-of-change calculating unit 18.
That is, the determination unit 19 determines that the detection target image 3 in the change detection period T2 has changed from the image for comparison 2 in the period for comparison T1 when the degree of change c(t2) calculated by the degree-of-change calculating unit 18 is equal to or greater than a threshold θ, and determines that the detection target image 3 in the change detection period T2 has not changed from the image for comparison 2 in the period for comparison T1 when the degree of change c(t2) is less than the threshold θ.
In addition, the determination unit 19 determines that the detection target image 3 in the change detection period T4 has changed from the image for comparison 2 in the period for comparison T3 when the degree of change c(t4) calculated by the degree-of-change calculating unit 18 is equal to or greater than the threshold θ, and determines that the detection target image 3 in the change detection period T4 has not changed from the image for comparison 2 in the period for comparison T3 when the degree of change c(t4) is less than the threshold θ.
As the threshold θ, a value of 0.5 or a value of 0.6 is used, for example. The threshold θ may be stored in an internal memory of the determination unit 19 or may be given from the outside of the image change detection device 4.
When determining that the detection target image 3 in the change detection period T2 has changed from the image for comparison 2 in the period for comparison T1, the determination unit 19 outputs information indicating that there is a change point in the radio wave image during the change detection period T2 to the outside. In addition, when determining that the detection target image 3 in the change detection period T4 has changed from the image for comparison 2 in the period for comparison T3, the determination unit 19 outputs information indicating that there is a change point in the radio wave image during the change detection period T4 to the outside.
The determination unit 19 outputs a determination result indicating whether or not the detection target image 3 has changed from the image for comparison 2 to each of the degree-of-change calculating unit 18 and the image generation unit 1.
In addition, when determining that the detection target image 3 in the change detection period T2 has not changed from the image for comparison 2 in the period for comparison T1, the determination unit 19 outputs the determination result indicating that the detection target image 3 in the change detection period T2 has not changed from the image for comparison 2 in the period for comparison T1 to the subspace updating unit 20.
In the image change detection device 4 illustrated in
The subspace updating unit 20 is implemented by, for example, a subspace updating circuit 39 illustrated in
When the determination unit 19 determines that the detection target image 3 in the change detection period T2 has not changed from the image for comparison 2 in the period for comparison T1, the subspace updating unit 20 updates the first subspace span(U1) by rotating the first subspace span(U1) so that the first subspace span(U1) approaches the second subspace span(U2).
The subspace updating unit 20 outputs the updated first subspace span(U1) to the degree-of-change calculating unit 18 as the first subspace span(U3′).
When the determination unit 19 determines that the detection target image 3 in the change detection period T2 has changed from the image for comparison 2 in the period for comparison T1, the subspace updating unit 20 does not update the first subspace span(U1).
In
Here, each of the matrix-for-comparison generating circuit 31, the first vector calculating circuit 32, the first space generation processing circuit 33, the detection target matrix generating circuit 34, the second vector calculating circuit 35, the second space generation processing circuit 36, the degree-of-change calculating circuit 37, the determination circuit 38, and the subspace updating circuit 39 is, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination of some of these circuits.
The components of the image change detection device 4 are not limited to be implemented by dedicated hardware, and the image change detection device 4 may be implemented by software, firmware, or a combination of software and firmware.
Software or firmware is stored in a memory of a computer as a program. The computer means hardware that executes the program, and may be, for example, a central processing unit (CPU), central processor, processing unit, computing unit, microprocessor, microcomputer, processor, or digital signal processor (DSP).
In a case where the image change detection device 4 is implemented by software, firmware, or the like, a program for causing the computer to execute the processing procedures performed by the matrix-for-comparison generating unit 12, the first vector calculating unit 13, the first space generation processing unit 14, the detection target matrix generating unit 15, the second vector calculating unit 16, the second space generation processing unit 17, the degree-of-change calculating unit 18, the determination unit 19, and the subspace updating unit 20 is stored in a memory 41. Then, a processor 42 of the computer executes the program stored in the memory 41.
Further,
Next, the operation of the image change detection device 4 illustrated in
The radio wave image illustrated in
In
For example, a red pixel extending in the horizontal direction near the center of the vertical axis is a pixel having a larger amplitude, and a pixel closer to blue is a pixel having a smaller amplitude.
Among the plurality of pixels constituting the radio wave image illustrated in
In the radio wave image illustrated in
The image generation unit 1 generates a radio wave image as illustrated in FIG. 5 on the basis of the observation radio wave of the radar.
The image generation unit 1 outputs a radio wave image in the period for comparison T1 among the generated radio wave images to the matrix-for-comparison generating unit 12 of the subspace generation unit 11 as the image for comparison 2 (see
The image generation unit 1 outputs a radio wave image in the change detection period T2 among the generated radio wave images to the detection target matrix generating unit 15 of the subspace generation unit 11 as the detection target image 3 (see
In
The entire space RD including the first subspace span(U1) and the second subspace span(U2) is expressed by a hemisphere.
When receiving the image for comparison 2 in the period for comparison T1 from the image generation unit 1, the matrix-for-comparison generating unit 12 generates a matrix for comparison U indicating the image for comparison 2 in the period for comparison T1 (step ST1 in
When the image for comparison 2 is, for example, a two-dimensional image having (x1×y1) pixel values, the matrix-for-comparison generating unit 12 generates a matrix having x1 elements in the row direction and y1 elements in the column direction as the matrix for comparison U1. Each of x1 and y 1 is an integer equal to or greater than 1.
The matrix-for-comparison generating unit 12 outputs the matrix for comparison U1 to the first vector calculating unit 13.
When receiving the matrix for comparison U1 from the matrix-for-comparison generating unit 12, the first vector calculating unit 13 calculates the left singular vectors LSV1 of the image for comparison 2 by performing singular value decomposition on the matrix for comparison U1 (step ST2 in
The processing for calculating the left singular vectors LSV1 by performing the singular value decomposition on the matrix for comparison U1 is known, and thus the detailed description thereof will be omitted.
The maximum number of left singular vectors LSV1 calculated by the first vector calculating unit 13 is determined by the rank of the matrix for comparison U1. The first vector calculating unit 13 may calculate any number of left singular vectors LSV1. Any number may be stored in the internal memory of the first vector calculating unit 13, or may be given from the outside.
Alternatively, the first vector calculating unit 13 may select the left singular vectors LSV1 in which the cumulative contribution ratio of the singular value is larger than a certain threshold from among the plurality of left singular vectors LSV1 that can be calculated.
The first vector calculating unit 13 outputs the calculated left singular vectors LSV1 to the first space generation processing unit 14.
When receiving the left singular vectors LSV1 from the first vector calculating unit 13, the first space generation processing unit 14 generates the first subspace span(U1) whose bases are the left singular vectors LSV1 (step ST3 in
The first subspace span(U1) is a subspace formed by a set of the top n left singular vectors LSV1 having a large singular value with respect to the matrix for comparison U1 among the plurality of left singular vectors LSV1 in the image for comparison 2. The number n is an integer equal to or greater than 2. Note that, since the plurality of left singular vectors LSV1 in the image for comparison 2 is orthogonal to each other and have a norm equal to 1, the basis of the first subspace span(U1) is an orthonormal basis.
The first space generation processing unit 14 outputs the first subspace span(U1) to each of the degree-of-change calculating unit 18 and the subspace updating unit 20.
When receiving the detection target image 3 in the change detection period T2 from the image generation unit 1, the detection target matrix generating unit 15 generates a matrix for detection U2 indicating the detection target image 3 (step ST4 in
When the detection target image 3 is, for example, a two-dimensional image having (x2×y2) pixel values, the detection target matrix generating unit 15 generates a matrix having x2 elements in the row direction and y2 elements in the column direction as the matrix for detection U2. Each of x2 and y2 is an integer equal to or greater than 1.
In the image change detection device 4 illustrated in
The detection target matrix generating unit 15 outputs the matrix for detection U2 to the second vector calculating unit 16.
When receiving the matrix for detection U2 from the detection target matrix generating unit 15, the second vector calculating unit 16 calculates the left singular vectors LSV2 of the detection target image 3 by performing singular value decomposition on the matrix for detection U2 (step ST5 in
The processing for calculating the left singular vectors LSV2 by performing the singular value decomposition on the matrix for detection U2 is known, and thus the detailed description thereof will be omitted.
The maximum number of left singular vectors LSV2 calculated by the second vector calculating unit 16 is determined by the rank of the matrix for detection U2. The second vector calculating unit 16 may calculate any number of left singular vectors LSV2. Any number may be stored in the internal memory of the second vector calculating unit 16, or may be given from the outside.
Alternatively, the second vector calculating unit 16 may select the left singular vectors LSV2 in which the cumulative contribution ratio of the singular value is larger than a certain threshold from among the plurality of left singular vectors LSV2 that can be calculated.
The second vector calculating unit 16 outputs the calculated left singular vectors LSV2 to the second space generation processing unit 17.
When receiving the left singular vectors LSV2 from the second vector calculating unit 16, the second space generation processing unit 17 generates the second subspace span(U2) whose bases are the left singular vectors LSV2 (step ST6 in
The second subspace span(U2) is a subspace formed by a set of the top m left singular vectors LSV2 having a large singular value with respect to the matrix for detection U2 among the plurality of left singular vectors LSV2 in the detection target image 3, m is an integer equal to or greater than 2. Note that, since the plurality of left singular vectors LSV2 in the detection target image 3 is orthogonal to each other and have a norm equal to 1, the basis of the second subspace span(U2) is an orthonormal basis.
The second space generation processing unit 17 outputs the second subspace span(U2) to each of the degree-of-change calculating unit 18 and the subspace updating unit 20.
The degree-of-change calculating unit 18 acquires the first subspace span(U1) output from the first space generation processing unit 14 and the second subspace span(U2) output from the second space generation processing unit 17.
The degree-of-change calculating unit 18 calculates a degree of change c(t2) between the first subspace span(U1) and the second subspace span(U2) (step ST7 in
The degree of change c(t2) corresponds to the distance (see
c(t2)=1−∥U1TU2∥F (1)
In Equation (1), ∥·∥
The degree-of-change calculating unit 18 outputs the degree of change c(t2) to the determination unit 19.
The determination unit 19 compares the degree of change c(t2) calculated by the degree-of-change calculating unit 18 with the threshold θ.
If the degree of change c(t2) is equal to or greater than the threshold θ (step ST8 in
When determining that there has been any change, the determination unit 19 outputs information indicating that there is a change point in the radio wave image during the change detection period T2 to the outside.
If the degree of change c(t2) is less than the threshold θ (step ST8 in
The determination unit 19 outputs the determination result indicating whether or not the detection target image 3 in the change detection period T2 has changed from the image for comparison 2 in the period for comparison T1 to each of the degree-of-change calculating unit 18 and the image generation unit 1.
When determining that there has been no change, the determination unit 19 outputs the determination result indicating that the detection target image 3 in the change detection period T2 has not changed from the image for comparison 2 in the period for comparison T1 to the subspace updating unit 20.
When the determination result output from the determination unit 19 indicates that the detection target image 3 in the change detection period T2 has changed from the image for comparison 2 in the period for comparison T1, the image generation unit 1 outputs the image for comparison 2 in the period for comparison T3 to the matrix-for-comparison generating unit 12 and outputs the detection target image 3 in the change detection period T4 to the detection target matrix generating unit 15.
When the determination result output from the determination unit 19 indicates that the detection target image 3 in the change detection period T2 has not changed from the image for comparison 2 in the period for comparison T1, the image generation unit 1 outputs the detection target image 3 in the change detection period T4 to the detection target matrix generating unit 15 without outputting the image for comparison 2 in the period for comparison T3 to the matrix-for-comparison generating unit 12.
Here, for convenience of description, it is assumed that the determination result output from the determination unit 19 indicates that the detection target image 3 in the change detection period T2 has not changed from the image for comparison 2 in the period for comparison T1. Therefore, the image generation unit 1 outputs the detection target image 3 in the change detection period T4 to the detection target matrix generating unit 15 without outputting the image for comparison 2 in the period for comparison T3 to the matrix-for-comparison generating unit 12.
When receiving, from the determination unit 19, the determination result indicating that the detection target image 3 in the change detection period T2 has not changed from the image for comparison 2 in the period for comparison T1, the subspace updating unit 20 updates the first subspace span(U1) (step ST11 in
That is, the subspace updating unit 20 acquires the first subspace span(U1) output from the first subspace generation processing unit 14 and the second subspace span(U2) output from the second subspace generation processing unit 17.
The subspace updating unit 20 updates the first subspace span(U1) by rotating the first subspace span(U1) so that the first subspace span(U1) approaches the second subspace span(U2).
In a case where the determination unit 19 has already determined twice or more, the subspace updating unit 20 may have received the previous determination result before receiving the current determination result from the determination unit 19. In a case where the subspace updating unit 20 already updates the first subspace span(U) because the previous determination result output from the determination unit 19 indicates that there has been no change, the subspace updating unit 20 further updates the updated first subspace span(U1) instead of the first subspace span(U1) output from the first subspace generation processing unit 14.
Equation (2) below is for updating the matrix for comparison U1 related to the first subspace span(U1). Equation (2) indicates the relationship between the matrix for comparison U1 related to the first subspace span(U1), the matrix for detection U2 related to the second subspace span(U2), and the matrix for comparison U3′ related to the first subspace span(U3′) which is the updated first subspace span(U1).
U3′=orth((1+μU2U2T)U1) (2)
In Equation (2), μ represents a rotation coefficient, and I represents an identity matrix.
The function orth( ) is an orthonormalization function. As the orthonormalization, Gram-Schmidt orthonormalization can be used, for example.
The subspace updating unit 20 outputs the updated first subspace span(U1) to the degree-of-change calculating unit 18 as the first subspace span(U3′).
In
By rotating the first subspace span(U1) so that the first subspace span(U1) approaches the second subspace span(U2), the information of the second subspace span(U2) is reflected in the first subspace span(U3′). Therefore, the first subspace span(U3′) is a subspace similar to the first subspace span(U3) related to the period for comparison T3. The first subspace span(U3) in the period for comparison T3 is a subspace assumed to be generated from the image for comparison 2 in the period for comparison T3 by the subspace generation unit 11.
When receiving the detection target image 3 in the change detection period T4 from the image generation unit 1, the detection target matrix generating unit 15 generates a matrix for detection U4 indicating the detection target image 3, and outputs the matrix for detection U4 to the second vector calculating unit 16 (step ST4 in
When receiving the matrix for detection U4 from the detection target matrix generating unit 15, the second vector calculating unit 16 calculates the left singular vectors LSV4 of the detection target image 3 by performing singular value decomposition on the matrix for detection U4 (step ST5 in
The second vector calculating unit 16 outputs the calculated left singular vectors LSV4 to the second space generation processing unit 17.
When receiving the left singular vectors LSV4 from the second vector calculating unit 16, the second space generation processing unit 17 generates the second subspace span(U4) whose bases are the left singular vectors LSV4 (step ST6 in
The second subspace span(U4) is a subspace formed by a set of the top m left singular vectors LSV4 having a large singular value with respect to the matrix for detection U4 among the plurality of left singular vectors LSV4 in the detection target image 3.
The second space generation processing unit 17 outputs the second subspace span(U4) to each of the degree-of-change calculating unit 18 and the subspace updating unit 20.
That is, the degree-of-change calculating unit 18 acquires the first subspace span(U3′) which is the updated first subspace span(U1) output from the subspace updating unit 20 and the second subspace span(U4) output from the second space generation processing unit 17.
The degree-of-change calculating unit 18 calculates a degree of change c(t4) between the first subspace span(U3′) and the second subspace span(U4) (step ST7 in
The degree of change c(t4) corresponds to the distance between the first subspace span(U3′) and the second subspace span(U4). The degree of change c(t4) is expressed by Equation (3) below.
c(t4)=1−∥U3′TU4∥F (3)
The degree-of-change calculating unit 18 outputs the degree of change c(t4) to the determination unit 19.
The determination unit 19 compares the degree of change c(t4) calculated by the degree-of-change calculating unit 18 with the threshold θ.
If the degree of change c(t4) is equal to or greater than the threshold θ (step ST8 in
When determining that there has been any change, the determination unit 19 outputs information indicating that there is a change point in the radio wave image during the change detection period T4 to the outside.
If the degree of change c(t4) is less than the threshold θ (step ST8 in
The determination unit 19 outputs the determination result indicating whether or not the detection target image 3 in the change detection period T4 has changed from the image for comparison 2 in the period for comparison T3 to each of the degree-of-change calculating unit 18 and the image generation unit 1.
When determining that there has been no change, the determination unit 19 outputs the determination result indicating that the detection target image 3 in the change detection period T4 has not changed from the image for comparison 2 in the period for comparison T3 to the subspace updating unit 20.
When the determination result output from the determination unit 19 indicates that the detection target image 3 in the change detection period T4 has changed from the image for comparison 2 in the period for comparison T3, the image generation unit 1 outputs the image for comparison 2 in the period for comparison T5 to the matrix-for-comparison generating unit 12 and outputs the detection target image 3 in the change detection period T6 to the detection target matrix generating unit 15.
When the determination result output from the determination unit 19 indicates that the detection target image 3 in the change detection period T4 has not changed from the image for comparison 2 in the period for comparison T3, the image generation unit 1 outputs the detection target image 3 in the change detection period T6 to the detection target matrix generating unit 15 without outputting the image for comparison 2 in the period for comparison T5 to the matrix-for-comparison generating unit 12.
Here, for convenience of description, it is assumed that the determination result output from the determination unit 19 indicates that the detection target image 3 in the change detection period T4 has changed from the image for comparison 2 in the period for comparison T3. Therefore, the image generation unit 1 outputs the image for comparison 2 in the period for comparison T5 to the matrix-for-comparison generating unit 12 and outputs the detection target image 3 in the change detection period T6 to the detection target matrix generating unit 15.
When receiving the image for comparison 2 in the period for comparison T5 from the image generation unit 1, the matrix-for-comparison generating unit 12 generates a matrix for comparison U5 indicating the image for comparison 2 in the period for comparison T5, and outputs the matrix for comparison U5 to the first vector calculating unit 13 (step ST1 in
When receiving the matrix for comparison U5 from the matrix-for-comparison generating unit 12, the first vector calculating unit 13 calculates the left singular vectors LSV5 of the image for comparison 2 by performing singular value decomposition on the matrix for comparison U5 (step ST2 in
The first vector calculating unit 13 outputs the calculated left singular vectors LSV5 to the first space generation processing unit 14.
When receiving the left singular vectors LSV5 from the first vector calculating unit 13, the first space generation processing unit 14 generates the first subspace span(U5) whose bases are the left singular vectors LSV5 (step ST3 in
The first subspace span(U5) is a subspace formed by a set of the top n left singular vectors LSV1 having a large singular value with respect to the matrix for comparison U5 among the plurality of left singular vectors LSV5 in the image for comparison 2.
The first space generation processing unit 14 outputs the first subspace span(U5) to each of the degree-of-change calculating unit 18 and the subspace updating unit 20.
When receiving the detection target image 3 in the change detection period T6 from the image generation unit 1, the detection target matrix generating unit 15 outputs a matrix for detection U6 indicating the detection target image 3 to the second vector calculating unit 16 (step ST4 in
When receiving the matrix for detection U6 from the detection target matrix generating unit 15, the second vector calculating unit 16 calculates the left singular vectors LSV6 of the detection target image 3 by performing singular value decomposition on the matrix for detection U6 (step ST5 in
The second vector calculating unit 16 outputs the calculated left singular vectors LSV6 to the second space generation processing unit 17.
When receiving the left singular vectors LSV6 from the second vector calculating unit 16, the second space generation processing unit 17 generates the second subspace span(U6) whose bases are the left singular vectors LSV6 (step ST6 in
The second subspace span(U6) is a subspace formed by a set of the top m left singular vectors LSV6 having a large singular value with respect to the matrix for detection U6 among the plurality of left singular vectors LSV6 in the detection target image 3.
The second space generation processing unit 17 outputs the second subspace span(U6) to each of the degree-of-change calculating unit 18 and the subspace updating unit 20.
The degree-of-change calculating unit 18 acquires the first subspace span(U5) output from the first space generation processing unit 14 and the second subspace span(U6) output from the second space generation processing unit 17.
The degree-of-change calculating unit 18 calculates a degree of change c(t6) between the first subspace span(U5) and the second subspace span(U6) (step ST7 in
The degree of change c(t6) corresponds to the distance between the first subspace span(U5) and the second subspace span(U6). The degree of change c(t6) is expressed by Equation (4) below.
c(t6)=1−∥U5TU6∥F (4)
The degree-of-change calculating unit 18 outputs the degree of change c(t6) to the determination unit 19.
The determination unit 19 compares the degree of change c(t6) calculated by the degree-of-change calculating unit 18 with the threshold θ.
If the degree of change c(t6) is equal to or greater than the threshold θ (step ST8 in
When determining that there has been any change, the determination unit 19 outputs information indicating that there is a change point in the radio wave image during the change detection period T6 to the outside.
If the degree of change c(t6) is less than the threshold θ (step ST8 in
The determination unit 19 outputs the determination result indicating whether or not the detection target image 3 in the change detection period T6 has changed from the image for comparison 2 in the period for comparison T5 to each of the degree-of-change calculating unit 18 and the image generation unit 1.
When determining that there has been no change, the determination unit 19 outputs the determination result indicating that the detection target image 3 in the change detection period T6 has not changed from the image for comparison 2 in the period for comparison T5 to the subspace updating unit 20.
The image change detection device 4 illustrated in
In the first embodiment described above, the image change detection device 4 includes the subspace updating unit 20 that updates the first subspace by rotating the first subspace so that the first subspace approaches the second subspace when the determination unit 19 determines that there has been no change, and the degree-of-change calculating unit 18 calculates the degree of change between the first subspace updated by the subspace updating unit 20 and the second subspace whose bases are the left singular vectors of the image in the change detection period next to the change detection period when the determination unit 19 determines that there has been no change. Therefore, the image change detection device 4 can determine, when an image in a certain change detection period has not changed from an image in a period for comparison, whether or not an image in a change detection period next to the certain change detection period has changed from an image in a period for comparison next to the period for comparison without performing singular value decomposition on the image in the period for comparison next to the period for comparison.
A second embodiment will describe an image change detection device 4 in which a subspace updating unit 21 updates the first subspace by rotating the first subspace more largely as the degree of change calculated by the degree-of-change calculating unit 18 is larger.
In
The subspace updating unit 21 is implemented by, for example, a subspace updating circuit 40 illustrated in
When the determination unit 19 determines that the detection target image 3 in the change detection period T2 has not changed from the image for comparison 2 in the period for comparison T1, the subspace updating unit 21 updates the first subspace span(U1) by rotating the first subspace span(U1) so that the first subspace span(U1) approaches the second subspace span(U2), as in the subspace updating unit 20 illustrated in
Note that, unlike the subspace updating unit 20 illustrated in
The subspace updating unit 21 outputs the updated first subspace span(U1) to the degree-of-change calculating unit 18 as a first subspace span(U3′).
In
The subspace updating circuit 40 is, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination of some of these circuits.
The components of the image change detection device 4 are not limited to be implemented by dedicated hardware, and the image change detection device 4 may be implemented by software, firmware, or a combination of software and firmware.
In a case where the image change detection device 4 is implemented by software, firmware, or the like, a program for causing the computer to execute the processing procedures performed by the matrix-for-comparison generating unit 12, the first vector calculating unit 13, the first space generation processing unit 14, the detection target matrix generating unit 15, the second vector calculating unit 16, the second space generation processing unit 17, the degree-of-change calculating unit 18, the determination unit 19, and the subspace updating unit 21 is stored in the memory 41 illustrated in
Further,
Next, the operation of the image change detection device 4 illustrated in
Since the configuration other than the subspace updating unit 21 is similar to that of the image change detection device 4 illustrated in
As in the first embodiment, the degree-of-change calculating unit 18 calculates a degree of change c(t2) between the first subspace span(U1) and the second subspace span(U2).
The degree-of-change calculating unit 18 outputs the degree of change c(t2) to each of the subspace updating unit 21 and the determination unit 19.
When receiving, from the determination unit 19, the determination result indicating that the detection target image 3 in the change detection period T2 has not changed from the image for comparison 2 in the period for comparison T1, for example, the subspace updating unit 21 updates the first subspace span(U1) as in the subspace updating unit 20 in
However, unlike the subspace updating unit 20 illustrated in
Therefore, unlike the subspace updating unit 20 illustrated in
The maximum rotation of the first subspace span(U1) indicates a rotation by which the first subspace span(U1) reaches a position overlapping the second subspace span(U2). The subspace updating unit 21 rotates the first subspace span(U1) so that the first subspace span(U3′) that is the updated first subspace span(U1) is located between the first subspace span(U1) and the second subspace span(U2).
The subspace updating unit 21 calculates the rotation coefficient μ as represented by Equation (5) below.
The rotation coefficient μ indicated in Equation (5) is calculated by the ratio between a length L1 of the vector projection of the first subspace span(U1) with respect to the second subspace span(U2) and a length L2 of the vector projection of the first subspace span(U1) with respect to an orthogonal complement span(U) as shown in
L1=∥TU1∥F2 (6)
L2=∥(1−T)U1∥F2 (7)
In the second embodiment described above, the image change detection device 4 illustrated in
It is to be noted that two or more of the above embodiments can be freely combined, or any component in the embodiments can be modified or omitted, within the scope of the present invention.
The present invention is suitable for an image change detection device and an image change detection method for determining whether or not an image has changed.
1: image generation unit, 2: image for comparison, 3: detection target image, 4: image change detection device, 11: subspace generation unit, 12: matrix-for-comparison generating unit, 13: first vector calculating unit, 14: first space generation processing unit, 15: detection target matrix generating unit, 16: second vector calculating unit, 17: second space generation processing unit, 18: degree-of-change calculating unit, 19: determination unit, 20, 21: subspace updating unit, 31: matrix-for-comparison generating circuit, 32: first vector calculating circuit, 33: first space generation processing circuit, 34: detection target matrix generating circuit, 35: second vector calculating circuit. 36: second space generation processing circuit, 37: degree-of-change calculating circuit, 38: determination circuit, 39, 40: subspace updating circuit
This application is a Continuation of PCT International Application No. PCT/JP2019/031690, filed on Aug. 9, 2019, which is hereby expressly incorporated by reference into the present application.
Number | Name | Date | Kind |
---|---|---|---|
20060045353 | Brand | Mar 2006 | A1 |
20080140751 | Ide | Jun 2008 | A1 |
20080243437 | Ide et al. | Oct 2008 | A1 |
20150055826 | Real | Feb 2015 | A1 |
20180059237 | Liu | Mar 2018 | A1 |
Number | Date | Country |
---|---|---|
2008-146353 | Jun 2008 | JP |
Entry |
---|
López-Rodríguez, Patricia, et al. “Non-cooperative target recognition by means of singular value decomposition applied to radar high resolution range profiles.” Sensors 15.1 (2014): 422-439. (Year: 2014). |
Narayanamurthy, Praneeth, and Namrata Vaswani. “Provable dynamic robust PCA or robust subspace tracking.” IEEE Transactions on Information Theory 65.3 (2018): 1547-1577. (Year: 2018). |
Idé et al., “Knowledge Discovery from Heterogeneous Dynamic Systems using Change-Point Correlations,” In Proceedings of 2005 SIAM International Conference on Data Mining (SDM 05), Oct. 4, 2004, total 5 pages. |
Simon et al., “On Updating Problems in Latent Semantic Indexing,” SIAM Journal on Scientific Computing, vol. 21, No. 2, Nov. 1997, total 12 pages. |
Number | Date | Country | |
---|---|---|---|
20220122270 A1 | Apr 2022 | US |
Number | Date | Country | |
---|---|---|---|
Parent | PCT/JP2019/031690 | Aug 2019 | US |
Child | 17565900 | US |