This application claims the benefit to Chinese patent application No. 201710751699.9 filed on Aug. 28, 2017, the entire disclosure of which is incorporated herein by reference.
The present disclosure relates to an image detection method, system and non-volatile computer readable medium.
An image is detected by establishing a target model when detecting whether there is a known target structure in the image. The target model or the target structure is generally constituted by lines, and each of the lines has length information and direction information. When the resolution of the image is low, the width of the individual line segments which constitute the target structure may be ignored, and therefore, this approach is effective in image detection. However, as the resolution of the image is higher and higher, the target template constructed by the above approach has gradually failed to accurately characterize the features of the target structure.
An embodiment of the present disclosure provides an image detection method including: establishing a data model based on mark point information of a group of first strip-like line segments, the mark point information of each first strip-like line segment including the length, the width, the direction and the coordinate of the center point of the first strip-like line segment, establishing a priori model based on the distribution and the number of a group of second strip-like line segments which constitute a target structure, establishing a probability density function for the data model and the priori model, and performing sampling and solution optimization to obtain a globally optimal solution, and detecting the image taking the globally optimal solution as a target model.
Optionally, the step of establishing the data model includes: dividing each first strip-like line segment into multiple sub-strip-like line segments, and calculating the homogeneity within the first strip-like line segment according to the position relationship between every two sub-strip-like line segments, calculating the heterogeneity between each first strip-like line segment and each of its neighboring first strip-like line segments according to the position relationship between the first strip-like line segment and each of its neighboring first strip-like line segments, and determining a first statistic of each first strip-like line segment based on the homogeneity and the heterogeneity.
Optionally, the step of establishing the data model includes: determining a second statistic of each first strip-like line segment based on gradient magnitude of the boundary of each first strip-like line segment, and determining a third statistic of each first strip-like line segment based on gradient direction of the boundary of each first strip-like line segment.
Optionally, the step of establishing the priori model includes: for any second strip-like line segment and another second strip-like line segment within a selected adjacent region of the second strip-like line segment, determining a first direction relationship based on the directions of the two second strip-like line segments.
Optionally, the step of establishing the priori model includes: for each second strip-like line segment and another second strip-like line segment outside a selected adjacent region of the second strip-like line segment, determining the degree of connection between the two second strip-like line segments based on the connection judgment region area of the near ends of the two second strip-like line segments, and determining a second direction relationship based on the directions of the two second strip-like line segments.
Optionally, the step of performing sampling includes: selecting a transfer kernel, and generating a new state space based on a current state space according to the selected transfer kernel, and determining whether to jump to the new state space according to the energy functions of the current state space and the new state space and the probability of jumping between the current state space and the new state space.
Optionally, the transfer kernel includes multiple different sub-kernels, which sub-kernels include at least one of uniform birth and death kernels and simple moving kernels.
An embodiment of the present disclosure provides an image detection system including one or more processor configured to execute computer instructions to perform one or more step of the following method: establishing a data model based on mark point information of a group of first strip-like line segments, the mark point information of each first strip-like line segment including the length, the width, the direction and the coordinate of the center point of the first strip-like line segment, establishing a priori model based on the distribution and the number of a group of second strip-like line segments which constitute a target structure, establishing a probability density function for the data model and the priori model, and performing sampling and solution optimization to obtain a globally optimal solution, and detecting the image taking the globally optimal solution as a target model.
An embodiment of the present disclosure provides a non-volatile computer readable medium configured to store a computer program product containing instructions which, when executed in a processor, implement one or more step of the following method: causing to establish a data model based on mark point information of a group of first strip-like line segments, the mark point information of each first strip-like line segment including the length, the width, the direction and the coordinate of the center point of the first strip-like line segment, causing to establish a priori model based on the distribution and the number of a group of second strip-like line segments which constitute a target structure, causing to establish a probability density function for the data model and the priori model, and perform sampling and solution optimization to obtain a globally optimal solution, and causing to detect the image taking the globally optimal solution as a target model.
In the following various embodiments of the present disclosure will be described in detail with reference to the drawings.
As shown in
At S101, a data model is established based on mark point information of a group of first strip-like line segments, the mark point information of each first strip-like line segment including the length, the width, the direction and the coordinate of the center point of the first strip-like line segment.
For example, the data model is constituted by a group of strip-like line segments. In the data model, there are various spatial topological relationships between these strip-like line segments. It may be possible to set a group of first strip-like line segments many times, thereby gradually approaching a target model. Under the algorithm framework of the mark point process, each first strip-like line segment in each group of first strip-like line segments may be considered as a mark point. With reference to
s=(p,l,w,θ) Eq. 1
In Eq. 1, p=(x, y)∈Λ⊂R2 represents the coordinate of the center point of the first strip-like line segment, Λ represents an image space, R2 represents two dimensions, l represents the length of the first strip-like line segment, w represents the width of the first strip-like line segment, and θ represents the direction of the first strip-like line segment, and its value is within the interval of [0, π].
In each group of first strip-like line segments, based on the mark point information of the individual first strip-like line segments, a data model may be established to represent respective features of the individual first strip-like line segments in the group of first strip-like line segments and the distribution relationship between them.
At S102, a priori model is established based on the distribution and the number of a group of second strip-like line segments which constitute a target structure.
For example, the priori model is established according to the features of the target structure. The target structure is considered to be constituted by a group of second strip-like line segments, and there are also various spatial topological relationships between these second strip-like line segments. The individual second strip-like line segments are considered as mark points, and may also be represented by the above Eq. 1. The priori model represents the number of the second strip-like line segments in the group of second strip-like line segments and the distribution relationship between the individual second strip-like line segments.
At S103, a probability density function is established for the data model and the priori model, and sampling and solution optimization is performed to obtain a globally optimal solution.
For example, the data model is denoted as Ud(S), the priori model is denoted as Up(S), wherein S is a combination of a group of strip-like line segments. Then, the Gibbs point process is employed for modeling. The probability density function established for the data model and the priori model is as follows:
f(S)∝βn exp(−U(S))=βn exp−((Up(S)+Ud(S))) Eq. 2
For example, the sampling may be performed by the Reversible Jump Markov Chain Monte Carlo (RJMCMC) method, jump is conducted between a current state space and a new state space, and the solution optimization is performed by the simulated annealing algorithm to obtain a globally optimal solution.
At S104, the image is detected taking the globally optimal solution as a target model.
When the globally optimal solution is calculated, the calculated optimal solution is taken as the target model, which may be used for automatically detecting the target structure represented by the target model from the image to be detected.
By the image detection method and system of the embodiments of the present disclosure, width information is added to individual line segments of the target model and they become strip-like line segments, which causes that the built target model accurately characterize the features of the target structure, and improves the accuracy of image detection.
In some embodiments, the data model Ud(S) may be represented as
wherein δi represents a statistic of any strip-like line segment si constituting S, S={s1, s2, . . . sn} represents the target structure, and γd represents a weight, which is a positive constant. For a given set S of strip-like line segments, any strip-like segment si therein is relatively independent of its neighboring strip-like line segment.
In some embodiments, the establishing a data model includes creating a first statistic characterizing the internal homogeneity and the external heterogeneity of a first strip-like line segment within a group.
As shown in
In
With reference to
An embodiment of the present disclosure may employ, for example, the Bhattacharyya distance, etc. to calculate the homogeneity of the internal region of each first strip-like line segment and the heterogeneity between each first strip-like line segment and each of its neighboring first strip-like line segments.
In statistics, the Bhattacharyya distance is used for measuring the separability of two discrete probability distributions, and its value is 1 when they completely match, and is 0 when they do not match at all. The calculation formula of the Bhattacharyya distance is as follows:
In the equation, si and sj represent two discrete probability distributions respectively, and in particular, in an embodiment of the present disclosure, represent a sub-strip-like line segment inside a first strip-like line segment and a strip-like line segment of an external neighborhood respectively, mi and mj represent corresponding mean values respectively, and σi and σj represent corresponding standard deviations respectively.
For any first strip-like line segment si, the homogeneity D1 between sub-strip-like line segments in its internal region may be calculated according to the following equation:
Di,1=max(d(Rrc,Rr1),d(Rrc,Rr2),d(Rr1,Rr2)) Eq. 5
Then, the heterogeneity D2 between any first strip-like line segment Rr and the neighborhoods Rb1 and Rb2 is calculated according to the following equation:
Di,2=min(d(Rr,Rb1),d(Rr,Rb2)) Eq. 6
And the synthetic characteristic Di of the homogeneity and the heterogeneity is calculated according to the following equation:
Two thresholds T1, T2 are set for Di, wherein T1<T2, and the first statistic of the data model is created:
In some embodiments, the establishing a data model includes creating a statistic characterizing the boundary characteristics of a first strip-like line segment within a group, which includes, as shown in
As shown in
In some embodiments, for the gradient magnitude, statistics is performed on the mean values me
mG=min(me
Then, similar to the first statistic, the second statistic of the data model is created:
For the gradient direction, statistics is performed on the gradient direction histograms H(θ1) and H(θ2), and for the gradient direction histograms of the pixel points of the boundaries e1 and e2 of each first strip-like line segment s1, the number n1 and n2 of the points on each boundary of which the gradient directions are orthogonal to the direction θi of the first strip-like line segment si are calculated respectively, wherein the total number of points of each boundary is n. Here, once the angle between the direction of a boundary point and θi is less than threshold Tθ, it is considered that the point belongs to the orthogonal range. The proportion of the points that belong to the orthogonal range is calculated according to the following equation:
Then, thresholds Tn1 and Tn2 are set, wherein Tn1<Tn2, and the third statistic of the data model is created:
In some embodiments, the statistics in the data model include the above three statistics, and alternatively or optionally, one or two of them. For example, when the above three statistics are included, the statistics in the data model may be δi=δi,1δi,2δi,3, with reference to the step S126 in
In an embodiment of the present disclosure, for a direction between two strip-like line segments, the adjacent region Rin of a strip-like line segment is first defined.
As shown in
In an embodiment of the present disclosure, for any second strip-like line segment and a further second strip-like line segment within the adjacent region Rin of the second strip-like line segment, that is, two second strip-like line segments in adjacent relationship, a first direction relationship is determined based on the directions of the two second strip-like line segments, i.e., the step S131 in
As shown in
The angle τij between two strip-like line segments si and sj may be calculated according to the following equation:
τij=min[|θi−θj|,π−|θi−θj|] Eq. 13
In the equation, τij∈[0, π/2], and θi and θj are the directions of si and sj respectively.
If the angle between the directions of two strip-like line segments is too small, it indicates that their relationship is too sharp and needs to be penalized, for example, to reduce the weights of the statistics of the two strip-like line segments. A calculation formula for the adjacent relationship between two strip-like line segments is as follows:
In the equation, τmin is a threshold for the angle judgment, the function β(x, m, M) is a monotonically decreasing function, the definition domain is [m, M], and the value domain is [0, 1]. A calculation formula for β(x, m, M) is as follows:
In some embodiments, for the external relationship ˜out between two strip-like line segments as shown in
The formula which directly calculates the distances between the end points of two line segments in a conventional point process method is already not applicable to this disclosure, and therefore, an embodiment of the present disclosure employs the following formula to calculate the degree of connection between the two strip-like line segments si and sj:
In the equation, Area(g) represents the connection judgment region area, and here, represents the area of the connection ends siA and sjC of si and sj. As shown in
If the angle between the directions of two strip-like line segments is too large, it indicates the curvature of the constituted structure is too large, and a penalty is needed, for example, to reduce the weights of the statistics of the two strip-like line segments. A calculation formula for the external relationship between two strip-like line segments is as follows:
gout(si,sj)=I(si,sj)+θout(si,sj) Eq. 17
In the equation, I(si, sj) represents the degree of connection between the two strip-like line segments, θout(si, sj) represents the second direction relationship between the two strip-like line segments, and similar to the first direction relationship, θout(si, sj) is represented as:
Therein, τmax is a threshold for judging the angle between the strip-like line segments.
At the step S133 as shown in
Therein, ωi (i=0, 1, . . . , 4) is a weight, N represents the total number of the strip-like line segments, Nf represents the number of connectionless strip-like line segments, Ns represents the number of single-connection strip-like line segments, <si,sj>˜in indicates that si and si belongs to adjacent relationship, and <si,sj>˜out indicates that si and sj belongs to an external relationship.
The second term and the third term in parentheses in Eq. 19 may also be omitted.
In an embodiment of the present disclosure, the RJMCMC method is employed to perform sampling, including selecting a transfer kernel, generating a new state space based on a current state space, and determining whether to jump to the new state space according to the energy functions of the current state space and the new state space and the probability of jumping between the current state space and the new state space.
In particular, suppose that Q(ω→•) is the transfer kernel, the current state space is co, and the new state space to be generated is ω′, and then a flow of the sampling algorithm is as follows (referring to
step S121, selecting a transfer kernel Q(ω→•).
step S122, generating a new state space ω′ according to the selected transfer kernel Q(ω→•), and
step S123, calculating the Green ratio, which is denoted as R, and transferring from the current state ω to the new state ω′ according to the transfer kernel Q(ω→ω′), wherein a calculation formula for the expression of a condition that the acceptance probability needs to satisfy a detailed balance condition to ensure that the algorithm converges to the density function of the point process is as follows:
In the equation, h(•) represents the energy function of a state space, and Q(ω→ω′) and Q(ω′→ω) represent the transfer probabilities of jumping from the state space ω to the state space ω′ and jumping from the state space ω′ to the state space ω, respectively.
Jump is accepted according to the probability min[1, R] to jump to the new state space.
In an embodiment of the present disclosure, the transfer kernel for jumping between different state spaces may be constituted by different transfer kernels, that is:
Q(ω→•)=Σpiqi(ω→•) Eq. 21
In the equation, qi(ω→•) is a sub-kernel constituting the transfer kernel Q(ω→•), and pi represents the probability of selecting the sub-kernel for jumping, wherein Σpi=1.
For example, the sampling algorithm in an embodiment of the present disclosure uses the uniform birth and death kernel as shown in
As shown in
the simulated annealing algorithm, of which the main flow is:
S201, initialization and setting parameters,
S202, judging whether to terminate generating a new solution (i.e., judging whether the termination condition is satisfied), and if yes, proceeding to S203, otherwise, proceeding to S204,
S203, outputting the current solution as a globally optimal solution,
S204, generating a new solution x′,
S205, judging whether the new solution is accepted, and if yes, proceeding to S206, otherwise, returning to S204 to regenerate a new solution,
wherein the increment Δx=C(x′)−C(x) is calculated for judging whether the new solution is accepted, and C(x) is an evaluation function, and wherein to obtain the globally optimal solution with a certain probability, the simulated annealing algorithm accepts a new solution worse than the current solution with a certain probability, that is, when the Δx<0, it is indicated that the new solution is better than the current solution, and at this point, the new solution is always accepted, otherwise, it is indicated that the new solution is worse than the current solution, and at this point, the new solution is accepted with a certain probability,
S206, generating a new current solution,
S207, judging whether to cool down, and if yes, proceeding to S208, otherwise, returning to S204 to regenerate a new solution, and
S208, cooling down and returning to S202.
As shown in
The embodiments of the present disclosure may take the form of all hardware embodiments, all software embodiments or embodiments including hardware and software units.
In some embodiments, the present disclosure is implemented by software, which includes, but is not limited to, firmware, resident software, microcode, etc. In addition, as shown in
In some embodiments, as shown in
The image detection system 1100 may be coupled to (not shown) or contain the computer usable or computer readable medium 1102 which provides program code.
In some embodiments of the image detection system of the present disclosure, the processor 1100 is configured to, when the data model is established, execute computer instructions to: divide each first strip-like line segment into multiple sub-strip-like line segments, and calculate the homogeneity within the first strip-like line segment according to the position relationship between every two sub-strip-like line segments, calculate the heterogeneity between each first strip-like line segment and each of its neighboring first strip-like line segments according to the position relationship between the first strip-like line segment and each of its neighboring first strip-like line segments, and determine a first statistic of each first strip-like line segment based on the homogeneity and the heterogeneity.
In some embodiments of the image detection system of the present disclosure, the processor 1100 is configured to, when the data model is established, execute computer instructions to: determine a second statistic of each first strip-like line segment based on the gradient magnitudes of the boundary of each first strip-like line segment, and determine a third statistic of each first strip-like line segment based on the gradient directions of the boundary of each first strip-like line segment.
In some embodiments of the image detection system of the present disclosure, the processor 1100 is configured to, when the priori model is established, execute computer instructions to: for any second strip-like line segment and another second strip-like line segment within a selected adjacent region of the second strip-like line segment, determine a first direction relationship based on the directions of the two second strip-like line segments.
In some embodiments of the image detection system of the present disclosure, the processor 1100 is configured to, when the priori model is established, execute computer instructions to: for each second strip-like line segment and another second strip-like line segment outside a selected adjacent region of the second strip-like line segment, determine the degree of connection between the two second strip-like line segments based on the connection judgment region area of the near ends of the two second strip-like line segments, and determine a second direction relationship based on the directions of the two second strip-like line segments.
For specific processes of various operations implemented by the image detection system of the embodiments of the present disclosure, reference may be made to the above embodiments described with reference to
For the sake of illustration, the computer usable or computer readable medium may be any apparatus which may contain, store, communicate, transmit or convey a program for being used by or being combined with an instruction execution system, apparatus or device (e.g., a processor). The medium may be an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system (or apparatus or device) or communications medium, that is, may be a non-volatile or volatile medium. Examples of the computer readable medium include a semiconductor or solid memory, tape, removable computer floppy disk, random access memory (RAM), read only memory (ROM), hard disk and optical disk. Examples of current optical disk include the compact disc-read only memory (CD-ROM), the compact disc-read/write (CD-R/W) and the DVD.
The processor adapted for executing program code includes at least one processor 1101 directly or indirectly coupled with the computer usable or computer readable medium 1102 via the system bus, which processor 1101 may be implemented by a circuit with a logic operation function, for example, may be a central processing unit CPU, a field programmable logic array FPGA, application specific integrated circuit ASIC, a microcontroller unit MCU or a digital signal processor DSP. The computer usable or computer readable medium 1102 may include a local memory deployed during actual execution of the program code, a mass storage device and a cache memory, which cache memory provides a temporary storage device for at least a certain kind of program code to reduce the number of times the code must be retrieved from the mass storage device during execution. An input/output or I/O device (including, but not limited to, a keyboard, a display, a pointing device, etc.) may be coupled to the system directly or via an intermediate I/O controller. A network adapter may also be coupled to the system, such that a data processing system can become coupled to other data processing system or remote printer or storage device via an intermediate private or public network. The modem, the cable modem and the Ethernet card are just a part of currently available types of network adapters.
Although the embodiments of the present disclosure have already been illustrated and described, it may be appreciated by the person having ordinary skills in the art that many changes, modifications, replacements and variations may be made to these embodiments without departing from the principles and purposes of the present disclosure, and the scope of the invention is defined by the claims and their equivalents.
Number | Date | Country | Kind |
---|---|---|---|
2017 1 0751699 | Aug 2017 | CN | national |
Number | Name | Date | Kind |
---|---|---|---|
20140307621 | Frenger | Oct 2014 | A1 |
20160242229 | Balachandran | Aug 2016 | A1 |
20160338005 | Lim | Nov 2016 | A1 |
20170055250 | Lim | Feb 2017 | A1 |
20170135005 | Basu Mallick | May 2017 | A1 |
Entry |
---|
“First Office Action,” Chinese Application No. 201710751699.9 dated Dec. 3, 2019. |
He, J., et al., “Road network extraction from remote sensing image based on modified marked point process,” 2013, 49(17)150-153; 4 pages (1 page of English translation of Abstract only). |
Linlin, L., “The Application of Marked Point Process in Road Extraction,” 2008; 93 pages (2 pages of English translation of Abstract only). |
Stoica et al., “A Gibbs Point Process for Road Extraction from Remotely Sensed Images”, International Journal of Computer Vision, vol. 57, No. 2, 2004, pp. 121-136. |
Lacoste et al., “Point Processes for Unsupervised Line Network Extraction in Remote Sensing”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, No. 10, Oct. 2005, pp. 1568-1579. |
Number | Date | Country | |
---|---|---|---|
20190065891 A1 | Feb 2019 | US |