Geometric calibration method and apparatus of computer tomography

Information

  • Patent Grant
  • 12142009
  • Patent Number
    12,142,009
  • Date Filed
    Monday, June 13, 2022
    3 years ago
  • Date Issued
    Tuesday, November 12, 2024
    8 months ago
Abstract
A geometric calibration apparatus detects points from projection regions onto which markers disposed on a phantom are projected, and calculates an output vector representing a probability distribution that gives a probability with which each point is a projection of each marker, by inputting data corresponding to each point to a learning model. The geometric calibration apparatus extracts a predetermined number of samples based on the probability distribution, obtains a candidate projection matrix by transforming correspondences between markers determined based on the samples among the markers and points determined based on the samples among the points, calculates points into which the markers are transformed by the candidate projection matrix, calculates a difference between a set of the transformed points and a set of the detected points, and designates the candidate projection matrix as a projection matrix when the difference is less than or equal to a threshold.
Description
CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to and the benefit of Korean Patent Application No. 10-2021-0080024 filed in the Korean Intellectual Property Office on Jun. 21, 2021, the entire contents of which are incorporated herein by reference.


BACKGROUND
(a) Field

The present disclosure relates to a geometric calibration method and apparatus of computer tomography.


(b) Description of the Related Art

A computer tomography (CT) device, for example, a cone-beam CT (CBCT) device, acquires multiple X-ray images by imaging a subject with X-rays in different directions and creates a three-dimensional tomographic image from the X-ray images. Information on the directions in which X-ray imaging is performed is essential in order to generate an accurate tomographic image. An operation of specifying all imaging direction information of the CBCT device is called geometric calibration.


Geometric calibration methods of the CBCT device can be classified into a method using a geometric calibration phantom and a method using no geometric calibration phantom. The geometric calibration phantom refers to an object manufactured in a certain shape for the geometric calibration.


Some methods using some representative values as the imaging direction information of the CBCT device have been proposed for geometric calibration. These methods, however, obtain the imaging direction information by parameterizing some representative values, so projection matrices that fully represent the imaging direction information of the CBCT device cannot be obtained. In addition, although a geometric calibration method using phantom markers of different sizes has been proposed, manufacturing and verification of the phantoms are complicated because different types of phantom markers are used.


SUMMARY

Some embodiments may provide a geometric calibration method and apparatus capable of calculating accurate imaging direction information from an X-ray image of a phantom acquired by a computer tomography device.


According to one embodiment, there may be provided a geometric calibration apparatus including a memory configured to store one or more instructions and a processor configured to execute the one or more instructions. By executing the one or more instructions, the processor may detect a plurality of first points from a plurality of projection regions, respectively, a plurality of first markers disposed on a phantom being projected onto the projection regions in two dimensions by a computer tomography device, calculate an output vector representing a probability distribution that gives a probability with which each of the first points is a projection of each of the first markers, by inputting data corresponding to each of the first points to a learning model, extract a predetermined number of first samples based on the probability distribution, obtain a candidate projection matrix by transforming correspondences between second markers determined based on the first samples among the first markers and second points determined based on the first samples among the first points, calculate third points into which the first markers are transformed by the candidate projection matrix, calculate a difference between a first set of the third points and a second set of the first points, and designate the candidate projection matrix as a projection matrix in response to the difference being less than or equal to a threshold.


In some embodiments, the processor may transform each of the first points into a histogram. In this case, the data corresponding to each of the first points may be data obtained by being transformed into the histogram.


In some embodiments, the processor may extract the first samples by applying weighted random sampling to the probability distribution.


In some embodiments, the processor may extract a plurality of second samples for the first points, respectively, from a third set including indices of the first markers, and extract the first samples from a fourth set including indices of the first points such that a probability with which each element of the fourth set is extracted is a normalized value of a probability with which a first point corresponding to said each element among the first points is a projection of a marker at a second sample for the first point among the second samples.


In some embodiments, the processor may extract the first samples and the second samples using a weighted random sampling technique.


In some embodiments, the second markers may be markers that have, as indices, second samples, corresponding to the first samples among the second samples, among the first markers, and the second points may be points having the first samples as indices among the first points.


In some embodiments, in response to the difference being greater than the threshold, the processor may repeat a process of extracting the first samples, obtaining the candidate projection matrix, calculating the second points, and calculating the difference.


In some embodiments, the processor may calculate a minimum-weight perfect matching for a bipartite graph in which the first set and the second set are set as vertex sets and an edge between a vertex of the first set and a vertex of the second set has a weight, and calculate a sum of weights obtained by the minimum-weight perfect matching as the difference.


In some embodiments, the learning model may be a model that has been trained by using training data that have, as input data, data corresponding to fourth points into which the first markers are respectively transformed by a randomly-generated projection matrix and have, as correct answer labels, values indicating classification of the first markers.


In some embodiments, the data corresponding to the fourth points may be data obtained by transforming the fourth points into a histogram.


In some embodiments, the processor may apply direct linear transformation to the correspondences to obtain the candidate projection matrix.


According to another embodiment, a geometric calibration method of a computer tomography device may be provided. The geometric calibration method may include detecting a plurality of points q′j(j=1, . . . ,N′) from a plurality of projection regions, respectively, a plurality of first markers disposed on a phantom being projected onto the projection regions in two dimensions by the computer tomography device, calculating an output vector yj=(P1j, . . . , PMj) having, as elements, a probability Pij(i=1, . . . , M) with which the q′j is a projection of each of the first markers, by inputting data corresponding to the q′j to a learning model, extracting samples s′k(k=1, . . . , 6) from a set Ω′={1, . . . , N′} based on the yj, obtaining a candidate projection matrix by transforming correspondences between second markers determined based on the samples s′k among the first markers and points determined based on the samples s′k among the q′j, calculating points {circumflex over (q)}i into which the first markers are transformed by the candidate projection matrix, calculating a difference d({circumflex over (Q)}, Q′) between a set {circumflex over (Q)}={{circumflex over (q)}1, . . . , {circumflex over (q)}, . . . , {circumflex over (q)}M} of the {circumflex over (q)}i and a set Q′={q′1, . . . , q′N′} of the q′j, and designating the candidate projection matrix as a projection matrix in response to the d({circumflex over (Q)}, Q′) being less than or equal to a threshold.


In some embodiments, the geometric calibration method may further include transforming the q′j into a histogram. In this case, the data corresponding to the q′j may be data obtained by being transformed into the histogram.


In some embodiments, extracting the s′k may include extracting samples sj∈Ω from a set Ω={1, . . . , M} such that a probability with which each element i of the set Ω is extracted for a given j is the Pij, normalizing a probability Psj,j at the sj into αPsj,j, and extracting the s′k such that a probability with which each element j of the Ω′ is extracted from the Ω′ is αPsj,j.


In some embodiments, the a may be a reciprocal of a sum of the Psj,j at the sj.


In some embodiments, calculating the d({circumflex over (Q)}, Q′) may include calculating a minimum-weight perfect matching for a bipartite graph in which the {circumflex over (Q)} and the Q′ are set as vertex sets and an edge between a vertex {circumflex over (q)}i of {circumflex over (Q)} and a vertex q′j of Q′ has a weight wij=∥{circumflex over (q)}i−q′j2, and calculating a sum of weights obtained by the minimum-weight perfect matching as the d({circumflex over (Q)}, Q′).


In some embodiments, the geometric calibration method may further include, in response to the d({circumflex over (Q)}, Q′) being greater than the threshold, repeating extracting the s′k, obtaining the candidate projection matrix, calculating the {circumflex over (q)}i, and calculating the d({circumflex over (Q)}, Q′).


According to yet another embodiment, a computer program executed by a computing device and stored in a non-transitory recording medium may be provided. The computer program may cause the computing device to execute detecting a plurality of first points from a plurality of projection regions, respectively, a plurality of first markers disposed on a phantom being projected onto the projection regions in two dimensions by a computer tomography device, calculating an output vector representing a probability distribution that gives a probability with which each of the first points is a projection of each of the first markers, by inputting data corresponding to each of the first points to a learning model, extracting a predetermined number of first samples based on the probability distribution, obtaining a candidate projection matrix by transforming correspondences between second markers determined based on the first samples among the first markers and second points determined based on the first samples among the plurality of first points, calculating third points into which the first markers are transformed by the candidate projection matrix, calculating a difference between a first set of the third points and a second set of the first points, and designating the candidate projection matrix as a projection matrix in response to the difference being less than or equal to a threshold.





BRIEF DESCRIPTION OF THE DRAWINGS


FIG. 1 is a drawing showing an example of a geometric calibration phantom according to an embodiment of the present invention.



FIG. 2 is a drawing showing an example of a learning apparatus according to an embodiment of the present invention.



FIG. 3 is a flowchart showing an example of a training data generation method in a learning apparatus according to an embodiment of the present invention.



FIG. 4 is a drawing showing an example of a learning model or learning method in a learning apparatus according to an embodiment of the present invention.



FIG. 5 is a flowchart showing an example of a geometric calibration method according to an embodiment of the present invention.



FIG. 6 is a drawing showing an example of detection of a marker projection region in a geometric calibration method according to an embodiment of the present invention.



FIG. 7 is a drawing showing an example of a probability distribution in a geometric calibration method according to an embodiment of the present invention.



FIG. 8 is a drawing showing an example of a bipartite graph in the geometric calibration method according to an embodiment of the present invention.



FIG. 9 is a drawing showing an example of a computing device according to an embodiment of the present invention.





DETAILED DESCRIPTION OF THE EMBODIMENTS

In the following detailed description, only certain embodiments of the present invention have been shown and described, simply by way of illustration. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. Accordingly, the drawings and description are to be regarded as illustrative in nature and not restrictive. Like reference numerals designate like elements throughout the specification.


As used herein, a singular form may be intended to include a plural form as well, unless the explicit expression such as “one” or “single” is used.


In flowcharts described with reference to the drawings, the order of operations or steps may be changed, several operations or steps may be merged, a certain operation or step may be divided, and a specific operation or step may not be performed.


First, a principle of geometric calibration to be used in a geometric calibration method according to an embodiment of the present invention is described with reference to FIG. 1.



FIG. 1 is a drawing showing an example of a geometric calibration phantom according to an embodiment of the present invention.


Referring to FIG. 1, a geometric calibration phantom 100 includes a plurality of phantom markers 110 and a main body 120 to which the markers 110 are fixed (or disposed). The markers 110 may include, for example, small balls having a radiopaque material (e.g., metal), and the main body 120 may include, for example, an object having a radiolucent material (e.g., acryl). When X-ray imaging 130 of the phantom 100 is performed, X-ray beams passing through the markers 110 are greatly attenuated compared to those that do not pass through the markers 110. As a result, when viewed from an X-ray image 140, regions onto which the markers 110 are projected are displayed with dark pixel values compared to their surroundings. Phantom-based geometric calibration can be regarded as a process of receiving a pattern 150 of these regions appearing in the two-dimensional X-ray image 140 and outputting geometric information about the X-ray imaging direction (hereinafter referred to as “X-ray imaging direction information”).


The X-ray imaging direction information is mathematically expressed as a projection matrix. The projection matrix is given as a 3×4 matrix of real components, and can also be called a camera matrix. Geometrically, the projection matrix represents mapping or transformation from a set of three-dimensional points to a set of two-dimensional points. That is, when the three-dimensional point p=(px, py, pz)∈custom character3 is given, this point is transformed as in Equation 1 by the projection matrix A.

{tilde over (q)}=A{tilde over (p)}  Equation 1


In Equation 1, {tilde over (p)} and {tilde over (q)} are homogeneous coordinates of points p and q, respectively. Equation 1 is explicitly written as Equation 2.










[





q
~

u







q
~

v







q
~

w




]

=

A
[




p
x






p
y






p
z





1



]





Equation


2







By equation 2, the two-dimensional point q∈custom character2 is given as in Equation 3.









q
=

[






q
~

u

/


q
~

w









q
~

v

/


q
~

w





]





Equation


3







Accordingly, if the phantom marker p=px, py,pz) is projected onto q=(qu, qv) by the X-ray imaging, Equation 1 is expressed as in Equation 4 because the projection matrix A representing the X-ray imaging direction satisfies Equation 1.

A11px+A12py+A13pz+A14=qu{tilde over (q)}w
A21px+A22py+A23pz+A24=qv{tilde over (q)}w
A31px+A32py+A33pz+A34={tilde over (q)}w  Equation 4


In equation 4, Aijcustom character is an element in ith row and jth column of the 3×4 matrix A. Eliminating {tilde over (q)}w from Equation 4 gives Equation 5. That is, two equations for the projection matrix A can be obtained from the correspondence of p↔q.

PxA11+PyA12+pzA13+A14−qu(pxA31+pyA32+pzA33+A34)=0
PxA21+PyA22+pzA23+A24−qv(pxA31+pyA32+pzA33+A34)=0  Equation 5


If K correspondences pk↔qk(k=1, . . . , K) are given, 2K equations for the projection matrix A can be obtained from the K correspondences and given as in Equation 6 when expressed as a matrix equation.

custom charactera=0  Equation 6


In Equation 6, a is a column vector of size 12, expressed as in Equation 7, and custom character is a matrix of size 2K×12 where a 2×12 partial matrix custom characterk corresponding to (2k−1)th and 2kth rows is given as in Equation 8.








Equation


7







a
=

[




A
11




A
12




A
13




A
14




A
21




A
22




A
23




A
24




A
31




A
32




A
33





A
34

]













Equation


8









k

=

[




p
x
k




p
y
k




p
z
k



1


0


0


0


0



-

p
x
k



q
u
k





-

p
y
k



q
u
k





-

p
z
k



q
u
k





-

q
u
k






0


0


0


0



p
x
k




p
y
k




p
z
k



1



-

p
x
k



q
v
k





-

p
y
k



q
v
k





-

p
z
k



q
v
k





-

q
v
k





]





From the above Equations, it can be observed that K must be equal to or greater than 6 (K≥6) in order to determine the solution a of Equation 6. The process of constructing the equation custom charactera=0 from the correspondences pk↔qk for the projection matrix A is called direct linear transformation (DLT).


One method for solving custom charactera=0 obtained by the DLT with respect to the unknown a may use, for example, singular value decomposition (SVD). That is, if the matrix custom character is decomposed as in Equation 9 by applying the SVD to the matrix custom character, a column of V corresponding to the smallest element among singular values appearing in diagonal matrix D becomes the solution of a. Rearranging a gives the projection matrix A.

custom character=UDVT  Equation 9


In Equation 9, U is an orthogonal matrix obtained by eigenvalue decomposition of AAT, V is an orthogonal matrix obtained by eigenvalue decomposition of ATA, D is a diagonal matrix having the singular values as diagonal elements, and AT is a transpose matrix of A.


A geometric calibration phantom may be designed for a geometric calibration method according to various embodiments of the present invention. The geometric calibration phantom design may mean defining spatial positions of a plurality of markers constituting the phantom. As described above, in the geometric calibration phantom design, M (M≥6) markers (e.g., 110 in FIG. 1) may be disposed on a main body of the geometric calibration phantom (e.g., 110 in FIG. 1) to be positioned at three-dimensional coordinates (p1, . . . , pM). Here, M and pi (i=1, . . . , M) may be values arbitrarily determined by a user. In some embodiments, the phantom may be determined based on the ease of fabrication or the size of a field of view in which the CBCT device can image. For example, as shown in Table 1, three-dimensional coordinates of 16 markers may be determined.









TABLE 1







Marker positions









x
y
z












11.131
29.021
62.179


9.851
11.078
84.876


28.609
58.157
11.798


26.290
89.173
59.756


36.147
64.311
86.668


76.712
20.840
2.826


56.729
1.279
35.583


87.399
54.607
2.605


38.432
86.550
12.123


10.308
51.591
61.399


49.733
37.451
98.660


18.009
67.792
47.432


15.280
25.564
1.724


44.632
95.417
11.597


12.424
73.945
57.123


74.076
89.189
78.380









Next, for a learning apparatus and a learning model according to various embodiments of the present invention are described with reference to FIG. 2 to FIG. 4.



FIG. 2 is a drawing showing an example of a learning apparatus according to an embodiment of the present invention, FIG. 3 is a flowchart showing an example of a training data generation method in a learning apparatus according to an embodiment of the present invention, and FIG. 4 is a drawing showing an example of a learning model or learning method in a learning apparatus according to an embodiment of the present invention.


Referring to FIG. 2, a learning apparatus 200 for geometric calibration is a computing device that performs machine learning on a neural network to generate a learning model that performs a target task. In some embodiments, the target task may include a task for geometric calibration. In some embodiments, the task for geometric calibration may be a task of finding a projection matrix.


In some embodiments, the learning model trained (generated) in the learning apparatus may be provided to a geometric calibration apparatus. The geometric calibration apparatus may perform geometric calibration based on the learning model. In some embodiments, the geometric calibration apparatus may be a computing device other than the learning apparatus. In some embodiments, the learning apparatus may perform functions of a geometric calibration apparatus.


In some embodiments, the computing device may be a desktop computer, a laptop computer, a server, or the like, but is not limited thereto and may be any kind of device having a computing function.


Although FIG. 2 shows the learning apparatus 200 is implemented as one computing device by way of example, in some embodiments, functions of the learning apparatus 200 may be implemented by a plurality of computing devices.


The learning apparatus 200 may train a neural network (e.g., a convolutional neural network, CNN) by using training data set 210 including a plurality of training data 211. Each training data 211 may include data in which a correct answer 211a is labeled. In some embodiments, the learning apparatus 200 may train the neural network by predicting a value by performing the target task by inputting the training data 211 to the neural network and backpropagating a loss between the predicted value and the correct answer 211a labeled in the training data to the neural network.


In some embodiments, the learning apparatus 200 may use a neural network learning model using a set of two-dimensional coordinates Q={q1, . . . , qN} as the training data (i.e., inputs). In this case, a correct answer label of the training data may be a value indicating classification of a phantom marker in a geometric calibration phantom, for example, an index of the phantom marker.


Referring to FIG. 3, the learning apparatus first initializes the training data set D at S310. In some embodiments, the learning apparatus may initialize the training data set D to an empty set at S310. That is, D←∅.


The learning apparatus randomly generates twelve real numbers and creates a 3×4 projection matrix A having the twelve real numbers as elements at S320. The learning apparatus determines whether the projection matrix A satisfies a predetermined constraint condition at S330. In some embodiments, the predetermined constraint conditions may be set by a user configurable. In some embodiments, the predetermined constraint condition may include, for example, a condition that the projection matrix is a matrix that satisfies a specification of a designed pattern. That is, since there is no need to train the learning model with the projection matrix out of the specification, the constraint condition may be set in order to train the learning model only with the projection matrix that satisfies the specification. When the projection matrix A does not satisfy the predetermined constraint condition, the learning apparatus performs the process of S320 again.


When the projection matrix A satisfies the predetermined constraint condition, the learning apparatus transforms each phantom marker picustom character3 into qicustom character2 through the projection matrix A at S340. In some embodiments, the learning apparatus may calculate a point qi to satisfy {tilde over (q)}i=A{tilde over (p)}i as described with reference to Equation 1 at S340. Next, the learning apparatus designates as in xƒ(q1, . . . , qM), y→(1, . . . , M) at S340. In some embodiments, the tuple x may be a set of two-dimensional coordinates of the points onto which the phantom markers are projected and correspond to data used as an input. The tuple y may correspond to data used as correct answer labels and be values indicating classification (e.g., indices) of the phantom markers.


In some embodiments, the learning apparatus may perform data augmentation on the training data at S350. The learning apparatus may generate augmented training data by changing x and y based on any one or a combination of two or more among various data augmentation techniques. In some embodiments, the learning apparatus may select any one or the combination of two or more among the various data augmentation techniques with a probability. In some embodiments, the various data augmentation techniques may include, for example, a technique of randomly selecting j to remove the j-th elements in x and y, a technique of randomly generating q∈custom character2 to add it as the last element of x and adding (N+1) as the last elements of y (N is the number of elements in each of x and y before the change), and a technique of randomly generating permutation σ and changing ordering of the elements in x and y according to the permutation σ. According to the third technique, it may be designated as in xƒ(qσ(1), . . . , qσ(N)), yƒ(σ(1), . . . , σ(N)).


In some embodiments, the learning apparatus may transform each element of x obtained in the process of S340 or S350 into a histogram at S360. The x and y obtained in the process of S340 or S350 may be expressed as in x=(x1, . . . , xN), y=, (y1, . . . , yN). The learning apparatus may transform each element of xjcustom character2 of x into the histogram Hj=H(xj; x) to facilitate training of a neural network model. The histogram transformation H(xj; x) may be defined in several ways to quantify spatial features of xj in relation to other points x. In some embodiments, the histogram transformation may be computed as in Equation 10 using shape context. Various histogram transformation techniques including, for example, “Shape context: A new descriptor for shape matching and object recognition” (Belongie, S. et al., The technique presented in Advances in neural information processing systems, 13, 831-837) may be used. The learning apparatus may transform x into the histogram by binding each element of Equation 10 and designating it as Hƒ(H1, . . . , HN).

Hj(k)=#{j′≈j:xj′−xj∈bin(k)}  Equation 10


The learning apparatus adds N training data items consisting of (Hj, yj) generated in the process of S360 to the training data set at S370. That is, the learning apparatus may update the training data set D as in Equation 11.

D←D∪{(H1,y1), . . . (HN,yN)}  Equation 11


If sufficient training data items are added to the training data set at S380, the learning apparatus ends the generation of training data. If the sufficient training data items are not added, the learning apparatus repeats the process from S320. In some embodiments, the learning apparatus may determine that the sufficient training data items have been added when the number of elements (e.g., training data) of the training data set reaches a threshold.


Through these processes, the learning apparatus may generate the training data set D and store the generated training data set D in a storage device.


Next, referring to FIG. 4, a neural network 400 may include a learning model, and the learning model may include a feature extraction layer 410 and an output layer 420. In some embodiments, the learning model may be a multi-class classification model.


Two-dimensional input data (e.g., data transformed into a histogram) 430 of the training data is input to the feature extraction layer 410. The feature extraction layer 410 extracts features from the input data 430. Such the feature may be expressed in the form of a feature map.


In some embodiments, the feature extraction layer 410 may include a plurality of layers. A first layer among the plurality of layers may receive the input data and extract the feature map from the input data. The other layers may extract the feature map again from the feature map transferred by the previous layer and transfer it to the next layer. In some embodiments, each layer may extract the feature map by applying an operation of the corresponding layer to the input data or the input feature map. In some embodiments, the feature extraction layer 410 may include a convolution layer. The convolution layer may extract the feature map by applying a convolution filter to the input data or the input feature map.


In some embodiments, the feature extraction layer 410 may further include other layers such as a pooling layer and an activation layer. The pooling layer can extract features by performing a pooling operation on input features.


The output layer 420 predicts a predicted value 440 according to the target task based on the feature map extracted from the feature extraction layer 410. In some embodiments, the predicted value may be given as an M-dimensional vector y indicating values indicating classification (e.g., indices) of the phantom markers predicted by the target task.


In some embodiments, the output layer 420 may include a fully-connected layer that performs classification from the feature map output through the feature extraction layer 410. In some embodiments, a plurality of fully-connected layers may be provided.


A loss 450 between the predicted value 440 of the output layer 420 and a correct answer label of the input data may be calculated and be backpropagated to the neural network 400 so that the neural network 400 can be trained.


After repeatedly training the learning model using the training data of the training data set, the learning apparatus may store the learning model in the storage device. Next, the geometric calibration apparatus may perform geometric calibration by generating a projection matrix based on the learning model learned by the learning apparatus.


Next, a geometric calibration method according to various embodiments of the present invention is described with reference to FIG. 5 to FIG. 8.



FIG. 5 is a flowchart showing an example of a geometric calibration method according to an embodiment of the present invention, FIG. 6 is a drawing showing an example of detection of a marker projection region in a geometric calibration method according to an embodiment of the present invention, FIG. 7 is a drawing showing an example of a probability distribution in a geometric calibration method according to an embodiment of the present invention, and FIG. 8 is a drawing showing an example of a bipartite graph in the geometric calibration method according to an embodiment of the present invention.


A geometric calibration apparatus detects projection regions (e.g., 150 in FIG. 1) of markers (e.g., 110 in FIG. 1) in an X-ray image obtained by imaging a phantom (e.g., 100 in FIG. 1) by a CBCT device at S510. In some embodiments, the projection regions of the markers may appear as dark brightness values in the X-ray image. In some embodiments, the geometric calibration apparatus may detect the marker projection regions using a general digital image processing technique (e.g., a thresholding technique, etc.). The geometric calibration apparatus may designate the number of detected marker projection regions as N and a center point of each marker projection region as q′jcustom character(j=1, . . . , N′). For example, as shown in FIG. 6, the geometric calibration apparatus may detect the center point q′j of each marker projection region in the X-ray image 610.


The geometric calibration apparatus transforms the detected point q′j and inputs data corresponding to the detected point q′j to the learning model at S520 and S530. In some embodiments, the geometric calibration apparatus may transform the detected point q′j into a histogram H′j; at S520. In some embodiments, the geometric calibration apparatus may designate as in xƒ(q′1, . . . , q′N′) and calculate the histogram H′j=H(q′j; x), for example, based on Equation 10. The geometric calibration apparatus may calculate an output vector yj=(P1j, . . . , PMj) by inputting the calculated histogram H′j to the learning model at S530. The geometric calibration apparatus may perform an operation of transforming the detected point q′j into the histogram H′j for each j(=1, . . . , N′) and calculating the output vector yj=(P1j, . . . , PMj), for each j(=1, . . . , N′), at S520 and S530.


When there is a set of two-dimensional points {q′1, . . . , q′N′}, the output vector yj may be given as an M-dimensional vector (P1j, . . . , PMj) if the point q′j is transformed into the histogram and input to the learning model. Here, Pij(i=1, . . . , M) means a probability with which the point q′j is a projection of the phantom marker pi. The output vector yj may be expressed as, for example, a probability distribution as shown in FIG. 7. Accordingly, the geometric calibration apparatus may estimate, based on the probability distribution yj, which of the phantom markers p1, . . . , pM generates each of the detected points q′1, . . . , q′N′. The geometric calibration apparatus may extract six samples (referred to as “first samples”) based on the probability distribution yj at S540 and S550. In some embodiments, the geometric calibration apparatus may extract the sample by applying a weighted random sampling technique to the probability distribution. Various weighted random sampling techniques including, for example, the technique presented in “Weighted random sampling with a reservoir” (Efraimidis, P. et al., Information Processing Letters, 97 5, 181-185) may be used.


In some embodiments, the geometric calibration apparatus may extract samples sj∈Ω (referred to as “second samples”) from a set Ω={1, . . . , M} such that a probability with which each element i of the set Ω is extracted for a given j is a probability Pij. In some embodiments, the geometric calibration apparatus may extract the sample sj using the weighted random sampling technique in order to achieve sampling according to the probability distribution. In this case, the geometric calibration apparatus may perform an operation of extracting the sample sj for each j(=1, . . . , N′). For example, when M=9 and N′=9, the samples sj may be extracted as shown in Table 2. In some embodiments, the weighted random sampling technique may allow the samples sj of a different pattern to be extracted the next time the sample sj is extracted.


















TABLE 2





i
j = 1
j = 2
j = 3
j = 4
j = 5
j = 6
j = 7
j = 8
j = 9







1
P11
P12
P13
P14
P15
P16
P17
P18
P19


2
P21
P22
P23
P24
P25
P26
P27
P28
P29


3
P31
P32
P33
P34
P35
P36
P37
P38
P39


4
P41
P42
P43
P44
P45
P46
P47
P48
P49


5
P51
P52
P53
P54
P55
P56
P57
P58
P59


6
P61
P62
P63
P64
P65
P66
P67
P68
P69


7
P71
P72
P73
P74
P75
P76
P77
P78
P79


8
P81
P82
P83
P84
P85
P86
P87
P88
P89


9
P91
P92
P93
P94
P95
P96
P97
P98
P99



s1 = 6
s2 = 3
s3 = 8
s4 = 1
s5 = 4
s6 = 9
s7 = 7
s8 = 2
s9 = 5









Next, the geometric calibration apparatus may extract samples {s′1, . . . , s′6}⊆Ω′ through sampling without replacement such that a probability with which each element j of a set Ω′={1, . . . , N′} is extracted from the set Ω′ is αPsj,j at S550. For example, the sample s′k may be extracted as shown in Table 3 based on the example of Table 2. That is, the geometric calibration apparatus may extract the samples {s′1, . . . , s′6} of size 6. In some embodiments, the geometric calibration apparatus may use a weighted random sampling technique to extract the samples s′k(k=1, . . . ,6). In some embodiments, the probability αPsj,j may be a probability into which the probability Psj,j at the sample sj (i.e., the probability with which the point q′j is a projection of the marker at sj) is normalized. In some embodiments, α denotes a normalization constant which may be given, for example, as a reciprocal of a sum of the probabilities at the samples sj as in Equation 12.











TABLE 3





j
sj
P







1
6
αP61


2
3
αP32


3
8
αP83


4
1
αP14


5
4
αP45


6
9
αP96


7
7
αP77


8
2
αP28


9
5
αP59




s′1 = 1, s′2 = 2, s′3 = 3, s′4 = 5, s′5 = 7, s′6 = 9
















α
=

1




j
=
1


N




P


s
j

,
j








Equation


12







Next, the geometric calibration apparatus creates correspondences between phantom markers determined based on the extracted samples among the plurality of phantom markers and points determined based on the extracted samples among the detected points, and transforms the correspondences to obtain a candidate projection matrix  at S560. In some embodiments, the geometric calibration apparatus may obtain the candidate projection matrix  by designating i(k)=ss′k′, j(k)=s′k (k=1, . . . , 6) using the extracted samples, creating six correspondences pi(k)↔q′j(k), and transforming the correspondences at S560. For example, the geometric calibration apparatus may create the correspondences as shown in Table 4 based on the example of Table 3. In some embodiments, as described above, the geometric calibration apparatus may apply DLT to the correspondences to obtain the projection matrix Â.











TABLE 4





k
pi(k)
q′j(k)







1
p6
q′1


2
p3
q′2


3
p8
q′3


4
p4
q′5


5
p7
q′7


6
p5
q′9









The geometric calibration apparatus calculates points {circumflex over (q)}i into which the markers pi are transformed by the candidate projection matrix Â, and calculates a difference d({circumflex over (Q)}, Q′) between a set of transformed points {circumflex over (Q)}={{circumflex over (q)}1, . . . , {circumflex over (q)}M} and a set of the detected points Q′={q′1, . . . , q′N′} at S570. The geometric calibration apparatus may calculate the difference between the two sets by various techniques. In some embodiments, as shown in FIG. 8, the geometric calibration apparatus may obtain a minimum-weight perfect matching for a bipartite graph G in which the sets {circumflex over (Q)} and Q′ are set as vertex sets and an edge ({circumflex over (q)}i, q′j) between a vertex of the set {circumflex over (Q)} and a vertex of the set Q′ has a weight wij=∥{circumflex over (q)}i−q′j2, and calculate a sum of weights obtained by the minimum-weight perfect matching as the difference d({circumflex over (Q)},Q′).


If the calculated difference d(({circumflex over (Q)},Q′) is less than or equal to a threshold (d≤d0) at S580, the geometric calibration apparatus designates the candidate projection matrix  as the projection matrix A at S590. If the calculated difference d({circumflex over (Q)},Q′) is greater than the threshold (d>d0) at S580, the geometric calibration apparatus may discard the candidate projection matrix  and repeat the process from S540.


As described above, according to some embodiments, the geometric calibration phantom can be designed with almost no constraints, and additional information other than the phantom design is not required during geometric calibration. Accordingly, the user can arbitrarily dispose the phantom markers on the space as needed. According to some embodiments, the geometric calibration phantom can be designed with only one type of phantom markers, without needing, for example, to adopt different sizes to allow the phantom markers to be distinguished from each other. According to some embodiments, the geometric calibration can be achieved by using only the phantom marker position information without needing to know approximate information about the imaging directions in advance.


In some embodiments, the phantom design and fabrication, and training of the learning model can be performed only once for each phantom, as a pre-work for the geometric calibration. Based on the phantom and the learning model, the geometric calibration can be performed for each X-ray image acquired by a computer tomography device.


Hereinafter, an example computing device for implementing a learning apparatus or a geometric calibration apparatus according to various embodiment of the present invention is described with reference to FIG. 9.



FIG. 9 is a drawing showing an example of a computing device according to an embodiment of the present invention.


Referring to FIG. 9, a computing device includes a processor 910, a memory 920, a storage device 930, a communication interface 940, and a bus 950. The computing device may further include other general components.


The processor 910 controls an overall operation of each component of the computing device. The processor 910 may be implemented with at least one of various processing units such as a central processing unit (CPU), an application processor (AP), a microprocessor unit (MPU), a micro controller unit (MCU), and a graphic processing unit (GPU), or may be implemented with parallel processing units. In addition, the processor 910 may perform operations on a program for executing the above-described training data generation method, learning method, or geometric calibration method.


The memory 920 stores various data, instructions, and/or information. The memory 920 may load a computer program from the storage device 930 to execute the above-described training data generation method, learning method, or geometric calibration method. The storage device 930 may non-temporarily store a program. The storage device 930 may be implemented as a non-volatile memory.


The communication interface 940 supports wired or wireless Internet communication of the computing device. In addition, the communication interface 940 may support various communication methods other than Internet communication.


The bus 950 provides a communication function between the components of the computing device. The bus 950 may be implemented as various types of buses, such as an address bus, a data bus, and a control bus.


The computer program may include instructions for causing the processor 910 to execute the training data generation method, the learning method, or the geometric calibration method when loaded into the memory 920. That is, the processor 910 may perform the training data generation method, the learning method, or the geometric calibration method by executing the instructions.


In some embodiments, the computer program may include one or more instructions of detecting a plurality of first points from a plurality of projection regions, respectively, a plurality of first markers disposed on a phantom being projected onto the projection regions in two dimensions by a computer tomography device, calculating an output vector representing a probability distribution that gives a probability with which each of the first points is a projection of each of the first markers, by inputting data corresponding to each of the first points to a learning model, extracting a predetermined number of first samples based on the probability distribution, obtaining a candidate projection matrix by transforming correspondences between second markers determined based on the first samples among the first markers and second points determined based on the first samples among the plurality of first points, calculating third points into which the first markers are transformed by the candidate projection matrix, calculating a difference between a first set of the third points and a second set of the first points, and designating the candidate projection matrix as a projection matrix in response to the difference being less than or equal to a threshold.


In some embodiments, the computer program may include one or more instructions of generating training data that have, as input data, data corresponding to fourth points into which the first markers are respectively transformed by a randomly-generated projection matrix and have, as correct answer labels, values indicating classification of the first markers.


In some embodiments, a computer program may include one or more instructions for training the learning model using the training data.


In some embodiments, the computer program may include one or more instructions of detecting a plurality of points q′j (j=1, . . . , N′) from a plurality of projection regions, respectively, a plurality of first markers disposed on a phantom being projected onto the projection regions in two dimensions by the computer tomography device, calculating an output vector yj=(P1j, . . . , PMj) having, as elements, a probability Pij (i=1, . . . , M) with which the q′j is a projection of each of the first markers, by inputting data corresponding to the q′j to a learning model, extracting samples s′k (k=1, . . . , 6) from a set Ω′={1, . . . , N′} based on the yj, obtaining a candidate projection matrix by transforming correspondences between second markers determined based on the samples s′k among the first markers and points determined based on the samples s′k among the q′j, calculating points {circumflex over (q)}i into which the first markers are transformed by the candidate projection matrix, calculating a difference d({circumflex over (Q)}, Q′) between a set {circumflex over (Q)}={{circumflex over (q)}1, . . . , {circumflex over (q)}M} of the {circumflex over (q)}i and a set Q′={q′1, . . . , q′N′,} of the q′j, and designating the candidate projection matrix as a projection matrix in response to the d({circumflex over (Q)}, Q′) being less than or equal to a threshold.


The training data generation method, the learning method or the geometric calibration method according to various embodiments may be implemented as a computer-readable program on a computer-readable medium. In one embodiment, the computer-readable medium may include a removable recording medium or a fixed recording medium. In another embodiment, the computer-readable program recorded on the computer-readable medium may be transmitted to another computing device via a network such as the Internet and installed in another computing device, so that the computer program can be executed by another computing device.


While this invention has been described in connection with what is presently considered to be practical embodiments, it is to be understood that the invention is not limited to the disclosed embodiments. On the contrary, it is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims
  • 1. A geometric calibration apparatus comprising: a memory configured to store one or more instructions; anda processor configured to, by executing the one or more instructions: detect a plurality of first points from a plurality of projection regions, respectively, a plurality of first markers disposed on a phantom being projected onto the projection regions in two dimensions by a computer tomography device;calculate an output vector representing a probability distribution that gives a probability with which each of the first points is a projection of each of the first markers, by inputting data corresponding to each of the first points to a learning model;extract a predetermined number of first samples based on the probability distribution;obtain a candidate projection matrix by transforming correspondences between second markers determined based on the first samples among the first markers and second points determined based on the first samples among the first points;calculate third points into which the first markers are transformed by the candidate projection matrix;calculate a difference between a first set of the third points and a second set of the first points; anddesignate the candidate projection matrix as a projection matrix in response to the difference being less than or equal to a threshold.
  • 2. The geometric calibration apparatus of claim 1, wherein the processor is configured to transform each of the first points into a histogram, and wherein the data corresponding to each of the first points may be data obtained by being transformed into the histogram.
  • 3. The geometric calibration apparatus of claim 1, wherein the processor is configured to extract the first samples by applying weighted random sampling to the probability distribution.
  • 4. The geometric calibration apparatus of claim 1, wherein the processor is configured to: extract a plurality of second samples for the first points, respectively, from a third set including indices of the first markers; andextract the first samples from a fourth set including indices of the first points such that a probability with which each element of the fourth set is extracted is a normalized value of a probability with which a first point corresponding to said each element among the first points is a projection of a marker at a second sample for the first point among the second samples.
  • 5. The geometric calibration apparatus of claim 4, wherein the processor is configured to extract the first samples and the second samples using a weighted random sampling technique.
  • 6. The geometric calibration apparatus of claim 4, wherein the second markers are markers that have, as indices, second samples corresponding to the first samples among the second samples, among the first markers, and wherein the second points are points having the first samples as indices among the first points.
  • 7. The geometric calibration apparatus of claim 1, wherein the processor is configured to repeat a process of extracting the first samples, obtaining the candidate projection matrix, calculating the second points, and calculating the difference, in response to the difference being greater than the threshold.
  • 8. The geometric calibration apparatus of claim 1, wherein the processor is configured to: calculate a minimum-weight perfect matching for a bipartite graph in which the first set and the second set are set as vertex sets and an edge between a vertex of the first set and a vertex of the second set has a weight; andcalculate a sum of weights obtained by the minimum-weight perfect matching as the difference.
  • 9. The geometric calibration apparatus of claim 1, wherein the learning model is a model that has been trained by using training data that have, as input data, data corresponding to fourth points into which the first markers are respectively transformed by a randomly-generated projection matrix and have, as correct answer labels, values indicating classification of the first markers.
  • 10. The geometric calibration apparatus of claim 9, wherein the data corresponding to the fourth points are data obtained by transforming the fourth points into a histogram.
  • 11. The geometric calibration apparatus of claim 1, wherein the processor is configured to apply direct linear transformation to the correspondences to obtain the candidate projection matrix.
  • 12. A geometric calibration method of a computer tomography device, the method comprising: detecting a plurality of points q′j(j=1, . . . , N′) from a plurality of projection regions, respectively, a plurality of first markers disposed on a phantom being projected onto the projection regions in two dimensions by the computer tomography device;calculating an output vector yj=(P1j, . . . , PMj) having, as elements, a probability Pij(i=1, . . . , M) with which the q′j is a projection of each of the first markers, by inputting data corresponding to the q′j to a learning model;extracting samples s′k(k=1, . . . , 6) from a set Ω′={1, . . . , N′} based on the yj;obtaining a candidate projection matrix by transforming correspondences between second markers determined based on the samples s′k among the first markers and points determined based on the samples s′k among the q′j;calculating points {circumflex over (q)}i into which the first markers are transformed by the candidate projection matrix;calculating a difference d({circumflex over (Q)}, Q′) between a set {circumflex over (Q)}={{circumflex over (q)}1, . . . , {circumflex over (q)}M} of the {circumflex over (q)}i and a set Q′={q′1, . . . , q′N′} of the q′j; anddesignating the candidate projection matrix as a projection matrix in response to the d ({circumflex over (Q)}, Q′) being less than or equal to a threshold.
  • 13. The method of claim 12, further comprising transforming the q′j into a histogram, wherein the data corresponding to the q′j are data obtained by being transformed into the histogram.
  • 14. The method of claim 12, wherein extracting the s′k comprises: extracting samples sj∈Ω from a set Ω={1, . . . , M} such that a probability with which each element i of the set Ω is extracted for a given j is the Pij;normalizing a probability Psj,j at the sj into αPsj,j; andextracting the s′k such that a probability with which each element j of the Ω′ is extracted from the Ω′ is αPsjj.
  • 15. The method of claim 14, wherein the α is a reciprocal of a sum of the Psj,j at the sj.
  • 16. The method of claim 12, wherein calculating the d({circumflex over (Q)}, Q′) comprises: calculating a minimum-weight perfect matching for a bipartite graph in which the {circumflex over (Q)} and the Q′ are set as vertex sets and an edge between a vertex {circumflex over (q)}i of {circumflex over (Q)} and a vertex of q′j of Q′ has a weight wij=∥{circumflex over (q)}i−q′j∥2; andcalculating a sum of weights obtained by the minimum-weight perfect matching as the d({circumflex over (Q)}, Q′).
  • 17. The method of claim 12, further comprising, in response to the d({circumflex over (Q)}, Q′) being greater than the threshold, repeating extracting the s′k, obtaining the candidate projection matrix, calculating the {circumflex over (q)}i, and calculating the d({circumflex over (Q)}, Q′).
  • 18. A computer program executed by a computing device and stored in a non-transitory recording medium, the computer program causing the computing device to execute: detecting a plurality of first points from a plurality of projection regions, respectively, a plurality of first markers disposed on a phantom being projected onto the projection regions in two dimensions by a computer tomography device;calculating an output vector representing a probability distribution that gives a probability with which each of the first points is a projection of each of the first markers, by inputting data corresponding to each of the first points to a learning model;extracting a predetermined number of first samples based on the probability distribution;obtaining a candidate projection matrix by transforming correspondences between second markers determined based on the first samples among the first markers and second points determined based on the first samples among the plurality of first points;calculating third points into which the first markers are transformed by the candidate projection matrix;calculating a difference between a first set of the third points and a second set of the first points; anddesignating the candidate projection matrix as a projection matrix in response to the difference being less than or equal to a threshold.
Priority Claims (1)
Number Date Country Kind
10-2021-0080024 Jun 2021 KR national
US Referenced Citations (17)
Number Name Date Kind
7016456 Basu et al. Mar 2006 B2
7147373 Cho et al. Dec 2006 B2
7844094 Jeung et al. Nov 2010 B2
7950849 Claus et al. May 2011 B2
8777485 Holt Jul 2014 B2
9572539 Carrat Feb 2017 B2
10311602 Choi Jun 2019 B2
10478147 Hawker et al. Nov 2019 B2
10643354 Wang et al. May 2020 B2
10835765 Tulik et al. Nov 2020 B2
11110301 Yang Sep 2021 B2
20050117708 Cho Jun 2005 A1
20120259204 Carrat Oct 2012 A1
20150216498 Schulze Aug 2015 A1
20170238897 Siewerdsen Aug 2017 A1
20180197315 Choi Jul 2018 A1
20200085404 Siewerdsen Mar 2020 A1
Foreign Referenced Citations (7)
Number Date Country
1760457 Mar 2007 EP
10-1027099 Apr 2011 KR
10-1524937 Jun 2015 KR
10-2018-0082027 Jul 2018 KR
10-2018-0117323 Oct 2018 KR
10-2019-0123865 Nov 2019 KR
2016-148351 Sep 2016 WO
Non-Patent Literature Citations (9)
Entry
Massimiliano Ferrucci et al., “Towards geometrical calibration of x-ray computed tomography systems—a review” Measurement Science and Technology 26.9 (Aug. 19, 2015): 092003.
Boris P. Selby et al., “Geometry calibration for x-ray equipment in radiation treatment devices and estimation of remaining patient alignment errors” Proc. SPIE 6913, Medical Imaging 2008: Physics of Medical Imaging. vol. 6913. SPIE (Mar. 2008).
Bernhard E. H. Claus, “Geometry calibration phantom design for 3D imaging” Proc. SPIE 6142, Medical Imaging 2006: Physics of Medical Imaging. vol. 6142 61422E (Feb. 28, 2006).
EPO, the extended European search report of the corresponding Application No. 22179761.6, dated Nov. 14, 2022.
Serge Belongie et al., “Shape context: A new descriptor for shape matching and object recognition”, Advances in Neural Information Processing Systems 13, 831-837, 2000.
Pavlos S. Efraimidis et al., “Weighted random sampling with a reservoir”, Information Processing Letters 97 (2006) 181-185, Dec. 2005.
Frederic Noo et al., “Analytic method based on identification of ellipse parameters for scanner calibration in cone-beam tomography” Physics in Medicine and Biology, vol. 45 (2000) 3489-3508, Nov. 2000.
Youngbin Cho et al., “Accurate technique for complete geometric calibration of cone-beam computed tomography systems” Medical Physics, vol. 32, No. 4, p. 968, Apr. 2005.
Xinhua Li et al., “A generic geometric calibration method for tomographic imaging systems with flat-panel detectors—A detailed implementation guide” Medical Physics, vol. 37, No. 7, p. 3844, Jul. 2010.
Related Publications (1)
Number Date Country
20220405970 A1 Dec 2022 US