The present disclosure relates to a method and a system for extracting a shape of an actual object, based on a radiation projection image of the actual object.
X-ray CT (Computed Tomography) is a technology with which it is possible to reconstruct an image of the inside of a subject (cross sectional image) based on X-ray intensity distribution (X-ray projection image) obtained by irradiating the subject (physical object) with X-rays from many directions. This type of CT has conventionally been widely used in healthcare applications, and recently has also been used in industry. By using X-ray CT, it is possible to identify a surface shape that cannot be seen from the outside, which makes it useful, for example, in the dimensional inspection and reverse engineering of products having complicated shapes.
In the reconstruction of cross-sectional images in X-ray CT, a regular lattice structure, for example, a mesh formed from a square lattice, is assumed. If a CT value of the outside of a component is made 0 and the CT value of the inside is made 1, in the case where a boundary line that delimits a surface shape (boundary line that separates the component and the outside) passes through a mesh, a CT value of that mesh will be greater than 0 and less than 1, and the surface shape will become blurred. In particular, there are many instances where if shapes of corner sections of a component are blurred, it makes identifying a shape difficult. With healthcare use, it is less problematic to have blurred components, but in industrial use, a solution to the problem of blurred images is necessary in order to identify the surface shape of a product.
On the other hand, the following foreign patent literature describes technology for acquiring images in conformity with the shape of a subject by causing lattice points to move in conformity with a specified shape: Japanese Patent Publication Nos. 2003-123057, 2002-24855, and 2002-74386. However, with each of these technologies, since it is a prerequisite that the shape of a subject has been obtained in advance in some way, it is difficult to apply to a subject of unknown shape. Also, the following foreign patent literature, describes technology for causing the lattice points to move based on a feature plane or feature edge line that has been designated by user: Japanese Patent No. 2007-102595. However, with this technology, even if it is possible to smooth the exterior appearance of an acquired subject shape, it is not always possible to match an acquired subject shape to an unknown subject shape. Also, the technology of Japanese Patent No. 2007-102595 is not for carrying out lattice deformation for matching to an unknown subject shape using values of an acquired cross-sectional image, which means it is not suitable for automatic or mechanical lattice point movement.
The present disclosure has been conceived in view of the above-described situation. An object of the present disclosure is to provide technology that can extract an unknown surface shape of a physical object with good precision.
Means for solving the above-described problem can be described as in the following aspects.
Aspect 1
A shape-extraction method comprising:
(1) a step of receiving image data of a projective image that has been acquired by radiation projection to an object;
(2) a step of using a predetermined mesh structure to acquire a cross-sectional image of the object from the image data of the projective image by tomographic reconstruction;
(3) a step of causing a lattice point that forms the mesh structure to move in conformity with the surface shape of the object based on the cross-sectional image that has been acquired by the reconstruction; and
(4) a step of carrying out step (2) using a mesh that has had the position of the lattice point corrected using step (3), and then repeatedly executing step (3) and step (2) as many times as required.
Here, the radiation is a concept that includes x-rays, a neutron beam, and an electron beam.
Aspect 2
The shape-extraction method of aspect 1, wherein the processing for causing the lattice point to move in conformity with the surface shape of the object, in step (3), includes the following steps:
(3a) a step of approximating a boundary line between the object and portions outside of the object that passes through a convex polygon constituting the mesh structure and that shares a specified lattice point, to a straight line for each convex polygon; and
(3b) a step of moving the specified lattice point to an intersection point of the straight line for each convex polygon, or to a point approximated to the intersection point.
Here, the convex polygon is a concept that includes a triangle. The convex polygon also includes a convex quadrilateral, pentagon, hexagon, etc.
Aspect 3
The shape extraction method of aspect 2, wherein the processing to approximate a boundary line between the object and a portion outside of the object to a straight line for each convex polygon, in step (3a), includes the following steps:
(3a1) a step of obtaining a normal to the boundary line, based on a gradient vector of image data of the convex polygon; and
(3a2) a step of obtaining a position of the boundary line such that a ratio of divided surface areas of the convex polygon that has been divided by the boundary line corresponds to a value of image data of the convex polygon.
Aspect 4
The shape-extraction method of aspect 1, wherein the mesh structure is a tetrahedron mesh for producing a three-dimensional shape.
Aspect 5
The shape-extraction method of aspect 4, wherein the processing for causing the lattice point to move in conformity with the surface shape of the object, in step (3), includes the following steps:
(3p) a step of obtaining barycentric positions for two adjacent tetrahedron meshes that also include a boundary surface;
(3q) a step of obtaining a point constituting a predetermined CT value, between the barycentric positions of two tetrahedron meshes, as a barycenter of a triangle constituting a boundary mesh;
(3r) a step of obtaining orientation of the boundary mesh such that a normal direction of a triangle constituting the boundary mesh conforms to a gradient direction of the CT values of the two adjacent tetrahedron meshes; and
(3s) a step of moving the lattice point using an apex position of the triangle constituting the boundary mesh that was obtained in steps 3q and 3r.
Aspect 6
A shape-extraction system, provided with a projective image-receiving section, a cross section reconstruction section, a lattice point movement section, and a surface shape output section, wherein:
the projective image-receiving section receives image data of a projective image that has been acquired by radiation projection to an object,
the cross section reconstruction section uses a predetermined mesh structure to acquire a cross-sectional image of the object from the image data of the projective image by reconstruction using tomography,
the lattice point movement section is configured to carry out:
(1) processing to cause a lattice point that forms the mesh to move in conformity with the surface shape of the object, based on the cross-sectional image that has been acquired by the reconstruction, and
(2) processing to send a mesh that has had the lattice point position corrected by the processing of (1) above to the cross section reconstruction section, and acquiring an updated cross-sectional image from the cross section reconstruction section, and
the surface shape output section outputs the updated cross-sectional image.
Aspect 7
The shape-extraction system of aspect 6, wherein the mesh structure is a tetrahedron mesh for producing a three-dimensional shape.
Aspect 8
A computer program for executing the shape-extraction method of any one of aspect 1 to aspect 5 using a computer.
According to the present disclosure, it is possible to provide technology that can extract an unknown surface shape of a physical object with good precision. Also, according to the present disclosure, since movement of lattice points is carried out based on cross-sectional images that have been acquired by reconstruction, it is easy to automate movement of lattice points.
The foregoing aspects and many of the attendant advantages of this invention will become more readily appreciated as the same become better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, wherein:
A shape-extraction system and a shape extraction device of a first embodiment of the present disclosure will be described in the following with reference to
A shape-extraction system of this embodiment is provided with a projective image-receiving section 1, a cross section reconstruction section 2, a lattice point movement section 3, and a surface shape output section 4. Each of these functional elements can basically be configured using computer programs.
The projective image-receiving section 1 receives image data of a projective image that has been acquired by radiation projection to an object. Here, the radiation is not only x-rays, but also includes a neutron beam and an electron beam. Also, the projection image is not only a transmission image, but also includes a reflected image.
The cross section reconstruction section 2 uses a predetermined mesh structure to acquire cross-sectional images of the subject from image data of the projective image by reconstruction using tomography (known as CT).
The lattice point movement section 3 is configured to carry out the following: (1) processing to move lattice points constituting the mesh in conformity with the surface shape of the object, based on cross-sectional images that have been acquired by reconstruction, and (2) processing to send a mesh that has had the lattice point position corrected by the processing of (1) above to the cross section reconstruction section 2, and acquiring an updated cross sectional image from the cross section reconstruction section 2.
Detailed operation of the lattice point movement section 3 will be described later.
The surface shape output section 4 outputs the updated cross-sectional image to a suitable memory or to a subsequent process.
Next, a shape-extraction method that uses the previously described system will be described in detail with reference to the flowchart of
Step SA-1 in
First, an x-ray projection image (with this embodiment, an x-ray transmission image) is obtained by a projection of irradiation to an object. A method of acquiring the x-ray projection image will be described with reference to
With the system of this embodiment, image data for a projection image that has been created as described above is received by the projective image-receiving section 1.
Step SA-2 in
Next, a predetermined mesh structure is used to acquire cross-sectional images of the subject from image data of the projective images by reconstruction using tomography. This step will be further described with reference to
Step SA-3 in
Next, lattice points constituting the mesh structure are moved in conformity with the surface shape of the object, based on the cross-sectional image that has been acquired by reconstruction. A procedure for moving the lattice points will be described in detail with reference to
Step SB-1 in
First, a boundary line passing through each triangle constituting the mesh structure is approximated to a straight line. As will be understood, the position of the actual boundary line is unclear in this step, and so some sort of estimation is necessary. With this embodiment therefore, as described in the following, a method is used whereby first a normal of the boundary line is calculated, and secondly a position of a straight line is calculated. It is possible to reverse the order of these calculations.
Calculation of the Boundary Line Normal
A normal of the estimated boundary line can be calculated using a gradient vector of the CT value.
First, it is assumed that a straight line approximation of a boundary of triangle T is a set of points x that satisfy the following equation:
nT·(x−pT)=0
[nT: unit normal vector, pT: point where straight line passes through]
Then, the entire mesh is subjected to translation by a microscopic amount in a positive direction of each axis (with this example, the x-axis and the y-axis), and CT reconstruction is carried out. Here, the amount of movement in the x-axis direction is made δx, and the amount of movement in the y-axis direction is made δy.
Next, a unit normal vector nT for the triangle T is calculated from the following equation (refer to
nT=(ITδx−IT,ITδy−IT)/√{square root over ((ITδx−IT)2+(ITδx−IT)2)}{square root over ((ITδx−IT)2+(ITδx−IT)2)}
IT: CT value with no translation
ITδx: CT value of mesh that has been moved in the x axis direction
ITδy: CT value of mesh that has been moved in the y axis direction
In this way it is possible to obtain a normal to the estimated boundary line. Specifically, it is possible to determine orientation of the boundary line using this normal.
Estimation of Boundary Line Position
The position of the boundary line can be estimated as follows:
First, as described above, a straight line approximation of a boundary surface of triangle T is a set of points x that satisfy the following equation.
nT·(x−pT)=0
[nT: unit normal vector, pT point passing through straight line]
Then, point pT is placed on an edge, of the three edges of triangle T, where an orthogonal projection to a straight line having a direction of vector nT becomes longest, and a ratio of surface areas of the triangle that has been divided by the straight line (specifically, an internal ratio) satisfies the following equation (refer to
AT+:AT−=W−IT:IT−B
AT+: for T divided by a straight line, surface area of forward positive side of nT
AT−: for T divided by a straight line, surface area of negative direction side of nT
IT: CT value of triangle
W: CT value (constant) of object
B: CT value (constant) of air
Using this condition, it is possible to estimate the position of a boundary line between the object and the air. If the position and orientation of the boundary line are known, it is possible to acquire a straight line that passes through the triangular region (namely, the boundary line).
Step SB-2 in
Next, intersection points of the straight lines that have been acquired for each region, or points that approximate thereto, are obtained. Specifically, it is possible to carry out calculations as follows.
Lattice point v is moved to a point x where E(x) below is minimum. Here, since the three or more straight lines generally do not cross at a single point; with this embodiment, a point where a square sum of distance to the straight lines is the minimum is substituted for the single point (refer to the following equation):
N(v): set of triangles having lattice point v as apex and CT value that is blurred
In this way, it is possible to place a single lattice point on the estimated boundary line. By applying this operation to the triangular meshes that are blurred, it is possible to move each lattice point. If the moved lattice points are connected, it is possible to acquire the estimated boundary line.
Step SB-3 in
Next, lattice points that have been moved with the previously described procedure are further moved on the boundary line. Specifically, a barycentric position for lattice points surrounding the lattice point is first calculated. Then, the lattice point is moved to a position that is closest to the barycentric point on the boundary line. In this way, there is the advantage that it is easy to carry out reconstruction using an ML-EL method.
Step SA-4 in
Next, it is determined whether or not the lattice point has actually moved in steps up to the aforementioned. In the case where the lattice point has moved, a moved lattice point is assumed and processing of the previous step, SA-2 to SA-4, is repeated. The result of carrying out lattice movement and repeating the reconstruction is shown in
In the case where the lattice point has not moved, a finally obtained cross-sectional image is output. The output destination can be set in accordance with the intended use of the data, but is, for example, a memory for data storage or a process for additional processing.
With conventional CT reconstruction, calculation was performed using a square and regular lattice. Differing from this, with the procedure of the embodiment described above, the shape of the lattice is a convex polygon, and calculation for reconstruction is carried out using an irregular (unstructured) lattice (although it is also possible to use a regular lattice in an initial state). Further, with this embodiment, the shape of this lattice can be changed dynamically. These points constitute the feature of the above-described procedure.
Next, with reference to
With the conventional method
According to the method of this embodiment, therefore, there is the advantage that it is possible in an industrial situation, for example, to accurately acquire the shape of a component, being the object, in an industrial application.
Next, a shape-extraction method and shape extraction device of a second embodiment of the present disclosure will be described with reference to
With the above-described first embodiment, it was possible to extract the surface shape of a subject by using a triangular mesh as a mesh structure. Differing from this, the second embodiment described in the following extracts a solid shape by using a tetrahedron mesh as the mesh structure. Also, with the first embodiment an ML-EL method was adopted as the CT reconstruction method. Differing from this, with the second embodiment an already-known Feldkamp method (Feldkamp, L. A., et al., “Practical Conebeam Algorithm,” J. Opt. Soc. Am. a, 1:612-619, 1984.) is used. In this way it is possible, with the second embodiment, to obtain CT values f(c) at a barycenter c of each tetrahedron constituting the mesh. Here, a CT value f(c) at the barycenter of each tetrahedron can be treated as a CT value representing the element (the mesh).
However, even in the case of extracting a solid shape, the device and method for extracting a solid shape are basically common to the above-described first embodiment. Specifically, each of the steps shown in
2D3D
polygonpolyhedron
Triangletetrahedron
Straight lineplanar surface
boundary lineboundary surface
surface areavolume
It is therefore necessary to slightly change the steps shown in SB-1 to SB-3 in
Step SC-1 in
First, processing to approximate a boundary surface passing through each tetrahedron constituting the mesh structure to a planar surface will be described.
Detection of Material Boundary Using Binarization
First, detection of a material boundary is carried out using CT values inherent to the material (already known). In the case where CT values for two types of material A and B are already known, a CT value Ft at a boundary surface between material A and material B is obtained as an average value of the CT values for A and B. With this example, since a component of a single material that has been placed in the atmosphere is assumed, material A is air and material B is the component. By comparing this value and the CT value of each tetrahedron, it is possible to differentiate a tetrahedron included in material A and a tetrahedron included in material B. It is then possible to obtain a boundary mesh M as a set of triangles (triangles constituting surfaces of the tetrahedrons), commonly contained in tetrahedrons of material A and tetrahedrons of material B. Accordingly, an individual boundary mesh is a planar surface mesh of triangles. In this state, the boundary line surface usually passes through the inner part of a tetrahedron mesh positioned on the boundary, but as will be understood, at what position of the mesh it passes is not known. Accordingly, at this point in time, a boundary mesh M does not usually conform to the boundary surface of the subject body.
Calculation of Planar Surface Representing Boundary Surface
Processing to approximate a position of the boundary mesh M to an ideal boundary surface by moving the position of the obtained boundary mesh M will be described in the following. As a prerequisite, the optimum position of the boundary mesh M is considered to be as follows. Specifically, in order to be the optimum (namely, corresponding to a boundary surface) boundary mesh M, the following are necessary:
a barycenter m, of a triangle constituting the boundary mesh must be on the material boundary; and
a normal of that triangle must be equivalent to a gradient direction of a CT value of that barycentric point (namely, a direction in which CT value increases or decreases).
How the barycentric position and normal direction of the detected boundary surface change at the correct material boundary is schematically shown in
Calculation of Optimal Material Boundary
A position m′I of a barycenter of a triangle constituting an optimum boundary mesh M is a point between barycenters ci and cj of two tetrahedrons including the boundary, where f(m′i)=Ft (refer to
Ft is generally a number that a user can easily designate. For example, in the case where two things that touch at the boundary are air and an object, then since respective CT values are themselves generally known, it is possible to make Ft between those two values.
Also, a normal to a triangle constituting the optimum boundary mesh is a vector n(m′i) for which a gradient ∇f(m′i) of a CT value for an obtained position m′i has been normalized. ∇f(m′i) can be obtained using central difference, as in the following equation:
As a result of the above described processing, it is possible to acquire a boundary mesh M positioned on the material boundary surface. Specifically, it becomes possible to approximate the boundary surface to a planar surface.
Step SC-2 in
Movement of Apex
Apexes of adjacent meshes M are expected to align at both ends of edges constituting the boundary. However, with a boundary mesh that has been acquired as described above, the apexes do not generally align.
QEM (quadric error metrics) is therefore used in order to cause the apexes of the boundary mesh M to move to an optimum position. Specifically, a position at which E(p), being a square of distance between each apex point p of the boundary mesh M and a planar surface of a boundary mesh that has been estimated as described above becomes minimum, is obtained. E(p) is represented by the following formula:
N(p) here represents a group of triangles that are adjacent to the apex p, while m′i and n(m′i) represent optimum barycentric position and normal of respective triangles ti.
Re-Mesh
In the case of moving an apex using QEM, there may be cases in which a drawback arises where the mesh is reversed. In this type of case, it is preferable to carry out generating a new mesh (known as re-mesh). An already known re-mesh method can be used, and so detailed description will be omitted.
Step SC-3 in
Next, using E(p) obtained as described above, an extension point is moved such that that value becomes minimum. This step corresponds to step SB-3 in
Subsequent processing is the same as that after SA-4 in
The present disclosure is not limited to the above-described embodiments, and various modifications can additionally be obtained within a scope that does not depart from the spirit of the disclosure.
For example, each of the above-described structural elements can exist as a functional block, and can also exist as independent hardware. Also, as a method of implementation, it is possible to use hardware or to use computer software. Further, a single functional element of the present disclosure may be realized as a set of a plurality of functional elements, and a plurality of functional elements of the present disclosure may be implemented by a single functional element.
It is also possible for each functional element constituting the present disclosure to exist separately. In the case of existing separately, necessary data can be exchanged by means of a network, for example. Similarly, it is also possible for each function of an internal part of each section to exist separately. For example, it is possible to implement each functional element, or some of the functional elements, of this embodiment using grid computing or cloud computing.
Also, in the above-described first embodiment, a mesh is constructed using triangles, but it is also possible to use different convex polygons instead of triangles.
Number | Date | Country | Kind |
---|---|---|---|
2011-150024 | Jul 2011 | JP | national |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/JP2012/054889 | 2/28/2012 | WO | 00 | 2/19/2014 |
Publishing Document | Publishing Date | Country | Kind |
---|---|---|---|
WO2013/005455 | 1/10/2013 | WO | A |
Number | Name | Date | Kind |
---|---|---|---|
7203267 | De Man | Apr 2007 | B2 |
Number | Date | Country |
---|---|---|
09-073558 | Mar 1997 | JP |
11-025293 | Jan 1999 | JP |
2002-024855 | Jan 2002 | JP |
2002-074386 | Mar 2002 | JP |
2003-123057 | Apr 2003 | JP |
2005-245830 | Sep 2005 | JP |
2007-102595 | Apr 2007 | JP |
2009083864 | Jul 2009 | WO |
Entry |
---|
Feldkamp, L. A., et al., “Practical Cone-Beam Algorithm,” Journal of the Optical Society of America A 1(6):612-619, Jun. 1984. |
Garland, M., and P. S. Heckbert, “Surface Simplification Using Quadric Error Metrics,” Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '97), ACM Press/Addison-Wesley Publishing, N.Y., pp. 209-216. |
Lorensen, W. E., and H. E. Cline, “Marching Cubes: A High Resolution 3D Surface Construction Algorithm,” Computer Graphics 21(4):163-169, Jul. 1987. |
Sitek, A., et al., “Tomographic Reconstruction Using an Adaptive Tetrahedral Mesh Defined by a Point Cloud,” IEEE Transactions on Medical Imaging 25(9):1172-1179, Sep. 2006. |
Yu, Y., et al., “Mesh Editing With Poisson-Based Gradient Field Manipulation,” ACM Transactions on Graphics—Proceedings of ACM SIGGRAPH 23(3):644-651, Aug. 2004. |
International Search Report mailed Apr. 24, 2012, in International Application No. PCT/JP2012/054889, filed Feb. 28, 2012, 2 pages. |
Extended European Search Report mailed Feb. 2, 2015, issued in corresponding European Application No. EP 12 808 211.2, filed Feb. 28, 2012, 5 pages. |
Soussen, C., and A. Mohammad-Djafari, “Polygonal and Polyhedral Contour Reconstruction in Computed Tomography,” IEEE Transactions on Image Processing 13(11):1507-1523, Nov. 2004. |
Number | Date | Country | |
---|---|---|---|
20140169651 A1 | Jun 2014 | US |