The present invention relates to a three-dimensional image processing apparatus, a three-dimensional image processing method, and a three-dimensional image processing program for generating three-dimensional shape model data from three-dimensional image data.
As a conventional three-dimensional image processing apparatus of this type, for example, a stereoscopic model data generation apparatus disclosed in PTL 1 is known. In this stereoscopic model data generation apparatus, a liver region extraction section and a structure extraction section extract a structure such as a liver region, hepatic artery, or hepatic vein from three-dimensional image data, and a surface data generation section generates surface data of the liver region and the structure. A pattern imparting section imparts an uneven pattern to at least one of the surfaces of the liver region and the structure, and a data generation section synthesizes the surface data of the liver region and the structure to which the uneven pattern has been imparted to generate stereoscopic model data. A stereoscopic model creation apparatus creates a stereoscopic model of the liver on the basis of the stereoscopic model data.
As a conventional modeled object generated on the basis of three-dimensional shape model data, for example, a three-dimensional stereoscopic model disclosed in PTL 2 is known. This three-dimensional stereoscopic model is formed of a soft material that represents an external structure of a stereoscopic object, and an external color and an internal structure and an internal color that cannot be observed from the outside, and the internal structure and the internal color can be observed by cutting and opening the soft material.
Meanwhile, metadata is added to three-dimensional image data in some cases. However, metadata has been lost in the process of creating three-dimensional shape model data from three-dimensional image data in a conventional three-dimensional image processing apparatus. Therefore, in order for a user to confirm a content of the metadata after creation of the three-dimensional shape model data, troublesome work such as checking by using the three-dimensional image data again is necessary.
Accordingly, an object of the present invention is to provide a three-dimensional image processing apparatus, a three-dimensional image processing method, and a three-dimensional image processing program capable of generating more convenient three-dimensional shape model data.
A first aspect of the present invention is directed to a three-dimensional image processing apparatus including: an acquisition section that acquires three-dimensional image data including a three-dimensional image and metadata; a separation section that separates the three-dimensional image data acquired by the acquisition section into the three-dimensional image and the metadata; a synthesis section that synthesizes information represented by the metadata separated by the separation section with the three-dimensional image separated by the separation section; and a conversion section that converts the three-dimensional image synthesized with the information represented by the metadata in the synthesis section into three-dimensional shape model data of a predetermined file format and of an object represented by the three-dimensional image.
A second aspect of the present invention is directed to a three-dimensional image processing method including: an acquisition step of acquiring three-dimensional image data including a three-dimensional image and metadata; a separation step of separating the three-dimensional image data acquired in the acquisition step into the three-dimensional image and the metadata; a synthesis step of synthesizing information represented by the metadata separated by the separation step with the three-dimensional image separated by the separation step; and a conversion step of converting the three-dimensional image synthesized with the information represented by the metadata in the synthesis step into three-dimensional shape model data of a predetermined file format and of an object represented by the three-dimensional image.
A third aspect of the present invention is directed to a three-dimensional image processing program that causes a computer to perform: an acquisition step of acquiring three-dimensional image data including a three-dimensional image and metadata; a separation step of separating the three-dimensional image data acquired in the acquisition step into the three-dimensional image and the metadata; a synthesis step of synthesizing information represented by the metadata separated in the separation step with the three-dimensional image separated in the separation step; and a conversion step of converting the three-dimensional image synthesized with the information represented by the metadata in the synthesis step into three-dimensional shape model data of a predetermined file format and of an object represented by the three-dimensional image.
According to each embodiment described above, a three-dimensional image processing apparatus, a three-dimensional image processing method, and a three-dimensional image processing program capable of creating more convenient three-dimensional shape model data can be provided.
Three-dimensional image processing apparatus 1 according to Embodiment 1 of the present invention will be described in detail below with reference to
<<1-1. Configuration of Three-Dimensional Image Processing Apparatus 1>>
As shown in
For example, medical image diagnostic apparatus 2 such as a computed tomography (CT) apparatus or a magnetic resonance imaging (MRI) apparatus can be connected to first input IF section 11. First input IF section 11 receives a three-dimensional image output from medical image diagnostic apparatus 2 under the control of control section 13 and stores the three-dimensional image in RAM 133.
Here, an example of the three-dimensional image will be described in detail. The three-dimensional image represents an image with a sense of depth, more specifically, a group of values (that is, volume data) assigned to each position on a three-dimensional space. This three-dimensional image also includes a collection of images (that is, two-dimensional tomographic images) that are obtained by medical image diagnostic apparatus 2 and are two-dimensional images stacked in a predetermined direction. In the present description, metadata is also added to the three-dimensional image. Although the metadata is not the three-dimensional image itself, but is additional information related to the three-dimensional image. Hereinafter, in the present description, data including a three-dimensional image and metadata added thereto are referred to as three-dimensional image data.
The three-dimensional image data has, for example, a DICOM format. The DICOM stands for digital imaging and communication in medicine and includes a standard defining the format of a medical image photographed by medical image diagnostic apparatus 2. As shown in
Reference is made to
Control section 13 includes at least program memory 131, CPU 132, and RAM 133. Program memory 131 is, for example, a nonvolatile memory and stores three-dimensional image processing program P. CPU 132 executes program P while using RAM 133 as a work region, whereby, as shown in
Reference is made to
Display apparatus 5 such as a 2D or 3D high resolution display can be connected to second output IF section 15. Second output IF section 15 transfers various display data generated by control section 13 to display apparatus 5. Display apparatus 5 performs screen display on the basis of the received display data.
<<1-2. Processing of Three-Dimensional Image Processing Apparatus 1>>
Three-dimensional image data output from medical image diagnostic apparatus 2 and having, for example, the DICOM format is input to first input IF section 11. In this description, it is assumed that the three-dimensional image data includes a two-dimensional tomographic image as an example of the three-dimensional image. Although the two-dimensional tomographic image represents a predetermined object, in the present description, it is assumed that the predetermined object is a predetermined human body site including at least the liver. In three-dimensional image processing apparatus 1, CPU 132 first functions as acquisition section 134 and controls input three-dimensional image data to first input IF section 11 to be transferred to RAM 133, whereby the three-dimensional data to be processed is acquired (step S001 in
Next, CPU 132 functions as discrimination and separation section 135, and when determining that the three-dimensional image data stored in RAM 133 conforms to the DICOM standard, on the basis of a value of each tag included in the three-dimensional image data or the like, separates a data element including the tag into a three-dimensional image (two-dimensional tomographic image in the present description) and metadata (step S002). As a result, RAM 133 separately stores the two-dimensional tomographic image photographed by medical image diagnostic apparatus 2 and the metadata related to the patient or examination, for example.
For the three-dimensional image (two-dimensional tomographic image) image data (image in step S003), CPU 132 processes as follows. By operating input apparatus 3, the user operates input apparatus 3 to transmit various instructions for extracting or deforming a necessary portion (that is, a human body site) from the two-dimensional tomographic image stored in RAM 133, or deleting an unnecessary portion to three-dimensional image processing apparatus 1. In this description, it is assumed that the user selects a two-dimensional tomographic image of the liver that is a human body site, and deletes other portions. In response to the operation of input apparatus 3, in three-dimensional image processing apparatus 1, CPU 132 functions as image selection section 136, processes the two-dimensional tomographic image as instructed from input apparatus 3 (step S004), and decides the portion to be a three-dimensional shape model (step S005). Thereafter, CPU 132 functions as 3D-VR generation section 137, performs 3D-VR (3D volume rendering) on the two-dimensional tomographic image decided in step S005 to generate a 3D-VR image (step S006). Here, the 3D-VR image represents a display image on display apparatus 5, and object L (see
In contrast to the above, CPU 132 processes metadata (metadata in step S003) as follows. That is, CPU 132 functions as metadata selection section 138 and selects and extracts necessary metadata from the metadata stored in RAM 133 according to an instruction from input apparatus 3 (step S007).
Next, CPU 132 functions as adjustment section 139 and generates information (for example, a character string) represented by the metadata selected in step S007 (step S008). Thereafter, the user operates input apparatus 3 to instruct three-dimensional image processing apparatus 1 a language, size, font, layout, etc. of the character string generated in step S008. As instructed from input apparatus 3, CPU 132 reflects the instruction of the user in the character string generated in step S008 (step S009). Thereafter, CPU 132 functions as 3D-VR generation section 1310 to generate 3D-VR metadata by performing 3D-VR on the information generated in step S008 (step S010). As similar to the 3D-VR image, the 3D-VR metadata also is display image data on display apparatus 5, and represents an object (character string in the present description) obtained by integrating and projecting density values and color information of pixels along a predetermined viewing direction. Since the procedure of 3D-VR is well-known, detailed description thereof will be omitted.
Upon completion of steps S006 and S010, CPU 132 functions as synthesis section 1311 to synthesis the 3D-VR metadata generated in S010 with the 3D-VR image generated in step S006 on RAM 133 to generate synthesized data (step S011 in
Next, CPU 132 transfers the synthesized data generated in step S011 to display apparatus 5 via second output IF section 15. In response to this, display apparatus 5 displays an image in which the information (character string in the present description) generated in step S008 has been synthesized with the portion (liver in the present description) decided in step S005 (step S012).
Next, CPU 132 determines whether the user has performed the decision operation with input apparatus 3 (step S013). The user refers to the display image in step S012 and starts modifying the character string for reasons such as poor visibility of the character string. In this case, CPU 132 determines that the user has not performed the decision operation, and reflects the instruction of the user to the information generated in step S008 as instructed from input apparatus 3 in step S009.
On the other hand, in step S013, when determining that the user has performed the decision operation by input apparatus 3, CPU 132 determines that there is no more modification, and converts the synthesized data generated in step S011 to three-dimensional shape model data of a standard triangulated language (STL) format (step S014). The STL format is generally called a stereolithography format.
The three-dimensional shape model data of the STL format generated by the above procedure is output to 3D modeling apparatus 4, for example, in order to formulate three-dimensional modeled object 6. Also, the configuration is not limited to this, and the three-dimensional shape model data may be stored in a portable storage medium or a remote server apparatus.
<<1-3. Three-Dimensional Shape Model Data and Three-Dimensional Modeled Object 6>>
Here, the three-dimensional shape model data in the STL format will be described. This three-dimensional shape model data expresses an object with an aggregate of polygon meshes made of, for example, minute triangles. As shown in
As a representative example of information represented by metadata, a method of converting “A” of the alphabet to three-dimensional shape model data is well known from before. In the case of creating three-dimensional data from the two-dimensional tomographic image data of medical image diagnostic apparatus 2, as shown in
First, when converting the alphabet “A” into the three-dimensional shape model data, the alphabet “A” is divided into polygon meshes of a predetermined size. At this time, depending on the size of the smallest polygon mesh, when the total number of polygon meshes is large, the three-dimensional shape of the character string can be represented more finely, and conversely when the total number of polygon meshes is small, only coarse three-dimensional shapes can be expressed. In the present description, it is assumed that an alphabet “A” is formed with about 100 polygon meshes. In
<<1-4. Effect of Three-Dimensional Image Processing Apparatus 1>>
As described above, according to three-dimensional image processing apparatus 1, discrimination and separation section 135 separates the three-dimensional image data into a three-dimensional image and metadata. 3D-VR generation section 137 performs 3D volume rendering on the three-dimensional image, and 3D-VR generation section 1310 performs 3D volume rendering on the separated metadata. Synthesis section 1311 synthesizes the 3D-VR metadata on the generated 3D-VR image to generate synthesized data. Conversion section 1312 creates three-dimensional shape model data in which information C is synthesized with object L on the basis of the synthesized data. Therefore, when three-dimensional modeled object 6 based on the three-dimensional shape model data is formed in 3D modeling apparatus 4, as shown in
According to the present embodiment, in the case of medical use, three-dimensional modeled object 6 faithfully reproducing an affected portion of a patient, ID information or name of the patient as an example of information C represented by the metadata, the examination date, and the like can be associated with each other automatically. Therefore, it is not necessary for the user to perform troublesome work such as writing the ID information and name of the patient, or the like on the three-dimensional modeled object manually after completing the three-dimensional modeled object. In addition to the above, three-dimensional image processing apparatus 1 exerts an exceptional effect that patient authentication in a preoperative plan can be reliably performed. Therefore, according to the present embodiment, it is possible to provide three-dimensional image processing apparatus 1 capable of generating three-dimensional shape model data that is easier to use than before.
<<1-5. Modification of Three-Dimensional Image Processing Apparatus 1>>
Three-dimensional modeled object 6 can also be used for training of resecting a tumor in a surgical operation, or the like. Here, in an upper part of
When three-dimensional modeled object 6 is used in training applications, it is not necessary to synthesize ID information and the like of the patient with three-dimensional modeled object 6. However, when the sex or age the name of a doctor of the patient is synthesized instead of the ID information and the like of the patient, with three-dimensional modeled object 6 as another example of information C represented by the metadata, understanding of knowledge and technique on the case is deepened and beneficial.
According to the present embodiment, it is possible to automatically impart, for example, a serial No. or a manufacturing number necessary for mass production and additional production in three-dimensional image processing apparatus 1, or to add a stamp of a bar code of the manufacturing No. described later. Thus, quantity management by a serial No. in mass produced in lesson, or the like, visual confirmation of the manufacturing No. and additional ordering by visual confirmation of the manufacturing No. and reading the bar code stamp, and cost such as labor cost can also be lowered. This enables more effective utilization of a 3D model that is an output created by the 3D printer.
In order to automatically derive the synthesizing position as described above, the user operates input apparatus 3 and instructs separation line C-D. In step S009 of
In the above description, it is described that information C is synthesized with the polygon mesh furthest from separation line C-D, but the configuration is not limited to this. Since a polygon mesh having the largest area or one or a plurality of polygon meshes having a small amount of deformation after separation also has distance from separation line C-D, information C may be synthesized with these polygon meshes.
In the above description, information C is synthesized with the polygon mesh furthest from separation line C-D. However, it is sufficient that the synthesis position of information C is determined with reference to a predetermined target position other than separation line C-D.
The synthesis position of information C derived by the above processing can be utilized as an initial position. More specifically, in step S011 in
Next, three-dimensional image processing apparatus 1A according to Embodiment 2 of the present invention will be described in detail with reference to
<<2-1. Configuration and Processing of Three-Dimensional Image Processing Apparatus 1A>>
Since a hardware configuration of three-dimensional image processing apparatus 1A is similar to that of three-dimensional image processing apparatus 1,
CPU 132 executes program P stored in program memory 131 while using RAM 133 as a work region, whereby, as shown in
In data input section 1313, data that is other than the metadata added to the three-dimensional image and is to be synthesized with object L is input.
As input data to data input section 1313, first, an enlargement ratio (0<enlargement ratio) is exemplified. The cost of producing three-dimensional modeled object 6 by 3D modeling apparatus 4 is governed by the size of three-dimensional modeled object 6. Therefore, three-dimensional modeled object 6 smaller than the actual size may be produced unless three-dimensional modeled object 6 is not necessarily to be the actual size, such as for education for anatomy or the like, endoscopic surgery training, preoperative plan, or the like. However, the size with respect to the actual size (that is, the enlargement ratio) is important information. When this enlargement ratio is synthesized with three-dimensional modeled object 6, the user can recognize the enlargement ratio instantaneously and usability is high. Note that the enlargement ratio is typically input by operating input apparatus 3 by the user and passed to adjustment section 139 via data input section 1313.
As another input data, the name of the site and the name of the operation (procedure) are exemplified. In the DICOM tag, the site name represented by the two-dimensional tomographic image data can be described. However, in the present embodiment, since image selection section 136 can select a portion to be the three-dimensional shape model, the site name of object L represented by the three-dimensional shape model data is not always described with a tag of DICOM. For example, at the time of being transferred from medical image diagnostic apparatus 2 to this three-dimensional image processing apparatus 1A, the two-dimensional tomographic image represents both of the right lung and the left lung, and the site name in the tag of DICOM is described as “lungs”. However, when the patient is suffering from emphysema in the left lung, three-dimensional shape model data is generated only for the left lung. In such a case, the user inputs the site name “left lung” just before the conversion in conversion section 1312 by operating input apparatus 3 or the like. The site name that has been input is passed to adjustment section 139 via data input section 1313.
Another input data is a name of creator, a creation date, or creation software name of three-dimensional modeled object 6, and there is naturally a difference in the quality of three-dimensional modeled object 6 depending on 3D modeling apparatus 4 and a material used for molding. When a 3D-VR image is generated from a two-dimensional tomographic image, a difference is generated in image quality or the like due to the performance of the CPU or the like. Therefore, even when the same two-dimensional tomographic image is used, there is a possibility that three-dimensional modeled objects 6 of the same quality may not be generated due to various factors. Accordingly, it can be considered from the standpoint of a doctor and a patient, that displaying such factors on three-dimensional modeled object 6 is desirable. The creator, creation date or creation software name can be input by the user operating input apparatus 3 in the same manner as the above-mentioned enlargement ratio. However, the configuration is not limited to this. User registration information and software license information held in three-dimensional image processing apparatus 1A may be transferred to data input section 1313 of control section 13 before conversion section 1312 converts the information into the STL format. As a result, it is possible to eliminate troublesome manual input by the user.
Another input data is information on 3D modeling apparatus 4. This information includes a manufacturer, an apparatus name, an apparatus number, or the like of 3D modeling apparatus 4. In order to acquire the information of 3D modeling apparatus 4, ID management section 1314 transmits a command to 3D modeling apparatus 4, acquires information from 3D modeling apparatus 4, and passes the information to adjustment section 139.
Adjustment section 139 treats each piece of external data received via data input section 1313 and ID management section 1314 in the same manner as the metadata. That is, adjustment section 139 generates information represented by the received external data in the same manner as the above-described step S008.
<<2-2. Effect of Three-Dimensional Image Processing Apparatus 1A>>
According to Embodiment 2, it is possible to provide three-dimensional image processing apparatus 1A capable of generating three-dimensional shape model data that is easier to use, since various kinds of information can be displayed on three-dimensional modeled object 6.
<<2-3. Modification of Three-Dimensional Image Processing Apparatus 1A>>
According to Embodiment 2, it is assumed that the information to be displayed on three-dimensional modeled object 6 increases. In this case, a one-dimensional code (so-called bar code) or a two-dimensional code (for example, a two-dimensional code standardized by JISX0510) generated from these pieces of information may be displayed on three-dimensional modeled object 6.
In some cases, it is preferable for a person to understand the meaning of the information displayed on three-dimensional modeled object 6, and there are cases where it is not so. Under such circumstances, as shown in
Next, three-dimensional image processing apparatus 1B according to Embodiment 3 of the present invention will be described in detail with reference to
<<3-1. Configuration and Processing of Three-Dimensional Image Processing Apparatus 1B>>
As shown in
The input and output IF section 16 stores in remote server apparatus 7 the three-dimensional shape model data generated by control section 13 by the method described in Embodiment 1 and Embodiment 2.
Server apparatus 7 is, for example, managed by a dealer who sells the three-dimensional shape model data stored therein, and can be accessed by various terminal apparatus 8 such as a personal computer. For example, suppose that a doctor who participated in a case study meeting want to use three-dimensional modeled object 6 as shown in
<<3-2. Effect of Three-Dimensional Image Processing Apparatus 1B>>
As described above, according to three-dimensional image processing apparatus 1B, since the three-dimensional shape model data generated by three-dimensional image processing apparatus 1B itself can be stored in remote server apparatus 7, the three-dimensional shape model data can be used for a wider application.
<<3-3. Appendix of Three-Dimensional Image Processing Apparatus 1B>>
When server apparatus 7 is also provided with three-dimensional image processing apparatus 1B, a serial number corresponding to the number of orders may be displayed as information C on three-dimensional modeled object 6 in order to prevent counterfeit products.
This application claims priority based on Japanese Patent Application Laid-Open No. 2016-022758 filed on Feb. 9, 2016 to the Japan Patent Office. The contents of Japanese Patent Application Laid-Open No. 2016-022758 are incorporated into this application by reference.
The three-dimensional image processing apparatus according to the present invention can create three-dimensional shape model data that is more convenient and is suitable for medical use and the like.
Number | Date | Country | Kind |
---|---|---|---|
2016-022758 | Feb 2016 | JP | national |
Number | Date | Country | |
---|---|---|---|
Parent | PCT/JP2017/004559 | Feb 2017 | US |
Child | 16058772 | US |