Patent Application
20250014176
This application is based upon and claims the benefit of priority of the prior Japanese Patent Application No. 2023-110444, filed on Jul. 5, 2023, the entire contents of which are incorporated herein by reference.
Embodiments discussed herein are related to a lesion detection method and a lesion detection program.
Medical images obtained by computed tomography (CT), magnetic resonance imaging (MRI), or the like are widely used for diagnosis of various diseases. For image diagnosis using the medical images, a doctor has to interpret a large number of images, which imposes a large burden on the doctor. For this reason, there is a demand for a technique for supporting diagnosis work of the doctor by a computer in some form.
U.S. Patent Application Publication No. 2019/0223725 is disclosed as related art.
According to an aspect of the embodiments, a lesion detection method executed by a computer includes: acquiring a first tomographic image obtained by imaging a first imaging position of an inside of a first human body; selecting any one of a first machine learning model that classifies whether or not a pixel included in an input tomographic image is a specific lesion region in units of the pixel and a second machine learning model that classifies whether or not an image block obtained by dividing the input tomographic image into certain sizes is the specific lesion region in units of the image block, based on, among a plurality of tomographic images which are obtained by imaging an inside of one or more second human bodies, are used as learning data when the first machine learning model is generated by machine learning, and each of which includes the specific lesion region, the number of tomographic images captured at a second imaging position of the inside of the one or more second human bodies, which corresponds to the first imaging position; and detecting the specific lesion region from the first tomographic image by using a machine learning model selected between the first machine learning model and the second machine learning model.
The object and advantages of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the claims.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention.
As an example of such a technique, there is a technique for detecting a lesion region from a medical image by using a trained model (machine learning model) generated by machine learning. For example, a system for heart segmentation that trains a neural network integrated with a long-short-term memory (LSTM) is proposed.
As an example of the machine learning model, there is a machine learning model that classifies whether or not each pixel included in a captured tomographic image is a specific lesion region. In the classification processing using the machine learning model, when one tomographic image is input to the machine learning model, a lesion region is classified in pixel units from the tomographic image.
A problem with such a machine learning model is that classification accuracy may be decreased depending on an imaging position of the tomographic image in a human body.
For example, when such a machine learning model is generated, the actually captured tomographic image of the human body is used as learning data. Normally, since the tomographic image is captured at equal intervals from a specific range inside any human body, the number of tomographic images used as the learning data is fixed regardless of the position in the human body. On the other hand, an appearance frequency of a lesion region may have a bias depending on the position inside the human body. Therefore, among the tomographic images used as the learning data, the number of tomographic images including the lesion region may vary depending on the position inside the human body. For a tomographic image captured at a position at which a sufficient number of tomographic images are not used as the learning data, there is a possibility that the classification accuracy of the lesion is decreased.
According to one aspect, an object of the present disclosure is to provide a lesion detection method and a lesion detection program capable of detecting a lesion region with a certain level of accuracy regardless of an imaging position of a tomographic image of an inside of a human body.
Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.
The storage unit 2 is a storage region secured in a storage device (not illustrated) included in the lesion detection apparatus 1. The storage unit 2 stores data indicating a machine learning model 2a and data indicating a machine learning model 2b.
The machine learning model 2a is a trained model that classifies whether or not a pixel included in an input tomographic image is a specific lesion region in units of the pixel. In the classification processing using the machine learning model 2a, when one tomographic image is input to the machine learning model 2a, a lesion region is classified in pixel units from the tomographic image.
On the other hand, the machine learning model 2b is a trained model that classifies whether or not an image block obtained by dividing the input tomographic image into certain sizes is a specific lesion region in units of the image block. When one tomographic image is given in the classification processing using the machine learning model 2b, the tomographic image is divided into image blocks. The image blocks are input to the machine learning model 2b one by one, and it is determined whether or not the input image block is a lesion region each time.
The processing unit 3 is, for example, a processor. In this case, the following process of the processing unit 3 is implemented by the processor executing a predetermined program. By selectively using any one of the machine learning models 2a and 2b, the processing unit 3 detects a lesion region from the captured tomographic image.
A tomographic image group 12 is a set of tomographic images including a lesion region, among tomographic images used as learning data when the machine learning model 2a is generated by machine learning. A tomographic image included in the tomographic image group 12 is an image obtained by imaging an inside of human bodies 11a, 11b, and . . . .
The processing unit 3 acquires a tomographic image 22a obtained by imaging a first imaging position of an inside a human body 21 as a target for lesion detection. Among the tomographic images included in the tomographic image group 12, the processing unit 3 selects any one of the machine learning models 2a and 2b, based on the number of tomographic images captured at a second imaging position corresponding to the first imaging position in the human bodies 11a, 11b, and . . . . By using the selected machine learning model, the processing unit 3 detects a specific lesion region from the tomographic image 22a.
An appearance frequency of the lesion region may have a bias depending on a position of the inside of the human body. Therefore, among the tomographic images used as learning data, the number of tomographic images including the lesion region may vary depending on the position of the inside of the human body. For a tomographic image captured at a position at which a sufficient number of tomographic images are not used as learning data, there is a possibility that classification accuracy of a lesion using the machine learning model 2a is decreased.
By the model selection described above, the processing unit 3 selects the machine learning model 2a in a case where it is determined that the number of tomographic images captured at the second imaging position is a sufficient number with which a certain or higher level of lesion classification accuracy is obtained from the machine learning model 2a. On the other hand, in a case where it is determined that the number of images is not sufficient to obtain the certain or higher level of lesion classification accuracy from the machine learning model 2a, the processing unit 3 selects the machine learning model 2b. Since the machine learning model 2b classifies a lesion region based on a feature in a division region instead of the entire tomographic image, the lesion classification accuracy does not depend on the imaging position of the tomographic image. Therefore, by the model selection described above, it is possible to detect a lesion region with a certain level of accuracy regardless of the imaging position of the tomographic image of the inside of the human body.
According to the example illustrated in
Noted that the division regions A1 to A3 are obtained by dividing a region between predetermined reference points in each of the human bodies 11a, 11b, and . . . . As such a region, a specific organ region may be applied.
In the example illustrated in
As an example, it is assumed that 20% of a total number (1000 in
On the other hand, for example, it is assumed that among the tomographic images included in the tomographic image group 22, the lesion detection process is executed on a tomographic image belonging to the division region B3. In this case, the processing unit 3 acquires the number of tomographic images used as learning data, which is counted for the division region A3 corresponding to the division region B3, among the division regions A1 to A3. Here, the number of images “100” is acquired for the division region A3. Since a ratio of the acquired number “100” of images to the total number “1000” is 10%, which is equal to or less than the threshold value, the processing unit 3 selects the machine learning model 2b. In this case, the processing unit 3 divides the tomographic image into image blocks, and inputs the divided image blocks one by one to the machine learning model 2b to classify whether each image block is a lesion region.
With such a process, it is possible to detect a lesion region with a certain level of accuracy regardless of an imaging position of a tomographic image of an inside of a human body.
Next, a system capable of detecting a lesion region of a lung from a CT image will be described.
The CT apparatus 50 captures an X-ray CT image of a human body. According to the present embodiment, the CT apparatus 50 captures a predetermined number of tomographic images of an axial plane in a chest region while changing a position (slice position) of a human body in a height direction (direction perpendicular to the axial plane) at a predetermined interval.
The diagnosis support apparatus 100 extracts a lung field region from each tomographic image captured by the CT apparatus 50, and detects a lesion region based on image information of the extracted lung field region. According to the present embodiment, a shadow region is detected as the lesion region. The diagnosis support apparatus 100 may classify a type of shadow. For example, the diagnosis support apparatus 100 causes a display device to display information indicating a detection result of the lesion region. Thus, the diagnosis support apparatus 100 supports the image diagnosis work of a user (for example, a radiologist).
The diagnosis support apparatus 100 detects a lesion region (or classifies a lesion) by using a trained model generated in advance by machine learning. The diagnosis support apparatus 100 is also capable of executing a process of generating the trained model by machine learning.
A hardware configuration of the diagnosis support apparatus 100 will be described below with reference to
The processor 101 comprehensively controls the entire diagnosis support apparatus 100. The processor 101 is, for example, a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or a programmable logic device (PLD). The processor 101 may also be a combination of two or more elements among the CPU, the MPU, the DSP, the ASIC, and the PLD. Noted that the processor 101 is an example of the processing unit 3 illustrated in
The RAM 102 is used as a main storage device of the diagnosis support apparatus 100. The RAM 102 temporarily stores at least a part of an operating system (OS) program and an application program to be executed by the processor 101. The RAM 102 also stores various kinds of data to be used in processes performed by the processor 101.
The HDD 103 is used as an auxiliary storage device of the diagnosis support apparatus 100. The OS program, the application program, and various kinds of data are stored in the HDD 103. Noted that a different type of non-volatile storage device such as a solid-state drive (SSD) may be used as the auxiliary storage device.
A display device 104a is coupled to the GPU 104. The GPU 104 causes the display device 104a to display an image in accordance with an instruction from the processor 101. Examples of the display device 104a include a liquid crystal display, an organic electroluminescence (EL) display, and the like.
An input device 105a is coupled to the input interface 105. The input interface 105 transmits a signal output from the input device 105a to the processor 101. The input device 105a includes a keyboard, a pointing device, and the like. The pointing device includes a mouse, a touch panel, a tablet, a touch pad, a track ball, and the like.
A portable-type recording medium 106a is removably attached to the reading device 106. The reading device 106 reads data recorded in the portable-type recording medium 106a, and transmits the data to the processor 101. The portable-type recording medium 106a includes an optical disc, a semiconductor memory, and the like.
The communication interface 107 transmits and receives data to and from another apparatus such as the CT apparatus 50 via a network.
The hardware configuration as described above is capable of implementing a processing function of the diagnosis support apparatus 100.
As described above, a lesion classification process from a medical image may be executed by using a trained model generated by machine learning, for example.
In the same manner, the Segmentation model 60 is generated by machine learning (for example, deep learning) using a tomographic image group 61 of the axial plane as learning data. A label indicating whether or not each pixel is a lesion region is added to each tomographic image used as learning data, and these labels are used as correct answer data in machine learning. By performing machine learning in tomographic image units by using such learning data, the Segmentation model 60 is generated.
Noted that for each pixel, the Segmentation model 60 may be a lesion classification model that classifies three or more types of lesions including a type indicating that the pixel is not a lesion. For example, the Segmentation model 60 classifies, for each pixel, one of six types of shadows of a ground glass opacity, a consolidation, a honeycomb, emphysema, other shadows, and a normal (not a shadow). In this case, a label indicating which of the shadow types described above is added to each tomographic image used as the learning data for each pixel. The lesion classification process using such a Segmentation model 60 is effective for, for example, diagnosis of a diffuse pneumonia.
The classification model 70 is generated by machine learning (for example, deep learning) using a tomographic image group 71 of an axial plane as learning data. Meanwhile, unlike the Segmentation method in
Noted that for each patch image, the Classification model 70 may be a lesion classification model that classifies three or more types of lesions including a type indicating that the patch image is not a lesion. For example, the Classification model 70 classifies, for each patch image, one of the six types of shadows described above. In this case, a label indicating one of the shadow types described above is added to each tomographic image used as the learning data for each patch image.
In this manner, the Classification model 70 classifies a lesion by using a patch image including a plurality of pixels as a unit. On the other hand, since the Segmentation model 60 classifies a lesion in pixel units, it is possible to classify the lesion by using a finer region in a tomographic image as a unit, as compared with the Classification model 70. Meanwhile, as illustrated in
A histogram 82 illustrated in
Where a lesion region is likely to appear in the organ region differs for each lesion as an classification target. In a case where an appearance frequency of the lesion region has a bias depending on a position in the organ region, the number of pieces of learning data of the lesion region included in the learning data set 81 described above also has a bias depending on the position of the organ region. In the example illustrated in
Based on an image feature in the entire one tomographic image such as a shape of an organ in the tomographic image or a positional relationship with another organ, the Segmentation model 60 classifies a lesion in pixel units. In order to classify the lesion with high accuracy, the sufficient number of pieces of learning data as learning data of the lesion region are desired for each position in the organ region. Meanwhile, in a case where there is a bias in the number of pieces of learning data for each position in the organ region as in the histogram 82, there is a problem in that, at a time of an inference, a difference in classification accuracy of the lesion occurs for each position in the organ region of the tomographic image input to the Segmentation model 60. As an example of the histogram 82, in a case where tomographic images of an upper portion and a lower portion in the organ region in the vertical direction are input, there is a possibility that the classification accuracy of the lesion is lowered.
On the other hand, the Classification model 70 classifies a lesion by using a patch image obtained by dividing a tomographic image as a unit. Since such a Classification model 70 classifies a lesion based on an image feature such as light and dark of an image, in a partial region in a tomographic image called a patch image, classification accuracy does not depend on a position of the patch image in an organ region.
Accordingly, the diagnosis support apparatus 100 according to the present embodiment selects and uses one of the Segmentation model 60 and the Classification model 70 at a time of an inference in accordance with the number of pieces of data in the organ region of the learning data of the lesion region, which is used at a time of training of the Segmentation model 60. For example, when a tomographic image of a certain division region in an organ region is set as an classification target, in a case where the number of pieces of learning data of a lesion region in the division region is large, lesion classification is executed by using the Segmentation model 60. On the other hand, in a case where the number of pieces of learning data of the lesion region in the division region is small, the lesion classification is executed by using the Classification model 70. Thus, it is possible to reduce a deterioration in classification accuracy of a lesion in a division region in which the number of pieces of learning data of a lesion region is small, and to classify the lesion with stable accuracy regardless of a position in an organ region.
In the example illustrated in
It is assumed that among the pieces of learning data included in the learning data set 81, the number of pieces of learning data corresponding to the middle organ portion is 800. In this case, since the number of pieces of learning data is more than the threshold value TH, lesion classification for a tomographic image corresponding to the middle organ portion is executed by using the Segmentation model 60.
It is assumed that among the pieces of learning data included in the learning data set 81, the number of pieces of learning data corresponding to the lower organ portion is 100. In this case, since the number of pieces of learning data is equal to or less than the threshold value TH, lesion classification for a tomographic image corresponding to the upper organ portion is executed by using the Classification model 70.
With such a process, it is possible to reduce a deterioration in classification accuracy of a lesion in a division region in which the number of pieces of learning data of a lesion region is small. As a result, the lesion may be classified with stable accuracy regardless of the position in the organ region in the tomographic image of the classification target.
Noted that as described in the examples in
A division direction of the organ region for generating the division regions is not limited to the vertical direction as described above. For example, the organ region may be equally divided in a direction perpendicular to the slice plane of the tomographic image (parallel direction of the tomographic image).
The learning data storage unit 110, the setting data storage unit 120, the aggregation data storage unit 130, and the model data storage unit 140 are storage regions secured in a storage device included in the diagnosis support apparatus 100, such as the RAM 102 or the HDD 103.
The learning data storage unit 110 stores learning data to be used when the Segmentation model 60 and the Classification model 70 are generated by machine learning.
Setting data referred to in a process of determining a lesion classification model to be used for each division region is stored in the setting data storage unit 120.
Among the pieces of learning data stored in the learning data storage unit 110, the aggregation data storage unit 130 stores aggregation data related to learning data of a lesion region (learning data including a tomographic image including the lesion region as teaching data). As the aggregation data, the number of pieces of learning data of the lesion region for each of the division regions obtained by dividing an organ region is stored. For each of the division regions, the aggregation data storage unit 130 also stores information indicating which of the Segmentation model 60 and the Classification model 70 is to be used.
A model parameter corresponding to each of an organ region extraction model that is a trained model for extracting an organ region, the Segmentation model 60, and the Classification model 70 is stored in the model data storage unit 140.
The processes of the data aggregation unit 151, the model determination unit 152, the model generation unit 153, the organ region extraction unit 154, the model selection unit 155, and the lesion classification unit 156 are implemented, for example, by the processor 101 executing a predetermined program.
The data aggregation unit 151, the model determination unit 152, and the model generation unit 153 operate in a learning phase in which the Segmentation model 60 and the Classification model 70 are generated by machine learning.
Among the pieces of learning data stored in the learning data storage unit 110, the data aggregation unit 151 counts the number of pieces of learning data of the lesion region for each division region, and registers the counting result in the aggregation data storage unit 130.
Based on the number of pieces of learning data for each division region registered in the aggregation data storage unit 130, the model determination unit 152 determines which one of the Segmentation model 60 and the Classification model 70 is to be used for each division region, and registers the determination result in the aggregation data storage unit 130.
By performing machine learning using the learning data stored in the model data storage unit 140, the model generation unit 153 generates the Segmentation model 60 and the Classification model 70, and registers a model parameter indicating the generated each lesion classification model in the learning data storage unit 110.
The organ region extraction unit 154, the model selection unit 155, and the lesion classification unit 156 operate in an inference phase for classifying a lesion by using the Segmentation model 60 and the Classification model 70.
The organ region extraction unit 154 extracts a tomographic image included in an organ region from a tomographic image set obtained by imaging a diagnosis target person by using the organ region extraction model.
For each of the tomographic images extracted by the organ region extraction unit 154, the model selection unit 155 selects any one of the Segmentation model 60 and the Classification model 70 as a lesion classification model to be used for lesion classification, based on information registered in the aggregation data storage unit 130.
By using the lesion classification model selected by the model selection unit 155, the lesion classification unit 156 classifies whether or not each pixel or each patch image is a lesion region from the tomographic image.
Learning data to be used to generate the Segmentation model 60 is registered in the learning data table 111. For each of a plurality of disease cases, a plurality of sets of learning data including image data and correct answer data are registered in the learning data table 111. The image data is data of a tomographic image, and an image data group corresponding to the disease case is an image data group of a plurality of tomographic images each obtained by one imaging for one person. The correct answer data is data in which a label indicating whether or not each pixel of the corresponding tomographic image is a lesion is associated with the pixel.
Learning data to be used to generate the Classification model 70 are registered in the learning data table 112. For each of the plurality of disease cases, a plurality of sets of learning data including image data and correct answer data as teaching data are registered in the learning data table 112. The image data is data of patch images obtained by dividing a tomographic image into predetermined sizes, and an image data group corresponding to the disease case is an image data group of patch images obtained from each of a plurality of tomographic images obtained by one imaging for one person. The correct answer data is a label indicating whether or not the corresponding patch image is a lesion.
The number of divisions indicates the number of divisions of an organ region (for example, the number of divided division regions). The threshold value is a value indicating a ratio of a threshold value (the threshold value TH described above) used to determine which lesion classification model is to be used to the total number of pieces of learning data for a lesion region. The number of divisions and the threshold value may be freely set by a user.
The division region range is a value indicating a range of a division region in an organ region with a ratio based on an upper end of the organ region. In the example of
The number of pieces of learning data indicates the number of pieces of learning data including a tomographic image included in a corresponding division region among pieces of learning data of a lesion region used when the Segmentation model 60 is generated.
The model flag is a flag indicating which of the Segmentation model 60 and the Classification model 70 is to be used when a lesion is classified for the tomographic image included in the corresponding division region. According to the present embodiment, it is assumed that the flag is set to 0 in a case where the Segmentation model 60 is to be used, and the flag is set to 1 in a case where the Classification model 70 is to be used.
The model structure data indicates a type of a machine learning model to be used and a configuration of a neural network. A file name of a file in which such information is described may be registered in the model data table 141 in practice. The weight data indicates a weight coefficient set between nodes in the neural network of the machine learning model. Actually, a file name of a file in which such a weight coefficient is described may be registered in the model data table 141.
The model structure data and the weight data for the organ region extraction model are registered in the model data table 141 in advance. Each model structure data for the Segmentation model 60 and the Classification model 70 is also registered in the model data table 141 in advance. On the other hand, each weight data for the Segmentation model 60 and the Classification model 70 is generated by the model generation unit 153, is registered in the model data table 141.
Next, a process of the diagnosis support apparatus 100 will be described with reference to a flowchart. First, a process in a learning phase will be described with reference to
[step S11] The data aggregation unit 151 acquires the number of divisions of an organ region from the setting data set 121 in the setting data storage unit 120.
[step S12] The data aggregation unit 151 creates records for the acquired number of divisions, in the aggregation data table 131 of the aggregation data storage unit 130, and sets a range (division region range) of each division region. Assuming that the number of divisions is d, the division region ranges are, in order from an upper side of the body, a range of 0 or more and 1/d or less, a range of more than 1/d and 2/d or less, and . . . , a range of more than d−1/d and 1 or less. The data aggregation unit 151 sets an initial value of 0 as the number of pieces of learning data of each record created in the aggregation data table 131.
[step S13] The data aggregation unit 151 selects one disease case from the learning data table 111 for the Segmentation model 60 stored in the learning data storage unit 110.
The data aggregation unit 151 assigns an image number starting from 0 to image data of a tomographic image included in the selected disease case, in order from image data at an upper end of the body. Among these pieces of image data, the data aggregation unit 151 sets an image number of the image data at the upper end of the organ region to Imin, and sets an image number of the image data at a lower end of the organ region to Imax.
Since all the tomographic images registered as the learning data are images in the organ region, Imin=0 and Imax=Imin+ (the number of pieces of image data included in the selected disease case)−1 are set in step S13.
[step S14] The data aggregation unit 151 selects one piece of image data included in the selected disease case. With this process, among the pieces of image data included in the disease case, one piece of unselected image data is selected sequentially from a head side (upper side).
[step S15] The data aggregation unit 151 determines whether or not the selected image data includes a lesion region. With this process, the data aggregation unit 151 acquires correct answer data corresponding to the selected image data, from the learning data table 111. When one or more pixels having a label indicating a lesion exist in the acquired correct answer data, the data aggregation unit 151 determines that the selected image data includes the lesion region. When it is determined that the selected image data includes the lesion region, the process proceeds to step S16, and when it is determined that the selected image data does not include the lesion region, the process proceeds to step S18.
[step S16] In order to determine in which division region the selected image data is included, the data aggregation unit 151 calculates an image number xafter after normalization for the selected image data. Assuming that the image number of the selected image data is x, the image number xafter is calculated by the following Equation (1). xafter=(x−Imin)/(Imax−Imin) . . . (1)
According to Equation (1), the image number x is normalized to a value equal to or more than 0 and equal to or less than 1. For example, the image number xafter calculated by Equation (1) indicates a ratio position of the selected image data in the entire organ region from the head side.
[step S17] The data aggregation unit 151 determines in which division region range set in the aggregation data table 131 in step S12 the calculated image number xafter is included. Thus, the data aggregation unit 151 determines a division region to which the selected image data belongs. Among the numbers of pieces of learning data in the aggregation data table 131, the data aggregation unit 151 counts up the number of pieces of learning data corresponding to the division region to which the selected image data belongs by 1.
[step S18] The data aggregation unit 151 determines whether all the image data included in the disease case selected in step S13 are selected. In a case where there is unselected image data, the process proceeds to step S14, and the image data on the head side among the unselected image data is selected. On the other hand, in a case where all the corresponding pieces of image data are selected, the process proceeds to step S19.
[step S19] The data aggregation unit 151 determines whether all the disease cases included in the learning data table 111 are selected. In a case where there is an unselected disease case, the process proceeds to step S13, and one of the unselected disease cases is selected. On the other hand, in a case where all the disease cases are selected, the data aggregation process ends.
With the process described above, the number of pieces of learning data for each division region is registered in the aggregation data table 131. By referring to the aggregation data table 131, the model determination process illustrated in
[step S21] The model determination unit 152 acquires a threshold value from the setting data set 121 in the setting data storage unit 120.
[step S22] The model determination unit 152 selects one of the division regions of which division region ranges are set in the aggregation data table 131 of the aggregation data storage unit 130.
[step S23] From the aggregation data table 131, the model determination unit 152 acquires the number of pieces of learning data corresponding to the selected division region.
[step S24] The model determination unit 152 determines whether or not the acquired number of pieces of learning data is equal to or less than the threshold value acquired in step S21. In a case where the number of pieces of learning data is equal to or less than the threshold value, the process proceeds to step S25, and in a case where the number of pieces of learning data is more than the threshold value, the process proceeds to step S26.
Noted that in a case where the threshold value registered in the setting data set 121 is a value indicating a ratio, the model determination unit 152 converts this threshold value into an absolute number of pieces of learning data, and compares the converted threshold value with the acquired number of pieces of learning data. Specifically, the model determination unit 152 sums up the numbers of pieces of learning data of the respective division regions registered in the aggregation data table 131, and multiplies the total value by a threshold value indicating a ratio to convert the total value into a threshold value indicating the absolute number of pieces of learning data.
[step S25] The model determination unit 152 sets 1 as a model flag corresponding to the selected division region in the aggregation data table 131. Thus, it is set that the Classification model 70 is to be used for lesion classification of a tomographic image included in the selected division region.
[step S26] For the aggregation data table 131, the model determination unit 152 sets 0 as the model flag corresponding to the selected division region. Thus, it is set that the Segmentation model 60 is to be used for the lesion classification of the tomographic image included in the selected division region.
[step S27] The model determination unit 152 determines whether all the division regions are selected. In a case where there is an unselected division region, the process proceeds to step S22, and one of the unselected division regions is selected. On the other hand, in a case where all the division regions are selected, the model determination process ends.
[step S31] The model generation unit 153 acquires learning data for the Segmentation model 60 from the learning data table 111.
[step S32] By executing machine learning using the acquired learning data, the model generation unit 153 generates the Segmentation model 60. Specifically, the model generation unit 153 calculates weight data of the Segmentation model 60 by machine learning, and registers the weight data in a record corresponding to the Segmentation model 60 among records included in the model data table 141.
[step S33] The model generation unit 153 acquires learning data for the Classification model 70 from the learning data table 112.
[step S34] By executing machine learning using the acquired learning data, the model generation unit 153 generates the Classification model 70. Specifically, the model generation unit 153 calculates weight data of the Classification model 70 by machine learning, and registers the calculated weight data in a record corresponding to the Classification model 70 among the records included in the model data table 141.
Noted that the model generation process in
Next, a process in an inference phase will be described with reference to
[step S41] The organ region extraction unit 154 acquires a tomographic image set of a diagnosis target person, which is captured by the CT apparatus 50. The organ region extraction unit 154 assigns an image number starting from 1 to each tomographic image included in the acquired tomographic image set, in order from an upper side of the body.
[step S42] The organ region extraction unit 154 inputs the acquired tomographic image set to the organ region extraction model based on the weight data registered in the model data table 141. Thus, the organ region extraction unit 154 extracts a tomographic image included in an organ region among the tomographic images included in the tomographic image set.
Among the extracted tomographic images, the model selection unit 155 sets the image number of a head (upper end) tomographic image to Imin, and sets the image number of a last (lower end) tomographic image to Imax.
[step S43] The model selection unit 155 selects one tomographic image of the organ region. With this process, unselected tomographic images are selected in order from the head side, among the tomographic images extracted in step S42.
[step S44] In order to determine in which division region the selected tomographic image is included, the model selection unit 155 calculates the image number xafter after normalization for the selected image data. Assuming that the image number of the selected tomographic image is x, the image number xafter is calculated by Equation (1) described above. Thus, a ratio position of the selected tomographic image from the head side in the entire organ region is calculated.
[step S45] Among the division region ranges registered in the aggregation data table 131, the model selection unit 155 determines in which division region range the calculated image number xafter is included. Thus, the model selection unit 155 determines a division region to which the selected tomographic image belongs.
Among the model flags in the aggregation data table 131, the model selection unit 155 acquires a model flag corresponding to the division region to which the selected tomographic image belongs, and determines whether the acquired model flag is 1. In a case where the model flag is 0, the process proceeds to step S46, and in a case where the model flag is 1, the process proceeds to step S47.
[step S46] The lesion classification unit 156 inputs the tomographic image selected in step S43 to the Segmentation model 60 based on the weight data registered in the model data table 141. Thus, the lesion classification unit 156 classifies whether or not each pixel in the tomographic image is a lesion.
[step S47] The lesion classification unit 156 divides the tomographic image selected in step S43 into patch images having a certain size.
[step S48] The lesion classification unit 156 inputs the divided patch images one by one to the Classification model 70 based on the weight data registered in the model data table 141. Thus, the lesion classification unit 156 classifies whether or not each patch image over the tomographic image is a lesion.
[step S49] The lesion classification unit 156 determines whether all tomographic images in the organ region are selected. In a case where there is an unselected tomographic image, the process proceeds to step S43, and one of the unselected tomographic images is selected. On the other hand, in a case where all the corresponding tomographic images are selected, the lesion classification process ends.
Noted that for example, in steps S46 and S48, the lesion classification unit 156 generates an classification result image to which information indicating whether or not each pixel of the selected tomographic image is a lesion is assigned. After that, in a case where a display of an classification result of the tomographic image is requested, the lesion classification unit 156 displays a region determined to have a lesion over the tomographic image in a manner distinguished from other regions based on the classification result image.
As described above, the Segmentation model 60 and the Classification model 70 may be lesion classification models for classifying three or more types of lesions including a type indicating that the model is not a lesion. In this case, in steps S46 and S48, for example, an classification result image to which information indicating one of the lesion types is assigned is generated for each pixel of the tomographic image.
With the process described above, it is possible to reduce a deterioration in classification accuracy of a lesion in a division region in which the number of pieces of learning data of a lesion region is small. As a result, the lesion may be classified with stable accuracy regardless of the position in the organ region in the tomographic image of the classification target.
Noted that the processes in
According to the second embodiment described above, the lesion classification model is selected depending on whether or not the number of pieces of learning data of the lesion region used at the time of training is large for the division region to which the captured tomographic image belongs. Meanwhile, in a third embodiment, when the Segmentation model 60 is generated, the classification accuracy of the Segmentation model 60 is verified for each of the division regions. In an inference phase, the lesion classification model is selected depending on whether or not the classification accuracy of the Segmentation model 60 is high for the division region to which the captured tomographic image belongs.
A diagnosis support apparatus 100a according to the third embodiment includes a learning data storage unit 110a, a setting data storage unit 120a, and an aggregation data storage unit 130a, instead of the learning data storage unit 110, the setting data storage unit 120, and the aggregation data storage unit 130 illustrated in
In the same manner as the learning data storage unit 110, the learning data storage unit 110a includes learning data to be used when the Segmentation model 60 is generated and learning data to be used when the Classification model 70 is generated. A part of the former learning data is used for verifying classification accuracy of the Segmentation model 60.
The setting data storage unit 120a is different from the setting data storage unit 120 in that a value of the classification accuracy is set as a threshold value.
Instead of the number of pieces of learning data, the number of verifications and the number of correct answers at a time of verifying the classification accuracy of the Segmentation model 60 are registered in the aggregation data storage unit 130a.
The diagnosis support apparatus 100a includes a data aggregation unit 151a and a model determination unit 152a, instead of the data aggregation unit 151 and the model determination unit 152 illustrated in
After the Segmentation model 60 is generated by the model generation unit 153, the data aggregation unit 151a verifies the classification accuracy of the Segmentation model 60 for each of the division regions by using the learning data for verification registered in the learning data storage unit 110a.
The model determination unit 152a determines a lesion classification model to be used for each of the division regions based on a result of verification by the data aggregation unit 151a.
The division region range and the model flag are the same as the division region range and the model flag in the aggregation data table 131 illustrated in
The data aggregation unit 151a executes the processes in steps S11 and S12 in
[step S51] The data aggregation unit 151a selects one disease case for verification from a learning data table for the Segmentation model 60 stored in a learning data storage unit 110a.
The data aggregation unit 151 assigns an image number starting from 0 to image data of a tomographic image included in the selected disease case, in order from image data at an upper end of the body. Among these pieces of image data, the data aggregation unit 151 sets an image number of the image data at the upper end of the organ region to Imin, and sets an image number of the image data at a lower end of the organ region to Imax.
Since all the tomographic images registered as the learning data are images in the organ region, Imin=0 and Imax=Imin+ (the number of pieces of image data included in the selected disease case)−1 are set in step S51.
[step S52] The data aggregation unit 151a selects one piece of image data included in the selected disease case. With this process, among the pieces of image data included in the disease case, one piece of unselected image data is selected sequentially from a head side (upper side).
[step S53] In order to determine in which division region the selected image data is included, the data aggregation unit 151a calculates the image number xafter after normalization for the selected image data. Assuming that the image number of the selected image data is x, the image number xafter IS calculated by Equation (1) described above. Thus, a ratio position of the selected image data from the head side in the entire organ region is calculated.
The data aggregation unit 151a determines in which division region range set in the aggregation data table 132 in step S12 the calculated image number xafter is included. Thus, the data aggregation unit 151a determines a division region to which the selected image data belongs. Among the numbers of verifications in the aggregation data table 132, the data aggregation unit 151a counts up the number of verifications corresponding to the division region to which the selected image data belongs by 1.
[step S54] The data aggregation unit 151a inputs the image data selected in step S52 to the Segmentation model 60 based on the weight data registered in the model data table 141. Thus, the data aggregation unit 151a classifies whether or not each pixel over the tomographic image is a lesion.
[step S55] The data aggregation unit 151a compares an classification result for each pixel in step S54 with a value for each pixel of correct answer data corresponding to the selected image data. A case where the classification result and the pixel value coincide with each other indicates a correct answer. For example, in a case where the number of correct pixels among the pixels of the image data is equal to or more than a certain ratio, the data aggregation unit 151a determines that classification accuracy for the selected image data is high. In this case, among the numbers of correct answers in the aggregation data table 132, the data aggregation unit 151a counts up the number of correct answers corresponding to the division region to which the selected image data belongs by 1.
Noted that in step S53, the number of pixels of the image data may be added to the corresponding number of verifications registered in the aggregation data table 132. In this case, in step S55, the number of correct pixels may be added to the corresponding number of correct answers registered in the aggregation data table 132.
[step S56] The data aggregation unit 151a determines whether all the image data included in the disease case selected in step S51 are selected. In a case where there is unselected image data, the process proceeds to step S52, and the image data on the head side among the unselected image data is selected. On the other hand, in a case where all the corresponding pieces of image data are selected, the process proceeds to step S57.
[step S57] The data aggregation unit 151a determines whether all the disease cases for verification included in the learning data table 111 are selected. In a case where there is an unselected disease case, the process proceeds to step S51, and one of the unselected disease cases is selected. On the other hand, in a case where all the disease cases are selected, the data aggregation process ends.
With the process described above, the number of verifications and the number of correct answers for each division region are registered in the aggregation data table 132. By referring to such an aggregation data table 132, the model determination process illustrated in
In the process in
[step S23a] The model determination unit 152a acquires the number of verifications and the number of correct answers from a record corresponding to the selected division region, among records in the aggregation data table 132. The model determination unit 152a calculates classification accuracy corresponding to the selected division region by a ratio of the number of correct answers to the number of verifications.
In next step S24, the calculated classification accuracy is compared with a threshold value.
According to the third embodiment described above, a lesion may be classified with stable accuracy, regardless of a position in an organ region in a tomographic image of an classification target.
Noted that the processing function of the apparatus (for example, the lesion detection apparatus 1 and the diagnosis support apparatuses 100 and 100a) described in each of the embodiments described above may be implemented by a computer. In such a case, a program describing the processing contents of the functions to be included in each apparatus is provided, and the processing functions described above are implemented over the computer by executing the program with the computer. The program describing the processing contents may be recorded in a computer-readable recording medium. Examples of the computer-readable recording medium include a magnetic storage device, an optical disc, a semiconductor memory, and the like. Examples of the magnetic storage device include a hard disk drive (HDD), a magnetic tape, and the like. Examples of the optical disc include a compact disc (CD), a Digital Versatile Disc (DVD), a Blu-ray disc (BD, registered trademark), and the like.
In a case where the program is distributed, for example, a portable-type recording medium such as a DVD or a CD on which the program is recorded is sold. The program may be stored in a storage device of a server computer and transferred from the server computer to another computer via a network.
The computer that executes the program stores, in a storage device thereof, the program recorded on the portable-type recording medium or the program transferred from the server computer, for example. The computer reads the program from the storage device thereof and executes the processing according to the program. Noted that the computer may also read the program directly from the portable-type recording medium and execute the processing according to the program. Each time the program is transferred from the server computer coupled to the computer via the network, the computer may also sequentially execute the processing according to the received program.
All examples and conditional language provided herein are intended for the pedagogical purposes of aiding the reader in understanding the invention and the concepts contributed by the inventor to further the art, and are not to be construed as limitations to such specifically recited examples and conditions, nor does the organization of such examples in the specification relate to a showing of the superiority and inferiority of the invention. Although one or more embodiments of the present invention have been described in detail, it should be understood that the various changes, substitutions, and alterations could be made hereto without departing from the spirit and scope of the invention.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2023-110444 | Jul 2023 | JP | national |
