1. Field of the Invention
The present invention relates to a similar case search device, a similar case search method, and a non-transitory computer readable medium.
2. Description of the Related Art
In the medical field, a similar case search device has been known which searches for a past case that is similar to an examination image on the basis of the examination image (for example, see JP2010-237930A and JP2012-118583A (US2012/134555A)). The examination image is, for example, an image captured by a modality, such as a computed tomography (CT) apparatus that performs tomography or a general X-ray apparatus that captures a simple transparent image, and is used to diagnose a patient, for example, to specify the disease of a patient. In some cases, in one examination operation using the general X-ray apparatus, only one examination image is captured or a plurality of examination images are captured. In one examination operation using the CT apparatus, a plurality of tomographic images (slice images) are acquired. Therefore, one examination data item includes one or more examination images. In many cases, the past examination data is accumulated to create a case. Therefore, data of one case includes one or more case images.
In a case in which a similar case search is performed, first, a user, such as a doctor, designates a region of interest in an examination image. The region of interest indicates a region in which the doctor is particularly interested in the examination image and which includes a lesion to be diagnosed. The similar case search device compares a feature amount which is obtained by quantifying the features of one region of interest designated in the examination image and a feature amount which is obtained by quantifying the features of one lesion in a case image and determines the similarity therebetween. Here, for convenience of explanation, a lesion which is included in the region of interest of the examination image is referred to as a target lesion and a lesion which is included in the case image is referred to as a case lesion. Then, the similar case search device searches for a case including a case lesion that is similar to the region of interest from a case database storing a plurality of cases.
In general, the users designate the region of interest including a target lesion, using different methods, and a variation in search, that is, a variation in the search result occurs due to the difference between individuals. JP2010-237930A discloses a technique which reduces the variation in search. Specifically, even in a case in which a region including the same target lesion is designated as the region of interest, the shape or size of the designated region varies due to the difference in how the user designates the region of interest. As a result, a feature amount is likely to be changed. In the event that the feature amount is changed, similarity is also changed, which results in a variation in search in which the search result varies depending on the user. In JP2010-237930A, in order to reduce the variation in search, for example, the feature amount of each of a plurality of regions of interest in which one target lesion is designated by different methods is calculated, similarity is calculated on the basis of the average value of the calculated feature amounts of the plurality of regions of interest, and a similar image is searched. According to this structure, it is possible to reduce a variation in search due to the difference in designation between the users.
JP2012-118583A (US2012/134555A) relates to a technique that outputs the search result which is more suitable than the subjective feeling of the user on similarity. Specifically, in a case in which the same type of target lesion is present in a plurality of examination images, in the event that a region of interest is designated, the regions of interest including a plurality of target lesions of the same type which the user feels to be similar to each other are put into one group as a group of the same type of target lesions. In one examination data item, a feature amount range including all of the feature amounts of a plurality of target lesions belonging to the group of the same type of target lesions is calculated and a similar case search is performed, using the feature amount range as a search condition. Since it is considered that the feature amount range of the group of the same type of target lesions is equal to that of the target lesions which the user subjectively feels to be similar to each other, the search result is more suitable than the subjective feeling of the user.
However, in some cases, a plurality of target lesions appear in an examination image depending on a disease, which is a basis for specifying a disease. For example, in the case of tuberculosis, a disease is specified on the basis of three types of target lesions, that is, a vomica shadow (cavity), a punctate shadow (small nodules), and a frosted glass shadow (ground glass opacity), which appear in an examination image. In the case of diffuse panbronchiolitis, a disease is specified on the basis of two types of target lesions, that is, an abnormal shadow of the bronchus and a punctate shadow. In the case of a cancer, a case that is similar to a single target lesion may be searched. In the case of non-cancerous diseases other than cancer, it is necessary to search for a case that is similar to a plurality of target lesions.
In the similar case search devices disclosed in JP2010-237930A and JP2012-118583A (US2012/134555A), attention is paid to one target lesion included in the examination image and a similar case is searched on the basis of the feature amount of the region of interest including one target lesion to which attention is paid. However, it is not considered that attention is paid to each of a plurality of target lesions included in the examination images.
As described above, in JP2010-237930A, the feature amount is calculated for each region of interest. A plurality of regions of interest are designated by different methods and have the same target lesion. Therefore, JP2010-237930A does not disclose a technique that pays attention to the feature amounts of a plurality of regions of interest including different target lesions and searches for a similar case. In addition, in JP2012-118583A (US2012/134555A), for a plurality of target lesions included in a plurality of examination images, one search condition is created for one group of the same type of target lesions and a similar case is searched under the created search condition. In other words, in JP2012-118583A (US2012/134555A), the feature amount common to the regions of interest including a plurality of target lesions of the same type is calculated according to the user's preference. However, JP2012-118583A (US2012/134555A) does not disclose a technique that pays attention to the feature amounts of a plurality of regions of interest including a plurality of target lesions and searches for a similar case.
As disclosed in JP2010-237930A and JP2012-118583A (US2012/134555A), in the technique that pays attention to the feature amount of one region of interest, in a case in which there are a plurality of regions of interest, it is difficult to appropriately search for a similar case.
An object of the invention is to provide a similar case search device and a similar case search method that can appropriately search for a similar case even in a case in which there are a plurality of regions of interest, and a non-transitory computer readable medium.
A similar case search device according to the invention searches for a similar case which is similar to an examination image used to diagnose a patient from a case database in which a plurality of cases, each of which includes one or more case images, are registered and comprises a feature amount acquisition unit, an individual similarity calculation unit, a total similarity calculation unit, and a similar case search unit. The feature amount acquisition unit acquires a feature amount of each of a plurality of regions of interest, each of which is designated so as to include one or more different target lesions that are lesions in the examination images, in examination data including one or more examination images. The individual similarity calculation unit compares the feature amount of each region of interest with a feature amount of a case lesion, which is a lesion in the case image registered in the case, and calculates an individual similarity for each region of interest. The total similarity calculation unit calculates a total similarity on the basis of a plurality of calculated individual similarities. The similar case search unit searches for the similar case on the basis of the total similarity.
In a case in which a plurality of case lesions are registered in one case, it is preferable that the individual similarity calculation unit sets a plurality of regions of interest and the plurality of case lesions so as to be in one-to-one correspondence with each other, compares the feature amounts, and calculates the individual similarities.
Here, the case in which a plurality of case lesions are registered in one case, that is, a plurality of case lesions are present in the case images includes a case in which a plurality of case lesions are present in one case image and a case in which the sum of the case lesions that are present in a plurality of case images is two or more, for example, a case in which one case lesion is present in each of two case images.
The total similarity calculation unit may create permutations corresponding to the number of regions of interest and the number of case lesions, using the individual similarities as elements, and calculate the total similarity for each of the permutations. It is preferable that the total similarity is a sum of a plurality of individual similarities included in the permutations.
The individual similarity calculation unit may create an individual similarity table, in which a plurality of individual similarities that are calculated by a correspondence between each region of interest and a plurality of case lesions are recorded, for each region of interest. The total similarity calculation unit may read out the individual similarities one by one from a plurality of individual similarity tables calculated for each region of interest and create the permutations, using the plurality of read individual similarities as elements.
The total similarity calculation unit may perform a weighting process for the total similarity according to values of the individual similarities which are elements for calculating the total similarity. In a case in which the individual similarity is equal to or greater than a threshold value, the weighting process may increase the total similarity.
It is preferable that the similar case search unit creates a similar case list which is a list of information related to the plurality of similar cases on the basis of the total similarity. It is preferable that, in the similar case list, the similar cases are sorted in an order of the total similarity.
Preferably, display items of the similar case list include a value of the total similarity and breakdown information related to the total similarity and the breakdown information includes a correspondence relationship between the region of interest and the case lesion for calculating the individual similarity.
Preferably, in addition to the value of the total similarity, values of the plurality of individual similarities which are elements for calculating the total similarity are displayed in the similar case list. It is preferable that images of the region of interest and the case lesion are displayed in the similar case list.
Preferably, in a case in which the number of designated regions of interest is changed, the similar case search unit can re-search for the similar case. Preferably, the similar case search unit stores data of a processing result created during a search and uses the stored data of the processing result to re-search the similar case.
Preferably, the similar case search unit can change at least one of a combination of a plurality of regions of interest or a sorting order of a plurality of regions of interest in the similar case list, in response to a request. Preferably, the similar case search unit excludes a case in which the number of registered case lesions is less than the number of designated regions of interest from a search target.
The similar case search device may further comprise a type information acquisition unit and an essential designation receiving unit. The type information acquisition unit acquires type information indicating the types of the target lesion and the case lesion. The essential designation receiving unit receives essential designation for designating at least one of the plurality of designated regions of interest, which is an essential region of interest, as a search condition. In this case, preferably, the similar case search unit searches for a case including a case lesion that is the same type as the target lesion in the region of interest designated as the essential region of interest, regardless of the number of registered case lesions.
Preferably, the similar case search device may further comprise a representative value determination unit that, in a case in which a plurality of total similarities are calculated by a one-to-one correspondence between one region of interest and a plurality of case lesions included in one case, determines one representative value from the plurality total similarities. In this case, preferably, the similar case search unit searches for the similar case on the basis of the representative value.
A similar case search method according to the invention searches for a similar case which is similar to an examination image used to diagnose a patient from a case database in which a plurality of cases, each of which includes one or more case images, are registered. The similar case search method comprises a feature amount acquisition step, an individual similarity calculation step, a total similarity calculation step, and a similar case search step. In the feature amount acquisition step, a feature amount of each of a plurality of regions of interest, each of which is designated so as to include one or more different target lesions that are lesions in the examination images, in examination data including one or more examination images is acquired. In the individual similarity calculation step, the feature amount of each region of interest is compared with a feature amount of a case lesion, which is a lesion in the case image registered in the case, and an individual similarity for each region of interest is calculated. In the total similarity calculation step, a total similarity is calculated on the basis of a plurality of calculated individual similarities. In the similar case search step, the similar case is searched on the basis of the total similarity.
A non-transitory computer readable medium according to the invention stores a computer-executable program enabling execution of computer instructions to perform operations for searching for a similar case which is similar to an examination image used to diagnose a patient from a case database in which a plurality of cases, each of which includes one or more case images, are registered. The operations include acquiring a feature amount of each of a plurality of regions of interest, each of which is designated so as to include one or more different target lesions that are lesions in the examination images, in examination data including one or more examination images, comparing the feature amount of each region of interest with a feature amount of a case lesion, which is a lesion in the case image registered in the case, and calculating an individual similarity for each region of interest, calculating a total similarity on the basis of a plurality of calculated individual similarities, and searching for the similar case on the basis of the total similarity.
The feature amounts of a plurality of regions of interest are compared with the feature amounts of each case lesion included in the case images to calculate the individual similarities. The total similarity is calculated on the basis of the calculated individual similarities. A similar case is searched on the basis of the total similarity. Therefore, it is possible to provide a similar case search device and a similar case search method that can appropriately search for a similar case even in a case in which there are a plurality of regions of interest, and a non-transitory computer readable medium.
A medical information system 9 illustrated in
The treatment department terminal 11 is operated by a doctor (to which letters “Dr” are attached in the drawings) in the treatment department 10 to input or browse electronic medical records and to issue an examination order for examination to the examination department 12. In addition, the treatment department terminal 11 is used as an image display terminal that displays an examination image 19 which has been captured in the examination department 12 and then stored in the examination image DB server 15 such that the doctor can browse the examination image 19.
In the examination department 12, the order management terminal 14 receives the examination order from the treatment department 10 and manages the received examination order. A technician in the examination department 12 takes a radiographic image of a patient using the modality 13 according to the content of the examination order. One or a plurality of examination images 19 are captured in response to one examination order. In the event that imaging ends, the modality 13 transmits the captured examination image 19 to the examination image DB server 15. In the event that examination ends, the doctor in the treatment department 10 is notified of the end of the examination from the examination department 12 and is also notified of the storage destination of the examination image 19 in the examination image DB server 15. The doctor in the treatment department 10 accesses the examination image DB server 15 through the treatment department terminal 11 and browses the examination image 19 using the treatment department terminal 11.
The examination image DB server 15 includes an examination image DB 20 that stores a plurality of examination images 19 and is a so-called picture archiving and communication system (PACS) server. The examination image DB 20 is a database which can be searched by keyword and transmits an examination image 19 matched with search conditions or a designated examination image 19 in response to, for example, a search request or a transmission request from the treatment department terminal 11.
As illustrated in
The examination order includes, for example, information about a request source, such as the ID (identification data) or position of the doctor in the treatment department 10, patient information, and the type of examination. An image file of the examination image 19 includes image data and accessory information such as a digital imaging and communication in medicine (DICOM) header. Examination order information is stored as the accessory information of the examination image 19. In addition, the accessory information includes an examination ID and an image ID which is given to each examination image 19. In the example illustrated in
The similar case search server 17 receives the examination image 19 as search conditions and searches for a case including a case image 22 that is similar to the received examination image 19. The case image 22 is an examination image that was used for diagnosis in the past. The case DB server 16 includes a case DB 23 that stores a plurality of cases such that the cases can be searched. The similar case search server 17 accesses the case DB server 16, reads out the cases one by one, compares the examination image 19 which has been received as the search conditions with the case image 22 in the case, and searches for a case that is similar to the examination image 19.
As illustrated in
The doctor in the treatment department 10 checks the case included in the examination result. The case includes a radiogram interpretation report associated with the case image 22. The doctor makes a definite diagnosis, such as the specification of a disease in the examination image 19, with reference to, for example, an opinion on the case image 22 which is written in the radiogram interpretation report.
As illustrated in
The case image 22 includes a lesion (case lesion CL) indicating the symptoms of a disease. One or more case lesions CL are registered in one case. In this example, three case lesions CL with No1 to No3 are registered in a case with a case ID “C101”, two case lesions CL are registered in a case with a case ID “C102”, and one case lesion CL is registered in a case with a case ID “C103”. The case lesion CL is a region that was designated as a lesion by the doctor in the event that the case image 22 was used as the examination image for diagnosis in the past and was registered as the case lesion CL by the doctor through a definite diagnosis. A method for designating the case lesion CL is the same as, for example, a method for designating a region of interest ROI which will be described below.
The feature amount DB 23B is a database that stores the feature amount CAC of an image of the case lesion CL. An ID including the case ID and a lesion number (No) is given to the feature amount CAC. For example, there are three case lesions CL in the case ID “C101” and serial numbers No1 to No3 in one case are given to each case lesion CL. A number following the feature amount CAC corresponds to the serial number in the case. A method for calculating the feature amount CAC is the same as, for example, a method for calculating the region of interest ROI which will be described below.
As illustrated in
The treatment department terminal 11, the order management terminal 14, the examination image DB server 15, the case DB server 16, and the similar case search server 17 are implemented by installing a control program, such as an operating system, or an application program, such as a client program or a server program, in computers, such as personal computers, server computers, or workstations.
As illustrated in
The storage device 43 is, for example, a hard disk drive (HDD) and stores a control program or an application program (hereinafter, referred to as an AP) 50. In addition to the HDD storing the programs, a disk array obtained by connecting a plurality of HDDs is provided as the storage device 43 for a DB in a server in which a DB is constructed. The disk array may be provided in the main body of the server, or it may be provided separately from the main body of the server and may be connected to the main body of the server through a cable or a network.
The memory 42 is a work memory that is used by the CPU 41 to perform processes. The CPU 41 loads the control program stored in the storage device 43 to the memory 42 and performs a process according to the program to control the overall operation of each unit of the computer. The communication I/F 44 is a network interface that controls communication with the network 18.
As the AP 50, a client program, such as electronic medical record software for browsing or editing electronic medical records or viewer software for browsing examination images or a similar case list, is installed in the treatment department terminal 11. The viewer software may be, for example, dedicated software or a general-purpose web browser.
As illustrated in
As illustrated in
The region designation button 52C is an operation button for designating the region of interest ROI in the examination image 19. In the event that the region designation button 52C is clicked by a pointer 56 of a mouse, a region designation operation which designates an arbitrary region of the examination image 19 can be performed. In this state, the pointer 56 is operated to designate the outer circumference of a region including a target lesion OL, using, for example, a spline. The spline is a smooth curve that passes through a plurality of designated control points and is input by the designation of the control points by the pointer 56. The region including the target lesion OL is designated as the region of interest ROI by the above-mentioned operation. The clear button 52D is a button for clearing the designated region of interest ROI.
A plurality of regions of interest ROI can be designated. In the example illustrated in
As illustrated in
The request receiving unit 61 receives the similar case search request transmitted from the treatment department terminal 11 and stores the received examination image 19 and the received region information of the region of interest ROI in, for example, the storage device 43 of the similar case search server 17. The feature amount calculation unit 62 calculates the feature amount of the region of interest ROI on the basis of the received examination image 19 and region information. Here, the feature amount calculation unit 62 functions as a feature amount acquisition unit.
As illustrated in
As illustrated in
As illustrated in
The discriminator output value indicates the likeness of the typical lesion pattern and indicates the probability of the typical lesion pattern being present in the region of interest ROI. Therefore, as the discriminator output value increases, the probability of the typical lesion pattern being present in the region of interest ROI increases. As the discriminator output value decreases, the probability of the typical lesion pattern being present in the region of interest ROI decreases. Specifically, a “positive (+)” discriminator output value indicates that the typical lesion pattern is present in the region of interest ROI and a “negative (−)” discriminator output value indicates that no typical lesion pattern is present in the region of interest ROI. In the event that the discriminator output value is “positive (+)” and becomes larger, the probability of the typical lesion pattern being present becomes higher.
As can be seen from the example illustrated in
Each of the discriminators corresponding to the typical lesion patterns can be created by a machine learning algorithm, such as “Ada-boost”, using, for example, a well-known feature amount described in “Document Name: Computer Vision and Image Understanding, vol. 88, pp. 119 to 151, December 2002, and Chi-Ren Shyu, Christina Pavlopoulou Avinash C. kak, and Cala E. Brodley, “Using Human Perceptual Categories for Content-Based Retrieval from a Medical Image Database”.
The feature amount calculation unit 62 calculates the feature amount RAC of each of a plurality of regions of interest ROI designated in the examination data 21 attached to the similar case search request.
As illustrated in
As illustrated in
As illustrated in
In the examination data 21 with the examination ID “O901”, three regions of interest ROI with No1 to No3 are designated. Three case lesions CL with No1 to No3 are registered in the case data 24 with the case ID “C101”. Therefore, in the example illustrated in
An identification code in parentheses which follow each individual similarity ISM is obtained by adding the serial number of each of the region of interest ROI and the case lesion CL to the case ID. For example, “C101-11” indicates an individual similarity ISM between the region of interest ROI with No1 and the case lesion CL with No1 which is registered in the case data 24 with the case ID “C101”. Similarly, “C101-12” indicates an individual similarity ISM between the region of interest ROI with No1 and the case lesion CL with No2 which is registered in the case data 24 with the case ID “C101”.
As illustrated in
In the event that a process of calculating the individual similarity ISM for the case with the case ID “C101” ends, the individual similarity calculation unit 65 calculates the individual similarity ISM for one case with a case ID “C102”. Two case lesions CL (No1 and No2) are registered in the case with the case ID “C102”. Therefore, in the event that two case lesions CL correspond to three regions of interest ROI, a total of six (=3×2) individual similarities ISM are calculated. This process is repeatedly performed the number of times corresponding to the number of cases.
In
Similarly, the individual similarity calculation unit 65 sets the regions of interest ROI with No1 to No3 and the case lesions CL of a case with a case ID “C103” and the subsequent cases so as to correspond to each other and calculates the individual similarities ISM.
As illustrated in
First, the individual similarity calculation unit 65 records each individual similarity ISM in the ISM table 71 in a calculation order. The individual similarities ISM are recorded in ascending order of the number of the case ID, such as in the order of “C101”, “C102”, and “C103”. The value of the individual similarity ISM is calculated by the correlation between the feature amount RAC of the region of interest ROI and the feature amount CAC of the case lesion CL. Therefore, as the value increases, the similarity increases.
As illustrated in
As illustrated in
In this example, in the examination data 21 with the examination ID “O901”, there are three regions of interest ROI with No1 to No3 and three ISM tables 71 in which the individual similarities ISM are recorded for each region of interest ROI are created. Since three case lesions CL are registered in the case data 24 with the case ID “C101”, the number of permutations of three regions of interest ROI and three case lesions CL is 6 (3P3=3×2×1). The total similarities TSM corresponding to the number of permutations are calculated.
Since the individual similarities ISM are values that vary depending on the one-to-one correspondence between three regions of interest ROI with No1 to No3 and three case lesions CL with No1 to No3, the total similarity TSM varies depending on each permutation. In this example, the breakdown of the permutations is as illustrated in
Each individual similarity ISM is obtained in a case in which the regions of interest ROI correspond one to one with the case lesions CL. Therefore, as the total similarity TSM increases, the average value of the individual similarities ISM between three regions of interest ROI and three case lesions CL increases. In this example, the individual similarity ISM is represented by a correlation value. Therefore, as the value increases, the similarity increases. As a result, as the value increases, the total similarity TSM increases.
In this example, among six total similarities TSM, the total similarity TSM with the identification code “C101-2” has the highest value of “2.04”. In contrast, in this example, the total similarity TSM with the identification code “C101-3” has the lowest value of “1.32”. The total similarity TSM with the identification code “C101-5” includes the individual similarity ISM (C101-13) having the highest value of “0.91” among the individual similarities ISM. However, the total similarity TSM with the identification code “C101-2” without including the individual similarity ISM with the highest value is higher than the other total similarities TSM since the average value of the individual similarities ISM is high.
Among six total similarities TSM, the total similarity TSM (C101-2) with the highest value is the sum of the individual similarity ISM (C101-11) between the region of interest ROI with No1 and the case lesion CL with No1, the individual similarity ISM (C101-23) between the region of interest ROI with No2 and the case lesion CL with No3, and the individual similarity ISM (C101-32) between the region of interest ROI with No3 and the case lesion CL with No2. Therefore, it can be evaluated that the similarity between the examination data 21 with the examination ID “O901” and the case with the case ID “C101” is the highest in a case in which the case lesions CL with No1 to No3 correspond to the regions of interest ROI with No1, No3, and No2.
As illustrated in
The reason is as follows. The total similarity TSM is an index for evaluating the case in which the average value of a plurality of individual similarities ISM is high to be a case with high similarity. Therefore, preconditions for calculating the total similarity TSM for the case in which the number of registered case lesions CL is equal to or greater than the number of regions of interest ROI are different from preconditions for calculating the total similarity TSM for the case in which the number of registered case lesions CL is less than the number of regions of interest ROI. For example, one of the two total similarities TSM is the sum of three individual similarities and the other total similarity TSM is the sum of two individual similarities. It is considered that the comparison between the above-mentioned total similarities TSM is not appropriate.
The total similarities TSM corresponding to the number of permutations of a plurality of regions of interest ROI and a plurality of case lesions CL are calculated. For the cases to be searched, the number of total similarities TSM calculated varies depending on the number of case lesions CL. For the case with the case ID “C101”, as described above, the number of permutations is 6 (3P3=3×2×1) and the number of total similarities TSM calculated is 6. For the case with the case ID “C105”, since the number of registered case lesions CL is “5”, the number of permutations is 60 (5P3=5×4×3) and the number of total similarities TSM calculated is 60. Similarly, for the case with the case ID “C106”, since the number of registered case lesions CL is “7”, the number of permutations is 210 (7P3=7×6×5) and the number of total similarities TSM calculated is 210.
As illustrated in
As illustrated in
As illustrated in
As illustrated in
The similar case search unit 67 is provided with a list creation unit 67A (see
The list creation unit 67A extracts the total similarities TSM from the TSM table 72 and creates the similar case list 74 in which similar cases are arranged in descending order of the total similarity TSM. Display items of the similar case list 74 include the value of each total similarity TSM, a rank based on each total similarity TSM, a case ID, and breakdown information related to each total similarity TSM. In this example, the breakdown information is the correspondence relationship between the region of interest ROI and the case lesion CL for calculating each individual similarity ISM which is an element for calculating the total similarity TSM.
In the similar case list 74, the total similarity TSM for the case with the case ID “C106” corresponds to a total similarity TSM with an identification code “C106-7” illustrated in
In addition, the display items of the similar case list 74 include the images of the case lesions CL with No1, No3, and No4. The lesion images are read from, for example, the ISM table 71. In addition, the examination images 19 including the regions of interest ROI with No1 to No3 are displayed above the similar case list 74.
For example, the top six cases are displayed in the similar case list 74. Of course, the cases in sixth place or lower may be displayed by, for example, a screen scroll operation. In addition, the number of cases which can be displayed at the same time may be changed such that the top ten cases are displayed.
The output control unit 69 (see
Next, the operation of the above-mentioned structure will be described with reference to
In the event that the similar case search server 17 receives the similar case search request, the request receiving unit 61 receives the similar case search request (S2100). Then, the feature amount calculation unit 62 calculates the feature amount of each region of interest ROI on the basis of the examination images 19 and the region information of the regions of interest ROI (S2200). Then, a similar case search process is performed (S2300).
As illustrated in
The total similarity calculation unit 66 creates the TSM table 72 on the basis of a plurality of ISM tables 71 created for each region of interest ROI (S2370). In the creation of the TSM table 72, as illustrated in
The similar case search unit 67 creates the similar case list 74 on the basis of the created TSM table 72 (S2380). In the creation of the list, first, as illustrated in
The list creation unit 67A creates the similar case list 74 in which similar cases are arranged in descending order of the total similarity TSM, on the basis of the TSM table 72 (S2383).
In
In a case in which the examination data 21 includes a plurality of target lesions OL and a case similar to the examination data 21 is searched, it is preferable that a search process is performed, comprehensively considering each of the feature amounts of a plurality of regions of interest ROI including each target lesion OL and a plurality of case lesions CL, in addition to paying attention to the feature amounts. For example, in a certain case in which one case lesion CL has high similarity to one region of interest ROI and another case lesion CL has a very low similarity to another region of interest ROI, the case is not appropriate as a similar case in the event that attention is to be paid to at least a plurality of regions of interest ROI.
In the invention, the individual similarities ISM between each region of interest ROI and each case lesion CL are calculated and the total similarity TSM is calculated on the basis of the calculated individual similarities ISM. Then, a similar case is searched. The total similarity TSM is an index for evaluating a case in which the average value of a plurality of individual similarities ISM is high to be a case with high similarity. The search of a similar case on the basis of the total similarity TSM makes it possible to appropriately search for a similar case with high similarity to the examination data 21 including a plurality of target lesions OL.
In the related art, only a similar case search process in which attention is paid to only the feature amount of one region of interest ROI is performed. Therefore, it is difficult to appropriately search for a similar case in a similar case search process in a case in which there are a plurality of regions of interest ROI. In contrast, in the invention, a similar case is searched on the basis of the total similarity TSM. Therefore, it is possible to provide a technique that is more useful than the related art in the similar case search process in a case in which there are a plurality of regions of interest ROI.
In some cases, a disease is specified on the basis of whether a plurality of target lesions OL appear. As such, the invention is useful to diagnose a disease in which attention needs to be paid to the feature amounts of a plurality of regions of interest ROI. In many cases, this disease is anon-cancerous disease, such as tuberculosis in which attention needs to be paid to three types of target lesions OL, that is, a vomica shadow (cavity), a punctate shadow (small nodules), and a frosted glass shadow (ground glass opacity), or diffuse panbronchiolitis in which attention needs to be paid to two types of target lesions OL, that is, an abnormal shadow of the bronchus and a punctate shadow. Therefore, the invention is particularly useful for the diagnosis of a non-cancerous disease.
In this example, a plurality of regions of interest ROI include different types of target lesions OL such as “vomica” and a “frosted glass shadow”. However, the same type of target lesion OL may be included in the regions of interest ROI as long as the target lesions OL are different from each other.
In this example, the total similarity TSM is the sum of a plurality of individual similarities ISM. However, the total similarity TSM may be the product of the individual similarities ISM.
In this example, the representative value determination unit 67B determines a representative value for each case from a plurality of total similarities TSM calculated for each case and the similar case search process is performed on the basis of only the representative values. The determination of the representative values makes it possible to obtain the effect of reducing the amount of data treated in the search process, such as the number of total similarities TSM recorded in the TSM table 72, to reduce the processing time. The diagnosis result, such as the doctor's opinion on the case lesion CL, which is described in the radiogram interpretation report, is present for each case. Therefore, the determination of the representative values makes it possible to provide the search results for each case and to appropriately and effectively perform a diagnosis on the basis of the similar case. However, the similar case search process may be performed without determining the representative value. In a case in which the representative value is not determined, a plurality of total similarities TSM of the same case may be displayed in the similar case list 74. The plurality of total similarities TSM are different combination patterns of the individual similarities ISM which are elements for calculating the total similarities TSM. Therefore, it is possible to refer to the plurality of total similarities TSM while changing a point of view for one case.
The display items of the similar case list 74 include breakdown information related to the total similarity TSM, in addition to the case ID of a similar case and the total similarity TSM. It is possible to check the correspondence relationship between the regions of interest ROI and the case lesions CL for calculating each individual similarity ISM, which is an element for calculating the total similarity TSM, from the breakdown information. The display of the correspondence relationship makes it possible to check the correspondence relationship between a plurality of regions of interest ROI and a plurality of case lesions CL used to calculate each individual similarity ISM which is an element for calculating the total similarity TSM. In addition, the examination image 19 or the image of the case lesion CL is also displayed in the similar case list 74. Therefore, it is easy to compare or refer to the image patterns and to intuitively determine the similarity between the image patterns.
As the breakdown information related to the total similarity TSM, the values of the individual similarities ISM, which are elements for calculating the total similarity TSM, may be displayed as in a similar case list 75 illustrated in
The total similarity TSM is an index for evaluating the case in which the average value of a plurality of individual similarities ISM is high to be a case with high similarity. Therefore, in the similar case list, the case in which the average value of a plurality of individual similarities ISM is high is ranked high and the case in which the average value is low, but one individual similarity ISM is particularly high is ranked low. In some cases, the subjective evaluation of the doctor on similarity is more greatly affected by the impression of the doctor on a specific case lesion CL and a specific region of interest ROI than by the average value. Therefore, in some cases, there is a difference between the subjective evaluation of the doctor and objective evaluation (rank) based on the total similarity TSM.
The display of the values of the individual similarities ISM in addition to the total similarity TSM as in the similar case list 75 makes it possible for the doctor to check the individual similarities ISM and to verify his or her subjective evaluation even assuming that the difference occurs. In addition, in the event that the values of the individual similarities ISM are displayed, the doctor can search for each similar case suitable for a diagnosis, while correcting the objective evaluation based on the total similarity TSM, using the similar case list 75, on the basis of the subjective evaluation of the doctor, considering the values of the individual similarities ISM.
A second embodiment illustrated in
For example, as illustrated in
The following effect is obtained by the weighting process. As described above, the total similarity TSM is an index for evaluating the case in which the average value of a plurality of individual similarities ISM is high to be a case with high similarity. Basically, the case in which the average value of a plurality of individual similarities ISM is high is evaluated to be a similar case with high similarity. However, in some cases, a case including any case lesion CL that is very similar to each region of interest ROI is useful as a similar case. The total similarity TSM including any individual similarity ISM that is equal to or greater than a predetermined threshold value is increased by the weighting process. In this way, a case having the total similarity TSM subjected to the weighting process moves up in the ranks in the TSM table 72 or the similar case list 74. As such, the use of the weighting coefficient W makes it possible to improve the comprehensive evaluation of a very useful case as a similar case. Therefore, it is easy to extract the case as a similar case.
As a method for multiplying the weighting coefficient W, the individual similarity ISM may be multiplied by the weighting coefficient W as illustrated in
In this example, as a similar case evaluation method, a method is used which highly evaluates the case including the individual similarity ISM that is equal to or greater than the predetermined threshold value. On the contrary, a method which lowly evaluates a case including the individual similarity ISM that is less than the predetermined threshold value is considered. That is, the level of comprehensive evaluation is reduced such that a case including any case lesion CL that has very low similarity to the region of interest ROI is evaluated to be a similar case. In this case, instead of a positive weighting coefficient, a negative weighting coefficient may be multiplied to reduce the total similarity TSM. In this way, the level of comprehensive evaluation on a case including the total similarity TSM multiplied by the negative weighting coefficient as a similar case is reduced.
As the similar case evaluation method, in the event that the actual condition of a similar case is considered, it is more preferable to multiply the case including the individual similarity ISM that is equal to or greater than the predetermined threshold value by a positive weight coefficient than to multiply the case including the individual similarity ISM that is less than the predetermined threshold value by a negative weighting coefficient, in order to increase the level of evaluation on the case.
In this example, since the individual similarity ISM is represented by a correlation value, an individual similarity ISM having a correlation value that is equal to or greater than a predetermined value is determined to be an individual similarity ISM that is equal to or greater than a predetermined threshold value. In contrast, in a case in which the individual similarity ISM is represented by a least square distance, as the value of the least square distance decreases, the similarity increases. In a case in which the value of the least square distance is equal to or less than a predetermined value, the similarity is determined to be an individual similarity ISM that is equal to or greater than the predetermined threshold value.
A third embodiment illustrated in
In this way, for example, the following method can be used: a method which designates one region of interest ROI, issues a similar case search request, adds a region of interest ROI while checking the search result, issues a similar case search request again, and checks the search result on the basis of two regions of interest ROI. The number of designated regions of interest ROI is sequentially added. Therefore, it is possible to narrow down the number of similar cases included in a similar case list 74 which is the search result. Specifically, a similar case search process will be described with reference to the flowchart illustrated in
As illustrated in
As illustrated in
In a case in which only one region of interest ROI is designated, all of the cases including a case (with a case ID “C103”) in which only one case lesion CL is registered are search targets as illustrated in
Then, the individual similarity calculation unit 65 determines whether the number of regions of interest ROI is one or two or more (S3040 in
The doctor checks the similar case list 74A. In a case in which a region of interest ROI is added, as illustrated in
As illustrated in
In a case in which there are a plurality of regions of interest ROI, the individual similarity calculation unit 65 transmits the ISM tables 71 to a total similarity calculation unit 66 (Y in S3040 of
As illustrated in
As illustrated in
The similar case search unit 67 stores the data (for example, the TSM table 72A) of the results of the processes including an intermediate process which has been created during the search process based on the added region of interest ROI. In addition, in a case in which the region of interest ROI is added, the similar case list 74 (see
In this example, the number of designated regions of interest ROI increases and then the re-search process is performed. However, the number of designated regions of interest ROI may be reduced and then the re-search process may be performed.
In some cases, the case DB 23 stores a small number of cases in which a plurality of case lesions CL are registered. In this case, even assuming that a plurality of regions of interest ROI are designated, the number of cases having the case lesions, of which the number is equal to or greater than the number of designated regions of interest ROI is limited, which makes it difficult to appropriately search for a similar case from the cases. In this case, as in this example, the re-search process can be performed while the number of designated regions of interest ROI is increased or decreased. Therefore, this structure can be used according to the number of cases stored in the case DB 23, which is usable.
In a case in which the re-search process is performed while the number of designated regions of interest ROI is increased or decreased as in the third embodiment, a combination of a plurality of regions of interest ROI or the sorting order of the regions of interest ROI on the search result display screen 76 may be appropriately changed as in a fourth embodiment illustrated in
As illustrated in
The sorting order of the selection boxes 82A correspond to the sorting order of the examination images 19 including each region of interest ROI on the search result display screen 80A. For example, an examination image 19 including a region of interest ROI with No1 which is selected in a selection box 82A with number “1” is displayed on the leftmost side. Similarly, an examination image 19 including a region of interest ROI with No2 which is selected in a selection box 82A with number “2” is displayed at the center. In the event that a region of interest ROI is selected in a selection box 82A with number “3”, an examination image 19 including the selected region of interest ROI is displayed on the rightmost side, as illustrated in
A re-search button 83 is used to instruct a similar case re-search request under the conditions selected in the ROI selection portion 82. In the event that the re-search button 83 is operated, a re-search request including the conditions selected in the ROI selection portion 82 is transmitted from the treatment department terminal 11 to the similar case search server 17. A request receiving unit 61 receives the re-search request including the selected conditions.
The similar case search server 17 performs an individual similarity calculation process and a total similarity calculation process and then performs a similar case re-search process in response to the re-search request. Then, as illustrated in
The similar case list 81A is the search result in a case in which two regions of interest ROI with No1 and No2 are designated. In the similar case list 81A, for the sorting order of the regions of interest ROI, the examination image 19 including the region of interest ROI with No1 is displayed on the rightmost side and the examination image 19 including the region of interest ROI with No2 is displayed at the center. The similar case list 81B is the search result in a case in which three regions of interest ROI with No1 to No3 are designated. In the similar case list 81B, for the sorting order of the regions of interest ROI, the examination images 19 including the regions of interest ROI with No2, No3, and No1 are arranged in this order from the left side. In a case in which a similar case search process has been performed, the similar case search server 17 stores data of the processing results including the data of the similar case list, as described in the third embodiment.
The doctor in the treatment department 10 checks the similar case list 81. Then, the doctor can search for similar cases again while changing the combination or sorting order of a plurality of regions of interest ROI, using the ROI selection portion 82, assuming that it is necessary. The similar case search server 17 stores the data of the processing results. Therefore, in a case in which the data can be reused, it is possible to transmit the result of the re-search process in a short time. In addition, the treatment department terminal 11 may store the data of the transmitted similar case list 81 and display the data again, without performing a re-search process.
In the above-described embodiments, the individual similarities ISM are calculated by the correspondence between the regions of interest ROI and the case lesions CL, without determining the type of target lesion OL included in the region of interest ROI, and then similar cases are searched. However, in a fifth embodiment illustrated in
As illustrated in
In the fifth embodiment, as illustrated in
As illustrated in
In this way, it is possible to reduce the calculation time of the individual similarity calculation unit 65. In addition, the size of the ISM table 71 is smaller than that in the first embodiment in which the individual similarity ISM is calculated without distinguishing the types of case lesions, which results in a reduction in the work area of a memory. Therefore, load applied to the CPU 41B of the similar case search server 17 is reduced. As a result, it is possible to reduce the search time.
However, in the aspect in which the type of lesion is determined in advance and only the individual similarity ISM between the lesions of the same type is calculated, in a case in which the accuracy of determining the type of lesion is low, so-called search omission in which the case lesion CL to be searched as a similar case is missing is likely to occur. In particular, as illustrated in
A sixth embodiment illustrated in
However, in some cases, a small number of cases having a large number of registered case lesions CL are stored in the case DB 23. In this case, in the event that all of the cases in which the number of registered case lesions is less than the number of regions of interest ROI are excluded from the search target, the number of search targets is too small and it is difficult to appropriately search for similar cases. Even in the cases in which the number of registered case lesions CL is less than the number of regions of interest ROI, each case lesion CL is likely to be useful for a diagnosis. In the sixth embodiment, the case in which the number of registered case lesions CL is less than the number of regions of interest ROI is included in the search target and can be extracted as a similar case.
Specifically, for example, it is assumed that three types of regions of interest ROI, that is, “B: vomica”, “F: a punctate shadow (small nodules)”, and “E: a frosted glass shadow (ground glass opacity)” are designated in examination data 21. In this case, among a plurality of types of regions of interest ROI, one type of region of interest ROI needs to be designated as an essential region of interest ROI as a similar case search condition. The essentially designated region of interest ROI is, for example, a region of interest ROI that is determined to have the highest degree of importance by the doctor during a diagnosis.
In this example, a region of interest ROI (No1) including a target lesion OL, of which the type is “B: vomica”, is designated as the essential region of interest ROI (which is represented by a thick frame in
In the similar case search server 17, the request receiving unit 61 receives the similar case search request including the designated essential region of interest. The request receiving unit 61 functions as an essential designation receiving unit. A lesion type determination unit 86 determines the types of target lesions OL in three regions of interest ROI included in the received request. Since the region of interest ROI with No1 is essentially designated as the search condition, the similar case search server 17 fixes the type of the region of interest ROI with No1 and searches a similar case.
Specifically, since the type of the region of interest ROI with No1 is “B: vomica”, the similar case search server 17 fixes “B: vomica” as the search condition. Then, the similar case search server 17 searches a case that includes the fixed type “B: vomica” as the type of case lesion CL. Any case including the fixed type (“B: vomica”) case lesion CL becomes a search target, regardless of the number of registered case lesions. A case without including the fixed type (“B: vomica”) of case lesion CL is excluded from the search target.
For example, the individual similarity calculation unit 65 selects a search target, that is, determines whether to include a case in the search target or to exclude a case from the search target. The individual similarity calculation unit 65 calculates the individual similarity ISM for only the case including the case lesion CL “B: vomica” and creates an ISM table 71. In a case in which the total similarity calculation unit 66 performs the above-mentioned process, the total similarity calculation unit 66 calculates the total similarity TSM for only the case including the case lesion CL “B: vomica” and creates a TSM table 72.
In this example, the following four patterns of cases are selected as the search targets by the above-mentioned search target selection process. The first case is a case that is matched with examination data in only one type, that is, “B: vomica”, among three types of regions of interest ROI. The second case is a case that is matched with examination data in two types, that is, “B: vomica” and “F: a punctate shadow” which is the type of the region of interest ROI with No2 among three types. The third case is a case that is matched with examination data in two types, that is, “B: vomica” and “E: a frosted glass shadow” which is the type of the region of interest ROI with No3 among three types. The fourth case is a case that is matched with examination data in all of three types.
The individual similarity calculation unit 65 sets the regions of interest ROI and the case lesions CL which are the same type as the regions of interest ROI so as to be in one-to-one correspondence with each other, on the basis of the selected case, and calculates the individual similarity ISM. Since “B: vomica” which is the type of the region of interest ROI with No1 is fixed, the selected case includes the case lesion CL “B: vomica”. Therefore, the individual similarities ISM between the region of interest ROI with No1 and all of the selected cases are calculated. In this way, the ISM table 71 corresponding to the region of interest ROI with No1 is created.
Then, the individual similarity calculation unit 65 calculates the individual similarities ISM for the region of interest ROI with No2. In the case of the region of interest ROI with No2, the individual similarities ISM are calculated for the second and fourth patterns of cases including “F: a punctate shadow”. The individual similarity calculation unit 65 creates an ISM table 71 corresponding to the region of interest ROI with No2, on the basis of the calculated individual similarities ISM. The individual similarities ISM are calculated for the region of interest ROI with No3. In the case of the region of interest ROI with No3, the individual similarities ISM are calculated for the third and fourth patterns of cases including “E: a frosted glass shadow”. The individual similarity calculation unit 65 creates an ISM table 71 corresponding to the region of interest ROI with No3, on the basis of the calculated individual similarities ISM.
As such, after the ISM tables 71 corresponding to the regions of interest ROI with No1 to No3 are calculated, the total similarity calculation unit 66 calculates the total similarity TSM on the basis of combinations including the fixed type “B: vomica”, that that is, a combination of the ISM tables 71 corresponding to the region of interest ROI with No1 and No2, a combination of the ISM tables 71 corresponding to the region of interest ROI with No1 and No3, and a combination of the ISM tables 71 corresponding to the region of interest ROI with No1 to No3, and creates three types of TSM tables 72A to 72C.
The similar case search unit 67 creates a similar case list 87 on the basis of the ISM table 71 corresponding to the region of interest ROI with No1 and three types of TSM tables 72A to 72C. Here, the individual similarities ISM are recorded in the ISM table 71 and it is difficult to compare the individual similarities ISM with the total similarity TSM which is the sum of a plurality of individual similarities ISM on the same basis. Each of the TSM tables 72A and 72B is the sum of two individual similarities ISM and the TSM table 72C is the sum of three individual similarities ISM. Therefore, it is difficult to compare the tables on the same basis. For this reason, the similar case search unit 67 performs normalization such that the individual similarities ISM and the total similarities TSM in the tables 71 and 72A to 72C can be compared with each other. The cases are ranked on the basis of normalized values which are the normalized similarities. The normalization is, for example, a process that divides each of the individual similarity ISM and the total similarity TSM by the number of individual similarities ISM.
First, in a case with a case ID “C111”, for the individual similarity ISM (“0.91”) extracted from the ISM table 71, since the number of individual similarities ISM is 1, the value (“0.91”) is used as a normalized value. In a case with a case ID “C112” or “C116”, each of the total similarities TSM (“2.32” and “2.24”) which are extracted from the TSM tables 72A and 72B, respectively, is the sum of two individual similarities ISM. Therefore, values (“1.16” and “1.12”) obtained by dividing the total similarities TSM by 2 are normalized values. In a case with a case ID “C114”, since the total similarity TSM (“3.63”) extracted from the TSM table 72C is the sum of three individual similarities ISM, a value (“1.21”) obtained by dividing the total similarity TSM by 3 is a normalized value.
In this example, a simple average value obtained by dividing the total similarity by the number of individual similarities ISM is used as the normalized value. However, for example, for a case that includes a case lesion CL having a predetermined individual similarity ISM or more with respect to the region of interest ROI, such as a case with a case ID “C114”, an average value may be weighted to calculate a normalized value such that similarity is highly evaluated.
According to this example, even in a case in which the number of cases that is equal to or more than the number of regions of interest ROI is small in the case DB 23, a similar case search process can be performed effectively using these cases. Since the designated essential region of interest ROI is received, it is possible to search a case including the region of interest ROI which is considered to be important by the doctor among the search targets. Therefore, it is possible to narrow down the similar cases which are particularly useful. In this example, one of a plurality of regions of interest ROI is designated as an essential region of interest. However, two or more essential regions of interest ROI may be designated.
In a seventh embodiment illustrated in
As illustrated in
As illustrated in
In the seventh embodiment, in a case in which the lesion determination process described in the fifth embodiment illustrated in
In each of the above-described embodiments, the similar case search device according to the invention has been described in the form of the similar case search server 17 that searches for similar cases on the basis of the request from the treatment department terminal 11. However, the similar case search server 17 may not be used and the treatment department terminal 11 may be provided with the similar case search function such that the treatment department terminal 11 accesses the case DB server 16 and searches for similar cases. In this case, the treatment department terminal 11 is the similar case search device.
In each of the above-described embodiments, the similar case search server 17 and the case DB server 16 are provided as individual servers. However, the similar case search server 17 and the case DB server 16 may be integrated into one server. As such, a plurality of functions may be integrated into one server or may be distributed to different servers.
The hardware configuration of the computer system can be modified in various ways. For example, the similar case search server 17 may be formed by a plurality of server computers which are separated as hardware components in order to improve processing capability or reliability. As such, the hardware configuration of the computer system can be appropriately changed depending on required performances, such as processing capability, safety, and reliability. In addition to hardware, a program, such as the case DB 23 or the AP 50, may be duplicated or may be dispersedly stored in a plurality of storage devices in order to ensure safety or reliability.
In each of the above-described embodiments, the similar case search server 17 is used in one medical facility. However, the similar case search server 17 may be used in a plurality of medical facilities.
Specifically, in each of the above-described embodiments, the similar case search server 17 is connected to client terminals that are installed in one medical facility, such as the treatment department terminals 11, through a LAN such that it can communicate with the client terminals and provides application services related to a similar case search on the basis of requests from the client terminals. The similar case search server 17 is connected to the client terminals installed in a plurality of medical facilities through a wide area network (WAN), such as the Internet or a public telecommunication network, such that it can communicate with the client terminals. In this way, the similar case search server 17 can be used in a plurality of medical facilities. Then, the similar case search server 17 receives requests from the client terminals in the plurality of medical facilities and provides application services related to a similar case search to each client terminal.
In this case, the similar case search server 17 may be installed and operated by, for example, a data center different from the medical facilities or by one of the plurality of medical facilities. In a case in which the WAN is used, it is preferable to construct a virtual private network (VPN) or to use a communication protocol with a high security level, such as hypertext transfer protocol secure (HTTPS), considering information security.
The invention is not limited to each of the above-described embodiments and can use various structures, without departing from the scope and spirit of the invention. For example, in this example, CT images, MRI images, and plain X-ray images are given as examples of the examination image. However, the invention may be applied to examination images which are captured by other modalities, such as a mammography system or an endoscope. In addition, the above-mentioned various embodiments or various modification examples may be appropriately combined with each other. The invention is also applied to storage medium that stores programs, in addition to the program.
Number | Date | Country | Kind |
---|---|---|---|
2014-066285 | Mar 2014 | JP | national |
This application is a Continuation of PCT International Application PCT/JP2015/056369 filed on 4 Mar. 2015, which claims priority under 35 USC 119(a) from Japanese Patent Application No. 2014-066285 filed on 27 Mar. 2014. The above application is hereby expressly incorporated by reference, in its entirety, into the present application.
Number | Date | Country | |
---|---|---|---|
Parent | PCT/JP2015/056369 | Mar 2015 | US |
Child | 15276309 | US |